Quantifying Learning Algorithms

We have stepped into an era where we are building smart or intelligent machines. This smartness or intelligence is infused into the machine with the help of smart algorithms based on mathematics/statistics. These algorithms enable the system or machine to learn automatically without any human intervention. As an example of this, today we are surrounded by a number of mobile applications. One of the prime messaging apps of today in WhatsApp (currently owned by Facebook). Whenever we type a message into a textbox of WhatsApp, and we type, for example, I am..., we get a few word prompts popping up, such as ..going home, Rahul, traveling tonight, and so on. Can we guess what's happening here and why? Multiple questions come up:

• What is it that the system is learning?
• Where does it learn from?
• How does it learn?

Let's answer all these questions in this chapter.

In this chapter, we will cover the following topics:

• Statistical models
• Learning curves
• Curve fitting
• Modeling cultures
• Overfitting and regularization
• Train, validation, and test
• Cross-validation and model selection
• Bootstrap method