Chapter 2. Approaching Simple Linear Regression

Having set up all your working tools (directly installing Python and IPython or using a scientific distribution), you are now ready to start using linear models to incorporate new abilities into the software you plan to build, especially predictive capabilities. Up to now, you have developed software solutions based on certain specifications you defined (or specifications that others have handed to you). Your approach has always been to tailor the response of the program to particular inputs, by writing code carefully mapping every single situation to a specific, predetermined response. Reflecting on it, by doing so you were just incorporating practices that you (or others) have learned from experience.

However, the world is complex, and sometimes your experience is not enough to make your software smart enough to make a difference in a fairly competitive business or in challenging problems with many different and mutable facets.

In this chapter, we will start exploring an approach that is different from manual programming. We are going to present an approach that enables the software to self-learn the correct answers to particular inputs, provided you can define the problem in terms of data and target response and that you can incorporate in the processes some of your domain expertise—for instance, choosing the right features for prediction. Therefore, your experience will go on being critical when it comes to creating your software, though in the form of learning from data. In fact, your software will be learning from data accordingly to your specifications. We are also going to illustrate how it is possible to achieve this by resorting to one of the simplest methods for deriving knowledge from data: linear models.

Specifically, in this chapter, we are going to discuss the following topics:

• Understanding what problems machine learning can solve
• What problems a regression model can solve
• The strengths and weaknesses of correlation
• How correlations extends to a simple regression model
• The when, what, and why of a regression model
• The essential mathematics behind gradient descent

In the process, we will be using some statistical terminology and concepts in order to provide you with the prospect of linear regression in the larger frame of statistics, though our approach will remain practical, offering you the tools and hints to start building linear models using Python and thus enrich your software development.