Linear Regression

It's essential that we get started with a simple yet extremely effective technique that's been used for a long time: linear regression. Albert Einstein is believed to have remarked at one time or another that things should be made as simple as possible, but no simpler. This is sage advice and a good rule of thumb in the development of algorithms for machine learning. Considering the other techniques that we'll discuss later, there's no simpler model than tried and tested linear regression, which uses the least squares approach to predict a quantitative outcome. We can consider it to be the foundation of all the methods that we'll discuss later, many of which are mere extensions. If you can master the linear regression method, well then quite frankly I believe you can master the rest of this book. Therefore, let's consider this as a good starting point for our journey towards becoming a machine learning guru.

This chapter covers introductory material and an expert in this subject can skip ahead to the next topic. Otherwise, ensure that you thoroughly understand this topic before venturing to other, more complex learning methods. I believe you'll discover that many of your projects can be addressed by just applying what's discussed in the following sections. Linear regression is probably the most straightforward model to explain to your customers, most of whom will have at least a cursory understanding of R-squared. Many of them will have been exposed to it at great depth and hence will be comfortable with variable contribution, collinearity, and the like.

The following are the topics that we'll be covering in this chapter:

• Univariate linear regression
• Multivariate linear regression