Understanding logistic regression

Despite its name, logistic regression can actually be used as a model for classification. It uses a logistic function (or sigmoid) to convert any real-valued input x into a predicted output value ? that takes values between 0 and 1, as shown in the following figure:

Rounding ? to the nearest integer effectively classifies the input as belonging either to class 0 or 1.

Of course, most often, our problems have more than one input or feature value, x. For example, the Iris dataset provides a total of four features. For the sake of simplicity, let's focus here on the first two features, sepal length--which we will call feature f₁--and sepal width--which we will call f₂. Using the tricks we learned when talking about linear regression, we know we can express the input x as a linear combination of the two features, f₁ and f₂:

However, in contrast to linear regression, we are not done yet. From the previous section we know that the sum of products would result in a real-valued, output--but we are interested in a categorical value, zero or one. This is where the logistic function comes in: it acts as a squashing function, σ, that compresses the range of possible output values to the range [0, 1]:

Now let's apply this knowledge to the Iris dataset!