官术网_书友最值得收藏!

Encoding categorical data

The trick to encode categorical data is to expand categorical data into multiple columns, each having a 1 or 0 representing whether it's true or false. This of course comes with some caveats and subtle issues that must be navigated with care. For the rest of this subsection, I shall use a real categorical variable to explain further.

Consider the LandSlope variable. There are three possible values for LandSlope:

  • Gtl
  • Mod
  • Sev

This is one possible encoding scheme (this is commonly known as one-hot encoding):

This would be a terrible encoding scheme. To understand why, we must first understand linear regression by means of ordinary least squares. Without going into too much detail, the meat of OLS-based linear regression is the following formula (which I am so in love with that I have had multiple T-shirts with the formula printed on):

Here,is an(m x n) matrix andis an (m x 1) vector. The multiplications, therefore, are not straightforward multiplications—they are matrix multiplications. When one-hot encoding is used for linear regression, the resulting input matrixwill typically be singular—in other words, the determinant of the matrix is 0. The problem with singular matrices is that they cannot be inverted.

So, instead, we have this encoding scheme:

Here, we see an application of the Go proverb make the zero value useful for being applied in a data science context. Indeed, clever encoding of categorical variables will yield slightly better results when dealing with previously unseen data.

The topic is far too wide to broach here, but if you have categorical data that can be partially ordered, then when exposed to unseen data, simply encode the unseen data to the closest ordered variable value, and the results will be slightly better than encoding to the zero value or using random encoding. We will cover more of this in the later parts of this chapter.

主站蜘蛛池模板: 安徽省| 新民市| 龙门县| 西贡区| 龙游县| 淮阳县| 周口市| 郎溪县| 正镶白旗| 杭锦旗| 布尔津县| 曲周县| 呼图壁县| 巴塘县| 北海市| 宜昌市| 化德县| 重庆市| 琼海市| 即墨市| 尼勒克县| 林口县| 泾源县| 永顺县| 奈曼旗| 元阳县| 桐乡市| 苍梧县| 曲水县| 中西区| 南皮县| 长丰县| 海安县| 分宜县| 仁化县| 浪卡子县| 乌苏市| 望奎县| 汉源县| 庆城县| 莲花县|