官术网_书友最值得收藏!

Encoding categorical data

The trick to encode categorical data is to expand categorical data into multiple columns, each having a 1 or 0 representing whether it's true or false. This of course comes with some caveats and subtle issues that must be navigated with care. For the rest of this subsection, I shall use a real categorical variable to explain further.

Consider the LandSlope variable. There are three possible values for LandSlope:

  • Gtl
  • Mod
  • Sev

This is one possible encoding scheme (this is commonly known as one-hot encoding):

This would be a terrible encoding scheme. To understand why, we must first understand linear regression by means of ordinary least squares. Without going into too much detail, the meat of OLS-based linear regression is the following formula (which I am so in love with that I have had multiple T-shirts with the formula printed on):

Here,is an(m x n) matrix andis an (m x 1) vector. The multiplications, therefore, are not straightforward multiplications—they are matrix multiplications. When one-hot encoding is used for linear regression, the resulting input matrixwill typically be singular—in other words, the determinant of the matrix is 0. The problem with singular matrices is that they cannot be inverted.

So, instead, we have this encoding scheme:

Here, we see an application of the Go proverb make the zero value useful for being applied in a data science context. Indeed, clever encoding of categorical variables will yield slightly better results when dealing with previously unseen data.

The topic is far too wide to broach here, but if you have categorical data that can be partially ordered, then when exposed to unseen data, simply encode the unseen data to the closest ordered variable value, and the results will be slightly better than encoding to the zero value or using random encoding. We will cover more of this in the later parts of this chapter.

主站蜘蛛池模板: 伊金霍洛旗| 汉沽区| 林西县| 漳州市| 甘德县| 浠水县| 凤城市| 巫山县| 苗栗县| 南昌县| 巩留县| 莱州市| 雷波县| 台中市| 合肥市| 清远市| 元谋县| 周至县| 平武县| 湘潭县| 琼中| 维西| 元朗区| 普宁市| 仪陇县| 电白县| 南阳市| 莱阳市| 壶关县| 彭水| 拉萨市| 兴文县| 临沂市| 宁津县| 辽宁省| 定远县| 铁岭市| 昌图县| 阿拉善左旗| 寿阳县| 当雄县|