官术网_书友最值得收藏!

Normalization

Machine learning algorithms incrementally update the model parameters by minimizing the error between the real value and the one predicted with the last iteration's parameters. To measure this prediction error we introduce the concept of loss functions. A loss function is a measure of the prediction error. For a certain algorithm, using different loss functions will create variants of the algorithm. Most common loss functions use the L2 or the L1 norm to measure the error:

●  L2 norm:

●  L1 norm:

Where yi and ? are the real and predicted values of the samples.

The measure of the prediction error can end up being skewed when the different predictors differ by an order of magnitude. The large predictors obfuscate the importance of the smaller valued ones, thus making it difficult to infer the relative importance of each predictor in the model. This impacts how the respective weights of the linear model converge to their optimal value and as a consequence the performance of the algorithm. Predictors with the highest magnitude will end up dominating the model even if the predictor has little predictive power with regard to the real outcome value. Normalizing the data is a way to mitigate that problem by forcing the predictors to all be on the same scale.

There are two common types of normalization; data can be normalized or standardized:

  • The min-max normalization, or normalization, which sets all values between [0,1]:
  • The z-score normalization, or standardization, which normalizes with respect to the standard deviation. All predictors will have a mean of 0 and a standard deviation of 1:

The tree-based methods (decision trees, random forests, boosted trees) are the only machine learning models whose performance is not improved by normalization or standardization. All other distance/variance-based predictive algorithms may benefit from normalization. It has been shown that standardization is particularly useful for SGD, as it ensures that all the weights will be adapted at the same speed.

Efficient BackProp Yann A. LeCun et al. in Neural Networks: Tricks of the Trade pp. 9-48, Springer Verlag

Amazon ML offers z-score standardization as part of the available data transformations.

主站蜘蛛池模板: 正阳县| 广灵县| 车险| 揭西县| 松阳县| 英超| 民和| 诸暨市| 南京市| 张家口市| 阿合奇县| 疏附县| 漾濞| 攀枝花市| 辽源市| 色达县| 南昌县| 中宁县| 郑州市| 墨脱县| 庆云县| 景东| 咸丰县| 忻州市| 泰州市| 烟台市| 比如县| 临澧县| 桃园市| 贡山| 阜城县| 商都县| 亳州市| 荆州市| 大田县| 隆尧县| 卓资县| 香格里拉县| 抚远县| 乡宁县| 定边县|