官术网_书友最值得收藏!

Random forests

A single Decision Tree can learn quite complex functions. However, decision trees are prone to overfitting--learning rules that work only for the specific training set and don't generalize well to new data.

One of the ways that we can adjust for this is to limit the number of rules that it learns. For instance, we could limit the depth of the tree to just three layers. Such a tree will learn the best rules for splitting the dataset at a global level, but won't learn highly specific rules that separate the dataset into highly accurate groups. This trade-off results in trees that may have a good generalization, but an overall slightly poorer performance on the training dataset.

To compensate for this, we could create many of these limited decision trees and then ask each to predict the class value. We could take a majority vote and use that answer as our overall prediction. Random Forests is an algorithm developed from this insight.

There are two problems with the aforementioned procedure. The first problem is that building decision trees is largely deterministic—using the same input will result in the same output each time. We only have one training dataset, which means our input (and therefore the output) will be the same if we try to build multiple trees. We can address this by choosing a random subsample of our dataset, effectively creating new training sets. This process is called bagging  and it can be very effective in many situations in data mining.

The second problem we might run into with creating many decision trees from similar data is that the features that are used for the first few decision nodes in our tree will tend to be similar. Even if we choose random subsamples of our training data, it is still quite possible that the decision trees built will be largely the same. To compensate for this, we also choose a random subset of the features to perform our data splits on.

Then, we have randomly built trees using randomly chosen samples, using (nearly) randomly chosen features. This is a random forest and, perhaps unintuitively, this algorithm is very effective for many datasets, with little need to tune many parameters of the model.

主站蜘蛛池模板: 阿瓦提县| 平乡县| 丘北县| 鄂尔多斯市| 樟树市| 富裕县| 阿勒泰市| 穆棱市| 易门县| 武威市| 崇州市| 顺平县| 佛山市| 射阳县| 普宁市| 客服| 青浦区| 大埔县| 西城区| 怀安县| 凤山市| 清水河县| 凤台县| 六安市| 裕民县| 临邑县| 化州市| 罗江县| 观塘区| 阿拉善盟| 大名县| 武川县| 阳高县| 台北市| 水富县| 鄄城县| 巴彦淖尔市| 女性| 吉木乃县| 漳平市| 赞皇县|