官术网_书友最值得收藏!

Random forests

A single Decision Tree can learn quite complex functions. However, decision trees are prone to overfitting--learning rules that work only for the specific training set and don't generalize well to new data.

One of the ways that we can adjust for this is to limit the number of rules that it learns. For instance, we could limit the depth of the tree to just three layers. Such a tree will learn the best rules for splitting the dataset at a global level, but won't learn highly specific rules that separate the dataset into highly accurate groups. This trade-off results in trees that may have a good generalization, but an overall slightly poorer performance on the training dataset.

To compensate for this, we could create many of these limited decision trees and then ask each to predict the class value. We could take a majority vote and use that answer as our overall prediction. Random Forests is an algorithm developed from this insight.

There are two problems with the aforementioned procedure. The first problem is that building decision trees is largely deterministic—using the same input will result in the same output each time. We only have one training dataset, which means our input (and therefore the output) will be the same if we try to build multiple trees. We can address this by choosing a random subsample of our dataset, effectively creating new training sets. This process is called bagging  and it can be very effective in many situations in data mining.

The second problem we might run into with creating many decision trees from similar data is that the features that are used for the first few decision nodes in our tree will tend to be similar. Even if we choose random subsamples of our training data, it is still quite possible that the decision trees built will be largely the same. To compensate for this, we also choose a random subset of the features to perform our data splits on.

Then, we have randomly built trees using randomly chosen samples, using (nearly) randomly chosen features. This is a random forest and, perhaps unintuitively, this algorithm is very effective for many datasets, with little need to tune many parameters of the model.

主站蜘蛛池模板: 廊坊市| 两当县| 达孜县| 汨罗市| 白玉县| 资阳市| 南昌县| 台南县| 沙坪坝区| 文昌市| 万年县| 蚌埠市| 东海县| 玛沁县| 慈利县| 奉新县| 吴旗县| 鹤峰县| 陵川县| 福贡县| 湖州市| 甘洛县| 新建县| 广元市| 高雄市| 红安县| 腾冲县| 沈丘县| 从化市| 福贡县| 措勤县| 五常市| 岳阳市| 汉寿县| 北流市| 威信县| 岫岩| 阳城县| 虞城县| 马鞍山市| 长兴县|