- Learning Data Mining with Python(Second Edition)
- Robert Layton
- 379字
- 2021-07-02 23:40:09
Random forests
A single Decision Tree can learn quite complex functions. However, decision trees are prone to overfitting--learning rules that work only for the specific training set and don't generalize well to new data.
One of the ways that we can adjust for this is to limit the number of rules that it learns. For instance, we could limit the depth of the tree to just three layers. Such a tree will learn the best rules for splitting the dataset at a global level, but won't learn highly specific rules that separate the dataset into highly accurate groups. This trade-off results in trees that may have a good generalization, but an overall slightly poorer performance on the training dataset.
To compensate for this, we could create many of these limited decision trees and then ask each to predict the class value. We could take a majority vote and use that answer as our overall prediction. Random Forests is an algorithm developed from this insight.
There are two problems with the aforementioned procedure. The first problem is that building decision trees is largely deterministic—using the same input will result in the same output each time. We only have one training dataset, which means our input (and therefore the output) will be the same if we try to build multiple trees. We can address this by choosing a random subsample of our dataset, effectively creating new training sets. This process is called bagging and it can be very effective in many situations in data mining.
The second problem we might run into with creating many decision trees from similar data is that the features that are used for the first few decision nodes in our tree will tend to be similar. Even if we choose random subsamples of our training data, it is still quite possible that the decision trees built will be largely the same. To compensate for this, we also choose a random subset of the features to perform our data splits on.
Then, we have randomly built trees using randomly chosen samples, using (nearly) randomly chosen features. This is a random forest and, perhaps unintuitively, this algorithm is very effective for many datasets, with little need to tune many parameters of the model.
- 從零開始構建企業級RAG系統
- Mastering AWS Lambda
- Cocos2D-X權威指南(第2版)
- Python 深度學習
- C語言程序設計(第2版)
- 編程珠璣(續)
- Python王者歸來
- Swift 3 New Features
- Functional Programming in JavaScript
- 持續輕量級Java EE開發:編寫可測試的代碼
- Python 3.7從入門到精通(視頻教學版)
- Natural Language Processing with Python Quick Start Guide
- Data Science Algorithms in a Week
- Clojure High Performance Programming(Second Edition)
- Visual C++程序設計全程指南