Making Decisions with Trees

In this chapter, we are going to start by looking at our first supervised learning algorithm—decision trees. The decision tree algorithm is versatile and easy to understand. It is widely used and also serves as a building block for the numerous advanced algorithms that we will encounter later on in this book. In this chapter, we will learn how to train a decision tree and use it for either classification or regression problems. We will also understand the details of its learning process in order to know how to set its different hyperparameters. Furthermore, we will use a real-world dataset to apply what we are going to learn here in practice. We will start by getting and preparing the data and apply our algorithm to it. Along the way, we will also try to understand key machine learning concepts, such as cross-validation and model evaluation metrics. By the end of this chapter, you will have a very good understanding of the following topics:

• Understanding decision trees
• How do decision trees learn?
• Getting a more reliable score
• Tuning the hyperparameters for higher accuracy
• Visualizing the tree's decision boundaries
• Building decision tree regressors