- Effective Amazon Machine Learning
- Alexis Perrier
- 534字
- 2021-07-03 00:17:52
Training and evaluation in Amazon ML
In the context of Amazon ML, the model is linear regression and the algorithm the Stochastic Gradient Descent (SGD) algorithm. This algorithm has one main meta parameter called the learning rate and often noted , which dictates how much of a new sample is taken into account for each iterative update of the weights. A larger learning rate makes the algorithm converge faster but stabilizes further from the optimal weights, while a smaller learning induces a slower convergence but a more precise set of regression coefficients.
Given a training and a validation dataset, this is how Amazon ML tunes and select the best model:
- Amazon trains several models, each with a different learning rate
- For a given a learning rate:
- The training dataset allows the SGD to train the model by finding the best regression coefficients
- The model is used on the validation dataset to make predictions
- By comparing the quality of the predictions of the different models on that validation set, Amazon ML is able to select the best model and the associated best learning rate
- The held-out set is used as final confirmation that the model is reliable
Usual splitting ratios for the training, validation, and held-out subsets are as follows:
- Training : 70% validation and held-out 15% each
- Training : 60% validation and held-out 20% each
Shuffling: It is important to make sure that the predictors and the outcome follow the same distribution in all three subsets. Shuffling the data before splitting it is an important part of creating reliable training, validation, and held-out subsets.
It is important to define the data transformations on the training dataset and apply the transformation parameters on the validation and held-out subsets so that the validation and held-out subsets do not leak information back in the training set.
Take standardization as an example: the standard deviation and the mean of the predictors should be calculated on the training dataset. And these values then applied to standardize the validation and held-out sets. If you use the whole original dataset to calculate the mean and SGD, you leak information from the held-out set into the training set.
A common Supervised Predictive Analytics workflow follows these steps - Let's assume we have an already extracted dataset and that we have chosen a metric to assess the quality of our predictions:
- Building the dataset
- Cleaning up and transforming the data to handle noisy data issues
- Creating new predictors
- Shuffling and splitting the data into a training, a validation and a held-out set
- Selecting the best model
- Choosing a model (linear, tree-based, bayesian , ...)
- Repeat for several values of the meta parameters:
- Train the model on the training set
- Assess the model performance on the validation set
- Repeat steps 1 and 2 with new data, new predictors, and other model parameters until you are satisfied with the performances of your model. Keep the best model.
- Final test of the model on the held-out subset.
In the context of Amazon ML, there is no possibility to choose a model (step 2) other than a linear regression one (logistic regression for classification).
- GitHub Essentials
- Redis使用手冊
- 漫話大數據
- 大數據技術基礎
- Python絕技:運用Python成為頂級數據工程師
- 從0到1:數據分析師養成寶典
- Visual Studio 2015 Cookbook(Second Edition)
- 分布式數據庫系統:大數據時代新型數據庫技術(第3版)
- MySQL從入門到精通(第3版)
- 數據中心數字孿生應用實踐
- 數據庫設計與應用(SQL Server 2014)(第二版)
- Visual Studio 2013 and .NET 4.5 Expert Cookbook
- Oracle高性能SQL引擎剖析:SQL優化與調優機制詳解
- openGauss數據庫核心技術
- Google Cloud Platform for Architects