- Deep Learning with PyTorch
- Vishnu Subramanian
- 188字
- 2021-06-24 19:16:26
Optimize the neural network
We started with random weights to predict our targets and calculate loss for our algorithm. We calculate the gradients by calling the backward function on the final loss variable. This entire process repeats for one epoch, that is, for the entire set of examples. In most of the real-world examples, we will do the optimization step per iteration, which is a small subset of the total set. Once the loss is calculated, we optimize the values with the calculated gradients so that the loss reduces, which is implemented in the following function:
def optimize(learning_rate):
w.data -= learning_rate * w.grad.data
b.data -= learning_rate * b.grad.data
The learning rate is a hyper-parameter, which allows us to adjust the values in the variables by a small amount of the gradients, where the gradients denote the direction in which each variable (w and b) needs to be adjusted.
Different optimizers, such as Adam, RmsProp, and SGD are already implemented for use in the torch.optim package. We will be making use of these optimizers in later chapters to reduce the loss or improve the accuracy.
- 電腦維護與故障排除傻瓜書(Windows 10適用)
- 電腦常見問題與故障排除
- 硬件產品經理成長手記(全彩)
- Learning Stencyl 3.x Game Development Beginner's Guide
- Mastering Adobe Photoshop Elements
- Practical Machine Learning with R
- 電腦高級維修及故障排除實戰(zhàn)
- 固態(tài)存儲:原理、架構與數(shù)據安全
- Istio服務網格技術解析與實踐
- Blender 3D By Example
- The Deep Learning with PyTorch Workshop
- 多媒體應用技術(第2版)
- 施耐德M241/251可編程序控制器應用技術
- 超炫的35個Arduino制作項目
- Unreal Engine 4 AI Programming Essentials