- Deep Learning with PyTorch
- Vishnu Subramanian
- 188字
- 2021-06-24 19:16:26
Optimize the neural network
We started with random weights to predict our targets and calculate loss for our algorithm. We calculate the gradients by calling the backward function on the final loss variable. This entire process repeats for one epoch, that is, for the entire set of examples. In most of the real-world examples, we will do the optimization step per iteration, which is a small subset of the total set. Once the loss is calculated, we optimize the values with the calculated gradients so that the loss reduces, which is implemented in the following function:
def optimize(learning_rate):
w.data -= learning_rate * w.grad.data
b.data -= learning_rate * b.grad.data
The learning rate is a hyper-parameter, which allows us to adjust the values in the variables by a small amount of the gradients, where the gradients denote the direction in which each variable (w and b) needs to be adjusted.
Different optimizers, such as Adam, RmsProp, and SGD are already implemented for use in the torch.optim package. We will be making use of these optimizers in later chapters to reduce the loss or improve the accuracy.
- Learning SQL Server Reporting Services 2012
- Intel FPGA/CPLD設(shè)計(jì)(基礎(chǔ)篇)
- Instant uTorrent
- Linux KVM虛擬化架構(gòu)實(shí)戰(zhàn)指南
- Creating Dynamic UI with Android Fragments
- 深入淺出SSD:固態(tài)存儲(chǔ)核心技術(shù)、原理與實(shí)戰(zhàn)
- 深入淺出SSD:固態(tài)存儲(chǔ)核心技術(shù)、原理與實(shí)戰(zhàn)(第2版)
- Artificial Intelligence Business:How you can profit from AI
- scikit-learn:Machine Learning Simplified
- 基于Apache Kylin構(gòu)建大數(shù)據(jù)分析平臺(tái)
- R Deep Learning Essentials
- 計(jì)算機(jī)組裝維修與外設(shè)配置(高等職業(yè)院校教改示范教材·計(jì)算機(jī)系列)
- 筆記本電腦維修300問
- 無蘋果不生活:OS X Mountain Lion 隨身寶典
- 單片機(jī)原理及應(yīng)用:基于C51+Proteus仿真