- Generative Adversarial Networks Projects
- Kailash Ahirwar
- 276字
- 2021-07-02 13:38:49
Batch normalization
Batch normalization is a technique that normalizes the feature vectors to have no mean or unit variance. It is used to stabilize learning and to deal with poor weight initialization problems. It is a pre-processing step that we apply to the hidden layers of the network and it helps us to reduce internal covariate shift.
Batch normalization was introduced by Ioffe and Szegedy in their 2015 paper, Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. This can be found at the following link: https://arxiv.org/pdf/1502.03167.pdf.
The benefits of batch normalization are as follows:
- Reduces the internal covariate shift: Batch normalization helps us to reduce the internal covariate shift by normalizing values.
- Faster training: Networks will be trained faster if the values are sampled from a normal/Gaussian distribution. Batch normalization helps to whiten the values to the internal layers of our network. The overall training is faster, but each iteration slows down due to the fact that extra calculations are involved.
- Higher accuracy: Batch normalization provides better accuracy.
- Higher learning rate: Generally, when we train neural networks, we use a lower learning rate, which takes a long time to converge the network. With batch normalization, we can use higher learning rates, making our network reach the global minimum faster.
- Reduces the need for dropout: When we use dropout, we compromise some of the essential information in the internal layers of the network. Batch normalization acts as a regularizer, meaning we can train the network without a dropout layer.
In batch normalization, we apply normalization to all the hidden layers, rather than applying it only to the input layer.
推薦閱讀
- 構建高質量的C#代碼
- Splunk 7 Essentials(Third Edition)
- Visualforce Development Cookbook(Second Edition)
- TestStand工業自動化測試管理(典藏版)
- 機艙監測與主機遙控
- STM32G4入門與電機控制實戰:基于X-CUBE-MCSDK的無刷直流電機與永磁同步電機控制實現
- 大學C/C++語言程序設計基礎
- Storm應用實踐:實時事務處理之策略
- Troubleshooting OpenVPN
- 教育機器人的風口:全球發展現狀及趨勢
- 啊哈C!思考快你一步
- 人工智能:語言智能處理
- Puppet 3 Beginner’s Guide
- 計算智能算法及其生產調度應用
- 實戰Windows Azure