Optimization for Neural Networks

A number of applications in deep learning require optimization problems to be solved. Optimization refers to bringing whatever we are dealing with towards its ultimate state. The problem solved through the use of an optimization process must be supplied with data, providing model constants and parameters in functions, describing the overall objective function along with some constraints.

In this chapter, we will look at the TensorFlow pipeline and various optimization models provided by the TensorFlow library. The list of topics covered are as follows:

• Optimization basics
• Types of optimizers
• Gradient descent
• Choosing the correct optimizer