Getting Started with Neural Networks

In this chapter, we will be focusing on the basics of neural networks, including input/output layers, hidden layers, and how the networks learn through forward and backpropagation. We will start with the standard multilayer perceptron networks, talk about their building blocks, and illustrate how they learn step-by-step. We will also introduce a few, popular standard models such as Convolutional Neural Networks (CNN), Restricted Boltzmann Machines (RBM), and recurrent neural network (RNN) as well as its variation Long Short-Term Memory (LSTM). We will outline the key, critical components for the successful application of the models, and explain some important concepts to help you gain a better understanding of why these networks work so well in certain areas. In addition to a theoretical introduction, we will also show example code snippets using TensorFlow on how to construct layers and activation functions, and how to connect different layers. In the end, we will demonstrate an end-to-end example of MNIST classification using TensorFlow. With the setup you learned from Chapter 2, Getting Yourself Ready for Deep Learning it’s time we jump into some real examples to get our hands dirty.

The outline of this chapter is as follows:

• Multilayer perceptrons:
  • The input layers
  • The output layers
  • Hidden layers
  • Activation functions
• How a network learns
• Deep learning models:
  • Convolutional Neural Networks
  • Restricted Boltzmann Machines
  • RNN/LSTM
• MNIST hands-on classification example