What happens if we use too few neurons?

Imagine the case where we had no hidden layers and only an input and output. We talked about this architecture back in Chapter 1, The Building Blocks of Deep Learning, where we showed how it wouldn't be able to model the XOR function. Such a network architecture that wouldn't be able to model any nonlinearities in the data couldn't be modeled by the network. Each hidden layer presents an opportunity for feature engineering more and more complex interactions.

If you choose too few neurons, the outcome will likely be as follows:

• A really fast neural network
• That has high bias and doesn't predict very well