Getting ready

To understand the reason batch size has an impact on model accuracy, let's contrast two scenarios where the total dataset size is 60,000:

• Batch size is 30,000
• Batch size is 32

When the batch size is large, the number of times of weight update per epoch is small, when compared to the scenario when the batch size is small.

The reason for a high number of weight updates per epoch when the batch size is small is that less data points are considered to calculate the loss value. This results in more batches per epoch, as, loosely, in an epoch, you would have to go through all the training data points in a dataset.

Thus, the lower the batch size, the better the accuracy for the same number of epochs. However, while deciding the number of data points to be considered for a batch size, you should also ensure that the batch size is not too small so that it might overfit on top of a small batch of data.