The convolution layer

Convolutional Neural Networks (CNNs) are neural networks that learn filters, tensors in , which are convolved with the data. In the image domain, a filter is usually square and with small sizes ranging from 3 x 3 to 9 x 9 in pixel size. The convolution operation can be interpreted as sliding a filter over the data and, for each position, applying a dot product between the filter and the data at that position. The following diagram shows an intermediary step of convolution with stride 1 where the kernel in green is convolved with the first area in the data, represented by the red grid:

A special characteristic of CNNs is that the weights of the filters are learned. For example, if the task at hand is classifying monochromatic handwritten digits from the MNIST dataset, the ANN might learn filters that look similar to vertical, horizontal, and diagonal lines.