What are siamese networks?

A siamese network is a special type of neural network and it is one of the simplest and most popularly used one-shot learning algorithms. As we have learned in the previous chapter, one-shot learning is a technique where we learn from only one training example per class. So, a siamese network is predominantly used in applications where we don't have many data points in each class. For instance, let's say we want to build a face recognition model for our organization and about 500 people are working in our organization. If we want to build our face recognition model using a Convolutional Neural Network (CNN) from scratch, then we need many images of all of these 500 people for training the network and attaining good accuracy. But apparently, we will not have many images for all of these 500 people and so it is not feasible to build a model using a CNN or any deep learning algorithm, unless we have sufficient data points. So, in these kinds of scenarios, we can resort to a sophisticated one-shot learning algorithm such as a siamese network, which can learn from fewer data points.

But how do siamese networks work? Siamese networks basically consist of two symmetrical neural networks both sharing the same weights and architecture and both joined together at the end using some energy function, E. The objective of our siamese network is to learn whether two input values are similar or dissimilar. Let's say we have two images, X₁ and X2, and we want to learn whether the two images are similar or dissimilar.

As shown in the following diagram, we feed the image X₁ to Network A and the image X₂ to another Network B. The role of both of these networks is to generate embeddings (feature vectors) for the input image. So, we can use any network that will give us embeddings. Since our input is an image, we can use a convolutional network for generating the embeddings, that is, for extracting features. Remember the role of the CNN here is only to extract features and not to classify. As we know that these networks should have the same weights and architecture, if our Network A is a three-layer CNN then our Network B should also be a three-layer CNN and we have to use the same set of weights for both of these networks. So, Network A and Network B will give us the embeddings for the input images X₁ and X₂ respectively. Then, we will feed these embeddings to the energy function, which tells us how similar the two inputs are. Energy functions are basically any similarity measure, such as Euclidean distance and cosine similarity.

Siamese networks are not only used for face recognition, but they are also used extensively in applications where we don't have many data points and tasks where we need to learn similarity between two inputs. The applications of siamese networks include signature verification, similar question retrieval, object tracking, and more. We will study siamese networks in detail in the upcoming section.