Architecture of siamese networks

Now that we have a basic understanding of siamese networks, we will explore them in detail. The architecture of a siamese network is shown in the following diagram:

As you can see in the preceding diagram, a siamese network consists of two identical networks both sharing the same weights and architecture. Let's say we have two inputs, X₁ and X₂. We feed our input X₁ to Network A, that is, f_w(X₁), and we feed our input X₂ to Network B, that is, f_w(X₂). As you will notice, both of these networks have the same weights, w, and they will generate embeddings for our input, X₁ and X₂. Then, we feed these embeddings to the energy function, E, which will give us similarity between the two inputs.

It can be expressed as follows:

Let's say we use Euclidean distance as our energy function, then the value of E will be less, if X₁ and X₂ are similar. The value of E will be large if the input values are dissimilar.

Assume that you have two sentences, sentence 1 and sentence 2. We feed sentence 1 to Network A and sentence 2 to Network B. Let's say both our Network A and Network B are LSTM networks and they share the same weights. So, Network A and Network B will generate the word embeddings for sentence 1 and sentence 2 respectively. Then, we feed these embeddings to the energy function, which gives us the similarity score between the two sentences. But how can we train our siamese networks? How should the data be? What are the features and labels? What is our objective function?

The input to the siamese networks should be in pairs, (X₁, X₂), along with their binary label, Y ∈ (0, 1), stating whether the input pairs are a genuine pair (same) or an imposite pair (different). As you can see in the following table, we have sentences as pairs and the label implies whether the sentence pairs are genuine (1) or imposite (0):

So, what is the loss function of our siamese network? Since the goal of the siamese network is not to perform a classification task but to understand the similarity between the two input values, we use the contrastive loss function.

It can be expressed as follows:

In the preceding equation, the value of Y is the true label, which will be 1 when the two input values are similar and 0 if the two input values are dissimilar, and E is our energy function, which can be any distance measure. The term margin is used to hold the constraint, that is, when two input values are dissimilar, and if their distance is greater than a margin, then they do not incur a loss.