Comparing PyTorch to other deep learning frameworks

PyTorch is one of the main frameworks used in deep learning today. There are other widely used frameworks available too, such as TensorFlow, Theano, and Caffe. While these are very similar in many ways, there are some key differences in how they operate. These include the following:

• How the models are computed
• The way in which the computational graphs are compiled
• The ability to create dynamic computational graphs with variable layers
• Differences in syntax

Arguably, the main difference between PyTorch and other frameworks is in the way that the models themselves are computed. PyTorch uses an automatic differentiation method called autograd, which allows computational graphs to be defined and executed dynamically. This is in contrast to other frameworks such as TensorFlow, which is a static framework. In these static frameworks, computational graphs must be defined and compiled before finally being executed. While using pre-compiled models may lead to efficient implementations in production, they do not offer the same level of flexibility in research and explorational projects.

Frameworks such as PyTorch do not need to pre-compile computational graphs before the model can be trained. The dynamic computational graphs used by PyTorch mean that graphs are compiled as they are executed, which allows graphs to be defined on the go. The dynamic approach to model construction is particularly useful in the field of NLP. Let's consider two sentences that we wish to perform sentiment analysis on:

Figure 2.10 – Model construction in PyTorch

We can represent each of these sentences as a sequence of inpidual word vectors, which would then form our input to our neural network. However, as we can see, each of our inputs is of a different size. Within a fixed computation graph, these varying input sizes could be a problem, but for frameworks like PyTorch, models are able to adjust dynamically to account for the variation in input structure. This is one reason why PyTorch is often preferred for NLP-related deep learning.

Another major difference between PyTorch and other deep learning frameworks is syntax. PyTorch is often preferred by developers with experience in Python as it is considered to be very Pythonic in nature. PyTorch integrates well with other aspects of the Python ecosystem and it is very easy to learn if you have prior knowledge of Python. We will demonstrate PyTorch syntax now by coding up our own neural network from scratch.