CPUs, GPUs, and other compute frameworks

Progress in AI has always been tied to our compute abilities. In this section, we will discuss CPUs and GPUs for powering AI applications, and how to set up your system to work with accelerated GPU processing. 

The main computational hardware in your computer is known as the central processing unit (CPUs); CPUs are designed for general computing workloads. While your local CPU can be used to train a deep learning model, you might find your computer hanging up on the training process for hours. When training AI applications on hardware, it's smarter to use the CPU's cousin, the Graphics Processing Unit (GPU). GPUs are designed to process in parallel, just as an ANN process in parallel. As we learned in the last chapter, AI applications require many linear algebra operations, the exact same type of operations that are required for video games. GPUs, originally designed for the gaming industry, provide us with thousands of cores to process these operations as well as parallelize them. In this manner, they lend themselves naturally to constructing deep learning algorithms. 

When selecting a GPU to utilize in deep learning applications, we're looking at three main characteristics:

• Processing power: How fast the GPU can compute; defined as cores x speed
• Memory: The ability of the GPU to handle various sizes of data
• RAM: The amount of data you can have on your GPU at any given time

In this section, we'll be focusing on utilizing the most popular GPU brand for deep learning, NVIDIA, whose CUDA toolkit makes out-of-the-box deep learning an easy task. As the major competitor to NVIDIA, Radeon AMD GPUs utilize a toolkit called OpenCL, which does not have direct compatibility with most deep learning libraries out of the box. While AMD GPUs provide great hardware at a reasonable price, it is best to go with an NVIDIA product to make getting up to speed easy. 

Should you have another GPU on your computer or no GPU at all, it is recommended that you utilize a GPU instance on AWS to follow the steps.