Applied math basics

When we talk about mathematics as related to deep learning and AI, we're often talking about linear algebra. Linear algebra is a branch of continuous mathematics that involves the study of vector space and operations performed in vector space. If you remember back to grade-school algebra, algebra in general deals with unknown variables. With linear algebra, we're extending this study into linear systems that have an arbitrary number of dimensions, which is what makes this a form of continuous mathematics.

AI relies on the basic building block of the tensor. Within AI, these mathematical objects store information throughout ANNs that allow them to operate; they are data structures that are utilized throughout AI. As we will see, a tensor has a rank, which essentially tells us about the indices of the data (how many rows and columns the data has). 

While many problems in deep learning are not formally linear problems, the basic building blocks of matrices and tensors are the primary data structures for solving, optimizing, and approximating within an ANN. 

Want to see how linear algebra can help us from a programmatic standpoint? Take a look at the following code block: 

import numpy as np
## Element-wise multiplication without utilizing linear algebra techniques

x = [1,2,3]
y = [4,5,6]

product = []
for i in range(len(x)):
    product.append(x[i]*y[i])

## Element-wise multiplication utilizing linear algebra techniques

x = np.array([1,2,3])
y = np.array([4,5,6])
x * y

We can eliminate strenuous loops by simply utilizing NumPy's built-in linear algebra functions. When you think of AI, and the thousands upon thousands of operations that have to be computed at the runtime of an application, the building blocks of linear algebra can also help us out programmatically. In the following sections, we'll be reviewing these fundamental concepts in both mathematical notation and Python.