QR decomposition of a matrix

The general linear regression model calculation requires us to find the inverse of the matrix, which can be computationally expensive for bigger matrices. A decomposition scheme, such as QR and SVD, helps in that regard.

QR decomposition breaks a given matrix into two different matrices—Q and R, such that when these two are multiplied, the original matrix is found.

In the preceding image, X is an n x p matrix with n rows and p columns, R is an upper diagonal matrix, and Q is an n x n matrix given by:

Here, Q1 is the first p columns of Q and Q2 is the last n – p columns of Q.

Using the Math.Net method QR you can find QR factorization:

Just to prove the fact that you will get the original matrix back, you can multiply Q and R to see if you get the original matrix back:

let myMatAgain = qr.Q * qr.R 

SVD of a matrix

SVD stands for Single Value Decomposition. In this a matrix, X is represented by three matrices (the definition of SVD is taken from Wikipedia).

Suppose M is an m × n matrix whose entries come from the field K, which is either the field of real numbers or the field of complex numbers. Then there exists a factorization, called a singular value decomposition of M, of the following form:

In this preceding formula, you will find the following:

• U is an m × m unitary matrix.
• Σ is an m × n diagonal matrix with non-negative real numbers on the diagonal.
• V* is an n × n unitary matrix over K (If K = R, unitary matrices are orthogonal matrices). V* is the conjugate transpose of the n × n unitary matrix, V.

The diagonal entries, σi, of Σ are known as the singular values of M. A common convention is to list singular values in descending order. In this case, the diagonal matrix, Σ, is uniquely determined by M (though not the matrices U and V).

Using Math.NET, you can find the decomposed values of SVD as shown next:

SVD and QR decomposition of a matrix allow you to perform the multiple linear regression without performing the inverse of the matrix. Math.NET offers a couple of methods called Svd and QR to perform multiple linear regression using these decomposition schemes.