- Neural Network Programming with TensorFlow
- Manpreet Singh Ghotra Rajdeep Dua
- 412字
- 2021-07-02 15:17:08
Singular value decomposition
When we decompose an integer into its prime factors, we can understand useful properties about the integer. Similarly, when we decompose a matrix, we can understand many functional properties that are not directly evident. There are two types of decomposition, namely eigenvalue decomposition and singular value decomposition.
All real matrices have singular value decomposition, but the same is not true for Eigenvalue decomposition. For example, if a matrix is not square, the Eigen decomposition is not defined and we must use singular value decomposition instead.
Singular Value Decomposition (SVD) in mathematical form is the product of three matrices U, S, and V, where U is m*r, S is r*r and V is r*n:

The following example shows SVD using a TensorFlow svd operation on textual data:
import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plts
path = "/neuralnetwork-programming/ch01/plots"
text = ["I", "like", "enjoy",
"deep", "learning", "NLP", "flying", "."]
xMatrix = np.array([[0,2,1,0,0,0,0,0],
[2,0,0,1,0,1,0,0],
[1,0,0,0,0,0,1,0],
[0,1,0,0,1,0,0,0],
[0,0,0,1,0,0,0,1],
[0,1,0,0,0,0,0,1],
[0,0,1,0,0,0,0,1],
[0,0,0,0,1,1,1,0]], dtype=np.float32)
X_tensor = tf.convert_to_tensor(xMatrix, dtype=tf.float32)
# tensorflow svd
with tf.Session() as sess:
s, U, Vh = sess.run(tf.svd(X_tensor, full_matrices=False))
for i in range(len(text)):
plts.text(U[i,0], U[i,1], text[i])
plts.ylim(-0.8,0.8)
plts.xlim(-0.8,2.0)
plts.savefig(path + '/svd_tf.png')
# numpy svd
la = np.linalg
U, s, Vh = la.svd(xMatrix, full_matrices=False)
print(U)
print(s)
print(Vh)
# write matrices to file (understand concepts)
file = open(path + "/matx.txt", 'w')
file.write(str(U))
file.write("\n")
file.write("=============")
file.write("\n")
file.write(str(s))
file.close()
for i in range(len(text)):
plts.text(U[i,0], U[i,1], text[i])
plts.ylim(-0.8,0.8)
plts.xlim(-0.8,2.0)
plts.savefig(path + '/svd_np.png')
The output of this is shown as follows:
[[ -5.24124920e-01 -5.72859168e-01 9.54463035e-02 3.83228481e-01 -1.76963374e-01 -1.76092178e-01 -4.19185609e-01 -5.57702743e-02]
[ -5.94438076e-01 6.30120635e-01 -1.70207784e-01 3.10038358e-0
1.84062332e-01 -2.34777853e-01 1.29535481e-01 1.36813134e-01]
[ -2.56274015e-01 2.74017543e-01 1.59810841e-01 3.73903001e-16
-5.78984618e-01 6.36550903e-01 -3.32297325e-16 -3.05414885e-01]
[ -2.85637408e-01 -2.47912124e-01 3.54610324e-01 -7.31901303e-02
4.45784479e-01 8.36141407e-02 5.48721075e-01 -4.68012422e-01]
[ -1.93139315e-01 3.38495038e-02 -5.00790417e-01 -4.28462476e-01
3.47110212e-01 1.55483231e-01 -4.68663752e-01 -4.03576553e-01]
[ -3.05134684e-01 -2.93989003e-01 -2.23433599e-01 -1.91614240e-01
1.27460942e-01 4.91219401e-01 2.09592804e-01 6.57535374e-01]
[ -1.82489842e-01 -1.61027774e-01 -3.97842437e-01 -3.83228481e-01
-5.12923241e-01 -4.27574426e-01 4.19185609e-01 -1.18313827e-01]
[ -2.46898428e-01 1.57254755e-01 5.92991650e-01 -6.20076716e-01
-3.21868137e-02 -2.31065080e-01 -2.59070963e-01 2.37976909e-01]]
[ 2.75726271 2.67824793 1.89221275 1.61803401 1.19154561 0.94833982
0.61803401 0.56999218]
[[ -5.24124920e-01 -5.94438076e-01 -2.56274015e-01 -2.85637408e-01
-1.93139315e-01 -3.05134684e-01 -1.82489842e-01 -2.46898428e-01]
[ 5.72859168e-01 -6.30120635e-01 -2.74017543e-01 2.47912124e-01
-3.38495038e-02 2.93989003e-01 1.61027774e-01 -1.57254755e-01]
[ -9.54463035e-02 1.70207784e-01 -1.59810841e-01 -3.54610324e-01
5.00790417e-01 2.23433599e-01 3.97842437e-01 -5.92991650e-01]
[ 3.83228481e-01 3.10038358e-01 -2.22044605e-16 -7.31901303e-02
-4.28462476e-01 -1.91614240e-01 -3.83228481e-01 -6.20076716e-01]
[ -1.76963374e-01 1.84062332e-01 -5.78984618e-01 4.45784479e-01
3.47110212e-01 1.27460942e-01 -5.12923241e-01 -3.21868137e-02]
[ 1.76092178e-01 2.34777853e-01 -6.36550903e-01 -8.36141407e-02
-1.55483231e-01 -4.91219401e-01 4.27574426e-01 2.31065080e-01]
[ 4.19185609e-01 -1.29535481e-01 -3.33066907e-16 -5.48721075e-01
4.68663752e-01 -2.09592804e-01 -4.19185609e-01 2.59070963e-01]
[ -5.57702743e-02 1.36813134e-01 -3.05414885e-01 -4.68012422e-01
-4.03576553e-01 6.57535374e-01 -1.18313827e-01 2.37976909e-01]]
Here is the plot for the SVD of the preceding dataset:

- SQL Server入門經(jīng)典
- Effective Amazon Machine Learning
- Architects of Intelligence
- Visual Studio 2015 Cookbook(Second Edition)
- Oracle RAC 11g實戰(zhàn)指南
- Python數(shù)據(jù)分析、挖掘與可視化從入門到精通
- 數(shù)據(jù)革命:大數(shù)據(jù)價值實現(xiàn)方法、技術(shù)與案例
- Hadoop 3.x大數(shù)據(jù)開發(fā)實戰(zhàn)
- gnuplot Cookbook
- 探索新型智庫發(fā)展之路:藍(lán)迪國際智庫報告·2015(下冊)
- 數(shù)據(jù)修復(fù)技術(shù)與典型實例實戰(zhàn)詳解(第2版)
- 活用數(shù)據(jù):驅(qū)動業(yè)務(wù)的數(shù)據(jù)分析實戰(zhàn)
- 大數(shù)據(jù)數(shù)學(xué)基礎(chǔ)(R語言描述)
- 利用Python進行數(shù)據(jù)分析(原書第2版)
- 數(shù)據(jù)挖掘算法實踐與案例詳解