(prerequisite-Fast R-CNN) Truncated SVD

• SVD(Singular Value Decomposition)

 

$$A=U\Sigma V^{\text{T}}$$

 

It is decomposed the matrix into three matrices when A is an m × n matrix.

---

Each of the three matrices meets the following conditions.

 

$U:\text{Orthogonal Matrix}m\times m(A{A}^{\text{T}}=U(\Sigma {\Sigma }^{\text{T}})U^{\text{T}})$

 

$V:\text{Orthogonal Matrix}n\times n(A^{\text{T}}A=V({\Sigma }^{\text{T}}\Sigma )V^{\text{T}})$

 

$\Sigma :\text{Rectangular Diagonal Matrix}m\times n$

 

• Transposed Matrix

 

$$M=\left[\begin{array}{c}12\\ 34\\ 56\end{array}\right]$$
$$M^{\text{T}}=\left[\begin{array}{c}135\\ 246\end{array}\right]$$

 

A matrix that changes rows and columns from the original matrix. 

 

The symbol $\text{T}$ is appended to the right of an existing matrix representation. For example, if an existing matrix is $M$, the transpose matrix is represented by $M^{\text{T}}$.

 

• Identity Matrix(Unit Matrix or Elementary matrix)

$$I=\left[\begin{array}{c}10\\ 01\end{array}\right]$$ $$I=\left[\begin{array}{c}100\\ 010\\ 001\end{array}\right]$$

 

It is the matrix which is n × n square matrix where the diagonal consist of ones and the other elements are all zeros. 

---

 

• Inverse Matrix

 

$$A\times A^{-1}=I$$ $$\left[\begin{array}{c}123\\ 456\\ 789\end{array}\right]\times \left[\begin{array}{c}\\ ?\\ \end{array}\right]=\left[\begin{array}{c}100\\ 010\\ 001\end{array}\right]$$ If matrix $A$ is multiplied by '?' matrix, and the result is an Identity Matrix, then the '?' matrix is called the Inverse Matrix of $A$, and we call it $A^{-1}$.

 

---

---

$A={\displaystyle \left[\begin{array}{cc}a& b\\ c& d\end{array}\right]}$ 에 대해 $ad-bc\ne 0$ 이면 역행렬 $A^{-1}$ 는 다음과 같이 계산된다. $$A^{-1}=\frac{1}{ad-bc}\left[\begin{array}{cc}d& -b\\ -c& a\end{array}\right]$$

 

• Orthogonal matrix

 

we have a matrix A,

 

 

and its transpose matrix $A^{\text{T}}$,

 

 

Now we will do the multiplication of both matrices in the following way,

 

 

실수 $n\times n$행렬 $A$ 에 대해서 $A\times A^T=I$를 만족하면서 $A^T\times A=I$을 만족하는 행렬 $A$를 직교 행렬이라고 합니다. 그런데 역행렬의 정의를 다시 생각해보면, 결국 직교 행렬은 $A^{-1}=A^T$를 만족합니다.

 

• Diagonal Matrix

$$\Sigma =\left[\begin{array}{c}a00\\ 0a0\\ 00a\end{array}\right]$$

 

A matrix in which the entries outside the main diagonal are all zero; the term usually refers to square matrices. Elements of the main diagonal can either be zero or nonzero. 

 

• Rectangular Diagonal Matrix

 

The term diagonal matrix may sometimes refer to a rectangular diagonal matrix, which is a m-by-n matrix.

---

• When m × n matrix, m > n

$$\Sigma =\left[\begin{array}{c}a00\\ 0a0\\ 00a\\ 000\end{array}\right]$$

---

• When m × n matrix, n > m

$$\Sigma =\left[\begin{array}{c}a000\\ 0a00\\ 00a0\end{array}\right]$$

---

• Singular Value

 

A diagonal element of a diagonal matrix Σ. 

When it expressed as σ1, σ2, σ3・・・, the Singular Value σ1, σ2, σ3・・・are sorted in descending order.

 

$$\Sigma =\left[\begin{array}{c}12.400\\ 09.50\\ 001.3\end{array}\right]$$

 

• Full SVD

 

---

 

• Reduced SVDs

 

The decomposition of matrix m×n matrix A into SVD (where m>n).

 

• Thin SVD

 

 

Remove columns of U not corresponding to rows of V*.

---

 

 

• Compact SVD

 

 

Remove vanishing singular values and corresponding columns/rows in U and V*.

---

 

Not only elements but also zero singular values are removed.

However, the calculated A becomes the same matrix as the original A.

---

• Truncated SVD

 

 

Keep only largest t singular values and corresponding columns/rows in U and V*.

---

 

Non-zero singular value is also removed, in which case the original A is not preserved, and an approximate matrix A' comes out. The matrix A′ may be used for data compression, noise removal.

 

https://en.wikipedia.org/wiki/Singular_value_decomposition

https://darkpgmr.tistory.com/106

https://wikidocs.net/24949