Singular value decomposition |
Suppose M is an m-by-n matrix whose entries come from the field K, which is either the field of real
numbers or the field of complex numbers. A non-negative real number
λ is a singular value for M if there exist non-zero
vectors u in Km and v in Kn such that
- Mv = λu and M*u = λv
where M* denotes the conjugate
transpose of M. The vectors u and v are called left-singular and
right-singular vectors for λ, respectively.
The singular-value decomposition
theorem says that M has a factorization of the form
- M = UΣ V*
where U is an m-by-m unitary matrix
over K, V is an n-by-n unitary matrix over K, and Σ is an
m-by-n diagonal matrix whose diagonal entries Σi,i are non-negative real
numbers. Such a factorization is called a singular-value decomposition of M.
In any such singular value decomposition, the diagonal entries of Σ are necessarily equal to the singular values of
M.
The columns u1,...,um of U are eigenvectors of MM* and are left singular vectors of M. The columns
v1,...,vn of V are eigenvectors of M*M
and are right singular vectors of M. Note however that different singular value decompositions of M can
contain different singular vectors.
The linear transformation T:
Kn → Km that takes a vector x to Mx has a
particularly simple description with respect to these orthonormal
bases: we have T(vi) = di ui, for i =
1,...,min(m,n), where di is the i-th diagonal entry of D, and
T(vi) = 0 for i > min(m,n).
The number of non-zero singular values is equal to the rank
r of M. These non-zero singular values are equal to the square
roots of the non-zero eigenvalues of the positive semi-definite matrix MM*, and also
equal to the square roots of the non-zero eigenvalues of M*M.
If we focus only on these r nonzero singular values, we can construct a singular-value decomposition of the following
type:
- M = GDH*
where G is an m-by-r orthonormal
matrix over K, H is an n-by-r orthonormal matrix over K and D is an
r-by-r diagonal matrix whose diagonal entries are
positive real numbers.
The sum of the k largest singular values of M is a matrix
norm, the Ky Fan k-norm of M. The Ky Fan 1-norm is just the operator norm of M as a linear operator with respect to the Euclidean norms of
Km and Kn.
- Add applications of singular value decomposition
See also
|