Optimization Models
Home
Software
Links
Index
Contact
1. Introduction
1.1. Overview
1.2. Optimization Models
2. Linear Algebra
2.1. Vectors
2.2. Matrices
2.3. Linear Equations
2.4.  Least-Squares
2.5. Eigenvalues
2.6  Singular Values
3. Convex Models
3.1. Convexity
3.2. LP and QP
3.3. SOCP
3.4. Robust LP
3.5. GP
3.6. SDP
3.7. Non-Convex Models
4. Duality
4.1. Weak Duality
4.2. Strong Duality
4.3.  Applications
5. Case Studies
5.1. Senate Voting
5.2. Antenna Arrays
5.3. Localization
5.4. Circuit Design

Singular Value Decomposition of General Matrices

alt text 

The singular value decomposition (SVD) generalizes the spectral theorem (available for a square, symmetric matrix), to any non-symmetric, and even rectangular, matrix. The SVD allows to describe the effect of a matrix on a vector (via the matrix-vector product), as a three-step process: a first rotation in the input space; a simple positive scaling that takes a vector in the input space to the output space; and another rotation in the output space. The figure on the left shows the SVD of a matrix of biological data.

The SVD allows to analyze matrices and associated linear maps in detail, and solve a host of special optimization problems, from solving linear equations to linear least-squares. It can also be used reduce the dimensionality of high-dimensional data sets, by approximating data matrices with low-rank ones. This technique is closely linked to the principal component analysis method.

Outline