Facts and tricks about matrices.
- Matrix as a linear mapping:
- A row corresponds to an output.
- A column corresponds to an input. - Matrix multiplication:
- A matrix times a column vector is the vertical stacking of inner products between the rows of the matrix and the column vector.
- A matrix times a column vector is also a linear combination of the columns of the matrix.
- The ith row of the matrix product AB is the ith row of A times B.
- The jth column of the matrix product AB is A times the jth column of B.
- Multiplying a diagonal matrix on the right scales the columns of a matrix.
- Multiplying a diagonal matrix on the left scales the rows of a matrix.
- A product of three matrices with a diagonal matrix in the middle is a linear combination of the outer products, where the kth outer product belongs to the kth column of the left matrix and the kth row of the right matrix.
- For more intuitive interpretations, see The Art of Linear Algebra. - Gram-Schmidt process for QR factorization:
- Pseudo inverse as the least-squares (approximate) solution to a full-rank overdetermined linear system:
- Incremental rank-one update:
- A multi-objective point is Pareto optimal if there’s no point that is better than it without trading off any of the objectives.
- For multi-objective least squares, the Pareto optimal points form a convex surface. - The least-norm solution to a full-rank underdetermined linear system:
- Solving a high-order linear ODE:
- Diagonalizable linear system:
- The size of a Jordan block in the canonical form of a linear system determines the degree of the polynomial factor for an eigenvalue in the solution.
- Solution to a linear system as a linear combination of generalized modes:
- Cayley-Hamilton theorem implies:
- Continuous-time linear time-invariant system:
- For a real symmetric matrix, all the eigenvalues are real numbers and eigenvectors for different eigenvalues are orthogonal.
- Matrix norm as the maximum gain:
- Singular value decomposition (SVD) leading to the generalized pseudo inverse:
- The ith largest singular value is the distance (in terms of matrix norms) from the original matrix to any matrix of rank i-1.
- The reachable subspace of a discrete-time linear dynamical system after k steps:
- A linear system is controllable if any state is reachable.
- For a discrete-time system, a reachable state can be reached in n or fewer steps.
- For a continuous-time system, a reachable state can be reached in any amount of time ahead, with varying amount of energy required.
- The sufficient and necessary condition is that the controllability matrix has full rank.
