Matrix Identities

Go to: Introduction, Notation, Index

Cauchy-Schwarz Inequality

• |x^Hy|² = x^Hyy^Hx <= x^Hxy^Hy for any complex vectors x and y.  [4.16]
• X^HY(Y^HY)^-1Y^HX <= X^HX where <= represents the Loewner partial order.  [4.17]

Convolution Identities

conv(a_[m],b_[n])_[m+n-1] is the convolution of a and b,. The r'th element is conv(a_[m],b_[n])_r = a_p:q^T(Jb)_r-q+1:r-p+1where p=max(1,r-n+1) and q=min(r,m).

• conv(a,b) = conv(b,a)
• conv(a+b,c)=conv(a,c)+conv(b,c)
• conv([a_[p] ; b_[q]],[c_[r] ; d_[s]]) = [conv(a,c); 0_[q+s]] + [0_[p]; conv(b,c); 0_[s]] + [0_[r]; conv(a,d); 0_[q]] + [0_[p+r]; conv(b,d)]
  • conv([a_[p] ; b],[c_[r] ; d]) = [conv(a,c); 0; 0] + [0_[p]; bc; 0] + [0_[r]; da; 0] + [0_[p+r]; bd]
  • conv([a_[m] ; b_[n]],c) = [conv(a,c); 0_[n]] + [0_[m]; conv(b,c)]
    • conv([a_[m] ; b],c) = [conv(a,c); 0] + [0_[m]; bc]
    • conv([a ; 0_[n]],c) = [conv(a,c); 0_[n]]
    • conv([0_[m] ; b],c) = [0_[m]; conv(b,c)]
• conv(Ja,Jb)=J conv(a,b)
• TOE(a_[m]) b_[n] = conv(a,b)_n:m

See also: Toeplitz for more identities relating conv() and TOE()

Durbin Recursion

If x satisfies R_[n#n] x = [1; 0] and R is a symmetric toeplitz matrix and hence of the form TOE([Jr; 0]+[0; r])_[n#n] for some r_[n], where J is the exchange matrix. Then

• We can find y to solve Sy = [1; 0] where
  • S is the augmented symetric toeplitz matrix given by TOE([s; Jr; 0]+[0; r; s])_[n+1#n+1]
  • k = [r; s]^T J [x; 0]
  • y_[n+1] = ([x; 0] - k J [x; 0])/(1-k^2)

This recursion gives an efficient way of solving equations of the form R x = [1; 0].

Inversion Sums

• A + B = A (A^-1 + B^-1) B = B (A^-1 + B^-1) A = A (A^-1 + A^-1BA^-1) A = B (B^-1 + B^-1AB^-1) B
  • A + I = A (A^-1 + I) = (A^-1 + I) A
  • A^-1 + B^-1 = A^-1 (A + B) B^-1 = B^-1 (A+ B) A^-1 = A^-1 (A + AB^-1A) A^-1 = B^-1 (B+ BA^-1B) B^-1
  • A^-1 + I = A^-1 (A + I) = (A + I) A^-1
• AB + BC = A (A^-1B + BC^-1) C
• (A^-1 + B^-1)^-1 = A (A + B)^-1B = B (A + B)^-1A = A - A (A + B)^-1A = B - B (A + B)^-1B
  • (A^-1 + I)^-1 = A (A + I)^-1 = (A + I)^-1A = A - A (A + I)^-1A = I - (A + I)^-1

Inversion Identities

These identities are useful because it says how a matrix changes if you add a bit onto its inverse. They are variously called the Matrix Inversion Lemma, Sherman-Morrison formula and Sherman-Morrison-Woodbury formula.

• [ (I+V^HAU) non-singular]: (A^-1 + UV^H)^-1 = A - AU(I + V^HAU)^-1V^HA     [4.2]
  • [ (I+V^HA^-1U) non-singular]: (A + UV^H)^-1 = A^-1 - A^-1U(I + V^HA^-1U)^-1V^HA^-1
  • [v^HAu != -1]: (A^-1 + uv^H)^-1 = A - Auv^HA/(1 + v^HAu)
  • [ (C+V^HAU) non-singular]: (A^-1 + UC^-1V^H)^-1 = A - AU(C + V^HAU)^-1V^HA
    • [ (C+V^HAV) non-singular]: (A^-1 + VC^-1V^H)^-1 = A - AV(C + V^HAV)^-1V^HA
  • [ (I+V^HAV) non-singular]: (A^-1 + VV^H)^-1 = A - AV(I + V^HAV)^-1V^HA
  • [ (I+V^HAU) non-singular]: (A^-1 + UV^H)^-1U = AU(I + V^HAU)^-1
  • [ (I+V^HAU) non-singular]: V^H(A^-1 + UV^H)^-1 = (I + V^HAU)^-1V^HA
  • [ (I+V^HAU) non-singular]: V^H(A^-1 + UV^H)^-1U = I-(I + V^HAU)^-1
• det(A^-1 + UV^H) = det(I + V^HAU) × det(A^-1) [4.3] sometimes called the Matrix Determinant Lemma
  • det(A^-1 + uv^H) = (1 + v^HAu) × det(A^-1)

Projection Matrix Identities

These identities show how the matrix that projects onto the column-space of X changes if extra columns are added to X. We define the projection matrices P(X)=X(X^HX)^#X^H and Q(X)=I-P(X), these respectively project onto the column space and null space of X.

• P([X Y]) = P(X) + P(Q(X)Y) = P(X) + Q(X) Y (Y^H Q(X) Y)^# Y^H Q(X)
  • [y^HQ(X)y = 0] P([X y]) = P(X)
  • [y^HQ(X)y != 0] P([X y]) = P(X) + P(Q(X)y) = P(X) + Q(X) y y^H Q(X) / (y^H Q(X) y)
  •  
    • [y^HQ(X)y != 0] z^HP([X y])z = z^HP(X)z + |z^HQ(X) y|² / (y^H Q(X) y)

• Q([X Y]) = Q(X) - Q(X) Y (Y^H Q(X) Y)^# Y^H Q(X)

Toeplitz and Hankel Identities

If a vector is multiplied by a triangular toeplitz  matrix, it is possible to exchange the components of the vector and matrix. All matrices below are n#n.

• TOE([0_[n-1]; p_[n]])q = TOE([0_[n-1]; q_[n]])p = conv(p,q)_1:n
• TOE([ p_[n]; 0_[n-1]])q = TOE([ q_[n]; 0_[n-1]])p = conv(p,q)_n:2n-1

Similar expressions may be obtained for hankel matrices by observing that if T is toeplitz then TJ is hankel and Tp=(TJ)(Jp)

In the identity below, the matrix on the left of the = is a symmetric toeplitz matrix with 2a₁ on the main diagonal. The matrix on the right is the sum of a lower triangular toeplitz and an upper triangular hankel matrix and has 2b as its first column.

• TOE([Ja_[n]; 0_[n-1]]+[0_[n-1]; a_[n]])_[n#n]b = (TOE([b_[n]; 0_[n-1]])_[n#n]J+TOE([0_[n-1]; b_[n]])_[n#n])a .