Matrix Relations

Go to: Introduction, Notation, Index

Congruence

Square matrices A and B are congruent if there exists a non-singular X such that B= X^TAX . Congruence is an equivalence relation.

For Hermitian congruence, see Conjuctivity.

Congruence implies equivalence.

• Congruence preserves symmetry, skewsymmetry and definiteness
• A is congruent to a diagonal matrix iff it is Hermitian.

Conjunctivity

Square matrices A and B are conjunctive or hermitely congruent or star-congruent if there exists a non-singular X such that B= X^HAX. Conjunctivity is an equivalence relation.

• If A is hermitian, it is conjunctive to a diagonal matrix of the form D=DIAG(I_p#p,-I_n#n,0_z#z). D is the intertia matrix of A and the inertia of A is the scalar triple (p,n,z).
  • Two Hermitian matrices are conjunctive iff they have the same inertia.
• If A is skew-hermitian, it is conjunctive to a matrix of the form DIAG(jI,-jI,0).
• A is conjunctive to I iff it is positive definite hermitian in which case A=U^HIU for some non-singular upper triangular U.

Conjugate

Two matrices are conjugate iff they are similar.

Direct Sum

The Direct sum of matrices A, B, ... is written A⊕B⊕... = DIAG(A, B, ...).

Equivalence

Two m#n matrices, A and B, are equivalent iff there exists a non-singular m#m matix, M, and a non-singular n#n matrix, N, with B=MAN. Equivalence is an equivalence relation.

• A and B, are equivalent iff they have the same rank.

Hadamard (or Schur) Product

The Hadamard product of two m#n matrices A and B, written in this website A • B, is formed by the elementwise multiplication of their elements. The matrices must be the same size. The functions DIAG(x) and diag(X) respectively convert a vector into a diagonal matrix and the diagonal of a square matrix into a vector. The function sum(X) sums the rows of X to produce a vector.

• If A and B are +ve definite then A • B is +ve definite.
• If A and B are +ve semi-definite then  A • B is +ve semi-definite and rank(A • B) <=rank(A)rank(B)
• A • B = B • A
• A^T • B^T = (A • B)^T
• (a • b)(c • d)^T = ac^T • bd^T = ad^T • bc^T
• a • b = DIAG(a) b
• DIAG(a • b) = DIAG(a) DIAG(b)
• (X • ab^T) = DIAG(a) X DIAG(b)
  • (X • ab^T)(Y • cd^T ) = ad^T • (X DIAG(b • c) Y)
  • (X • ab^T)y = a • ( X(b • y) )
• diag(XY) = sum(X • Y^T)

Kronecker Product

The Kronecker product of A_[m#n] and B_[p#q], written A ⊗ B or KRON(A,B), is equal to the mp#nq matrix [a(1,1)B • a(1,n)B ; • ; a(m,1)B • a(m,n)B ]. It is also known as the direct product or tensor product of A and B. The Kronecker Product operation is often denoted by a • sign enclosed in a circle which we approximate with ⊗. Note that in general A ⊗ B != B ⊗ A. In the expressions below a : suffix denotes vectorization.

• Associative: A ⊗ B ⊗ C = A ⊗ (B ⊗ C) = (A ⊗ B) ⊗ C
• Distributive: A ⊗ (B+C) = A ⊗ B + A ⊗ C
• Not Commutative: A ⊗ B = B ⊗ A iff A = cB for some scalar c
• det(A_[m#m] ⊗ B_[n#n]) = det(A)ⁿdet(B)^m.
• tr(A_[n#n] ⊗ B_[n#n]) = tr(A) tr(B)
• A^T ⊗ B^T = (A ⊗ B)^T
• A^H ⊗ B^H = (A ⊗ B)^H
• A^-1 ⊗ B^-1 = (A ⊗ B)^-1
  • A ⊗ B  is singular iff A or B is singular.
• A ⊗ B = I iff A = cI and B = c^-1I  for some scalar c.
  • I_[m#m] ⊗ I_[n#n] =  I_[mn#mn]
• A ⊗ B is orthogonal iff cA and c^-1B are orthogonal for some scalar c.
• A ⊗ B is diagonal iff A and B are diagonal.
• If Aa=pa and Bb=qb then (A ⊗ B)(a ⊗ b)=pq(a ⊗ b). The algebraic multiplicity of the eigenvalue pq is the product of the corresponding multiplicities of p and q.
• If Aa=pa and Bb=qb then (A ⊗ I + I ⊗ B)(a ⊗ b)=(p+q)(a ⊗ b)
• (ba^T): = (a^T ⊗ b): =a ⊗ b  where the : suffix denotes vectorization.
• a ⊗ (BC)= (a ⊗ B)C
  • a^T ⊗ (BC)= B(a^T ⊗ C)
  • (AB) ⊗ c= (A ⊗ c)B
    • (AB) ⊗ c^T= A(B ⊗ c^T)
  • a ⊗ b^T = ab^T
    • a^T ⊗ b = ba^T
• a^T ⊗ BC^T ⊗ d = (B ⊗ d)(a ⊗ C)^T
• a ⊗ BC^T ⊗ d^T = (a ⊗ B)(C ⊗ d)^T
•  (a ⊗ b)(c ⊗ d)^T = (ba^T): (dc^T):^T = a ⊗ bc^T ⊗ d^T= c^T ⊗ ad^T ⊗ b =  ac^T ⊗ bd^T
• AB ⊗ CD = (A ⊗ C)(B ⊗ D)
  • A_[m#n] ⊗ B_[p#q] = (A ⊗ I_[p#p])(I_[n#n] ⊗ B) = (I_[m#m] ⊗ B)(A ⊗ I_[q#q])
    • a_[m] ⊗ B_[p#q] = (a ⊗ I_[p#p])B
    • A_[m#n] ⊗ b_[p] = (I_[m#m] ⊗ b)A
    • a_[m] ⊗ B_[p] = (a ⊗ I_[p#p])b = (I_[m#m] ⊗ b)a
  • I_[n#n] ⊗ AB = (I_[n#n] ⊗ A)(I_[n#n] ⊗ B)
  • AB ⊗ I_[n#n] = (A ⊗ I_[n#n])(B ⊗ I_[n#n])
  • ab^H ⊗ cd^H = (a ⊗ c)(b ⊗ d)^H = (ca^T):(db^T):^H
  • a^Hbc^Hd = a^Hb ⊗ c^Hd = (a ⊗ c)^H(b ⊗ d) = (ca^T):^H(db^T):
  • (A ⊗ B)^H(A ⊗ B) = A^HA ⊗ B^HB
• (ABC): = (C^T ⊗ A) B:
  • (AB): = (I ⊗ A) B: = (B^T ⊗ I) A:= (B^T ⊗ A) I:
  • (Abc^T): = (c ⊗ A) b = c ⊗ Ab
  • ABc = (c^T ⊗ A) B:
  • a^TBc = (c ⊗ a)^T B: = (ac^T):^T B: = (c ⊗ a)^T B: = (a ⊗ c)^T B^T: =  B:^T (a ⊗ c) = B:^T (ca^T):
• (ABC):^T =  B:^T (C ⊗ A^T)
  • (AB):^T = B:^T (I ⊗ A^T)  = A:^T (B ⊗ I) = I:^T (B ⊗ A^T)
  • (Abc^T):^T =  b^T(c^T ⊗ A^T) = c^T ⊗ b^TA^T
  • a^TB^TC = B:^T (a ⊗ C)
• ((ABC)^T): =(C^TB^TA^T): = (A ⊗ C^T) B^T:
  • ABc = (A ⊗ c^T) B^T:
  • B_[m#n]c = (I_[m#m] ⊗ c^T) B^T:
• If Y=AXB+CXD+... then Y: = (B^T ⊗ A + D^T ⊗ C+...) X: however this is a slow and often ill-conditioned way of solving such equations for X.

In the identities below, I_n = I_[n#n] and T_m,n = TVEC(m,n) [see vectorized transpose]

• B_[p#q] ⊗  A_[m#n] = T_p,m (A  ⊗  B) T_n,q
  • (A_[m#n]  ⊗  B_[p#q]) T_n,q  = T_m,p (B ⊗ A)
  • a_[m]  ⊗  B_[p#q]  = (a ⊗ I_p)B = T_m,p (B ⊗ a)
  • A_[m#n] ⊗ b_[p] = (I_m] ⊗ b)A = T_m,p (b ⊗ A)
  • a_[m] ⊗ b_[p] = (a  ⊗ I_p)b = (I_m] ⊗ b)a
• (A ⊗ b): = A: ⊗ b
• (a_[m] ⊗ B_[p,q]): = (T_q,m  ⊗ I_p)(a ⊗ B:) = (I_q ⊗ a ⊗ I_p)B:
• (A ⊗ B): = (I_n ⊗ T_q,m ⊗ I_p)(A: ⊗ B:) = (I_n ⊗ T_q,m ⊗ I_p)(A ⊗ B:): = (T_n,q ⊗ I_mp)(A: ⊗ B): = (I_nq ⊗ T_m,p)(B: ⊗ A):

Kroneker Sum

The Kronecker sum of two square matrices, A_[m#m] and B_[n#n], is equal to  (A ⊗ I_n) +  (I_m ⊗  B). It is sometimes written A⊕B but in these pages, this notation is reserved for the direct sum.

Loewner Partial Order

We can define a partial order on the set of Hermitian matrices by writing A>=B iff A-B is positive semidefinite and A>B iff A-B is positive definite.

• The partial order is:
  • reflexive: A>=A for all A.
  • antisymmetric: A>=B and B>=A are both true iff A=B.
  • transitive: If A>=B and B>=C then A>=C.
• Any pair of hermitian matrices, A and B, satisfy precisely one of the following:
  1. None of the relations A<B, A<=,B A=B, A>=B, A>B is true.
  2. A<B and A<=B only are true.
  3. A<=B only is true.
  4. A=B, A<=B and A>=B only are true.
  5. A>=B only is true.
  6. A>B  and A>=B only are true.
• A>=B iff x^HAx >= x^HBx for all x where >= has its normal scalar meaning (likewise for >)
• A>=B iff D^HAD >= D^HBD for any, not necessarily square, D. (not true for >).
• A>B iff D^HAD > D^HBD for any non-singular D.

Orthogonal Similarity

Real square matrices A and B are orthogonally similar if there exists an orthogonal Q such that B= Q^TAQ .

Orthogonal similarity implies both similarity and congruence.

See also: Unitary similarity

Similarity

Square matrices A and B are similar (also called conjugate) if there exists a non-singular X such that B=X^-1AX . Similarity is an equivalence relation, i.e. it is reflexive, symmetric and transitive.

Similar matrices represent the same linear transformation in a different basis. Similarity implies equivalence .

• Similar matrices have the same trace, determinant, rank, nullity, eigenvalues, characteristic polynomial and minimum polynomial.
• Two square matrices are similar iff their Jordan forms contain the same hypercompanion blocks (possibly in a different order).
• Two square matrices, A and B, are similar iff there exist matrices X and Y (not both singular) such that A = XY and B = YX.

Unitary Similarity or Unitary Equivalence

Square matrices A and B are unitarily similar if there exists a unitary Q such that B= Q^HAQ . Unitary similarity is an equivalence relation and implies both similarity and conjunctivity.

• If two unitarily similar matrices, A and B, are both real then they are orthogonally similar which also implies congruence.
• Unitarily similar matrices have the same Frobenius Norm.
• A_n#n is unitarily similar to a diagonal matrix iff it is normal (this includes diagonal, hermitian, skew-hermitian and unitary matrices).