From A First Course in Linear Algebra
Version 1.32
© 2004.
Licensed under the GNU Free Documentation License.
http://linear.ups.edu/
This section is contributed by Elizabeth Million.
You may have once thought that the natural definition for matrix multiplication would be entrywise multiplication, much in the same way that a young child might say, “I writed my name.” The mistake is understandable, but it still makes us cringe. Unlike poor grammar, however, entrywise matrix multiplication has reason to be studied; it has nice properties in matrix analysis and additionally plays a role with relative gain arrays in chemical engineering, covariance matrices in probability and serves as an inertia preserver for Hermitian matrices in physics. Here we will only expore the properties of the Hadamard product in matrix analysis.
Definition HP
Hadamard Product
Let and
be
matrices. The
Hadamard Product of
and is
defined by
for all ,
.
(This definition contains Notation HP.)
As we can see, the Hadamard product is simply “entrywise multiplication”. Because of this, the Hadamard product inherits the same benefits (and restrictions) of multiplication in . Note also that both and need to be the same size, but not necessarily square. To avoid confusion, juxtaposition of matrices will imply the “usual” matrix multiplication, and we will use “” for the Hadamard product.
Example HP
Hadamard Product
Consider
Then
Now we will explore some basics properties of the Hadamard Product.
Theorem HPC
Hadamard Product is Commutative
If and
are
matrices
then .
Proof The proof follows directly from the fact that multiplication in is commutative. Let and be matrices. Then
With equality of each entry of the matrices being equal we know by Definition ME that the two matrices are equal.
Definition HID
Hadamard Identity
The Hadamard identity is the
matrix
defined by
for all ,
.
(This definition contains Notation HID.)
Theorem HPHID
Hadamard Product with the Hadamard Identity
Suppose is
an matrix.
Then .
Proof
With equality of each entry of the matrices being equal we know by Definition ME that the two matrices are equal.
Definition HI
Hadamard Inverse
Let be an
matrix and
suppose
for all ,
. Then the
Hadamard Inverse,
, is given by
for all ,
.
(This definition contains Notation HI.)
Theorem HPHI
Hadamard Product with Hadamard Inverses
Let be an
matrix
such that
for all ,
. Then
.
Proof
With equality of each entry of the matrices being equal we know by Definition ME that the two matrices are equal.
Since matrices have a different inverse and identity under the Hadamard product, we have used special notation to distinguish them from what we have been using with “normal” matrix multiplication. That is, compare “usual” matrix inverse, , with the Hadamard inverse , and the “usual” matrix identity, , with the Hadamard identity, . The Hadamard identity matrix and the Hadamard inverse are both more limiting than helpful, so we will not explore their use further. One last fun fact for those of you who may be familiar with group theory: the set of matrices with nonzero entries form an abelian (commutative) group under the Hadamard product (prove this!).
Theorem HPDAA
Hadamard Product Distributes Across Addition
Suppose ,
and
are
matrices.
Then .
Proof
With equality of each entry of the matrices being equal we know by Definition ME that the two matrices are equal.
Theorem HPSMM
Hadamard Product and Scalar Matrix Multiplication
Suppose ,
and and
are
matrices.
Then .
Proof
With equality of each entry of the matrices being equal we know by Definition ME that the two matrices are equal.
We can relate the Hadamard product with matrix multiplication by considering diagonal matrices, since if and only if both and are diagonal (Citation!!!). For example, a simple calculation reveals that the Hadamard product relates the diagonal values of a diagonalizable matrix with its eigenvalues:
Theorem DMHP
Diagonalizable Matrices and the Hadamard Product
Let be a diagonalizable
matrix of size
with eigenvalues .
Let
be a diagonal matrix from the diagonalization of
,
, and
be a vector
such that
for all .
Then
That is,
Proof
With equality of each entry of the matrices being equal we know by Definition ME that the two matrices are equal.
We obtain a similar result when we look at the singular value decomposition of square matrices (see exercises).
Theorem DMMP
Diagonal Matrices and Matrix Products
Suppose ,
are
matrices,
and and
are diagonal
matrices of size
and ,
respectively. Then,
Proof
With equality of each entry of the matrices being equal we know by Definition ME that the two matrices are equal.
Also,
With equality of each entry of the matrices being equal we know by Definition ME that the two matrices are equal.
T10 Prove that
if and only if both
and
are diagonal matrices.
Contributed by Elizabeth Million
T20 Suppose , are matrices, and and are diagonal matrices of size and , respectively. Prove both parts of the following equality hold:
Contributed by Elizabeth Million
T30 Let be a square matrix of size with singular values . Let be a diagonal matrix from the singular value decomposition of , (Theorem SVD). Define the vector by , . Prove the following equality,
Contributed by Elizabeth Million
T40 Suppose , and are matrices. Prove that for all ,
Contributed by Elizabeth Million
T50 Define the diagonal matrix of size with entries from a vector by
Furthermore, suppose ,
are
matrices.
Prove that
for all .
Contributed by Elizabeth Million