From A First Course in Linear Algebra
Version 1.30
© 2004.
Licensed under the GNU Free Documentation License.
http://linear.ups.edu/
Theorem SLSLC tells us that a solution to a homogeneous system of equations is a linear combination of the columns of the coefficient matrix that equals the zero vector. We used just this situation to our advantage (twice!) in Example SCAD where we reduced the set of vectors used in a span construction from four down to two, by declaring certain vectors as surplus. The next two definitions will allow us to formalize this situation.
Definition RLDCV
Relation of Linear Dependence for Column Vectors
Given a set of vectors ,
a true statement of the form
is a relation of linear dependence on . If this statement is formed in a trivial fashion, i.e. , , then we say it is the trivial relation of linear dependence on .
Definition LICV
Linear Independence of Column Vectors
The set of vectors
is linearly dependent if there is a relation of linear dependence on
that
is not trivial. In the case where the only relation of linear dependence on
is the trivial one, then
is a linearly independent
set of vectors.
Notice that a relation of linear dependence is an equation. Though most of it is a linear combination, it is not a linear combination (that would be a vector). Linear independence is a property of a set of vectors. It is easy to take a set of vectors, and an equal number of scalars, all zero, and form a linear combination that equals the zero vector. When the easy way is the only way, then we say the set is linearly independent. Here’s a couple of examples.
Example LDS
Linearly dependent set in
Consider the set of
vectors from ,
To determine linear independence we first form a relation of linear dependence,
We know that is a solution to this equation, but that is of no interest whatsoever. That is always the case, no matter what four vectors we might have chosen. We are curious to know if there are other, nontrivial, solutions. Theorem SLSLC tells us that we can find such solutions as solutions to the homogeneous system where the coefficient matrix has these four vectors as columns,
Row-reducing this coefficient matrix yields,
We could solve this homogeneous system completely, but for this example all we need is one nontrivial solution. Setting the lone free variable to any nonzero value, such as , yields the nontrivial solution
completing our application of Theorem SLSLC, we have
This is a relation of linear dependence on that is not trivial, so we conclude that is linearly dependent.
Example LIS
Linearly independent set in
Consider the set of
vectors from ,
To determine linear independence we first form a relation of linear dependence,
We know that is a solution to this equation, but that is of no interest whatsoever. That is always the case, no matter what four vectors we might have chosen. We are curious to know if there are other, nontrivial, solutions. Theorem SLSLC tells us that we can find such solutions as solution to the homogeneous system where the coefficient matrix has these four vectors as columns,
Row-reducing this coefficient matrix yields,
From the form of this matrix, we see that there are no free variables, so the solution is unique, and because the system is homogeneous, this unique solution is the trivial solution. So we now know that there is but one way to combine the four vectors of into a relation of linear dependence, and that one way is the easy and obvious way. In this situation we say that the set, , is linearly independent.
Example LDS and Example LIS relied on solving a homogeneous system of equations to determine linear independence. We can codify this process in a time-saving theorem.
Theorem LIVHS
Linearly Independent Vectors and Homogeneous Systems
Suppose that
is an matrix
and is the set of
vectors in that are
the columns of .
Then
is a linearly independent set if and only if the homogeneous system
has a unique
solution.
Proof () Suppose that has a unique solution. Since it is a homogeneous system, this solution must be the trivial solution . By Theorem SLSLC, this means that the only relation of linear dependence on is the trivial one. So is linearly independent.
() We will prove the contrapositive. Suppose that does not have a unique solution. Since it is a homogeneous system, it is consistent (Theorem HSC), and so must have infinitely many solutions (Theorem PSSLS). One of these infinitely many solutions must be nontrivial (in fact, almost all of them are), so choose one. By Theorem SLSLC this nontrivial solution will give a nontrivial relation of linear dependence on , so we can conclude that is a linearly dependent set.
Since Theorem LIVHS is an equivalence, we can use it to determine the linear independence or dependence of any set of column vectors, just by creating a corresponding matrix and analyzing the row-reduced form. Let’s illustrate this with two more examples.
Example LIHS
Linearly independent, homogeneous system
Is the set of vectors
linearly independent or linearly dependent?
Theorem LIVHS suggests we study the matrix whose columns are the vectors in ,
Specifically, we are interested in the size of the solution set for the homogeneous system . Row-reducing , we obtain
Now, , so there are free variables and we see that has a unique solution (Theorem HSC, Theorem FVCS). By Theorem LIVHS, the set is linearly independent.
Example LDHS
Linearly dependent, homogeneous system
Is the set of vectors
linearly independent or linearly dependent?
Theorem LIVHS suggests we study the matrix whose columns are the vectors in ,
Specifically, we are interested in the size of the solution set for the homogeneous system . Row-reducing , we obtain
Now, , so there are free variables and we see that has infinitely many solutions (Theorem HSC, Theorem FVCS). By Theorem LIVHS, the set is linearly dependent.
As an equivalence, Theorem LIVHS gives us a straightforward way to determine if a set of vectors is linearly independent or dependent.
Review Example LIHS and Example LDHS. They are very similar, differing only in the last two slots of the third vector. This resulted in slightly different matrices when row-reduced, and slightly different values of , the number of nonzero rows. Notice, too, that we are less interested in the actual solution set, and more interested in its form or size. These observations allow us to make a slight improvement in Theorem LIVHS.
Theorem LIVRN
Linearly Independent Vectors,
and
Suppose that
is an matrix
and is the set of
vectors in that are
the columns of .
Let
be a matrix in reduced row-echelon form that is row-equivalent to
and let
denote the number of
non-zero rows in . Then
is linearly independent
if and only if .
Proof Theorem LIVHS says the linear independence of is equivalent to the homogeneous linear system having a unique solution. Since is consistent (Theorem HSC) we can apply Theorem CSRN to see that the solution is unique exactly when .
So now here’s an example of the most straightfoward way to determine if a set of column vectors in linearly independent or linearly dependent. While this method can be quick and easy, don’t forget the logical progression from the definition of linear independence through homogeneous system of equations which makes it possible.
Example LDRN
Linearly dependent,
Is the set of vectors
linearly independent or linearly dependent? Theorem LIVHS suggests we place these vectors into a matrix as columns and analyze the row-reduced version of the matrix,
Now we need only compute that to recognize, via Theorem LIVHS that is a linearly dependent set. Boom!
Example LLDS
Large linearly dependent set in
Consider the set of
vectors from ,
To employ Theorem LIVHS, we form a coefficient matrix, ,
To determine if the homogeneous system has a unique solution or not, we would normally row-reduce this matrix. But in this particular example, we can do better. Theorem HMVEI tells us that since the system is homogeneous with variables in equations, and , there must be infinitely many solutions. Since there is not a unique solution, Theorem LIVHS says the set is linearly dependent.
The situation in Example LLDS is slick enough to warrant formulating as a theorem.
Theorem MVSLD
More Vectors than Size implies Linear Dependence
Suppose that is the
set of vectors in ,
and that . Then
is a linearly
dependent set.
Proof Form the coefficient matrix that has the column vectors , as its columns. Consider the homogeneous system . By Theorem HMVEI this system has infinitely many solutions. Since the system does not have a unique solution, Theorem LIVHS says the columns of form a linearly dependent set, which is the desired conclusion.
We will now specialize to sets of vectors from . This will put Theorem MVSLD off-limits, while Theorem LIVHS will involve square matrices. Let’s begin by contrasting Archetype A and Archetype B.
Example LDCAA
Linearly dependent columns in Archetype A
Archetype A is a system of linear equations with coefficient matrix,
Do the columns of this matrix form a linearly independent or dependent set? By Example S we know that is singular. According to the definition of nonsingular matrices, Definition NM, the homogeneous system has infinitely many solutions. So by Theorem LIVHS, the columns of form a linearly dependent set.
Example LICAB
Linearly independent columns in Archetype B
Archetype B is a system of linear equations with coefficient matrix,
Do the columns of this matrix form a linearly independent or dependent set? By Example NM we know that is nonsingular. According to the definition of nonsingular matrices, Definition NM, the homogeneous system has a unique solution. So by Theorem LIVHS, the columns of form a linearly independent set.
That Archetype A and Archetype B have opposite properties for the columns of their coefficient matrices is no accident. Here’s the theorem, and then we will update our equivalences for nonsingular matrices, Theorem NME1.
Theorem NMLIC
Nonsingular Matrices have Linearly Independent Columns
Suppose that is a
square matrix. Then
is nonsingular if and only if the columns of
form a linearly
independent set.
Proof This is a proof where we can chain together equivalences, rather than proving the two halves separately.
Here’s an update to Theorem NME1.
Theorem NME2
Nonsingular Matrix Equivalences, Round 2
Suppose that
is a square matrix. The following are equivalent.
Proof Theorem NMLIC is yet another equivalence for a nonsingular matrix, so we can add it to the list in Theorem NME1.
In Subsection SS.SSNS we proved Theorem SSNS which provided vectors that could be used with the span construction to build the entire null space of a matrix. As we have hinted in Example SCAD, and as we will see again going forward, linearly dependent sets carry redundant vectors with them when used in building a set as a span. Our aim now is to show that the vectors provided by Theorem SSNS form a linearly independent set, so in one sense they are as efficient as possible a way to describe the null space. Notice that the vectors , first appear in the vector form of solutions to arbitrary linear systems (Theorem VFSLS). The exact same vectors appear again in the span construction in the conclusion of Theorem SSNS. Since this second theorem specializes to homogeneous systems the only real difference is that the vector in Theorem VFSLS is the zero vector for a homogeneous system. Finally, Theorem BNS will now show that these same vectors are a linearly independent set. we’ll set the stage for the proof of this theorem with a moderately large example. Study the example carefully, as it will make it easier to understand the proof.
Example LINSB
Linearly independence of null space basis
Suppose that we are interested in the null space of the a
matrix,
,
which row-reduces to
The set is the set of indices for our four free variables that would be used in a description of the solution set for the homogeneous system . Applying Theorem SSNS we can begin to construct a set of four vectors whose span is the null space of , a set of vectors we will reference as .
So far, we have constructed as much of these individual vectors as we can, based just on the knowledge of the contents of the set . This has allowed us to determine the entries in slots 3, 4, 6 and 7, while we have left slots 1, 2 and 5 blank. Without doing any more, lets ask if is linearly independent? Begin with a relation of linear dependence on , and see what we can learn about the scalars,
Applying Definition CVE to the two ends of this chain of equalities, we see that . So the only relation of linear dependence on the set is a trivial one. By Definition LICV the set is linearly independent. The important feature of this example is how the “pattern of zeros and ones” in the four vectors led to the conclusion of linear independence.
The proof of Theorem BNS is really quite straightforward, and relies on the “pattern of zeros and ones” that arise in the vectors , in the entries that correspond to the free variables. Play along with Example LINSB as you study the proof. Also, take a look at Example VFSAD, Example VFSAI and Example VFSAL, especially at the conclusion of Step 2 (temporarily ignore the construction of the constant vector, ). This proof is also a good first example of how to prove a conclusion that states a set is linearly independent.
Theorem BNS
Basis for Null Spaces
Suppose that
is an
matrix, and
is a row-equivalent matrix in reduced row-echelon form with
nonzero rows.
Let and
be the sets of
column indices where
does and does not (respectively) have leading 1’s. Construct the
vectors
,
of
size
as
Define the set . Then
Proof Notice first that the vectors , are exactly the same as the vectors defined in Theorem SSNS. Also, the hypotheses of Theorem SSNS are the same as the hypotheses of the theorem we are currently proving. So it is then simply the conclusion of Theorem SSNS that tells us that . That was the easy half, but the second part is not much harder. What is new here is the claim that is a linearly independent set.
To prove the linear independence of a set, we need to start with a relation of linear dependence and somehow conclude that the scalars involved must all be zero, i.e. that the relation of linear dependence only happens in the trivial fashion. So to establish the linear independence of , we start with
For each , , consider the equality of the individual entries of the vectors on both sides of this equality in position ,
So for all , , we have , which is the conclusion that tells us that the only relation of linear dependence on is the trivial one. Hence, by Definition LICV the set is linearly independent, as desired.
Example NSLIL
Null space spanned by linearly independent set, Archetype L
In Example VFSAL we previewed Theorem SSNS by finding a set of two vectors such
that their span was the null space for the matrix in Archetype L. Writing the matrix
as ,
we have
Solving the homogeneous system resulted in recognizing and as the free variables. So look in entries 4 and 5 of the two vectors above and notice the pattern of zeros and ones that provides the linear independence of the set.
Is S linearly independent or linearly dependent? Explain why.
Is S linearly independent or linearly dependent? Explain why.
Determine if the sets of vectors in Exercises C20–C25 are linearly independent
or linearly dependent.
C20
Contributed by Robert Beezer Solution [399]
C21
Contributed by Robert Beezer Solution [399]
C22
Contributed by Robert Beezer Solution [400]
C23
Contributed by Robert Beezer Solution [400]
C24
Contributed by Robert Beezer Solution [401]
C25
Contributed by Robert Beezer Solution [402]
C30 For the matrix below, find a set that is linearly independent and spans the null space of , that is, .
Contributed by Robert Beezer Solution [403]
C31 For the matrix below, find a linearly independent set so that the null space of is spanned by , that is, .
Contributed by Robert Beezer Solution [405]
C50 Consider each archetype that is a system of equations and consider the
solutions listed for the homogeneous version of the archetype. (If only
the trivial solution is listed, then assume this is the only solution to the
system.) From the solution set, determine if the columns of the coefficient
matrix form a linearly independent or linearly dependent set. In the case of
a linearly dependent set, use one of the sample solutions to provide a
nontrivial relation of linear dependence on the set of columns of the coefficient
matrix (Definition RLD). Indicate when Theorem MVSLD applies and
connect this with the number of variables and equations in the system of
equations.
Archetype A
Archetype B
Archetype C
Archetype D/Archetype E
Archetype F
Archetype G/Archetype H
Archetype I
Archetype J
Contributed by Robert Beezer
C51 For each archetype that is a system of equations consider
the homogeneous version. Write elements of the solution set in
vector form (Theorem VFSLS) and from this extract the vectors
described in Theorem BNS. These vectors are used in a span
construction to describe the null space of the coefficient matrix for
each archetype. What does it mean when we write a null space as
?
Archetype A
Archetype B
Archetype C
Archetype D/Archetype E
Archetype F
Archetype G/Archetype H
Archetype I
Archetype J
Contributed by Robert Beezer
C52 For each archetype that is a system of equations consider the homogeneous
version. Sample solutions are given and a linearly independent spanning set is
given for the null space of the coefficient matrix. Write each of the sample
solutions individually as a linear combination of the vectors in the spanning set
for the null space of the coefficient matrix.
Archetype A
Archetype B
Archetype C
Archetype D/Archetype E
Archetype F
Archetype G/Archetype H
Archetype I
Archetype J
Contributed by Robert Beezer
C60 For the matrix below, find a set of vectors so that (1) is linearly independent, and (2) the span of equals the null space of , . (See Exercise SS.C60.)
Contributed by Robert Beezer Solution [406]
M50 Consider the set of vectors from , , given below. Find a set that contains three vectors from and such that .
Contributed by Robert Beezer Solution [408]
T10 Prove that if a set of vectors contains the zero vector, then the set is
linearly dependent. (Ed. “The zero vector is death to linearly independent sets.”)
Contributed by Martin Jackson
T12 Suppose that
is a linearly independent set of vectors, and
is a subset
of ,
(Definition SSET).
Prove that
is linearly independent.
Contributed by Robert Beezer
T13 Suppose that
is a linearly dependent set of vectors, and
is a subset
of ,
(Definition SSET).
Prove that
is linearly dependent.
Contributed by Robert Beezer
T15 Suppose that is a set of vectors. Prove that
is a linearly dependent set.
Contributed by Robert Beezer Solution [410]
T20 Suppose that is a linearly independent set in . Prove that
is a linearly independent set.
Contributed by Robert Beezer Solution [411]
T50 Suppose that
is matrix with linearly independent columns and the linear system
is
consistent. Show that this system has a unique solution. (Notice that we are not
requiring
to be square.)
Contributed by Robert Beezer Solution [413]
C20 Contributed by Robert Beezer Statement [392]
With three vectors from ,
we can form a square matrix by making these three vectors the columns of a
matrix. We do so, and row-reduce to obtain,
the identity matrix. So by Theorem NME2 the original matrix is nonsingular and its columns are therefore a linearly independent set.
C21 Contributed by Robert Beezer Statement [392]
Theorem LIVRN says we can answer this question by putting theses
vectors into a matrix as columns and row-reducing. Doing this we obtain,
With (3 vectors, 3 columns) and (3 leading 1’s) we have and the theorem says the vectors are linearly independent.
C22 Contributed by Robert Beezer Statement [392]
Five vectors from .
Theorem MVSLD says the set is linearly dependent. Boom.
C23 Contributed by Robert Beezer Statement [392]
Theorem LIVRN suggests we analyze a matrix whose columns are the vectors of
,
Row-reducing the matrix yields,
We see that , where is the number of nonzero rows and is the number of columns. By Theorem LIVRN, the set is linearly independent.
C24 Contributed by Robert Beezer Statement [392]
Theorem LIVRN suggests we analyze a matrix whose columns are the vectors
from the set,
Row-reducing the matrix yields,
We see that , where is the number of nonzero rows and is the number of columns. By Theorem LIVRN, the set is linearly dependent.
C25 Contributed by Robert Beezer Statement [393]
Theorem LIVRN suggests we analyze a matrix whose columns are the vectors
from the set,
Row-reducing the matrix yields,
We see that , where is the number of nonzero rows and is the number of columns. By Theorem LIVRN, the set is linearly dependent.
C30 Contributed by Robert Beezer Statement [393]
The requested set is described by Theorem BNS. It is easiest to find by using the
procedure of Example VFSAL. Begin by row-reducing the matrix, viewing it as
the coefficient matrix of a homogeneous system of equations. We obtain,
Now build the vector form of the solutions to this homogeneous system (Theorem VFSLS). The free variables are and , corresponding to the columns without leading 1’s,
The desired set is simply the constant vectors in this expression, and these are the vectors and described by Theorem BNS.
C31 Contributed by Robert Beezer Statement [393]
Theorem BNS provides formulas for
vectors that will meet the requirements of this question. These vectors are the
same ones listed in Theorem VFSLS when we solve the homogeneous system
,
whose solution set is the null space (Definition NSM).
To apply Theorem BNS or Theorem VFSLS we first row-reduce the matrix, resulting in
So we see that and , so the vector form of a generic solution vector is
So we have
C60 Contributed by Robert Beezer Statement [395]
Theorem BNS says that if we find the vector form of the solutions to the homogeneous
system , then
the fixed vectors (one per free variable) will have the desired properties. Row-reduce
,
viewing it as the augmented matrix of a homogeneous system with an invisible
columns of zeros as the last column,
Moving to the vector form of the solutions (Theorem VFSLS), with free variables and , solutions to the consistent system (it is homogeneous, Theorem HSC) can be expressed as
Then with given by
Theorem BNS guarantees the set has the desired properties.
M50 Contributed by Robert Beezer Statement [396]
We want to first find some relations of linear dependence on
that
will allow us to “kick out” some vectors, in the spirit of Example SCAD.
To find relations of linear dependence, we formulate a matrix
whose
columns are .
Then we consider the homogeneous sytem of equations
by
row-reducing its coefficient matrix (remember that if we formulated the
augmented matrix we would just add a column of zeros). After row-reducing, we
obtain
From this we that solutions can be obtained employing the free variables and . With appropriate choices we will be able to conclude that vectors and are unnecessary for creating via a span. By Theorem SLSLC the choice of free variables below lead to solutions and linear combinations, which are then rearranged.
Since and can be expressed as linear combinations of and we can say that and are not needed for the linear combinations used to build (a claim that we could establish carefully with a pair of set equality arguments). Thus
That the is linearly independent set can be established quickly with Theorem LIVRN.
There are other answers to this question, but notice that any nontrivial linear combination of will have a zero coefficient on , so this vector can never be eliminated from the set used to build the span.
T15 Contributed by Robert Beezer Statement [397]
Consider the following linear combination
This is a nontrivial relation of linear dependence (Definition RLDCV), so by Definition LICV the set is linearly dependent.
T20 Contributed by Robert Beezer Statement [398]
Our hypothesis and our conclusion use the term linear independence, so it will get
a workout. To establish linear independence, we begin with the definition
(Definition LICV) and write a relation of linear dependence (Definition RLDCV),
Using the distributive and commutative properties of vector addition and scalar multiplication (Theorem VSPCV) this equation can be rearranged as
However, this is a relation of linear dependence (Definition RLDCV) on a linearly independent set, (this was our lone hypothesis). By the definition of linear independence (Definition LICV) the scalars must all be zero. This is the homogeneous system of equations,
Row-reducing the coefficient matrix of this system (or backsolving) gives the conclusion
This means, by Definition LICV, that the original set
is linearly independent.
T50 Contributed by Robert Beezer Statement [398]
Let .
is consistent, so we know the system has at least one solution
(Definition CS). We would like to show that there are no more than
one solution to the system. Employing Technique U, suppose that
and
are two solution
vectors for .
By Theorem SLSLC we know we can write,
Then
This is a relation of linear dependence (Definition RLDCV) on a linearly independent set (the columns of ). So the scalars must all be zero,
Rearranging these equations yields the statement that , for . However, this is exactly how we define vector equality (Definition CVE), so .