Class Notes on Linear Algebra

8/12/2019 Class Notes on Linear Algebra

1/15

:::Linear Algebra Lectures ::: MAL101 :::January 2014

Lecture 1

(1) We study system of linear equations with complex coefficients. But for most of the casesthe coefficients will be rational numbers. Give examples of such systems. Explain what is asolution of a system of linear equations. Geometrically describe solution of an equation whennumber of unknowns is two or three and the coefficients are real numbers (respectively as aline in a plane and a plane in space). Also describe what is a solution of a system of linear

equations when the number of unknowns is two or three (as intersection of lines or planes etc.).(2) Examples of systems which have (a) no solutions, (b) has a unique solution, (c) infinitely many

solutions (when number of unknowns is two or three or four):a) x1+ x2+ x3= 3, x1+ 2x2+ 3x3= 6, x2+ 2x3= 1;b) x1+ x2+ x3= 3, x1+ 2x2+ 3x3= 6, x1+ x2+ 2x3= 4;c) x1+ x2+ x3 = 3, x1+ 2x2+ 3x3 = 6, x2+ 2x3 = 3. Verify that the first system has nosolution, (1, 1, 1) is the only solution of the second system and the third equation has infinitelymany solutions. Indeed, for any real number , the tuple (, 32, ) is a solution.

(3) Students should recall Crammers rule and any other methods they learnt to solve a systemof equations. The Crammers rule is applicable when number of unknowns and equations aresame and the determinant of the coefficient matrix is nonzero. Discuss the limitations. We willnot disucss Crammers rule in the class.

(4) We write a general system of m linear equations with real (or complex) coefficients with nunknownsx1, x2, . . . , xn as

a11x1 +a12x2 + + a1nxn = b1

a21x1 + a22x2 + + a2n xn = b2

am1x1+ am2x2+ +amnxn = bm

ai,j (1 i m, 1 j n) are called coefficients and b1, b2, . . . , bm are called constantterms of the equations. We call the system a homogenous system if bi = 0 for each i and a

nonhomogeneous system if it is not homogenous.(5) The above system can be described by the following matrix equation: AX = B where A =

a11 a12 a1na21 a22 a2n am1 am2 amn

, B =

b1b2 bm

and Xis the matrix (or column) of unknowns

x1x2. . .

xn

. A is

called the coefficient matrix and B the matrix (or column) of constants. The matrix (A|B) iscalled the augmented matrix of the system.

In a), A=

1 1 11 2 3

0 1 2

, B =

36

1

, X=

x1x2

x3

and (A|B) =

1 1 1 | 31 2 3 | 6

0 1 2 | 1

.

In b), A=

1 1 11 2 31 1 2

, B =

364

, X=

x1x2x3

and(A|B) =

1 1 1 | 31 2 3 | 61 1 2 | 4

.

In c), A=

1 1 11 2 30 1 2

, B =

363

, X=

x1x2x3

and (A|B) =

1 1 1 | 31 2 3 | 60 1 2 | 4

.

Remark: i) A solution of a system of equations in n unknowns is an n-tuple whose componentsare from R (or C or Q).ii) The homogeneous equations always has a solution (namely, the zero tuple).iii) When scalars are complex numbers geometric description of the solution set is not what wehave when scalars are real.Warning: Before we write the matrix equation we should arrange the unknowns in the same


2/15

2

Lecture 2

(6) Recall that the collection R of all the real numbers is not just a set. There are two binaryoperations (namely, addition and multiplications) along with certain nice properties (namely,commutativity, associativity, distributivity of multiplication over addition, also difference ofnumbers and division by nonzero real numbers etc.)

Note that R can be replaced by C (complex numbers) or Q (rational numbers). However,mostly we will use R. For solving a system of linear equations using N (natural numbers) orZ (integers) only will not be considered. When we solve linear equations, we take differenceor/and ratio, so we always take some field (in this course, R, Qor C).

(7) To have a geometric idea we consider R as a straight line and R2 as plane and R3 as the spaceetc. If x= (x1, x2, . . . , xn),y = (y1, y2, . . . , yn) Rn and a R, define x+ y = (x1+y1, x2+y2, . . . , xn+ yn) andax= (ax1, ax2, . . . , a xn).

(8) Introducematricesand the symbolMmn(R). Recall properties of matrix addition and matrixmultiplication and recall trace and determinant of square matrices. Define column androw matrix. IfA is an n n matrix with (i, j)-th entry equal to ai,j then tr(A) =

n

i=1aii

and det(A) =n

i=1aiiAii whereAij is (1)

i+j det of the matrix obtained from A by simplyremoving the i-th row and j-th column. It is not difficult to verify using the definition thattr(A + B) = tr(A) + trB and tr(AB) = tr(BA). It may be harder to verify that det(AB) =det A det B.

(9) Define row reduced echelon matrix (RRE). A matrix is called row reduced echelon if thefollowing properties hold.

(a) Every zero row is below every nonzero row.(b) The leading coefficient of every nonzero row is 1.(c) A column which contains leading nonzero entry (which is 1) of a row has all other coeffi-

cients equal to zero.(d) Suppose the matrix has r nonzero rows (and remaining m r rows are zero). If leading

nonzero entry ofi-th row (1 i r) occurs in the ki-th column, then k1< k2 < < kr.

Consider the following matrices: a)

1 0 20 0 00 1 0

is not RRE since it has a zero row preceding a

nonzero row, b)

1 0 10 2 00 0 0

is not RRE since, he second row is nonzero but its leading non zero

entry is 2 (= 1), c)

1 1 20 1 10 0 0

is not RRE since the leading nonzero coefficient of the second

row is in the second column but the second column has another nonzero coefficien (in the first

row), d)

1 0 20 1 30 0 1

is not RRE since the the leading nonzero coefficient of the third row is

in the third column and the third column has another nonzero coefficients , e)

1 0 20 1 30 0 0

is

RRE, f )

0 1 2

1 0 30 0 0

is not RRE since in this there are two nonzero rows and k1= 2 andk2= 1violatingK1 < k2.

(10) Recall: What is an equivalent relation (a relation which is reflexive, symmetric and transitive).We are going to describe an equivalence relation (called row equivalence) on the collectionMn(F) ofn n matrices with coefficients from F = R,Cor Q.

(11) Define elementary row operations. Namely,(a) Multiplying the i-th row by a nonzero scalar denoted by Ri Ri.(b) Interchanging i-th row andj -th row denoted by Ri Rj.(c) Replacing the i-th row by the sum of i-th row and multiple of j-th row denoted by

Ri Ri+ Rj.


3/15

3

Lecture 3

(12) Students should describe how determinant of a square matrix would change if an elementaryrow operation is applied (in each of the three cases).

(13) If B is obtained from A by an elementary row operation then find how to get A from B.Students have to describe the inverse operations in each cases. Students should note that anelementary row operation is an invertible map from Mmn(R) to itself.

(14) Denote an elementary row operation by . If A Mmn(R) then (A) = (I) A (pre-multiplication of A by (I)) where I is the m m identity matrix. Students should verifythis fact in each case. If is an elementary row operation then (I) is referred to as anelementary matrix.

(15) Use the above rule successively on a matrix and get the following two statements for any finitelymany elementary row operations 1, 2, . . . , s:(a) (s 2 1)(A) = (s 2 1)(I)A.(b) (s 2 1)(A) =s(I) 2(I)1(I)A.

(16) A matrixA is said to berow equivalentto the matrixB if there is a finitely many elementaryrow operations1, 2, . . . , s such that B = (s 2 1)(A).

(17) Observe that row equivalence is an equivalence relation.(a) Ais row equivalent to itself. Recall that the identity map is composition of two elementary

row operations (for instance, composition ofRi Rj with itself).(b) Ais row equivalent toB if and only ifB is row equivalent to A. IfB = (s 21)(A),

then A = (11 1s1

1s )(B) and inverse of an elementary row operation is an

elementary row operation.(c) IfA is row equivalent to B andB is row equivalent toCthenA is row equivalent to C. IfB = (s 21)(A) andC= (s+t s+2s+1)(B) thenC= (s+t 21)(A).

IfA is row equivalent to B we will often write A B .(18) One can write an algorithm to show, Every matrix is row equivalent to a unique row reduced

echelon matrix. This statement is same as Every equivalence class of row equivalent matricescontains a unique row reduced echelon matrix.

Step 1: Appy interchange of rows to push down the zero rows to the end of the matrix sothat no zero row is before a nonzero row.Step 2: Find the first nonzero column (from left) (suppose it is k1).Step3: Again apply interchange of rows to push up a row whose leading nonzero coefficientoccurs in first nonzero column (i.e. in the k1-th column), to the first row. Divide the first row

by the leading nonzero coefficient so that the leading nonzero coefficient becomes 1.Step 4. Next apply apply Ri Ri R1for suitable values ofi and so that the first nonzerocolumn has nonzero coefficient only in the first row.Step5: Find the first nonzero column (from left) when we ignore the first row (suppose it isk2).Apply interchange of rows to push up a row whose leading nonzero coefficient occurs in columnk2, to the second row. Divide the second the second row by the leading nonzero coefficient.Step 6: ApplyRi R2for suitable values ofi and so thatk2column has nonzero coefficientsonly in the second row.Step 7: Find the first nonzero row when we ignore the first two rows. continue. When there isno nonzero rows left, stop.

0 0 4 10 3 0 10 0 0 00 4 2 0

Apply R3 R4. Get

0 0 4 10 3 0 10 4 2 00 0 0 0

. Apply R1 R2. Get

0 3 0 10 0 4 10 4 2 00 0 0 0

.Next

apply R1 1/3R1 . Get

0 1 0 1/30 0 4 10 4 2 00 0 0 0

. Apply R3 R3 4R1, get

0 1 0 1/30 0 4 10 0 2 4/30 0 0 0

.

Appy R2 1/4R2, get

0 1 0 1/30 0 1 1/40 0 2 4/30 0 0 0

. Apply R3 R3 2R2, get

0 1 0 10 0 1 1/40 0 0 11/60 0 0 0

.


4/15

4

Apply,R3 6/11R3, get

0 1 0 10 0 1 1/40 0 0 10 0 0 0

. ApplyR1 R1 R3 andR2 R2 1/4R3 get

0 1 0 00 0 1 00 0 0 10 0 0 0

, which is RRE.

Remark: There are more than one ways of getting the row reduced echelon form of a matrix.

Lecture 4(19) Following applications of row equivalence of matrices are to be discussed:(a) Finding rank of a matrix. IfA is row equivalent to the row reduced echelon matrix R then

the number of nonzero rows ofR is called the rank ofA. In particular, the rank of a rowreduced echelon matrix is the number of nonzero rows. We will see later that the row rankof a matrix (i.e. dimension of the row space) is the rank of the matrix.

The rank of

0 0 4 10 3 0 10 0 0 00 4 2 0

is 3. The Rank of

1 1 1 1 11 2 1 3 10 1 0 2 01 0 1 1 1

is two. Student must

show that this matrix is row equivalent to

1 0 1 1 10 1 0 2 0

0 0 0 0 00 0 0 0 0

which is row reduced echelon(so that the rank of the given matrix is 2).Remark: We will see use of rank of a matrix.

(b) Determining whether a square matrix is invertible. In case it is invertible finding theinverse. Theorem: SupposeR is row reduced echelon and row equivalent to A. ThenAis invertible if and only ifR is the identity matrix. In fact,Algorithm: Suppose (A| I) is row equivalent to (R| B) and R is row reduced echelon.ThenA is invertible if and only ifR = Iand in this case B = A1.If (k 2 1)(A) = I, then (k 2 1)(I) A = I. Therefore A

1 =(k 2 1)(I).

Example: Suppose A =1 1 11 2 1

1 2 3

. Apply elementary row operations on the matrix

(A|I) to find (R|B) where R is the RRE matrix row equivalent to A.

ApplyR2 R2 R1 and thenR3 R1 on

1 1 1 | 1 0 01 2 1 | 0 1 01 2 3 | 0 0 1

and get

1 1 1 | 1 0 00 1 0 | 1 1 00 1 2 | 1 0 1

. Then apply R1 R1 R2 and R3 R3 R1 and get

1 0 1 | 2 1 00 1 0 | 1 1 00 0 2 | 0 1 1

. Then applyR3 1/3R3 and then R1 R1 R3 and get

1 0 0 | 2 1/2 1/20 1 0 | 1 1 00 0 1 | 0 1/2 1/2

. Conclude that A1 =

2 1/2 1/21 1 00 1/2 1/2

.

Remark: Algorithm to find row reduced echelon form of a matrix, also determines whethera square matrix is invertible and if it is invertible, it can be used to find the inverse.

(c) Solving a system of linear equations. Obviously, AX = 0 always has a solution (forinstance,xi= 0 i). Suppose we have the following system of linear equations: AX=B .Observe the following statements:


5/15

5

i) If (A| B) is row equivalent to (A| B), then the two systems AX = B and AX = B

have the same solutions.Reasoning: Since(P Q) = (P)Q we have AX = B if and only (A)X = (B) for anycomposition of finitely many elementary row operations.ii) IfA is row reduced echelon matrix then one can determine whether AX = B has asolution and what are all solutions just by inspection.Reasoning: Suppose A is a row reduced echelon matrix. If there is a row i of (A|B) suchthati-th row ofA a zero row but i-th row ofB nonzero thenAX=B has no solution.Suppose AX = B has a solution. Assume that A has r nonzero rows and the leadingnonzero coefficient of i-th row is in the ki-th coefficients. Callxk1, . . . xkr dependent un-knowns. Assign arbitrary values to all xi fori {k1, k2, . . . , kr} then writexk1 , . . . , xkr asa linear combination as the independent unknowns.Discuss enough examples.(We will discuss later, if the system is homogenous then the dimension of the solutionspace isn r.)


6/15

6

Lecture 5

(20) Observation: If (A|B) (A|B) (row equivalent) then the the following system have exactlysame set of solutions: AX=B and AX=B .Reasoning: If = k k1 1 is a composition of elementary row operations then(AX) =(B) (A)X=(B) AX=B .Suppose A Mmn(F) is a row reduced echelone matrix having r nonzero rows (and m rzero rows). Further assume that the leading nonzero coefficient of the i-th nonzero row occursin theki-th column. Call the xk1, xk2, . . . , xkr to bedependent unknownsand the remainingunknowns as independent or free unknowns.The system has no solution if any one of the last m r rows ofB is nonzero because such arow gives an inconsistent equation 0 = br+i for some i 1. If the last m r rows are zerorows then the system has a solution. In this case, assign arbitrary values to the independentunknowns and then write the dependent unknowns in terms of arbitrary values assigned to theindependent unknowns.

(21) Thus, AX = B has a solution if and only if rank(A) = rank(A|B). Further, AX = B has aunique solution if and only if rank(A) = rank(A|B) = n, the number of unknowns (so thatn r= 0 is the number of free unknowns).

(22) Now we discuss the three system of equations in the first lectures using the method.

In a), A =

1 1 11 2 30 1 2

, B=

361

, X=

x1x2x3

and (A|B) =

1 1 1 | 31 2 3 | 60 1 2 | 1

.

Apply elementary row operations on (A|B) to find the row reduced echelon form of the first

block. (A|B)R2R2R1

1 1 1 | 30 1 2 | 30 1 2 | 1

R1R1R2,R3R3R2

1 0 1 | 00 1 2 | 30 0 0 | 2

.

Since the third row of the first block is zero but the third entry of the second column is nonzerothe system has no solution.

In b), A =

1 1 11 2 31 1 2

, B =

364

, X=

x1x2x3

and(A|B) =

1 1 1 | 31 2 3 | 61 1 2 | 4

.

In this case, (A|B) =R2R2R1,R3R3R1

1 1 1 | 30 1 2 | 30 0 1 | 1

R1R1R2

1 0 1 | 00 1 2 | 30 0 1 | 1

R1R1+R3,R2R22R3

1 0 0 | 10 1 0 | 10 0 1 | 1

. Thus, r = 3 = n = m, ki = i for each i = 1, 2, 3

so that there is no independent unknown. In fact the solution is x1= x2= x3 = 1 (unique).

In c), A =

1 1 11 2 30 1 2

, B =

363

, X =

x1x2x3

and (A|B) =

1 1 1 | 31 2 3 | 60 1 2 | 3

R2R2R1

1 1 1 | 30 1 2 | 30 1 2 | 3

R1R1R2,R3R3R2

1 0 1 | 00 1 2 | 30 0 0 | 0

.

In this case, n= m = 3, r = 2, k1 = 1, k2 = 2. x3 is the independent unknown so assign any

arbitrary value to it, say x3= . Then x1 = 0 andx2+ 2= 3. Thus the general solutionof the system is (, 3 2, ).

(23) Denote by ei, (1 i n) the n-tuple with 1 in the i-th component and 0 in the other

components. Then (a1, a2, . . . , an) =n

i=1

aiei. In fact anyn-tuple is a unique linear combination

ofeis (recall that twon-tuples are equal if and only if each component of the first tuple is equalto the corresponding component of the second tuple). Supposef1= e1+e2+e3, f2= e2+e3, f3=e3 R

3. Then (x,y,z) =af1+ bf2+ cf3for somea,b, c R. In fact,a = x, b= y x, c= z y.Further, choices for a,b, care unique (prove this).


7/15

7

More generally, the set {e1, e2, . . . , en} can be replaced by some other setX={f1, f2, . . . , f n}ofn elements from Rn so that every element ofRn can be written as a linear combination oftuples belonging to X.Note that that not any choice ofn vectors fi will do. We need a set ofn linearly independentset of vectors. Xis said to be linear independent if a linear combination of vectors belongingto X is zero only when each coefficient is zero. Further the number of elements in X cannotbe changed if we want every vector ofRn is represented as a linear combination of elements ofX in a unique way.

Explain geometrically when n = 1, 2, 3.


8/15

8

Lecture 6

(24) Suppose V is a nonempty set. A mapVV V is said to be a binary operation. Usinga binary operation on Vwe can combine any two elements ofVto get a third element ofV.Suppose + is a binary operation on V. SupposeF= R,Qor C and there is a map F V V(i.e., there is a way to multiply an element ofFto an element ofV). Then we say that V is avector space over Fif the following conditions are satisfied.(a) v+w = w +v for any v, w V(addition of vectors is commutative),(b) u+ (v+w) = (u+v) +w for any u, v,w V(addition of vector is associative),(c) there is a unique vector denoted (often) by 0 (same notation as the zero ofF) such that

v+ 0 =v for any v V.(d) for every vector v Vthere is a vector denoted by v V such that v+v = 0,(e) a(bv) =ab)v for any a, b F and v V,(f) (a+b)v= av +bv for any a, b F and v V,(g) a(v+w) =av +aw for any a F, v, w V.(h) 1v= v for each v V(where 1 is the usual 1 in F).

The following examples can be understood easily: Rn over R, Fn over F, Mmn(R) over R,

Mmn(F) over F. In these, addition of vectors is component wise addition and scalar multipli-

cation is by multiplying every component by the scalar. The space C[0, 1] is the collection ofcontinuous real valued functions on the interval [0, 1] where f+g is the function which takes

valuef(x) + g(x) at xand for any scalar a R, afis the function which takes value af(x) atx. F[X] denotes the collection of polynomials in X with coefficients from F. One can verifythat F[X] is a vector space over F under usual addition of polynomials (also we know how tomultiply a scalar to a polynomial). We define for every n N, P

n:={f F[X] : deg f < n}.

Denote the space of all the sequences in Fby Maps(N,F) and F :={{xn}: xn= 0 for all butfinitely many n N}.

Students should understand the vector space structure of these.(25) Remark: Cn is a vector space over R as well as over C. To specify the base field we may use

the notation Cn(R) and Cn(C) (respectively).Remark: Not difficult to show that when the zero scalar is multiplied to a vector, the out putis the zero vector. For instance, 0v= (0 + 0)v 0v= 0v+ 0v. Adding 0v to both sides we

have 0 = 0v. Similarly, justify that if a scalar is multiplied to the zero vector the out put isthe zero vector.

(26) Suppose V is a vector space over F. A nonempty subset W of V is said to be a subspaceif W is a vector space under the operations obtained by restricting the operations of V toW. Thus we require, u, v W u+ v W and a F, u W au W. E.g.,W1 :={(x, y) F

2 : x + 2y = 0} is a subspace ofF2; but W2 :={(x, x+ 1) : x R} is not asubspace ofR2. For instance, (1, 2), (2, 3) W2 but (1, 2) + (2, 3) = (3, 5) / W3.

W3:={

a bc d

M22(F) :a = c +d, b= c d} is a subspace ofM22(F).

F is a subspace of Maps(N,F).Pn is a subspace ofF[X].Remark: 1. R is not a vector space over C. Because if a real number (= 0) is multiplied bya non-real complex number then we get a non-real complex number.2. Every subspace must contain the zero vector. Thus a subset which does not contain thezero vector is not a subspace.3. The singleton set {0} is a vector space over evey field. The only element is the zero vectorof this space. This space is called the zero space. The zero space is a subspace of every vectorspace.


9/15

9

Lecture - 7 and 8

(27) A Criterion for a subset to be a subspace- Theorem: A necessary and sufficient condition fora non-empty subset Wof a vector space over Fis that u, v W anda F au + v W.Proof: This condition is necessary because ifW is a subspace ofV, sum of any two vectors inWmust be in Wand every scalar multiple of an element ofW is also in W.

Conversely, suppose u, v W and a F au+ v W.Since W is nonempty, may picku W. Then (1)u+ u W and 0 = (1)u+ u, hence 0 W. Let u, v W. Thenu+v = 1u+v and 1 F so that u+v W. Further, u W, a F au+ 0 W andau + 0 =au so that au W.

Theorem: A necessary and sufficient condition for a nonempty subset Sof a vector space tobe a subspace is a, b F, u , v S au + bv S.

(28) Suppose A Mmn(F). ThenW = {X Fn : AX = 0} is a subspace ofFn. This is called

the solution space of AX = 0. (Warning: Note that here a vector in Fn is written as acolumn matrix rather than ann-tuple.) Conversely, every subspace ofFn is a solution space ofa homogenous system of linear equations in n unknowns.

(29) Suppose S is a subset of the vector space V over F. Denote by S the intersection of allthe subspaces ofV which contain S, then S is a subspace ofV. (Proof: If u, v are in theintersection of all the subspaces containing S, then u, v and hence au+v is in each subspacecontainingS for any a F. Thus, au + v is in the intersection of all the subspaces containingS

. This shows that the intersection of all the subspaces containingS

is a subspace (using thecriterion above.) ) We call S the subspace generated by S in V. By definition it is thesmallest subspace containing S. IfS= , the empty set, then S= {0}, the zero space.Define span(S) :={a1v1+ a2v2+ + arvr : r N, ai R, vi S}. Show that span(S) is asubspace containing S. By convention, ifSis empty, we write spanS={0}, the zero space.Observations: (i) S S, (ii) S span(S), (iii) Sand span(S) are both subspaces ofV.

Theorem: S= span(S).Proof: By definition,S is contained in every subspace which contains S. Since span(S) is asubspace containing S, S span(S). To show the other containment, an arbitrary elementofspan(S) is a finite linear combination of elements ofS. Such an element is contained in everysubspace containing S. Therefore, in particular span(S) S (Recall that X Y X, Y).Exercise: Geometrically describe the possible subspaces ofR2 and R3.Find span(S) for the following S: a) S = {(1, 1, 1), (1, 2, 3)}, b) S = {(1, 1, 1), (1, 1, 2)}.Write the space as solution space for a suitable matrix.

(30) A subset Xof a vector space V over F is said to be linearly dependent if there exist a finitesubset {v1, v2, . . . , vm} ofXand scalarsa1, a2, . . . , am Fsuch that at least one of these scalarsis nonzero anda1v1 + a2v2 + + amvm= 0. E.g., in R

3, the set X={(n,n,n) :n N, n 2}is linearly dependent. Since (2, 2, 2), (3, 3, 3) Xand 3(2, 2, 2) 2(3, 3, 3) so that X is linearlydependent. A subset Xof a vector space is said to be linearly independent if it is not linearlydependent. Thus a subset of a vector space is either linear independent or linear dependent (andcannot be both). Thus, a subset Xof a vector space Vis said to be linearly independent if itsevery subset is linearly independent. A finite subset X={v1v2, . . . , vm} is linearly independentif and only ifa1v1+ a2v2+ + amvm= 0 a1 = a2= = am= 0.

The set X={(1, 2, 3), (2, 3, 4), (1, 1, 2)} is linearly independent. To see this consider a(1, 2, 3)+b(2, 3, 4) + c(1, 1, 2) = 0. Then (a + 2b+ c, 2a + 3b+ c, 3a+ 4b + 2c) = (0, 0, 0). Thus,a + 2b + c= 0, 2a + 3b + c= 0, 3a + 4b + 2c= 0. Next, we have to apply the method of solvinga system of linear equations to show that a = 0, b= 0, c= 0 is the only possible solution. Findthe row reduced echelon form of the coefficient matrix and find all the solutions. Students maysee the QWS 2 (in the course webpage) to understand this concept.

Remark: Students must observe the following (equivalent) statements:(a) A subset of a linearly independent set is linearly independent.(b) A superset of a linearly dependent set is linearly dependent.


10/15

10

(31) A subset B of a vector space Vis said to be a basisofV ifB is linearly independent and everyvector in Vis a linear combination of finitely many vectors belonging to B. We have already

seen that {e1, e2, . . . , en} is a basis ofFn for any field F. Definefi =

i

j=1

ej for i = 1, 2, . . . , n.

Show that{f1, f2, . . . , f n}is a basis ofRn and the set{1, X , X 2, . . . , X n, . . . }is a basis ofF[X].

Remarks: Students should make the following observations:(a) A basis of a vector spaceVis a maximal linearly independent subset ofV . In other words,

a superset of a basis is linearly dependent.Proof: Letv V butv B . Thenv = a1v1 + a2v2 + + arvr for somev1, v2, . . . , vr B

anda1, a2, . . . , ar F. Hence,v a1v1 a2v2 arvr = 0 which assures that B {v}is linearly dependent.

(b) A maximal linearly independent subset of a vector space is a basis.Proof: Suppose S is a maximal linear independent subset of a vector space V. Letv V. If v S, then v is a linear combination of element of S. If v S then byassumptionS {v}is linearly dependent. So there are vectorsv1, v2, . . . , vr Ssuch thatav+a1v1+a2v2+ +arvr = 0 for somea, a1, a2, ar F. Observe that ifa = 0 the eachaiiszero asSis linearly independent. Thusa = 0. Thenv = a1a1v1a

1a2v2 a1arvr

so that v is a linear combination of vectors in S.(c) IfB is a basis of a vector space V, then every vector in Vis a unique linear combination

of elements ofB . That is, there is only one way of writing a vector as a linear combinationof basis vectors.Proof: Supposev = a1v1 + a2v2 + + amvm= b1v1 + b2v2 + + bmvm(One may wonderwhat if a different set of vectors occur in the second expression! You supply zero coefficientsto certain vectors if necessary.) Then we have (a1b1)v1+(a2b2)v2+ +(ambm)vm= 0.Since {v1, v2, . . . , vm} B , it is linearly independent and soa1b1= a2b2 = ambm= 0so that ai = bi for each (1 i m).

We assume the following fact without proof: Theorem Every vector space has a basis.

(32) LemmaAny linearly independent finite set of vectors is a part of a basis. In other words, anyfinite linearly independent set of vectors can be extended to a basis of the vector space.Proof: Suppose S := {v1, v2, . . . , vm} is a linear indepent set of vectors and B is a basis. Wewill show that we get a new basis ofVby substituting a suitable vector ofB by v1. There exists

w1, w2, . . . , wr B such that v1 = a1w1+ a2w2+ +arwr for some a1, a2, . . . ar(= 0) R.Thenw1= a

1

1 v1a

1

1 a2w2 a

1

1 arwr. ThenB1:= B \{w1} {v1} is a basis ofVand it is

an easy check. Next we can get a new basisB2 fromB1substituting a suitable vector (differentfromv1) byv2(using the same procedure). For instance, v2 = b1u1+b2u2+ +btut+av1whereat least one of b1, b2, . . . , bt is nonzero (since {v1, v2} is a linearly independent set). Supposeb1= 0 and define, B2= {v2} B1 \ {u1}

This process will stop in m steps and we get a basis Bn ofV which contains Sas a subset.

(33) IfVhas a finite basis with n vectors then any other basis is finite and has n vectors.Proof: Suppose B1 and B2 are two bases of a vector space. Assume without loss of generalitythat |B1| |B2| (otherwise interchange the role ofB1 and B2). SupposeS is a subset ofB2containing as many elements as B1 possesses. SinceB2 is linearly independent, S is linearly

independent. By the procedure described in the above Lemma, we can construct a basis B

from B1 such that S B and |B| = |B1|. Since |B1| = |S|, we must have S = B. If B2properly contains S then B2 is linearly dependent (since S is a basis). HenceS=B2 so that|B1|= |B2|.

(34) If a vector space has no finite basis, we say that it is an infinite dimensional vector space.F[X], Maps(N,R),C[0, 1] are infinite dimensional vector space. We have proved that if a vectorspace has a finite basis then any two basis have the same number of elements. This number iscall the dimension of the vector space. We write dim(V) or dimF(V). Show the following byproducing a basis in each case: dimR(R

n) =n, dimC(Cn) =n, dimR(C

n) = 2n, dimF(Pn) =n,dimFMmn(F) =mn.


11/15

11

Lecture 9 and 10

(35) A basis B (of a vector space) with an ordering of the elements (ofB) is called an orderedbasis. Now on we will consider finite dimensional vector spaces (unless otherwise stated) andordered bases of vector spaces.

Suppose B = {v1, v2, . . . , vn}be a basis of a vector spaceVover the field F. Fix the orderingof elements inB as they are listed. Then we know thatv = a1v1 + a2v2 + + anvn (recall thatthe coefficients are uniquely determined). We write [v]B for the column matrix (a1 a2 an)

T

which is in Fn. [v]B is called the coordinate vector ofv with respect to the basis B. Withrespect to the standard basis {e1, e2, . . . , en} the coordinates of (a1, a2, . . . , an) is the column(a1 a2 an)

T. Observe that B = {(1, 1), (1, 1)} is a basis of R2. Let v = (a, b). Note,(a, b) =x(1, 1) + y(1, 1) if an only ifx + y= a and x y = b if and only ifx= a+b

2 , y= ab

2 .

Hence [(a, b)]B = (a+b2

ab2

)T =

a+b2

ab2

. Next, supposeB ={(1, 2), (2, 1)}. ThenB is another

basis ofR2. Then [(a, b)]B is

2ba3

2ab3

.

Thus, the coordinate vector of a vector depends upon the basis. Next we will see how thecoordinate vector changes with a change of basis..

(36) Change of basis (relation between coordinates with respect to old and new bases). SupposeB = {v1, v2, . . . , vn} and B

= {w1, w2, . . . , wn} are bases of a vector space V over a field F.Suppose

(1) vj =

ni=1

pijwi

Suppose for a vector v Vand [v]B = (a1 a2 an)T. Then

v=n

j=1

ajvj =n

j=1

aj

ni=1

pi,jwi=n

i=1

(n

i=1

pi,jaj)wi.

Thus [v]B =

ni=1

p1,jaj

ni=1

p2,jaj

n

i=1

pn,jaj

=P

a1a2

an

whereP is the n nmatrix whose (i, j)-th entry is pi,j .

Then [v]B =P[v]B.

Remark: The matrix Pwhich occur in the change of basis is invertible. Suppose

(2) wk =n

j=1

qjkvj

for (1 k n). Denote by Q the matrix whose (i, j)-th coefficient is qi,j. One can see thatP Q= QP=I. Indeed, use (1) in (2) to get

wk =

nj=1

qjk

ni=1

pijwi

=n

i=1

nj=1

pijqjk

wi

Since {w1, w2, . . . , wn} is a basis, we must haven

j=1

pijqjk = i,k. Therefore, P Q = I. In

particular,P is invertible and Q= P1.Exercise: Find Pfor the example in subsection 35 and verify the relation.

Hint: If (1, 1) = a(1, 2) +b(2, 1) and (1, 1) = c(1, 2) +d(2, 1) then P =

a cb d

.


12/15

12

Let ei,j Mmn(F) be the matrix whose (i, j)-th coefficient is 1 and any other coefficient iszero. Show that {ei,j : 1 i m, 1 j n} is a basis ofMmn(F).

(37) Let A Mmn(F). Denote the i-th row (1 i m) by Ri. Then Ri M1n(F). Arow can be thought as an n-tuples. With this understanding M1n(F) is identified with F

n.span(R1, R2, . . . , Rm) as a subspace ofF

n is called the row space ofA. The dimension of therow space ofA is called the row rank ofA. SupposeS1, S2 are subspaces of a vector space.IfS1 span(S2) and S2 span(S1) then span(S1) = span(S2). This statement can be usedto understand the following statement: If A and B are row equivalent matrices then theirrow spaces and hence their row ranks are same. How will you prove it? IfA and B are rowequivalent then every row ofA is a linear combination of rows ofB (hence in the span of rowsofB ) and every row ofB is a linear combination of rows ofA (hence in the span of rows ofA).

In particular, let B be the row reduced echelon form ofA. Then rankA= rankB = row rankofB=row rank ofA. Thus, rankA= row rank ofA.Remark: If a matrix is row reduced echelon, the nonzero rows form a basis of the row space.(Students should prove this statement.) Thus, to find a basis of the row space of a matrix A,we require to find the row reduced echelon form ofA.

(38) Theorem: The dimension of the solution space ofAX= 0 is n r where n is the number ofunknowns and r is the rank ofA.Proof: Suppose R is row reduced echelon form ofA. Then the solution space ofAX= 0 issame as the solution space ofRX= 0. We need to show that the dimension of the solution

space ofRX

= 0 isn

r

wherer

is number of nonzero rows ofR

. Suppose for 1i

r

, theleading nonzero coefficient of thei-th row is in theki-th column. SinceRis row reduced echelon,k1< k2


13/15

13

Lecture 11 and 12

(39) Suppose W1, W2 are subspaces of a vector space V over F. Then define

W1+W2:= {w1+w2: w1 W1, w2 W2}.

This is a subspace ofV and it is call the sum ofW1and W2. Students must verify that W1+W2is a subspace ofV(use the criterion for a subspace).

Examples:

(a) Let V = R2, W1 = {(x, x) : x R} and W2 = {(x,x) : x R}. ThenW1+ W2 = R2.

Indeed, (x, y) = ( x+y2

, x+y2

) + (xy2

,xy2

).

(b) Next, let V = R4, W1= {(x , y , z , w) :x+y+z= 0, x+ 2y z= 0}, W2= {(s, 2s, 3s, t) :s, t R}. How to describe W1+W2 (e.g., find a basis)?

The following theorem tells us the dimension ofW1 + W2 and the proof of the theorem suggesthow to write its bases.

Theorem: IfW1, W2 are subspaces of a vector space V, then

dim(W1+W2) = dimW1+ dimW2 dim(W1 W2).

Proof: LetSbe a basis ofW1W2(ifW1W2is the zero space thenS= .). For eachi= 1, 2,extend Sto a basis Bi ofWi. Let S= {u1, u2, . . . , ur}, B1= {u1, u2, . . . , ur, v1, v2, . . . , vs}andB2= {u1, u2, . . . , ur, w1, w2, . . . , wt}. Then dim(W1 W2) =r, dimW1= r + s, dimW2= r + t.

Let B = {u1, u2, . . . , ur, v1, v2, . . . , vs, w1, w2, . . . , wt}. It is enough to show that B is a basisof W1 +W2 because then we have dim(W1 +W2) = r + s + t = (r + s) + (r + t) r =dimW1+ dimW2 dim(W1 W2).

To show that B is linearly independent let

ri=1

aiui+s

j=1

bjvj+t

k=1

ckwk = 0.

Thenr

i=1

aiui+s

j=1

bjvj = t

k=1

ckwk.

Now the LHS is in W1 and the RHS is in W2. So this element is in W1W2. Thus

tk=1

ckwk =t

k=1

diui so thatr

i=1

diui+t

k=1

ckwk = 0 which implies di = 0 and ck = 0 for each i and k (since

B2 is linearly independent). Therefore,r

i=1

aiui+s

j=1

bjvj = 0 which implies ai = 0, bj = 0 for

each i and each j (since B1 is linearly independent). Thus, B is linearly independent. Let

w W1+W2. Then w = w1+w2 for some wi Wi for i= 1, 2. Then w1 =r

i=1

piui+s

j=1

qjvj

and w2 =r

i=1

giui+t

j=1

hjwj for pi, qi, gi, hi F. Now w =r

i=1

(pi+ gi)ui+s

j=1

qjvj +t

k=1

hkwk

which is in span(B).

(40) The sum W1+W2 is called direct ifW1 W2 = {0}. In particular, a vector space V is said tobe the direct sum of two subspaces W1 andW2 ifV =W1+ W2 andW1 W2= {0}. WhenVis a direct sum ofW1 and W2 we write V =W1 W2.

Theorem: SupposeW1and W2are subspaces of a vector space Vso thatV =W1 + W2. ThenV =W1W2if and only if every vector in Vcan be written in a unique way as w1 + w2 wherewi Wi.

Proof: Since V = W1+ W2, every vector in V is a sum of a vector in W1 and a vector inW2. Suppose that for every v V , there is only one pair (w1, w2) with wi Wi such thatv = w1+ w2. If W1 W2 is nonzero, pick a nonzero vector u W1 W2. Then u = u+ 0


14/15

14

with u W1, 0 W2 and u = 0 + u with 0 W1, u W2. This contradicts our uniquenessassumption.

Conversely, suppose V =W1W2. ThenV =W1 + W2 and W1W2= {0}. If for v V wehave v = w1+w2 = w

1+w

2 for w1, w

1 W1 and w2, w

2 W2 then w1 w

1 =w

2 w2. TheLHS is in W1 and the RHS in W2; therefore, this vector is in W1 W2. Since by assumptionW1 W2 = {0}, we have w1 w

1= 0 andw

2 w2= 0 so that w1= w

1 andw2 = w

2.

Examples:

(a) V = R2, W1 = {(x, 2x) : x R}, W2 = {(x, 3x) : x R}. Then V = W1 W2.Indeed, (x, y) = (3x y, 6x 2y) + (y 2x, 3y 6x) (Hint: Find a, b F such that

(x, y) = (a, 2a) + (b, 3b)) and W1 W2 = 0 (Hint: Let (x, y) W1 W2 then (x, y) =(a, 2a) = (b, 3b) for some a, b F. By comparing the first component we geta = b andcomparing the second we get a = 0).

(b) Suppose n 2, V = Rn, W1 = {(a1, a2, . . . , an) Rn : a1 +a2 + an = 0} and

W2 = {(a1, a2, . . . , an) Rn : a1+a2 + (1)

nan = 0}. Show that V =W1+W2.Further show that whenn= 2, V =W1 W2 and when n >2 the sum is not direct.

(c) V =Mn(R),W1is the subspace of all the upper trangular matrices and W2is the subspaceof all the lower trangular matrices over R(this sum is not direct).

(d) V =Mn(R),W1 is the subspace of all the symmetric n nmatrices over Rand W2 is thesubspace of all the skew-symmetric n nmatrices over R(in this, the sum is direct).

Exercise: Write a basis ofW1+W2 of example 39b.

(41) Suppose V and W are vector spaces over the same field F. A map T : V W is called alinear transformationifT(au+bv) =aT(u) +bT(v) for any a, b F and any u, v V.

Examples:

(a) Let V = W = R2, F = R. Then T1(x, y) = (ax+ by,cx+ dy) for any a,b,c, d Ris a linear transformation (verify). But T2(x, y) = (x+ y + 1, x y) is not a lineartransformation (why?). If T : R2 R2 is a linear transformation then T = T1 forsome choices ofa,b, c, d R. Indeed, T(x, y) = T(xe1+ ye2) = xT(e1) + yT(e2). SinceT(e1), T(e2) R

2, we have T(e1) = (p, q) and T(e2) = (r, s) for some p,q,r, s R. Thus,T(x, y) =x(p, q) +y(r, s) = (px+ry, qx+sy).

(b) For any vector spaces V , W over F, map from V to W defined by v 0 for all v V is a

linear transformation and it is called thezero map

(or zero transformation).(c) IdV : V V defined by v v for all v V is a linear tranformation; it is called theidentity operator.

(d) For each 1 i n,pi: Rn R defined bypi(a1, a2, . . . , an) =aiis a linear transformation

and it is called the i-th projection.

Remarks: IfT :V Wis a linear transformation then T(0) = 0, i.e., the image of the zerovector ofV is the zero vector ofW. Indeed,T(0) =T(00) = 0T(0) = 0 (where we have written00, the first zero is the scalar zero and the second zero is the zero vector in the domain spaceV ofT. T : R2 R2 given by T(x, y) = (x + y+ 1, x y) is not linear because T(0, 0) = (1, 0)which is not the zero vector ofR2.

(42) Suppose T :V Wis a linear transformation. Then define

ker(T) := {v V :T(v) = 0}and T(V) := {T(v) :v V}.

Show that ker(T) < V and T(V) < W for any linear transformation T. The spaces ker(T)andT(V) are called respectively null space(or kernel) and image space(or range space)ofT. The dimension of ker(T) is call the nullity ofTand the dimension ofT(V) is call therankofT.

Theorem: A linear tranformation is injective if and only if its null space is the zero space.Proof: Suppose T : V W is a linear tranformation. Suppose that the null space is thezero space. If T(u) = T(v) for u, v V then T(u v) = 0 so that u v ker(T) = {0}which implies u v = 0 u = v. Conversely, assume that T is injective. Ifu ker(T) thenT(u) = 0 =T(0) which implies u= 0 (since T is invertible).


15/15

15

Theorem: (Rank-nullity Theorem) Suppose T :V Wis a linear tranformation. Then

rank(T) + Nullity(T) = dim(V).

Proof: Suppose {u1, u2, . . . , um} is a basis of the null space ker(T). Extend this basis to abasis of V. Let {u1, u2, . . . , um, v1, v2, . . . , vr} be a basis of V so that dim(V) = s+ r. Wewill show that B = {T(v1), T(v2), . . . , T (vr)}is a basis of the range space T(V) (to prove that

dim(T(V)) = r). By definition B T(V). Supposer

i=1

aiT(vi) = 0. Then T(r

i=1

aivi) = 0 (since

T is linear). Hencer

i=1

aivi ker(T) so thatr

i=1

aivi=m

i=1

bjuj orm

j=1

(bj)uj+r

i=1

aivi= 0. Since

{u1, u2, . . . , um, v1, v2, . . . , vr} is linearly independent, the coefficients are all zero. In particular,ai= 0 for each 1 i r. Thus,B is linearly independent. Ifw T(V), there existsv V such

thatT(v) = w. Since{u1, u2, . . . , um, v1, v2, . . . , vr} is a basis ofV, v =mi=1

piui+r

j=1

qjvj where

pi, qj F. Since T(ui) = 0 for each i, we get, by applying Tboth sides, T(v) =r

j=1

qjT(vj).

Examples: Let T : R3 R3 defined by T(x,y,z) = (x+y z, x y+z, y z). The null ofT is{(x,y,z) :x +y z= 0, x y+z= 0, y z= 0} which is the solution space of certainhomogeneous system of linear equations. We know how to solve a system of homogenous linearequations. Show that ker(T) = {(x,y,z) : x = 0, y = z} = {(0, t , t) : t R}. {(0, 1, 1)} is a

basis of ker(T) so that Nullity(T) = 1. Using the rank-nullity theorem rank(T) = dim(R3)1 =3 1 = 2.

How to compute rank(T) directly by displaying a basis of T(V): First pick a basis of V,for instance, {e1, e2, e3}. Then T(V ) is generated by Y = {T(e1), T(e2), T(e3)} which is{(1, 1, 0), (1,1, 1), (1, 1,1)}. We have to pick a basis of T(V) using Y. Consider the

matrixA =

1 1 01 1 11 1 1

which is obtained from Yby converting elements ofY into rows.

Then a basis of the row space ofAis a basis of the range space. To find a basis of the row space

find the row reduced echelon form ofA which is

1 0 1

2

0 1 12

0 0 0

. Hence{(1, 0, 1

2), (0, 1,1

2)} is

a basis of the range space. Thus rank(T) = 2. Thus we have verified the rank-nullity theoremfor the given linear tranformation.

Applications of rank-nullity theorem: Since for any linear transformation T, rank(T) 0,Nullity(T) 0, we have, rank(T) dim(V) and Nullity(T) dim(V). A linear tranforma-tion T : V W is injective if and only if Nullity = {0}; and T is surjective if and only ifrank(T) = dim(W). Using rank-nullity theorem we have the following statements.

(a) There is no injective linear tranformation from Rm to Rn ifm > n.(b) There is no surjective linear transformation from Rm to Rn ifn > m.(c) There is an isomorphism (a bijective linear tranformation) from Rm to Rn if and only if

m= n.Proof: (a) IfT : Rm Rn is injective, then Nullity(T) = 0 so rank(T) + 0 = dim(Rm) = m(by rank-nullity theorem). Since T(V) < Rn, rank(T) n. Therefore,m n. Equivalentely,ifm > n, there is no injective linear transformation from Rm to Rn.(b) IfT : Rm Rn is surjective (i.e.,T(V) =W), then rank(T) = n. By rank-nullity theorem,n+ Nullity(T) =m. So m n 0 or m n (since Nullity(T) 0).(c) Suppose T : Rm Rn is an isomorphism (a bijective linear transformation). Since T isinjective, by part (a), m nand sinceT is surjective m n. Therefore, m = n.

Next, suppose m = n. Then the identity map is a linear transformation from Rm to itself.

Remark: This completes the syllabus ofMinor Test I. Go through the questions with answersappeared in the course page. Soon more of QWA are to appear.

Date post:	03-Jun-2018
Category:	Documents
Upload:	kartikeya-gupta
View:	229 times
Download:	0 times

Class Notes on Linear Algebra

Documents