LW 1115 Math245notes(Waterloo)

7/29/2019 LW 1115 Math245notes(Waterloo)

1/50

mlbaker presentslambertw.wordpress.com

MATH 245Advanced Linear Algebra 2

Instructor: Stephen NewTerm: Spring 2011 (1115)University of Waterloo

Finalized: August 2011

Disclaimer : These notes are provided as-is , and may be incomplete or contain errors.

Contents

1 Affine spaces 11.1 Affine spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Affine independence and span . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Convex sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1.4 Simplices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2 Orthogonality 52.1 Norms, distances, angles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Orthogonal complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Orthogonal projection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 Applications of orthogonality 93.1 Circumcenter of a simplex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Polynomial interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Least-squares best t polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4 Cross product in Rn 14.1 Generalized cross product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Parallelotope volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5 Spherical geometry 17

6 Inner product spaces 176.1 Abstract inner products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

6.2 Orthogonality and Gram-Schmidt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Orthogonal complements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Orthogonal pro jections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7 Linear operators 287.1 Eigenvalues and eigenvectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Dual spaces and quotient spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1


2/50

8 Adjoint of a linear operator 358.1 Similarity and triangularizability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Self-adjoint operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.3 Singular value decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9 Bilinear and quadratic forms 419.1 Bilinear forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9.2 Quadratic forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

10 Jordan canonical form 49

Administrative

Website : http://math.uwaterloo.ca/~snew . Office : MC 5163. Email : [email protected]

Textbook : We will cover most of Chapters 6 and 7 from Linear Algebra by Friedberg, Insel and Spenc(Sections 6.1 6.8, 7.1 7.4).

Midterm : Tuesday, June 7.

1 Affine spaces

1.1 Affine spaces

The set Rn is given by

Rn = x =x1...

xn: x iR

A vector space in Rn is a set of the form U = span {v1, . . . , v }.Denition 1.1. An affine space in Rn is a set of the form p + U = { p + u | uU}for some point pRn and somvector space U in Rn .Theorem 1.2. Let P = p + U and Q = q + U be two affine spaces. We have p + Uq +

V if and only if q pand UV.Proof. Suppose that p + Uq +

V. We have p = p + 0 p +U. Hence pq +

V. Hence p = q + v for some vby denition. From that equation, q p = vV. Let uU. Consider p + u. This is an element of p + U. Bu p + U

q + V. So we can write p + u = q + v for some v

V. Hence u = ( q

p) + v, which is in V since q

p, v

This nishes the proof of one direction.Conversely, suppose q pV and U V. We will show p + U q + V. Let u U. p + u = p + u + q q q + ( u (q p))q + V.Corollary 1.3. p + U = q + V if and only if q pU and U = V.Denition 1.4. The vector space U is called the vector space associated to the affine space p + U . The dimensionof an affine space is the dimension of its associated vector space. A 0-dimensional affine space in Rn is a point . 1-dimensional affine space in Rn is called a line (this denes the word line). A 2-dimensional affine space in Rn

called a plane . An (n 1)-dimensional affine space in Rn is called a hyperplane .

2


3/50

Remark 1.5. To calculate the dimension of a solution space, you count the number of parameters (columns with nopivots in the row-reduced matrix).

Remark 1.6. A vector is the name given to an element of a vector space. In an affine space, they are called points

1.2 Affine independence and span

Denition 1.7. Let a0, . . . , a be points in Rn . We let a0, . . . , a denote the smallest affine space in Rn that containthose points (that is, the intersection of all affine spaces in Rn which contain each a i ). This is called the affine spanof the points.

Remark 1.8. To be rigorous, we should prove that an intersection of countably many affine spaces is always an affinespace.

Denition 1.9. We say that the points a0, . . . , a are affinely independent if dim a0, . . . , a = .

Theorem 1.10. Let a0, . . . , a be points in Rn . Let uk = ak a0 (for 1 k ). Then a0, . . . , a = a0span{u1, . . . , u }.Proof. Let U = span {u1, . . . , u }. We rst show that a0 + U contains all the points a i . Note a0 = a0 + 0 a0 +and for the rest we can write ak = a0 + ak a0 = a0 + uka0 + U . This proves that a0, . . . , a a0 + U.Next, we show that a0 + U a0, . . . , a . So we need to show that a0 + U q + V for every affine space q +which contains a0, . . . , a . So let q + V be such an affine space. We must show that q

a0

V and U is a subspacof V. Since a0q +

V we have q a0V. Also, we have ak q + V for k 1. Say ak = q + vk for vk V. Nouk = ak a0 = q + vk a0. So ukV since q a0, vkV. Hence U is a subspace of V.

a0

a1

u1a2

u2

a3u3

Figure 1: The setup for l = 3

Corollary 1.11. We have the following:

(i) Note:

a0, . . . , a = a0 + span {u1, . . . , u k}= a0 +i=1

t i u i | t iR =i=0

s i a i | s iR ,i=0

s i = 1

(ii) {a0, . . . , a }is affinely independent if and only if {u1, . . . , u }is linearly independent where uk = ak a0.1.3 Convex sets

Denition 1.12. Let a, b

Rn . The line segment [a, b] is the set

{a + t(ba) | 0 t 1}.A set C R

n is called convex if for all a, bC , we have [a, b]C .

Denition 1.13. For a set S Rn , the convex hull of S , denoted by [S ], is the smallest convex set in Rn containin

S , that is, the intersection of all convex sets which contain S . (Should prove that the intersection of convex sets isconvex for full rigour; this is left as an exercise).

Remark 1.14. When S = {a0, . . . , a }we write [S ] as [a0, . . . , a ]. Note that this agrees with the notation for theline segment (convex hull of two points).

3


4/50

Figure 2: Convex and non-convex subsets of the plane

1.4 Simplices

Denition 1.15. An (ordered, non-degenerate) -simplex consists of an ordered (l + 1) -tuple (a0, . . . , a ) of poina i R

n with {a0, . . . , a }affinely independent, together with the convex hull [a0, . . . , a ]. A 0-simplex is a point,1-simplex is an ordered line segment . A 2-simplex is an ordered triangle . A 3-simplex is an ordered tetrahedronTheorem 1.16. Let {a0, . . . , a }be affinely independent points in Rn . Then [a0, . . . , a ] can be written

a0 +n

i=1

t i u i | 0 t iR ,i=1

t i 1 =i=0

s i a i | 0 s iR ,i=0

s i = 1

where uk = ak a0.Proof. Exercise. Check the statement of the theorem and then prove it.

Remark 1.17 (triangle facts) . The medians of a triangle meet at a point called the centroid . The perpendicularbisectors meet at the circumcenter (the center of the circle in which the triangle is inscribed). The angle bisectorsmeet at the incenter (the center of the circle inscribed in the triangle). Furthermore, the altitudes meet at thorthocenter . The cleavers meet at the cleavance center . We will think about higher-dimensional generalizationsof these notions.

Figure 3: The medial hyperplane M 1,2 (shaded) in a 3-simplex.

Denition 1.18. Let [a0, . . . , a ] be an -simplex in Rn . For 0 j < k , the ( j, k ) medial hyperplane is deneto be the affine space M j,k which is the affine span of the points a i (for i = j, k ) and the midpoint 12 (a j + ak ).Theorem 1.19. Let [a0, . . . , a ] be an -simplex in Rn . The medial hyperplanes M j,k with 0 j < k haveunique point of intersection g. This point g is called the centroid of the -simplex, and it is given by the the averageof the points a i ,

g =1

l + 1i=0

a i .

4


5/50

Proof. We rst claim that g lies in each M j,k . We have

g =1

l + 1i=0

a i =1

l + 1i= j,k

a i +1

l + 1(a j + ak ) =

1l + 1

i= j,k

a i +2

l + 1a j + ak

2

and the sum of the coefficients is(l 1)

11 + l

+2

l + 1=

l + 1l + 1

= 1 .

Hence g

M j,k for all 0

j < k

, hence

g j,k

M j,k .

Further, we claim g is unique; it is the only point of intersection. Note that each M j,k really is a hyperplane. To sethis, we show that the points a i (i = j, k ) and 12 (a j + ak ) are affinely independent. Suppose

i= j,k

s i a i + sa j + ak

2= 0

ands +

i= j,k

s i = 0 .

Thens0a0 + . . . +

s2

a j + . . . +s2

ak + . . . + s a = 0 .

Also, the sum of these coefficients is zero, therefore each coefficient is zero since {a0, . . . , a }is affinely independenThus {a i | i = j, k } {12 (a j + ak )} is affinely independent. This shows that each M j,k is indeed a hyperplane odimension l 1.Note ak does not lie in the medial hyperplane M 0,k = 12 (a0 + ak ), a1, . . . , a k1, a k+1 , . . . , a . Suppose to the contrarythat akM 0,k , say

ak = s1a1 + . . . + sk1ak1 + ska0 + ak

2+ sk+1 ak+1 + . . . + s a

with i=1 s i = 1 . Thensk2

a0 + s1a1 + . . . + sk1ak1 +sk2 1 ak + sk+1 ak+1 + . . . + s a = 0

and the sum of the coefficients is zero (since we moved ak to the other side). Since {a0, . . . , a }is affinely independenall of the coefficients sk2

, s1, . . . , s k1,sk2 1, sk+1 , . . . , s

must be zero. However, it is impossible that both s k2 ands k2 1 are zero. So we have reached a contradiction, provingthat ak /M 0,k .

We now construct a sequence of affine spaces (formed by taking intersections of the medial hyperplanes) by putting

Vk :=ki=1 M 0,i for 1 k . Hence V1 = M 0,1, V2 = M 0,1 M 0,2, and so on. To nish the proof, we note thabecause ak+1 Vk but ak+1 /Vk+1 , it follows that

k+1

i=1

M 0,ik

i=1

M 0,i

In other words, each Vk+1 is properly contained within Vk (for 1 k 1). We know dim( M 0,1) = 1 angV (as was previously shown), so dim( V ) 0. By the above, dim( Vk ) = k and hence dim( V ) = 0 . This hencdemonstrates g is the unique point of intersection of the hyperplanes M 0,k (1 k ). Thus g is the unique point ointersection of the hyperplanes M j,k for 1 j < k .

5


6/50

2 Orthogonality

2.1 Norms, distances, angles

Denition 2.1. Let u, vRn . We dene the dot product of u with v by

u v := u t v = vt u =n

i=1

u i vi .

Theorem 2.2 (Properties of Dot Product) . The dot product satises, for all u ,v,wR

n

and tR:

1. [positive denite] u u 0, holding with equality if and only if u = 0 .2. [symmetry] u v = v u.3. [bilinearity] (tu ) v = t(u v) = u (tv), (u + v) w = u w + v w, and u (v + w) = u v + u w.

Proof. Easy.

Denition 2.3. For uRn we dene the length (or norm ) of u to be

|u| := u u.Theorem 2.4 (Properties of Length) . Length satises, for all u ,v,w

Rn and tR:

1. [positive denite] |u| 0, holding with equality if and only if u = 0 .2. [homogeneous] |tu | = |t||u|.3. [polarization identity] u v = 12 (|u + v|2 |u|2 |v|2) = 14 (|u + v|2 |u v|2). (Law of Cosines)4. [Cauchy-Schwarz inequality ] |u v| |u||v|, with equality if and only if {u, v}is linearly dependent.5. [triangle inequality] |u + v| |u|+ |v|, and also ||u| |v|| |u + v|.

Proof. The rst two are trivial; we will prove properties 3 to 5.

3. Note that

|u + v|2 = ( u + v) (u + v) = u u + 2( u v) + v v = |u|2 2(u v) + |v|2whereby the rst equality follows immediately. We also have

|u v|2 = |u|2 2(u v) + |v|2Therefore |u + v|2 |u v|2 = 4( u v), and the second equality falls out.

4. Suppose {u, v}is linearly dependent, say v = tu . Then

|u v| = |u (tu )| = |t(u u)| = |t||u u| = |t||u|2 = |u||tu | = |u||v|.Now suppose {u, v} is linearly independent. We will prove strict equality. Note that for all t R we havu + tv = 0 by independence. Hence

0 < |u + tv|2 = ( u + tv) (u + tv) = |u|2 + 2 t(u v) + t2|v|2which is a quadratic in t with no real roots. Hence the discriminant must be negative, that is,

(2(u v)) 2 4|v|2|u|2 < 0whereby, taking square roots, we are done.

6


7/50

5. Note that

|u + v|2 = |u|2 + 2( u v) + |v|2 |u|2 + 2 |u v|+ |v|2 |u|2 + 2 |u||v|+ |v|2 = ( |u|+ |v|)2by applying Cauchy-Schwarz. Taking square roots, the proof follows. For the other equality, note

|u| = |(u + v) v| |u + v|+ |v|so that |u| |v| |u + v|. Similarly, |v| |u| |u + v|.

We are done.Denition 2.5. For a, bR

n , the distance between a and b is dened to be

dist (a, b) := |ba|.Theorem 2.6 (Properties of Distance) . Distance satises, for all a,b,cR

n :

1. [positive denite] dist (a, b) 0, holding with equality if and only if a = b.2. [symmetric] dist (a, b) = dist (b, a).

3. [triangle inequality] dist (a, c) dist (a, b) + dist (b, c).Proof. Exercise.

Denition 2.7. Let uRn . We say u is a unit vector if it has length 1.

Denition 2.8. For 0 = u, vRn , we dene the angle between them to be the angle

(u, v) := arccosu v|u||v|

[0, ].

Theorem 2.9 (Properties of Angle) . Angle satises for all 0 = u, vRn :

1. The following:

(u, v)[0, ]. (u, v) = 0 if and only if u = tv for some 0 < t R . (u, v) = if and only if u = tv for some 0 > t R . (u, v) = 2 if and only if u v = 0 .

2. (u, v) = (v, u ).

3. (tu,v ) = (u,tv ) =(u, v) if 0 < t R (u, v) if 0 > t R .

4. The Law of Cosines holds. Put := (u, v). Then

|v u|2 = |u|2 + |v|2 2|u||v|cos .5. Trigonometric ratios. Put := (u, v) and suppose (v u) u = 0 . Then

cos = |u||v|

and sin = |v u||v|

.

Proof. Exercise.

Remark 2.10. We have the following:

7


8/50

1. For points a,b,cRn with a = b and b = c, we dene

abc := (a b, cb).2. For 0 = u, vR

2 we can dene the oriented angle from u to v to be the angle [0, 2] with

cos =u v|u||v|

and sin =det( u, v)

|u||v|=

u1v2 u2v1|u||v|

.

In this case it is understood thatdet( u, v) := det u1 v1u2 v2

.

Denition 2.11. For u, vRn , we say that u and v are orthogonal (or perpendicular) when u v = 0 .

2.2 Orthogonal complements

Denition 2.12. For a vector space U in Rn we dene the orthogonal complement of U in Rn to be the vectospace

U= {vRn : v u = 0 for all uU}.Theorem 2.13 (Properties of Orthogonal Complement) . We have the following:

1. Uis a vector space in Rn .

2. If U = span {u1, . . . , u }where each u iRn , thenU= {vRn : v u i = 0 for all i}.

3. For AM kn (R), we have (Row A)= null A. For

A =r t1...

rt

k

Row A = span {r 1, . . . , r k} Rn . Note that

Ax =r t1x

...r tk x

=r 1 x...r k x

4. dim U + dim U= n, and UU= Rn .

5. (U)= U.

6. (null A)= Row A.Proof. We will prove #5; the rest are left as exercise.

5. Let xU. By denition of U, x v = 0 for all vU. By denition of (U), we have x(U)therefo

U(

U).

On the other hand by #4, dim U +dim( U) = n. Also, dim U+dim( U)= n. Therefore, dim U = dim( U)Since U is a subspace of (U)and dim U = dim( U), it follows that U = ( U).

8


9/50

2.3 Orthogonal projection

Theorem 2.14. Let AM kn (R). Then null( A

t A) = null( A).

Proof. For xRn , xnull( A) implies Ax = 0 , so A

t Ax = 0 . Therefore xnull(At A).

If xnull(At A) then At Ax = 0 , then x t At Ax = 0 , so (Ax)t (Ax) = 0 , so (Ax) (Ax) = 0 , so |Ax|2 = 0 , so |Ax| = so Ax = 0 , so xnull(A).

Remark 2.15. For A = ( u1, . . . , u n ), we have

At A =u t1...

u tn

(u1, . . . , u n ) =u1 u1 u1 u2 u1 u3 . . .u2 u1 u2 u2 . . ....

Theorem 2.16 (Orthogonal Projection Theorem ). Let U be a subspace of Rn . Then for any x Rn there exi

unique vectors u, vRn with u

U and vUand u + v = x.

Proof. (uniqueness) Let xRn . Suppose u

U, vU, u + v = x. Let {u1, . . . , u }be a basis for U. Let A be thmatrix with columns u1 up to u (AM n ), so U = col( A). Since uU = col( A), we have u = At for some tRSince vU= col( A)= null At , we have At v = 0 . We have

u + v = xAt + v = x

At At = At x since At v = 0t = ( At A)1At x

since {u1, . . . , u }is linearly independent, so rank( At A) = rank( A) = and At AM (R). Sou = At = A(At A)1At x

and v = x u.(existence) Again, let {u1, . . . , u }be a basis for U and let A = ( u1, . . . , u ), and let

u = A(At A)1At x

Then clearly ucol(A) =U and u + v = x. We need to show that v

U= (col A)= null At .

At v = At (x u) = At x At A(At A)1At x = At x (At A)(At A)1At x = At x At x = 0 .So the proof is complete.

Denition 2.17. Let U be a vector space in Rn . Let xRn . Let u and v be the vectors of the above theorem with

uU, v

U, u + v = x. Then u is called the orthogonal projection of x onto U and we write

u = Proj U (x).

Note that since U = ( U)it follows thatv = Proj U (x)

Example 2.18. When U = span {u}with 0 = uRn , we can take A = u, henceProj U (x) = A(At A)1At x = u(u t u)1u t x = u(|u|2)1u t x =

uu t x

|u|2=

u(u x)|u|2

=u x|u|2

u.

We also writeProj u (x) = Proj U (x) =

u x|u|2

u

9


10/50

Theorem 2.19. u := Proj U (x) is the unique point in U nearest to x.

Proof. Let wU with w = u. Note that (w u) (x u) = 0 since u, wU so w uU and v = x uUbthe denition of u = Proj U (x). By Pythagoras theorem (special case of the Law of Cosines), we note that

|w x|2 = |w u|2 + |x u|2 > |x u|2since |w u|2 > 0 (because w = u).Denition 2.20. Let U be a subspace of Rn and let x

Rn . We dene the reection of x in U to be

ReU (x) = Proj U (x) Proj U (x) = x 2Proj U (x) = 2Proj U (x) xFor an affine space P = p + UR

n and a point xRn we can also dene

Proj p+ U (x) = p + Proj U (x p)Re p+ U (x) = p + Re U (x p).

3 Applications of orthogonality

We started off by talking about affine spaces (when you solve systems of equations, the solution set is, in general, anaffine space: a vector space translated by a point). We also dened -simplices, and saw that we can apply the ideasof linear algebra to talk about geometry in higher dimensions. We then talked about inner products (the dot productin Rn and its relationship to lengths and angles, orthogonality, orthogonal complements of vector spaces, orthogonalprojections). Geometric applications of these include nding the circumcenter of a simplex by nding an intersectionof perpendicular bisectors, and nding best-t and interpolating polynomials.

3.1 Circumcenter of a simplex

Aside 3.1. In geometry, we study curves and surfaces and higher-dimensional versions of those. There are three mainways: a graph (draw a graph of a function of one variable, you get a curve; a graph of a function of two variables, youget a surface), you can do an implicit description, or you can do it parametrically.

For f : U Rk R , the graph of f denoted Graph( f ) = {(x, y) Rk+ | y = f (x)} Rk+ ; usually this isk-dimensional version of a surface. The equation y = f (x) is known as an explicit equation (it can be thought of as

equations, actually). The kernel of f is {xRk | f (x) = 0}. Usually this is (k )-dimensional. We also dene thimage of f , which is the set {f (x) | xU} R . Usually this is k-dimensional. (It is described parametrically, byparameter x).For example, the top half of a sphere can be described by z = r 2 (x2 + y2). An implicit equation describing thewhole sphere would be x2 + y2 + z2 r 2 = 0 . The sphere is the kernel of f (x ,y,z ) = x2 + y2 + z2 r 2.Denition 3.2. Let P be an affine space in Rn and let a, b be points in Rn with a = b. The perpendicular bisectorof [a, b] in P is the set of all x

P such that x a + b2 (ba) = 0 .Theorem 3.3. x is on the perpendicular bisector of [a, b] if and only if dist (x, a ) = dist (x, b).

Proof. For x P , x lies on the perpendicular bisector of [a, b] if and only if (x a + b2 ) (b a) = 0 if and onif (2x (a + b)) (b a) = 0 . This holds iff 2x b 2x a a b + a a b b + b a = 0 . This holds a a 2x a = bb2x b. But this holds iff a a 2x a + x x = bb2x b + x x iff |x a|2 = |x b|2

|x a| = |x b|.Theorem 3.4 (Simplicial Circumcenter Theorem ). Let [a0, a1, . . . , a ] be an -simplex in Rn . For 0 j < k write B j,k for the perpendicular bisector of the [a j , a k ]. Then the affine spaces B j,k with 0 j < k have a uniqupoint of intersection in a0, . . . , a . This point is denoted by and is called the circumcenter of the -simplex.

10


11/50

By the above theorem, is the unique point in a0, . . . , a which is equidistant from each a i . Theres an ( 1dimensional sphere centered at passing through each of the points a i .Proof. (uniqueness) Suppose such a point exists. Then lies on each B0,k for 1 k . We have a0, . . . , aSay = a0 + t1u1 + . . . + t u where uk = ak a0 and tkR . That is,

= a0 + At

where A is the matrix with the column vectors u1 to u . Since lies on B0,k , where 1 k , we use the denitioof B0,k and write the equation that satises.

a0 + ak

2 (ak a0) = 0We rewrite this as

((a0 + At) (a0 +ak a0

2)) (ak a0) = 0

This gives

(At 12

uk ) uk = 0

(At) uk =12 |uk |

2

(At) u1...(At) u

=12

|u1|2...|u |2

At At =12

|u1|2...|u |2

Where we note that At denotes the transpose and not A raised to the power of t (the letter t is being used two different ways here). Since [a0, . . . , a ] is a (non-degenerate) -simplex, we know that {u1, . . . , u } is linearindependent so that rank( A

tA) = rank( A) = and A

tA is an matrix, so it is invertible. Therefore,

t = ( At A)1v

where

v =12

|u1|2...|u |2

and therefore = a0 + At = a0 + A(At A)1v.(existence) We need to show that this point = a0 + A(At A)1v lies on all the perpendicular bisectors B j,k wi0

j < k

. So let 0

j < k

. Since

B0,j and

B0,k , we have

dist (, a 0) = dist (, a j )dist (, a 0) = dist (, a k )

hence dist (, a k ) = dist (, a j ) so that lies on B j,k .

11


12/50

3.2 Polynomial interpolation

The following theorem gives us information about the so-called polynomial interpolation of data points. This processproduces a polynomial which actually passes through the points.

Theorem 3.5. Let (a0, b0), (a1, b1), . . . , (an , bn ) be n + 1 points with the a i distinct. Then there exists a uniquepolynomial of degree n with p(a i ) = bi for all i.

Proof. For p(x) = c0 + c1x + c2x2 + . . . + cn xn , then we have p(a i ) = bi for all i if and only if

c0 + c1a0 + c2a20 + . . . + cn an0 = b1...

c0 + c1an + c2a22 + . . . + cn ann = bn

if and only if Ac = b, where

A =

1 a0 a 20 . . . a n01 a1 a 21 . . . a n1...1 an a 2n . . . a nn

and

c =

c0c1...

cn

, b =

b0b1...

bn

This matrix A is called the Vandermonde matrix for a0, a1, . . . , a n . We must show that A is invertible. We claimthat

det A =0 j


13/50

For each 0 i k 1, by subtracting the ith row from the last, we see that

D (x) = det Ak = det

1 a0 a 20 . . . a k01 a1 a 21 . . . a k1...1 ak1 a

2k1 . . . a

kk10 x a i x2 a2i . . . xk aki

.

So (x a i ) is a factor of each term on the last row, so (x a i ) is a factor of D (x) for each 0 i k 1. It followthat

D (x) = C (x a0)(x a1) (x ak1)=

0i


14/50

Proof. For f (x) = c0 + c1x + . . . + c x , we have

f (a1)...

f (an )=

c0 + c1a1 + c2a21 + . . . + c a1...

c0 + c1an + c2a2n + . . . + c an

= Ac,

where

A =

1 a1 a21 . . . a11 a2 a22 . . . a2...1 an a2n . . . an

M n

( +1) (R) and c =

c0c1...

cTo minimize the sum

n

i=1(bi f (a i ))2,

we must minimize

dist

b1...

bn,

f (a1)...

f (an )= dist (b,Ac)

We must have that Ac is the (unique) point in col(A) which lies nearest to b. Thus

Ac = Proj col( A ) (b).

Since + 1 of the a i are distinct, it follows that the corresponding + 1 rows of A form a Vandermonde matrix on+ 1 distinct points. This ( + 1) ( + 1) Vandermonde matrix is invertible by the previous theorem, so these +rows are linearly independent. Therefore, the rank of the matrix A is + 1 , which means the columns of A are linear

independent. Therefore A is one-to-one, and hence there is a unique c.

We now seek a formula for c. We look for u, v R+1 with u

U = col A, vU= null At , and u + v = b. S

u = Ac.

u + v = bAc + v = b

At Ac = At b

c = ( At A)1At b.

Since c is the vector of coefficients, the proof is complete. Put f (x) = c0 + . . . + c x .

4 Cross product in Rn

A familiar notion is the cross product of two vectors in R3. In this section, we generalize this to the cross product ofn 1 vectors in Rn , and see some results about the connections between cross products, determinants, and the volumeof the parallelotope generated by vectors.4.1 Generalized cross product

Denition 4.1. Let u1, . . . , u n 1 be vectors in Rn . We dene the cross product of these vectors to be

X(u1, . . . , u n 1) = formal det u1 . . . un 1e 1...

en

where ek is the kth standard basis vector. This is equal ton

i=1(1)i+ n det( Ai )ei

14


15/50

where A = ( u1, . . . , u n 1)M n (n 1) and Ai is the (n 1) (n 1) matrix obtained from A by removing the irow.

Example 4.2. In R2, for u = u1u2 R2 we write uX = X(u) = formal det u1 e1u2 e2

. In R3, for u, vR3 we writ

X(u, v) = formal detu1 v1 e1u2 v2 e2u3 v3 e3

= det u2 v2u3 v3e1 det u1 v1u3 v3 e2 + det u

1 v1u2 v2

e3.

=u2v3 u3v2u3v1 u1v3u1v2 u2v1

.

This particular cross product gives the area of the parallelogram generated by the vectors u, v .

Theorem 4.3 (Properties of Cross Product) . We have the following, for u1, . . . , u n 1, vRn , and tR:

1. X(u1, . . . , t u k , . . . , u n 1) = tX(u1, . . . , u k , . . . , u n 1).

2. X(u1, . . . , u , . . . , u k , . . . , u n 1) = X(u1, . . . , u k , . . . , u , . . . , u n 1), that is, interchanging two vectors ips thesign of the cross product. (This means the cross product is skew-symmetric ).3. X(u1, . . . , u k + v, . . . , u n1) = X(u1, . . . , u k , . . . , u n 1) + X(u1, . . . , v , . . . , u n 1). This, together with property 1

says that the cross product is multilinear.

4. We haveX(u1, . . . , u n 1) v = det( u1, . . . , u n 1, v)

hence for 1 i n 1, we haveX(u1, . . . , u n 1) u i = 0 .

5. X(u1, . . . , u n 1) = 0 if and only if

{u1, . . . , u n 1

}is linearly dependent. Furthermore, when X(u1, . . . , u n 1) =

the set{u1, . . . , u n 1, X(u1, . . . , u n 1)}

is what we would call a positively oriented basis for Rn which means

det( u1, . . . , u n 1, X(u1, . . . , u n 1)) > 0.

Proof. We prove 4 and 5.

4. Note that

X(u1, . . . , u n

1)

v =

n

i=1

(

1)i+ n det( Ai )ei

v

=n

i=1

(1)i+ n det( Ai )(ei v)

=n

i=1(1)i+ n det( Ai )vi

= det( u1, . . . , u n 1, v)

15


16/50

5. {u1, . . . , u n 1}is linearly independent if and only if A has rank n 1, if and only if row space has dimension 1, if and only if some n 1 rows of A are linearly independent, if and only if one of the matrices Aiinvertible, if and only if X(u1, . . . , u n1) = 0 .

X(u1, . . . , u n 1) =

+ |A1||A2|+ |A3|...

|An 1

|

= 0 .

Also,

det( u1, . . . , u n 1, X(u1, . . . , u n1)) = X(u1, . . . , u n 1) X(u1, . . . , u n 1)= |X(u1, . . . , u n )|2 > 0

whenever X(u1, . . . , u n1) = 0 .The proof is complete.

4.2 Parallelotope volume

Denition 4.4. For vectors u1, . . . , u k we dene the parallelotope (or parallelepiped ) on the vectors u1, . . . , u kRn to be the set

k

i=1

t i u i | 0 t i 1 for all i .We dene the k-volume of this parallelotope, written volk (u1, . . . , u k ) inductively by vol1(u1) = |u1| and for k 2 wdene

volk

(u1, . . . , u k ) = volk1

(u1, . . . , u k1)|uk |sin ,where is the angle between the vector uk and the vector space U = span {u1, . . . , u k1}, or alternatively

volk

(u1, . . . , u k ) = volk1

(u1, . . . , u k1) |Proj U (uk )| .Theorem 4.5 (Parallelotope Volume Theorem ). Let u1, . . . , u k

Rn . Then

volk

(u1, . . . , u k ) = det( At A),where A is the matrix with columns u1, . . . , u k .

Proof. To see the truth of the base case, note vol1(u1) = |u1| = u1 u1 = u t1u1 = det( u t1u1) = det( At Awhere A = u1. Next, x k 2, and suppose inductively thatvolk1

(u1, . . . , u k1) = det( At A)where A = ( u1, . . . , u k1). Let B = ( u1, . . . , u k ) = ( A, u k ), that is, B is the matrix obtained from A by inserting thvector uk as its last column. Now, put U = span {u1, . . . , u k1}and then dene the projections

p := Proj U (uk )col A

q := Proj U (uk )(col A)= null( At )

and hence p + q = uk . Then of course B = ( A, p + q ). Since pcol A, the matrix B can be dened from the matrix(A, q ) by performing elementary column operations of type 3. Hence B = ( A, p + q ) = ( A, q )E where E is a produof type 3 elementary matrices. Hence E is k k, and det( E ) = 1 . It follows that

det( B t B ) = det( E t (A, q )t (A, q )E ) = det At

q t (A, q )

16


17/50

since det E = 1 . This then becomes

det( B t B ) = det At A At q

q t A q t q = detAt A 0

0 |q |2= det( At A)|q |2.

Therefore taking square roots, we apply the induction hypothesis to obtain

det( B t B ) = det( At A)|q | = volk1(u1, . . . , u k1) |q | = volk (u1, . . . , u k )hence completing the proof.

Theorem 4.6 (Dot Product of Two Cross Products ). Let u1, . . . , u n 1, v1, . . . , vn 1Rn . Then

X(u1, . . . , u n1) X(v1, . . . , vn 1) = det( At B )where A is the matrix with columns u1, . . . , u n 1 and B is the matrix with columns v1, . . . , vn 1.Proof. First, we dene

x := X(u1, . . . , u n 1)y := X(v1, . . . , vn 1).

We now calculate x

y twice:

x y = X(u1, . . . , u n 1) y = det( A, y)and also

x y = x X(v1, . . . , vn 1) = X(v1, . . . , vn 1) x = det( B, x ).Therefore, we see that

(x y)2 = det( A, y)det( B, x ) = det (A, y)t (B, x ) = detAt

yt (B, x )

and this becomes(x y)2 = det

At B At xyt B yt x = det

At B 00 x y

.

To justify that At x = 0 , observe that since x = X(u1, . . . , u n 1), it is the case that x u i = 0 for all i. This meanthat x (span {u1, . . . , u n 1})= (col A)= null At . Also, y null B t . It is now clear that either x y = 0 , x y = det( At B ).Suppose that x y = 0 with one of x or y being zero. If x = X(u1, . . . , u n 1) = 0 , then {u1, . . . , u n 1}is lineardependent, so rank( A) < n 1 (the proof of this is an exercise). Therefore At B is non-invertible, implying det( At B ) 0. Similarly, if y = 0 we also obtain det( At B ) = 0 .Suppose next that x y = 0 with both x, y nonzero. Since

0 = x y = X(u1, . . . , u n 1) y = det( A, y)we have y

col(A) (since x = 0 so the columns of A are linearly independent). Similarly, x

col(B ), say x = BNote that t = 0 , since x = 0 . Note that since x = X(u1, . . . , u n 1), we know its perpendicular to col(A) which meanx null(A

t ), therefore At Bt = At x = 0 . Since t = 0 and (At B )t = 0 , we see that At B has nontrivial nullspaceproving At B cannot be invertible. Hence det( At B ) = 0 .

In all cases, we have obtained x y = det( At B ). So the proof is complete.Corollary 4.7. For u1, . . . , u n 1R

n ,

|X(u1, . . . , u n 1)| = det( At A) = voln1(u1, . . . , u n 1).This is the end of the material that will be tested on the course midterm .

17


18/50

5 Spherical geometry

In this section we study some spherical geometry. A useful reference for this material is available at

http://www.math.uwaterloo.ca/~snew/math245spring2011/Notes/sphere.pdf .

In view of this, these notes do not themselves include information on spherical geometry.

6 Inner product spaces

6.1 Abstract inner products

Denition 6.1. Let F be a eld. We dene the dot product of two vectors u, vFn by

u v :=u1...

un

v1...

vn=

n

i=1u i vi = vt u = u t v.

Remark 6.2. Let u,v,w Fn and tF . The dot product is symmetric and bilinear, but it is not positive denite,that is, it is not the case that u u 0.

For example, if F = Z5

then12

12 = 1

2 + 2 2 = 0

or on the other hand if F = C then1i

1i = 1

2 + i2 = 0 .

For AM k (F) and xF , if

A =r t1...

r tk

then we observe that

Ax =r 1 x...r k x

.

For u, vCn , we could dene

uv =u1...

un

v1...

vn=

x1 + iy1...

xn + iyn

r 1 + is 1...

r n + is n= x1r 1 + y1s1 + x2r 2 + y2s2 + . . .

Here we are identifying Cn with R2n and using the dot product in R2n . This dot product is symmetric and R-bilineabut is not C-bilinear. Note that for z

C we have |z|2 = zz.Denition 6.3. For u, vC

n , we dene the (standard) inner product of u and v to be

u, v =n

i=1u i vi = vt u = vu

where v= vt .

This product has the following properties:

18


19/50

It is conjugate-symmetric. It is linear in the rst variable, and conjugate linear in the second (we call this hermitian or sesquilinearThat is,

u + v, w = u, w + v, wu, v + w = u, v + u, w

tu,v = t u, vu,tv = t u, v

It is positive denite: we have u, u 0 with u, u = 0 if and only if u = 0 .For A = ( u1, . . . , u )

M n (C), and xCn , we have

Ax = At x =u t1...

u tx =

x, u 1...

x, u

and for B = ( v1, . . . , v )M n (C) we have

BA =v

t1...

vt(u1, . . . , u ) =

u1, v1 . . . u , v1... . . ....

u1, v . . . u , v

where A= At = At , that is,(A)ij = A ji .

When A M n (R) then we simply have A= A

t since complex conjugation, when restricted to the reals, is theidentity map.

Denition 6.4. Let F be R or C . Let V be a vector space over the eld F . Then we can dene an inner product onV as a map , : V

V

F such that for all u ,v,w

V and all t

F we have:

u, v = v, uu, v + w = u, v + u, wu + v, w = u, w + v, w

tu,v = t u, vu,tv = t u, vu, u 0 with equality iff u = 0

A vector space V over F (R or C) together with an inner product is called an inner product space over F .

Example 6.5. We have the following:

Rn is an inner product space using the dot product (also called the standard inner product) on Rn . Cn is an inner product space using its standard inner product. The standard inner product on M k (F) is given by

A, B =a11

. . .ak

,b11

. . .bk

= a11 b11 + a12 b12 + . . . + ak bk = tr( BA).

19


20/50

Example 6.6. Let F be R or C . Then F is the vector space of sequences a = ( a1, a2, . . .) with a iF , where a i =for only nitely many i. This vector space has the standard basis

{e1, e2, e3, . . .}where ek = ( ek1, ek 2, . . .) with eki = ki (Kronecker delta notation). The standard inner product on F is

a, b =

i=1

a i bi

where we note that this sum is indeed nite since only nitely many a i , bi are nonzero.

Example 6.7. Let a < b be real. Then we dene C([a, b], F) to be the vector space of continuous functions f : [a, b]F . The standard inner product on C([a, b], F) is given by

f, g = ba f gwhere we note that |f |2 = f, f = ba f f = ba |f |2 0.Example 6.8. Let P n (F) denote the vector space of polynomials with coefficients in F of degree at most n . Let P (denote the vector space of all polynomials over F . In

P n (F) we have several inner products:

We can dene ni=0

a i x i ,n

j =0b j x j =

n

i=0a i bi .

For a < b real, we can putf, g = ba f g

For distinct points a0, . . . , a n F we can dene

f, g =

n

i=0 f (a i )g(a i ).

Note that the rst and second of these inner products generalize to inner products on P (F).Denition 6.9. Let U be an inner product space over F (where F is R or C). For u, v

U we say that u and v aorthogonal when u, v = 0 . Also, for u

U we dene the norm (or length ) of u to be

|u| = u, unoting that when |u| = 1 we call u a unit vector .Denition 6.10. Let U be an inner product space. Let u, x

U. We dene the orthogonal projection of x on

u to beProj u x =

x, u

|u|2u not the same as

u, x

|u|2u if F = C!

Proposition 6.11. x Proj u x is orthogonal to u.Proof. This is an easy computation. Just expand x Proj u x, u and use the properties of the inner product.Theorem 6.12 (Properties of Norm) . Let U be an inner product space over F , with F = R or F = C . Then for au,v,w

U and tF we have

1. |tu | = |t||u| (note that these are 2 different norms).

20


21/50

2. |u| 0 with equality if and only if u = 0 .3. The Cauchy-Schwarz inequality holds, that is, | u, v | |u||v|.4. The triangle inequality holds: ||u| |v|| |u + v| |u|+ |v|.5. The polarization identity holds. If F = R then we have

u, v =14 |u + v|

2 |u v|2

whereas if F = C we have (?)

u, v =14 |u + v|

2 + i|u + iv|2 |u v|2 i|u iv|2

6. Pythagoras theorem holds: if u, v = 0 then |u + v|2 = |u|2 + |v|2. Note that in the complex setting, theconverse is not true!Proof of Cauchy-Schwarz. If {u, v}is linearly dependent, then one of u and v is a multiple of the other, say v =with tF . Then

| u, v |= | u,tu |= |t u, u |= |t||u|2 = |u||tu | = |u||v|.Next, suppose

{u, v

}is linearly independent. Consider

w = v Proj u v = v v, u

|u|2u = 1 v

v, u

|u|2u

Since {u, v}is linearly independent, we have w = 0 , hence |w|2 > 0, hence w, w > 0v

v, u

|u|2u, v

v, u

|u|2u > 0

hence

v, v v, u

|u|2v, u

v, u

|u|2u, v +

v, u

|u|2v, u

|u|2u, u > 0

therefore

v, v u, v u, v

|u|2 u, v u, v

|u|2+

u, v u, v

|u|4u, u > 0

hence

|v|2 2| u, v |2

|u|2+ | u, v |2

|u|4 |u|2 > 0

therefore multiplying through by |u|2 we obtain|u|2|v|2 |u, v |2 > 0

nally yielding

| u, v |2 < |u|2|v|2as required.

Proof of triangle inequality. We will prove that |u + v| |u|+ |v|. Note that|u + v|2 = u + v, u + v = u, u + u, v + v, u + v, v = u, u + u, v + u, v + v, v

and therefore this is equal to

|u|2 + 2 Re u, v + |v|2 |u|2 + 2 |Re u, v |+ |v|2 |u|2 + 2 |u||v|+ |v|2 = ( |u|+ |v|)2.So the proof is complete.

21


22/50

Proof of Pythagoras theorem. We will prove that if u, v = 0 then |u + v|2 = |u|2 + |v|2. As above, we have

|u + v|2 = |u|2 + 2 Re u, v + |v|2We see that

|u + v|2 = |u|2 + |v|2if and only if Re u, v = 0 .

Remark 6.13. Let F be R or C and U be a vector space over F . Then a norm on U is a map

| |: U

R such tha

for all u, vU and all t

F ,

1. |tu | = |t||u|.2. |u + v| |u|+ |v|.3. |u| 0 with equality if and only if u = 0 .

A vector space U over F with a norm is called a normed linear space .

Example 6.14. Some norms do not arise from inner products. In Rn for uRn we can dene the norm of u to b

|u

|=

n

i=1 |u

i |.

This is a norm on Rn (that is different from the standard one).

Denition 6.15. Let U be an inner product space (it could be a normed linear space) over F (R or C). For a, bwe dene the distance between them to bedist (a, b) = |ba|

The distance function has the following properties for all u ,v,wU:

1. dist (a, b) = dist (b, a).

2. dist (a, c)

dist (a, b) + dist (b, c).

3. dist (a, b) 0 with equality if and only if a = b.Remark 6.16. A metric on a set X is a map dist : X X R which satises properties 1, 2 and 3 above. A subseU X is open when for all aU , there exists r > 0 such that B r (a)U , where

B r (a) := {xX : dist (a, x ) < r }It has the following properties:

1. , X are open.

2. If U 1, . . . , U n are open then so isn

i=1

U i .

3. If U is open for all A then so is

A

U .

A topology on a set X is a set T of subsets of X which we call the open sets in X such that 1, 2, and 3 hold. Atopological space is a set X together with a topology T .

22


23/50

6.2 Orthogonality and Gram-Schmidt

Denition 6.17. Let W be an inner product space over F (R or C). Let U W . We say U is orthogonal wheu, v = 0 for all u = v U . We say U is orthonormal if it is orthogonal and furthermore |u| = 1 for all u U.Example 6.18. For U = {u1, . . . , u } Rn , let A = ( u1, . . . , u ), then U is orthogonal if and only if At A is diagonasince

At A =u1 u1 . . . u u1... . . . ...u1 u . . . u uand U is orthonormal if and only if At A = I .

Example 6.19. Recall that if U is a basis for the vector space U , then for xU = span U , we can writex = t1u1 + t2u2 + . . . + t u

with each t iF and u i distinct elements in U . When U = {u1, . . . , u }and x = t1u1 + . . . + t u we write

x U

= t =t1...tF .

The map : U F given by (x) = x U is a vector space isomorphism, so U = F .Remark 6.20. For U = {u1, . . . , u } Rn , A = ( u1, . . . , u ), and xU = span U = col A with x = t1u1 + . . . + t uAt , note x

U = t. To nd t = x

U we solve At = x. Hence

At At = At x

and thus x U

= t = ( At A)1At x. As a remark, note that (At A)1At is a left inverse for A. If U is orthonormal thenAt A = I so x

U = At x which gives

x U =

u t1...

u tx =

x u1...

x u.

Also, for xRn , we have

Proj U x = A(At A)1At x.If U is orthonormal so that At A = I , then Proj U x = AAt x, which is equal to

u1 . . . ux u1...x u

.

Theorem 6.21 (Gram-Schmidt Procedure) . LetU

be a nite (or countable) dimensional inner product space over(R or C). Let U = {u1, . . . , u }(or U = {u1, . . .}) be a basis for U. Let v1 = u1, and for k 1, let

vk = uk k1

i=1

Proj vi (uk ) = uk k1

i=1

uk , vi

|vi |2vi .

Then

V = {v1, . . . , v }(or V = {v1, . . .})is an orthogonal basis for U with span{v1, . . . , vk}= span {u1, . . . , u k}for all k 1.

23


24/50

Proof. We will prove inductively that {v1, . . . , vk}is an orthogonal basis for span{u1, . . . , u k}. The base case holdsince v1 = u1. Fix k 2 and suppose that {v1, . . . , vk1}is an orthogonal basis for span{u1, . . . , u k1}. Let

vk = uk k1

i=1

uk , vi

|vi |2vi .

Since vk is equal to uk plus a linear combination of v1, . . . , vk1 and since span{v1, . . . , vk1}= span {u1, . . . , u k1it follows thatspan

{v1, . . . , vk

}= span

{v1, . . . , vk

1, uk

}= span

{u1, . . . , u k

1, uk

}and hence {v1, . . . , vk}is a basis for span{u1, . . . , u k}. Next, we seek to show that {v1, . . . , vk}is orthogonal. By ouinduction hypothesis, vi , v j = 0 for all i = j less than k. So it remains to show that vk , vi = 0 for 1 i k We have for 1 j k 1 that

vk , v j = uk k1

i=1

uk , vi

|vi |2vi , v j = uk , v j

k1

i=1

uk , vi

|vi |2vi , v j = uk , v j

k1

i=1

uk , vi

|vi |2vi , v j

but this is, since vi , v j = 0 for all i = j , becomes

uk , v j

uk , v j

|v j |2 v j , v j = 0 .

This completes the proof.

Corollary 6.22. Every nite (or countable) dimensional inner product space U over F (R or C) has an orthonormabasis.

Proof. Let U = {u1, . . . , u }(or U = {u1, . . .}) be a basis for U. Apply the Gram-Schmidt procedure to U to obtaian orthonormal basis V = {v1, . . . , v }(or V = {v1, . . .}) for U , then for all k 1 letwk =

vk

|vk |to obtain an orthonormal basis

W =

{w

1, . . . , w

}(or

W =

{w

1, . . .

}).

Remark 6.23. The previous corollary does not hold for uncountable dimensional vector spaces.

Corollary 6.24. Let W be a nite (or countable) dimensional inner product space over R or C and let U be nite dimensional subspace. Then any orthogonal (or orthonormal) basis U can be extended to an orthogonal (ororthonormal) basis W for W .Proof. Extend a given orthogonal basis U = {u1, . . . , u }to a basis {u1, . . . , u +1 , . . .}for W , then apply the GramSchmidt procedure to obtain an orthogonal basis {v1, . . . , v , v +1 , . . .}for W and verify that vi = u i for 1 i .Remark 6.25. This result does not always hold when U is countable dimensional.

Example 6.26. Let W be an inner product space. Let

U =

{u

1, . . . , u

} W . If

U is an orthogonal set of nonzer

vectors, then for xspan U , sayx = t1u1 + . . . + t u =

i=1

t i u i

with each t iF (R or C), we have for each k (1 k ) that

x, u k =i=1

t i u i , uk =i=1

t i u i , uk = tk uk , uk = tk |uk |2.

24


25/50

Thus,

tk =x, u k

|uk |2so U is linearly independent and we have

[x] U =x, u 1

|u1|2. . .

x, u

|u |2t

If

U is orthonormal, then

[x] U = x, u 1 . . . x, u t

6.3 Orthogonal complements

Denition 6.27. Let W be an inner product space. Let U be a subspace of W . The orthogonal complement of UW is the vector space

U= {xW : x, u = 0 for all uU}Note that if U = {u1, . . . , u }is a basis for U, then

U= {xW : x, u i = 0 for all i = 1 , . . . , }and also if U is a (possibly innite) basis for

U, then

U= {xW : x, u = 0 for all u U}.Remark 6.28. Let W be a nite dimensional inner product space and let U be a subspace of W . Let U = {u1, . . . , ube an orthogonal (or orthonormal) basis for U. Extend U to an orthogonal (or orthonormal) basis

W = {u1, . . . , u , v1, . . . , vm }for W . Then V = {v1, . . . , vm }is an orthogonal (or orthonormal) basis for U.Proof. For x = t1u1 + . . . + t u + s1v1 + . . . + sm vm , we have

tk = x, u k

|uk |2and sk = x, vk

|vk |2.

If x U , then each tk = 0 so x span V . If x span V then x = s1v1 + . . . + sm vm , then for each k, x, u ks i vi , uk = s i vi , uk = 0 .Remark 6.29. As a consequence, we see that in the case above,

1. UU= W .

2. dim U + dim U= dim W .

3. U = ( U).

Proof of #3. x, v = 0 for all vU(from the denition of U), and so x(

U)(from the denition of (U)thus U(

U). Also dim U = dim W dim U= dim W (dim W dim( U)) = dim( U). Thus U = ( U).Note carefully that the rst part of the proof above does not use nite-dimensionality.

Remark 6.30. When U is innite dimensional we still have U(U)but in general U = ( U).

Since UU= W , it follows that given x

W there exist unique vectors u, v with uU, v

Usuch that u + v =

25


26/50

Example 6.31. Consider the inner product space W = R. This is the set of sequences (a1, a2, . . .) with each a iand only nitely many a i are nonzero. W has basis {e1, e2, . . .}. Let U be the subspaces of sequences whose sumi=1 a i = 0 (note that this is a nite sum). U has basis

{ek e1 : k 2}= {e2 e1, e3 e1, . . .}and

U= {xR : x, a = 0 for all aU}Note that

x, a = (x1, x2, . . .), (a1, a 2, . . .) = i=1

x i a i .

SoU= {xR : x, e k e1 = 0 for all k 2}= {x = ( x1, x1, x1, . . .)R}= {0}

since only nitely many x i are nonzero.

6.4 Orthogonal projections

Theorem 6.32 (Orthogonal Projection Theorem ). Let W be a (possibly innite dimensional) inner product space.Let U be a nite dimensional subspace of W . Let U = {u1, . . . , u }be an orthogonal basis for U . Given xW , theexist unique vectors u, v

W with u U, v

Usuch that u + v = x. The vector u is called the orthogonalprojection of x onto U and is denoted Proj U x. The projection is given by

u = Proj U x =i=1

x, u i

|u i |2u i .

Also, u is the unique vector in U which is nearest to x.

Proof. (uniqueness) Suppose such u, v exist. So uU , v

Uand u + v = x. Say

u =i=1

t i u i , so that t i =u, u i

|u i

|2 .

We havex, u i = u + v, u i = u, u i + v, u i = u, u i

since vUso v, u i = 0 therefore

u =i=1

u, u i

|u i |2u i =

i=1

x, u i

|u i |2u i .

This completes the proof of uniqueness.

(existence) Given xW , let

u =i=1

x, u i

|u i |2 u iso we have

u, u i

|u i |2=

x, u i

|u i |2then let v = x u. Clearly uU = span {u1, . . . , u }and u + v = x. We must verify vU. We have

v, u i = x u, u i = x, u i u, u i = 0 .This completes the proof of existence.

26


27/50

We claim u is the unique point in U nearest to x. Let wU with w = u. Note w u U, since w, u U. Alx u = vUso x u, w u = 0 . By Pythagoras theorem,

|(w u) (x u)|2 = |w x|2 = |x u|2 + |w u|2 > |x u|2since w = u.

Example 6.33. Let a0, . . . , a n be n + 1 distinct points in F (R or C). Consider P n (F) with the inner productf, g =

n

i=0

f (a i )g(a i ).

For 0 k n, letgk (x) =

i= k

x a iak a i

so that we havegk (a i ) = ki

(Kronecker delta notation). Note that {g0, . . . , gn }is an orthonormal basis for P n (F). For f P n (F),f =

n

k=0

f, g k

|gk

|2 gk

and

f, g k =n

i=0f (a i )gk (a i ) = f (ak ), since gk (a i ) = ki .

So then

f =n

k=0

f (ak )gk .

Example 6.34. Find the polynomial of degree 2 ( f P 2(R)) which minimizes

1

1(f (x) |x|)2 dx

Solution. Consider the vector space C([1, 1], R) with its standard inner productf, g = 11 fg.

We need to nd the unique f P 2(R) which minimizes dist (f, g ) where g(x) = |x|. We must takef = Proj P 2 g.

Let p0 = 1 , p1 = x, p2 = x2. So { p0, p1, p2}is the standard basis for P 2(R). Apply the Gram-Schmidt procedure:q 0 = p0 = 1 .

q 1 = p1 p1, q 0

|q 0|2q 0 = p1 = x.

q 2 = p2 p2, q 0

|q 0|2q 0

p2, q 1

|q 1|2q 1 = x2

23

12 1 0 = x

2 13

.

We now have an orthogonal basis {q 0, q 1, q 2}. So we takef = Proj P 2 g =

g, q 0

|q 0|2q 0 +

g, q 1

|q 1|2q 1 +

g, q 2

|q 2|2q 2 = . . . =

1516

x2 +3

16.

We are done.

27


28/50

Denition 6.35. Let U and V be inner product spaces. An inner product space isomorphism from U to V isbijective linear map L : U V which preserves the inner product, that is, L(x), L(y) = x, y for all x, yU.Proposition 6.36. Note that if U = {u1, . . . , u }and V = {v1, . . . , v }are orthonormal bases for U and V respectivelythen the linear map L : U V given by L(u i ) = vi (that is, L( t i u i ) = t i vi ) is an inner product space isomorphismProof. For x = s i u i , y = t j u j we have

x, y = s i u i , t j u j = i,j s i t j u i , u j = i s i t i =

s1...

s,

t1...t

= x U , y U

and we have

L(x) = L( s i u i ) = s i L(u i ) = s i vi

L(y) = t j v j

L(x), L(y) = s i t j .


As a corollary to the Gram-Schmidt orthogonalization procedure, we see that every n-dimensional inner product spaceis isomorphic to Fn . If you have any (innite dimensional) vector space, it has a basis, and you can use that basis toconstruct an inner product. Say U is a basis. We dene an inner product as follows: for

x =uU

su u and y =uU

tu u

we denex, y =

uU su tu .

Every vector space, we can construct an inner product such that a given basis is orthonormal in it. However, not allinner product spaces (namely, the innite dimensional ones) furnish an orthogonal basis!

7 Linear operators

7.1 Eigenvalues and eigenvectors

We now want to consider the following problem. Given a linear map L : U V nd bases U and V which are relateto the geometry of L so thatthe matrix L U

V is in some sense simple.

Example 7.1. Let U and V be nite-dimensional vector spaces over any eld F . Let L be a linear map. Show thawe can choose bases U and V for U and V so that

L U V

= I 00 0 .

Solution. Suppose r = rank( L). Choose a basis

{u r +1 , . . . , u k}for ker( L) = null( L). Extend this to a basis

{u1, . . . , u r , u r +1 , . . . , u k}=: U

28


29/50

for U. Let vi = L(u i ) (for i with 1 i r ). Verify that{v1, . . . , vr }

is linearly independent and hence forms a basis for the range of L. Extend this to a basis

V = {v1, . . . , vr , . . . , v }.Thus

L U V

= L(u1) V . . . L(uk ) V

= I r 00 0and we are done.

Remark 7.2. L U V

is the matrix such that

L(x)V

= L U V

x U

.

When U = V we sometimes simply write L U . Note that for U = {u1, . . . , u k}and V = {v1, . . . , v }we haveL(u i ) V

= L U V

u i U = L U

V ei = ith column of L U V

thereforeL U

V = L(u1) V

. . . L(uk ) V M k .

Also, for U( U )

L

V(V )M

W(W )we have

ML U W

= M V W

L U V

.

Similarly, forU

( U 1 )I

U( U 2 )L

V(V 2 )I

V(V 1 )we have

L U 1V 1

= IV V 2V 1

L U 2V 2

IU U 1 U 2

Warning . Some calculational examples follow. Im not completely sure about the correctness of these calculations;

sorry.Example 7.3. Let u1 = (1 , 1, 2)t , u2 = (2 , 1, 3)t , U = {u1, u2}, and U = span( U ). Let F = Re U , that is, F : R3 Ris given by

F (x) = Re U (x) = x 2Proj U (x) = Proj U (x) Proj U (x) = 2Proj U (x) x.We wish to nd F .

Solution. There are three methods.

1. Let A = ( u1, u2). We have Proj U (x) = A(At A)1At x. ThereforeF (x) = 2Proj U (x) x = (2 A(At A)1At I )x

so thatF = 2 A(At A)1At I.

Calculating, we see that

At A = 1 1 22 1 3

1 21 12 3

= 6 99 14

and eventually we arrive at

F =13

1 2 22 1 22 2 1

.

29


30/50

2. Use Gram-Schmidt to construct an orthonormal basis and perform the projection this way.

3. We have F (u1) = u1, F (u2) = u2. Choose u3 = u2 u1, so that u3 = (1 , 1, 1)t . Then F (u3) = u3. FV = {u1, u2, u3},

FV

= F (u1) V F (u2) V

F (u3) V =

1 0 00 1 00 0 1

So

F (u1, u2, u3) = ( u1, u2, u3) = ( u1, u2, u3)1 0 00 1 00 0 1

(u1, u2, u3)1

and then we get

F = ( u1, u2, u3)1 0 00 1 00 0 1

(u1, u2, u3)1.

We are done.

Example 7.4. Let R be the rotation in R3 about the vector u = (1 , 0, 2)t by / 2 (with the direction given by theright hand rule). We wish to nd R .Solution. Choose u2 = (2 , 0, 1)t so that u u2 = 0 and choose u3 = u1 u2 = (0 , 5, 0)t . Let

v1 =u1

|u1|=

1 5

10

2

v2 =u2

|u2|=

1 5

201

v3 =u3

|u3|=

010

and observe that V = {v1, v2, v3}is orthonormal. Then R(v1) = v1, R(v2) = v3, R(v3) = v2, so we get

RV

=1 0 00 0 10 1 0

.

Then

R (v1, v2, v3) = ( v1, v2, v3)1 0 00 0 10 1 0

so thatR = ( v1, v2, v3)

1 0 00 0 10 1 0

(v1, v2, v3)1.

Note that (v1, v2, v3)1 = ( v1, v2, v3)t for orthonormal vectors.

Example 7.5. Let L : R3 R3 be the linear map which scales by a factor of 2 in the direction of u1 := (1 , 1, 2)t , xvectors in the direction of u2 := (2 , 1, 3)t , and annihilates vectors in the direction of u3 := (1 , 1, 1)t . Find L = L

30


31/50

Solution. Let U = {u1, u2, u3}. Note L(u1) = 2 u1, L(u2) = u2, and L(u3) = 0 . Then

L U

=2 0 00 1 00 0 0

but note thatL

S = I U S L U U I S

U .

SinceI U

S = ( u1, u2, u3) andI S

U = ( u1, u2, u3)1

, we calculate

LS =

13

2 4 25 7 27 11 4

.

We are done.

Remark 7.6. Note that

L U

= diag( 1, . . . , n ) L(u1) U

. . . L(un ) U =

1 . . . 0... . . .

...

0 . . . n

which occurs if and only if L(u i ) = i u i for all i (1 i n).Denition 7.7. For a linear operator L : U U, we say F is an eigenvalue of L and u is an eigenvector offor when 0 = u and L(u) = u .Denition 7.8. For a linear operator L on a nite dimensional vector space U, the characteristic polynomial ofis the polynomial

f (t) = f L (t) = det( L tI).Denition 7.9. For F an eigenvalue, the eigenspace of is the space

E := null( L I).Remark 7.10 (eigenvalue characterizations) . Let L be a linear operator on U. The following are equivalent:

1. is an eigenvalue of L.

2. L(u) = u for some nonzero uU .

3. (L I)(u) = 0 for some nonzero uU.4. L I has a nontrivial kernel (that is, E is nontrivial).

If we assume further that U is nite dimensional then we can add the following to the list:

6. det( L I) = 0 .7. is a root of the characteristic polynomial, f L .

Note also that a nonzero uU is an eigenvector for if and only if uE .

31


32/50

Remark 7.11. Suppose dim( U) = n. If L : U U is diagonalizable, sayL

U = diag( 1, . . . , n )

then we have

f L (t) = det( L tI) = det L tI U = det( L U tI) = det1 t . . . 0... . . . ...

0 . . . n

t

= ( 1)nn

i=1

(t i ).

Note that the isomorphism U : U Fn given by U (x) = x U maps null( L) onto null( L U ) and hence mapnull( L I) ontonull

1 . . . 0... . . . ...0 . . . n

= span {ei | i = }Thus,

null( L I) = span {u i | i = }so that

dim E = dim null( L I) = ( # of i such that i = ) = multiplicity of in f L (t).Theorem 7.12 (Eigenvector Independence ). Let L be a linear operator on a vector space U. Let 1, . . . , k be distinceigenvalues of L. Let u1, . . . , u k be the corresponding eigenvectors. Then the set {u1, . . . , u k}is linearly independentProof. We proceed by induction. Note that {u1}is linearly independent, since u1 = 0 by denition of an eigenvectorSuppose {u1, . . . , u k1}is linearly independent and assume

t1u1 + t2u2 + . . . + tk1uk1 + tk uk = 0

where each t iF . Operate on both sides by the transformation (L k I). We obtain

t1(L(u1) k u1) + . . . + tk1(L(uk1) k uk1) + tk (L(uk ) k uk ) = 0and therefore

t1(1 k )u1 + . . . + tk1(k1 k )uk1 + tk (k k )uk = 0 .Since the i are distinct, and {u1, . . . , u k1}is linearly independent, it follows that

t1 = t2 = . . . = tk1 = 0

and hence tk = 0 as required.

Corollary 7.13. If 1, . . . , k are distinct eigenvalues for a linear operator L on a vector space U and if U i{u i, 1, . . . , u i, i }is a basis for E i then

k

i=1 U i is a basis for E 1 + E 2 + . . . + E k =k

i=1E i .

Theorem 7.14 (Dimension of an Eigenspace ). Let L be a linear operator on a nite dimensional vector space U. L be an eigenvalue of L. Then

1 dim E mult (f L ).

32


33/50

Proof. Since is an eigenvalue, we haveE = null( L I) = {0}.

Hence dim E 1. Let {u1, . . . , u }be a basis for E (so that dim E = ). Extend this to a basis U = {u1, . . . , u n }

for U. Then L U

has the block form

L U

= I A0 B

where O is a zero matrix. So

f L (t) = det( L U tI) = det

I A0 B = ( t) det( B tI).

Hence (t ) divides f L (t), therefore mult (f L ). This completes the proof.Corollary 7.15. For a linear operator L on a nite dimensional vector space U , L is diagonalizable if and only if splits (factors completely into linear factors) and dim( E ) = mult (f L ) for each eigenvalue .

Example 7.16. Let L(x) = Ax, where

A = 2 4 2

5 7 27 11 2

.

Diagonalize L.

Solution. Find the eigenvalues:

det( A I) = det2 4 2

5 7 27 11 4

= ( 2)(7 )(4 ) 56110 22(2)+20(4 )+14(7 which is equal to ( 3)( 6). Hence the eigenvalues are = 0 , 3, 6. Find a basis for each eigenspace:

( = 0 ): Note that

null(A I) = null2 4 25 7 27 11 4 = null

1 0 1

0 1 10 0 0

by a row reduction. So a basis is {(1, 1, 1)t}. ( = 3 ): Note that

null( A I) = null5 4 25 4 27 11 1

= null1 0 2/ 30 1 1/ 30 0 0

hence a basis is {(2, 1, 3)t}.And so on.

Aside 7.17. Dene(u, v) = cos 1 u, v

|u||v|[0, / 2]

for 0 = u, v Cn . Then in general we have to deal with the complex cos1 function on the unit disk, so we ge

complex angles.

Remark 7.18. Read section 5.4.

Theorem 7.19 (Cayley-Hamilton Theorem) . For a linear map L : U U of nite dimensional vector spaces, f L (L) = (Aside: if f L (t) = a0 + a1t + . . . + an tn then by denition f L (L) = a0I + a1L + a2L2 + . . . + an Ln so indeed f L is a lineoperator on L).

33


34/50

7.2 Dual spaces and quotient spaces

Denition 7.20. Let W be a vector space over a eld F and let U be a subspace of W . Then we dene the quotientspace W / U to be the vector space

W / U = {x + U : xW}with addition given by

(x + U) + ( y + U) = ( x + y) + U

and the zero given by 0 + U = U.

Example 7.21. Show that if U is a basis for U and we extend this to a basis W = U V for W (where U and V adisjoint) then {v + U : v V}is a basis for W / U .Corollary 7.22. We have

dim U + dim( W / U) = dim W .

In the nite dimensional case, if W is an inner product space, we have an isomorphism

U= W / U given by the map : W / U U dened by (x + U) = Proj U (x) = x Proj U (x).Denition 7.23. The codimension of U in W is dened to be dim( W / U).

Recall that for vector spaces U and V, the set

L(U, V) =

{linear maps from U to V

}is a vector space. This space i

also sometimes denoted Hom(U , V) (due to the word homomorphism, since linear maps are technically vector spacehomomorphisms). In the nite dimensional case, if we choose bases U and V for U and V we obtain an isomorphism

: L(U , V) M kwhere k = dim V, = dim U, given by

(L) = L U V

.

Denition 7.24. For a vector space U over a eld F , the dual space of U is

U= Hom( U, F)

that is, the space of linear maps from the vector space to the underlying eld (such maps are often called linearfunctionals on U).

Denition 7.25. Given a basis U = {u1, . . . , u }for U , for each k = 1 , . . . , dene f kUbyf k (u i ) = ki

where ki is the Kronecker delta notation. Observe the effect of these maps:

f ki

t i u i =i

t i f k (u i ) = tk .

Remark 7.26. Note that L =

{f

1, . . . , f

}is a basis for U

. Indeed, if t1f

1+ . . . + t f = 0 in U

, then

t1f 1(x) + . . . + t f (x) = 0

for all xU, so i t i f i (uk ) = 0 for all k. Hence tk = 0 for all k. Also, given L

U(that is, L : U F linear), notthat for x = i t i u i , we get

L(x) = L(i

t i u i ) =i

t i L(u i ) =i

f i (x)L(u i ) =i

L(u i )f i (x)

and hence L = i L(u i )f i . This basis L is called the dual basis to the basis U .

34


35/50

Remark 7.27. Note that for xU, say x = i t i u i , we have tk = f k (x), so

x U

=f 1(x)

...f (x)

and for LU we have

LL =

L(u1)...L(u )

Denition 7.28. Dene eval : U U by (eval( x))( f ) = f (x)for x

U, f U. This is called the evaluation map . This map is always a monomorphism (injective linear map),

but in the nite dimensional case is actually a vector space isomorphism.

Example 7.29. Show that in the case that U is nite dimensional, the evaluation map is an isomorphism.

Denition 7.30. Let W be a vector space over a eld F and let U be a subspace of W . Then we dene the annihilatoof U in Wto be

U = {gW: g(x) = 0 , xU}Example 7.31. Show that when W is nite dimensional,

dim U+ dim U = dim W

Denition 7.32. Let U, V be vector spaces over a eld F . Let L : U V be a linear map. Then the transpose of(or the dual map ) is the linear map Lt : VUgiven byLt (g)(x) = g(L(x)) g

V, xU.

Hence in terms of linear maps,Lt (g) = ( g

L)

g

V.

Theorem 7.33. Let U and V be nite dimensional vector spaces over F . Let L : U V be linear. Let U , V be basfor U and V respectively. Let F , Gbe the dual bases for Uand V. ThenLt G

F = L U

V t

Proof. We have

Lt GF

= Lt (g1) F . . . Lt (g )

F =

Lt (g1)(u1) . . . Lt (g )(u1)...

Lt (g1)(uk ) . . . Lt (g )(uk )=

g1(L(u1)) . . . g (L(u1))...

g1(L(uk )) . . . g (L(uk ))

and also

L U V

= L(u1) V . . . L(uk ) V

=g1(L(u1)) . . . g1(L(uk ))

...g (L(u1)) . . . g (L(uk )) .


35


36/50

V W

V W

L

V W

L t

L

8 Adjoint of a linear operator

Denition 8.1. Let U be an inner product space over F = R or C . We dene a map

= U : U U given by (u) = , uby which we mean, of course, that

(u)(x) = x, u

for all u, x U.

Note that when F = R , is linear in u, however when F = C it is conjugate linear in u. However, in either case, anmap (u) (u

U) is linear, and also is injective, since for uU, (u) = 0 implies that

(u)(x) = 0 for all x x, u = 0 for all x

hence in particular u, u = 0 , implying that u = 0 . In the case that U is nite dimensional, the map is alsurjective. To see this, suppose U is nite dimensional, and choose an orthonormal basis U = {u1, . . . , u }for U. LL

Uso L : U F . For x = t i u i , L(x) = L( t i u i ) = t i L(u i ).For u = s i u i , we have

(u)(x) = x, u =i

t i u i , j

s j u j =i,j

t i s j u i , u j =i

t i s i .

To get (u) = L, that is (u)(x) = L(x) for all x, choose s i = L(u i ), so that

u = L(u i )u i .

Denition 8.2. Let U and V be nite dimensional inner product spaces over F = R or C . Let L : U V be a linemap. We dene the adjoint (or conjugate transpose) of L to be the linear map L: V U, dened byL= 1U Lt V ,

where we note that 1U and V are conjugate linear. Hence Lis linear, and we have

U L= Lt V=(U

L)(y) = ( Lt V )(y) yV

=(UL)(y)(x) = ( Lt V )(y)(x) x

U

=U (L(y))( x) = Lt (V (y))( x) = V (y)(L(x))

= x, L(y) = L(x), y xU, yVLis the unique linear map from V to U satisfying the last line above for all x

U and yV.

Denition 8.3. More generally, for any (possibly innite dimensional) vector spaces U and V and for a linear maL : U V, if there exists a map L: V U such that

L(x), y = x, L(y) xU , yVthen Lis indeed unique and linear (prove this as an exercise) and we call it the adjoint of L.

36


37/50

Remark 8.4. For AM k (C) that is, A : C Ck , we have for all xC and yCk ,

Ax,y = yAx = yAx = ( Ay)x = x, Ay .

Theorem 8.5. Let U and V be nite dimensional inner product spaces over F = R or C . Let L : U V be lineaLet U and V be orthonormal bases for U and V respectively. ThenLV U

= L U V .

Proof. We simply calculate the matrices:

LV U = L(v1) U

. . . L(v ) U =

L(v1), u1 . . . L(v ), u1... . . .

...L(v1), uk . . . L(v ), uk

=u1, L(v1) . . . u1, L(v )

... . . ....

uk , L(v1) . . . uk , L(v )

which becomesL(u1), v1 . . . L(u1), v

... . . ....

L(uk ), v1 . . . L(uk ), v.

However, we have

L U V

= L(u1) V . . . L(uk ) V

=L(u1), v1 . . . L(uk ), v1

... . . ....

L(u1), v . . . L(uk ), v

which completes the proof.

8.1 Similarity and triangularizability

Remark 8.6. If L : U U is linear and we are given bases U , V for U , thenL V

V = I V

U L U

U I U

V .

Denition 8.7. For A, B M n n (F), we say A and B are similar when B = P 1AP for some invertible matri

P M n n (F). Note that if U = {u1, . . . , u n }is an orthonormal basis for an inner product space U, then U : U Fgiven by

U (x) = x U is an inner product space isomorphism. As a result, we see that the following are equivalent:

1. V = {v1, . . . , vn }is orthonormal.2. {v1 U , . . . , vn U }is orthonormal.3. Q := I V

U satises QQ = I , or in other words, Q= Q1.

4. P := I U V

satises P P = I .

Denition 8.8. For P M nn (F) where F = R or C , we call P an orthonormal matrix if P P = I . When F =

we call P a unitary matrix . When F = R , P = P t , hence P P = I if and only if P t P = I , and P is callorthogonal .

For A, B M nn (F) where F = R or C , we say A and B are orthonormally similar when B = P AP (= P

1APfor some orthonormal matrix P .

37


38/50

Denition 8.9. For L : U U linear when U is a nite dimensional inner product space, we say L is orthonormallydiagonalizable when there exists an orthonormal basis U for U such that L U is diagonal.For A

M n n (F) where F = R or C , we say A is orthonormally diagonalizable when A is orthonormally similato a diagonal matrix, that is, when P AP is diagonal for some orthonormal matrix P .

Theorem 8.10 (Schurs Theorem) . Let U be a nite dimensional inner product space over F = R or C . Let L : U be linear. Then L is orthonormally triangularizable (i.e. there exists an orthonormal basis U for U such that L Uupper triangular) if and only if f L (t) splits over F .We now recast the above theorem as an equivalent statement about square matrices over F = R or C , and then provthat.

Theorem 8.11 (Schurs Theorem ). For AM n n (F), F = R or C , A is orthonormally triangularizable if and only if

f A (t) splits.

Proof. Suppose A is orthonormally triangularizable. Choose an orthonormal matrix P such that T = P AP is upptriangular. Note that f A (t) = f T (t), which is equal to

det

t11 t t12 . . . t1n0 t22 t . . . t 2n... . . . . . . ...0 . . . 0 tnn t

=n

i=1

(t ii t)

which shows that f T (t) and hence f A (t) splits.

Conversely, suppose f A (t) splits. Choose an eigenvalue 1 of A with a corresponding unit eigenvector u1 for 1. TheAu1 = 1u1. Extend {u1}to an orthonormal basis {u1, . . . , u n }for Fn . Dene

P = ( u1, . . . , u n )

so that P is an orthonormal matrix. Also, let Q = ( u2, . . . , u n ) so that P = ( u1, Q). Then

P AP = u1Q

A(u1, Q) =u1Q

(1u1, AQ) =1u1u1 u1AQ

1Qu1 QAQ= 1 u1AQ

0 Bwhere the last equation is obtained by putting B = QAQ and noting that Qu1 is a matrix of dot products which are alzero due to orthonormality. Also u1u1 = 1 as remarked above so that 1u1u1 = 1. Now note that f A (t) = ( 1t)f B (tso f B (t) splits. Assume inductively that B is orthonormally triangularizable. Choose R M (n 1)(n 1) (F) wiRR = I so that RBR is upper triangular. Then we have

1 00 R

P AP 1 00 R =

1 00 R

1 u1AQ0 B

1 00 R =

1 u1AQR0 RBR

which is upper triangular. Note that

P 1 00 R P 1 00 R = 1 00 R P P 1 00 R = 1 00 RR = I.

The proof is complete.

Denition 8.12. Let U be a nite-dimensional inner product space over F = R or C . Let L : U U be linear. TheL is called normal when L commutes with L(that is, LL = LL). Similarly, for A

M n n (F), with F = R or C ,is called normal when AA = AA.

Denition 8.13. The spectrum of a linear map L : U U is the set of eigenvalues of L.

38


39/50

Theorem 8.14 (Orthonormal Diagonalization of Normal Matrices ). Let U be a nite-dimensional inner product spaceover F = R or C . Let L : U U be linear. Then L is orthonormally diagonalizable if and only if L is normal and f L (splits.Proof. Suppose L is orthonormally diagonalizable. Then choose an orthonormal basis U so that L U = Ddiag(1, . . . , n ). Then Lcommutes with L, since L U = D= diag( 1, . . . , n ) which commutes with D = L and f L (t) splits since

f L (t) = f D (t) =n

i=1

( i t).Conversely, suppose that LL = LLand f L (t) splits. Since f L (t) splits, L is orthonormally upper triangularizable bySchurs theorem. Choose an orthonormal basis U for U so that T = L U is upper triangular. Since LL = LL, whave T T = T T . We shall show that this implies T is diagonal. Suppose

T = T 11 T 12 T 13 . . .0 T 22 T 23 . . .so that T =

T 11 0 . . .T 12 T 22 . . .T 13 T 23 . . .

...... . . .

Note that since T T = T T , we clearly have (T T )11 = ( T T )11 and so

|T 11 |2 + |T 12|2 + |T 13|2 + . . . = |T 11 |and thus |T 12|2 = |T 13|2 = . . . = 0 . Now (T T )22 = ( T T )22 and hence

|T 22|2 + |T 23|2 + |T 24|2 + . . . = |T 12|2 + |T 22|2 = |T 22|2 by the previous equationwhereby T 23 = T 24 = . . . = 0 and so on. This completes the proof.

Denition 8.15. For 0 = u R3 and R , extend {u|u |} to an orthonormal basis U = {u1, u2, u3}. The

Ru, : R3 R3 is the mapRu,

U =

1 0 00 cos

sin

0 sin cos Remark 8.16. Given

x0, x1, x2, and the recurrence xn +3 = 6 xn + 5 xn +1 2xn +2 ,we have f (x) = 6 + 5 x 2x2 x3 and we solve for the roots ,, . The solution is of the form

A n + B n + C n .

8.2 Self-adjoint operators

Denition 8.17. Let U be a nite-dimensional inner product space over F = R or C . A linear operator L on Ucalled Hermitian or self-adjoint when L= L. For A

M n

n (F) with F = R or C we say that A is Hermitian

self-adjoint when A= A and we say A is symmetric when At = A.

Theorem 8.18 (Spectral Theorem for Hermitian Maps) . Let U be a nite dimensional inner product space overF = R or C . Let L be a linear operator on U . Then L is orthonormally diagonalizable and every eigenvalue of L is reif and only if L is Hermitian.

Proof. Suppose L is orthonormally diagonalizable and every eigenvalue of L is real. Choose an orthonormal basisfor U so that

L U

= D = diag( 1, . . . , n )

where each iR . We have D= diag( 1, . . . , n ), and soL U

= D= D = L U . Therefore, L= L.

39


40/50

Proof of other direction : Suppose L= L. We claim every eigenvalue of L is real. To see this, proceed as followChoose an orthonormal basis U for U and let A = L U . Note that A= A, so A commutes with A. By previotheorem, A is orthonormally diagonalizable over C . Choose a change of basis matrix P M n n (I) with P P =such that

P AP = D = diag( 1, . . . , n )

with each iC . We have D= ( P AP )= P AP = P AP = D.

Since D= diag( 1, . . . , n ) = diag( 1, . . . , n ) = D we have i = i for all i, so each i is real.

Alternate proof : Suppose L= L. Let be any eigenvalue of L and let u be an eigenvector for . Then

u, u = u,u = L(u), u = u, Lu = u, L(u) = u,u = u, u

therefore = since u, u = 0 , therefore R .Since the eigenvalues are all real, f L splits (even when F = R). Since L= L, L commutes with L. Therefore Lnormal. Therefore L is orthonormally diagonalizable.

Remark 8.19. Recall that for a linear operator L on a nite dimensional inner product space U, the following aequivalent:

1. L is an isometry.

2. L preserves norm.

3. L is an isomorphism of inner product spaces.

4. Given an orthonormal basis U = {u1, . . . , u n }for U , the set {L(u1), . . . , L(un )}is an orthonormal basis.5. The columns of A = L

U are an orthonormal basis.

6. AA = I .

Denition 8.20. Let U be a (nite dimensional) inner product space over F = R or C . Let L be a linear operator oU . Then L is called unitary , or orthonormal if it satises any of the (equivalent) conditions above.

For AM nn (F) where F is R or C , A is unitary if AA = I , and we say A is orthogonal when At A = I .

Remark 8.21. Here is some notation not used in the course. For any eld F ,

The general linear group , GL(n, F) = {AM n n (F) : det A = 0}. The special linear group , SL (n, F) = {AGL(n, F) : det A = 1}. The orthogonal group , O(n, F) = {AGL(n, F) : At A = I }. The special orthogonal group , SO (n, F) = {AO(n, F) : det A = 1}. When F = C , we also have the unitary group , U (n) = {AGL(n, C) : AA = I }.

And also the special unitary group , SU (n) = {AU (n) : det A = 1}.Theorem 8.22 (Spectral Theorem for Unitary Maps) . Let U be a nite dimensional inner product space over F =or C . Let L be a linear operator on U. Then L is orthonormally diagonalizable and every eigenvalue of L has unnorm if and only if L is unitary and f L splits.

Proof. This is an immediate corollary to the Spectral Theorem for Normal Matrices since if LL = I so L1 = LthuL commutes with L. Also for D = diag( 1, . . . , n ), we have

DD = diag( 1, . . . , n ) diag( 1, . . . , n ) = diag( |1|2, . . . , |n |2).So DD = I which is equivalent to each | i | = 1 .

40


41/50

Remark 8.23. Suppose U is a nite dimensional inner product space, and VU a subspace. Say {u1, . . . , u k}is orthonormal basis for V and we extend it to a basis U = {u1, . . . , u n }for the bigger space. Then

Proj V U = I k 00 0 and also ReV U

= I k 00 I n kOrthogonal scaling map:

Scale, V =I k 00 I n k

For L : U U , if U = {u1, . . . , u n }is an orthonormal basis for U such that L U = D = diag( 1, . . . , n ), thenL =

n

i=1

i Proj u i =distinct

eigenvalues

Proj E =n

i=1

Scale i ,u i =distinct

eigenvalues

Scale,E

Also, for L : U U where U is a nite-dimensional inner product space,L is an orthogonal reection (that is, L = Re V for some subspace V

U)

if and only if L = Land LL = I if and only if L = Land L2 = I. Furthermore,

L is an orthogonal projection (that is, L = Proj V for some subspace V

U)

if and only if L = Land L2 = L.

8.3 Singular value decomposition

Theorem 8.24 (Singular Value Theorem ). Let U and V be nite dimensional inner product spaces over F = R orLet L : U V be linear. Then there exist orthonormal bases U and V for U and V such that

L U V

is in the form diag(1, . . . , r ) 00 0

where the i are real ( r = rank L) with 1 2 . . . r > 0. The values i are uniquely determined from L.Proof. (uniqueness) Suppose that {u1, . . . , u k}and V = {v1, . . . , v }are orthonormal bases for U and V such that

L U V

= diag( 1, . . . , r ) 00 0

with 1 2 . . . r > 0. Then L(u i ) = i vi for i r and L(u i ) = 0 for i > r . Also,LV U

= diag(1, . . . , r ) 00 0

so L(vi ) = i u i for i r and also L(vi ) = 0 for i > r , therefore LL(u i ) = Li vi = i L(vi ) = 2i u i , so each 2i is eigenvalue for LL and each u i is an eigenvalue for 2i . This completes the proof for uniqueness.(existence) Note that rank( L) = rank( L) = rank( LL). Indeed,

null( L) = null( LL) by your homework.

Note that the eigenvalues LL are all non-negative since if LL(u) = u where 0 = uU then

|L(u)|2 = L(u), L(u) = u, LL(u) = u,u = u, u = |u|2therefore

= |L(u)|2|u|2

0

41


42/50

and hence 0. Also note that LL is self-adjoint, since (LL)= LL= LL. Thus we can orthonormallydiagonalize LL, so choose an orthonormal basis U for U so thatLL U

= D = diag( 1, . . . , n )

with 1 2 . . . n > 0 and i = 0 for i > r . For each i r , leti =

i and vi =

L(u i )i

.

Note that {v1, . . . , vr }is orthonormal sincevi , v j =

L(u i )i

,L(u j )

j=

1i j

L(u i ), L(u j ) =1

i ju i , LL(u j ) =

1i j

u i , 2 j u j = ji

ij = ij .

Extend {v1, . . . , v r }to an orthonormal basis V = {v1, . . . , vr , . . . , v }for V. ThenL U

V = diag( 1, . . . , r ) 00 0

since L(u i ) = i vi for i r and L(u i ) = 0 for i > r .Remark 8.25. When we choose U and V to put L U V in the above form, the above proof shows the following:

The i are the positive square roots of the eigenvalues i (which are non-negative) of LL. The vectors u i are eigenvectors of LL. For i = 1 , . . . , r ,

vi =L(u i )

iand u i =

L(vi )i

{u r +1 , . . . , u k}is an orthonormal basis for null L {u1, . . . , u r }is an orthonormal basis for (null L) {v1, . . . , v r }is an orthonormal basis for range(

L)

{vr +1 , . . . , v }is an orthonormal basis for range( L)= null L.For A

M k (F), F = R or C we can nd unitary (or orthogonal when F = R) matrices P and Q where P = ( u1, . . . , uand Q = ( u1, . . . , u ) (for u i , vi are above) so that A = Q P where

= diag(1, . . . , r ) 00 0

with 1 2 . . . r > 0. Such a factorization A = Q P is called a singular value decomposition of A, anthe numbers1 2 . . . r > 0

are called the singular values of A, or the singular values of L for LHom(U , V).

9 Bilinear and quadratic forms

9.1 Bilinear forms

Denition 9.1. Let U, V, W be vector spaces over any eld F . Then a map F : U V W is called bilinear whethe following are satised: F (u + v, w) = F (u, w) + F (v, w)

42


43/50

F (u, v + w) = F (u, v) + F(u, w) F (tu,v ) = tF(u, v) F (u,tv ) = tF(u, v)

for all u ,v,w in the appropriate spaces (notational note the small letters do not correspond to the large ones).

The set of bilinear mapsF : U V W

is a vector space, which we denote by Bilin( U V, W ).Theorem 9.2. Let U and V be nite-dimensional vector spaces over F . Then for a bilinear map F : U V F , if wchoose bases U and V then there is a unique matrix F U V such that for all uU and vV we have

F (u, v) = v tV

F U V

u U

Moreover the map U ,V : Bilin(U V, F) M k (F) where k = dim U , and = dim V given by U ,V (F ) = F U V

is an isomorphism.

Proof. (uniqueness) SupposeF(u, v) = yt Ax

where y = vV

, x = u U

for all uU , v

V. Say U = {u1, . . . , u k}and V = {v1, . . . , v }. ThenF (u i , v j ) = et j Aei = A ji .

(existence) Given FBilin(U V, F) let A be the matrix given by

A ji = F (u i , v j )

For u = x i u i , v = y j v j so that u U

= x, vV

= y we have

F (u, v) = F x i u i , y j v j =i,j

x i y j F (u i , v j ) =i,j

x i y j A ji = yt Ax.

Verify that this map is indeed linear and bijective (vector space isomorphism).

Remark 9.3. U

V = Bilin( UV, F) is called the tensor product of U and V.Denition 9.4. Let U be a vector space over a eld F . A bilinear form on U is a bilinear map F : U U F . bilinear form F : U U F is symmetric when

F (u, v) = F (v, u )

for all u, vU . It is skew-symmetric when

F (u, v) = F (v, u )When U is nite-dimensional and U is a basis for U we write

F U

= F U U

.

As an exercise, verify that symmetry (resp. skew-symmetry) of bilinear forms is equivalent to symmetry (resp. skew-symmetry) in their matrices.

43


44/50

Example 9.5. Let U be a nite dimensional vector space over a eld F . Let

F : U U Fbe a bilinear form on U . For bases U and V for U , determine how F U and F V are related.Solution. For x, y

U,

F(x, y ) = y tV

FV

xV

= I U V

y U

tF

V I U

V x

U = y t

U I U

V t

FV

I U V

x U

thereforeF

U = I U

V t

FV

I U V

and we are done.

Denition 9.6. For A, B M n n (F) when A = P

t BP for some invertible matrix P

Date post:	14-Apr-2018
Category:	Documents
Upload:	user2357
View:	216 times
Download:	0 times

LW 1115 Math245notes(Waterloo)

Documents