3.VI.1. Orthogonal Projection Into a Line3.VI.2. Gram-Schmidt Orthogonalization3.VI.3. Projection Into a Subspace
3.VI. Projection
3.VI.1. & 2. deal only with inner product spaces.
3.VI.3 is applicable to any direct sum space.
3.VI.1. Orthogonal Projection Into a Line
Definition 1.1: Orthogonal ProjectionThe orthogonal projection of v into the line spanned by a nonzero s is the vector.
ˆ ˆproj s v v s s
v ss
s s
ˆ
s ss
s s s
Example 1.3: Orthogonal projection of the vector ( 2 3 )T into the line y = 2x.
2 11 1
2 63 25 5
proj
s
2
x
x
s 2 24 5x x x s11
ˆ25
s
18
25
Example 1.4: Orthogonal projection of a general vector in 3 into the y-axis
2
0
1
0
e2
0 0
1 1
0 0
x x
proj y y
z z
e
0
1
0
y
0
0
y
Example 1.5: Project = Discard orthogonal componentsA railroad car left on an east-west track without its brake is pushed by a wind blowing toward the northeast at fifteen miles per hour; what speed will the car reach?
1proj ev w
115
02
115
12
w
15
2speed v
Example 1.6: Nearest PointA submarine is tracking a ship moving along the line y = 3x + 2.Torpedo range is one-half mile. Can the sub stay where it is, at the origin on the chart, or must it move to reach a place where the ship will pass within range?
Ship’s path is parallel to vector11
ˆ310
s
Point p of closest approach is
0 0
2 2proj
sp
0 16 1
2 310 10
0 13
2 35
31
15
110
5p 0.632 (Out of range)
Exercises 3.VI.1.
1. Consider the function mapping a plane to itself that takes a vector to itsprojection into the line y = x. (a) Produce a matrix that describes the function’s action.(b) Show also that this map can be obtained by first rotating everything in the plane π/4 radians clockwise, then projecting into the x-axis, and then rotating π/4 radians counterclockwise.
3.VI.2. Gram-Schmidt Orthogonalization
Given a vector s, any vector v in an inner product space can be decomposed as
proj proj s sv v v v // v v // 0 v vwhere
Definition 2.1: Mutually Orthogonal Vectors
Vectors v1, …, vk n are mutually orthogonal if
vi · vj = 0 i j
Theorem 2.2:A set of mutually orthogonal non-zero vectors is linearly independent.
Proof:i i
i
c v 0 0j i i j j ji
c c v v v v→
→ cj = 0 j
Corollary 2.3:A set of k mutually orthogonal nonzero vectors in V k is a basis for the space.
Definition 2.5: Orthogonal Basis An orthogonal basis for a vector space is a basis of mutually orthogonal vectors.
Example 2.6: Turn1 0 1
1 , 2 , 0
1 0 3
into an orthogonal basis for 3.
1
1
1
1
κ
12
0 0
2 2
0 0
proj
κκ
0 12
2 13
0 1
1 23
1 1 1
0 0 0
3 3 3
proj proj
κ κκ
1
0 1 0 11
2 1 2 13
0 1 0 1
proj
κ1 1 3 κ κ
12
13
1
12
23
1
2 2
24 8
9 3 κ κ
1
1 1 1 11
0 1 0 13
3 1 3 1
proj
κ
14
13
1
2
1 1 1 13 2 2
0 2 0 28 3 3
3 1 3 1
proj
κ
12
23
1
1 1 14 2
0 1 23 3
3 1 1
1
0
1
1 1 1
21 , 2 , 0
31 1 1
K
Theorem 2.7: Gram-Schmidt OrthogonalizationIf β1 , …, βk is a basis for a subspace of n then the κj s calculated from the following scheme is an orthogonal basis for the same subspace.
1 1κ β
12 2 2proj κκ β β 2 12 1
1 1
β κ
β κκ κ
1 23 3 3 3proj proj κ κκ β β β 3 1 3 23 1 2
1 1 2 2
β κ β κ
β κ κκ κ κ κ
1
1j
k
k k kj
proj
κκ β β1
1
kk j
k jj j j
β κ
β κκ κ
1 12
1 1
κ κI β
κ κ
1 1 2 22
1 1 2 2
κ κ κ κI β
κ κ κ κ
1
1
kj j
kj j j
κ κI β
κ κ
Proof: For m 2 , Let β1 , …, βm be mutually orthogonal, and
11 1
1
mm j
m m jj j j
β κ
κ β κκ κ
11 1
1
mm j
i m i m i jj j j
β κ
κ κ κ β κ κκ κ
Then 1 1i m m i κ β β κ 0 1, ,i m QED
If each κj is furthermore normalized, the basis is called orthonormal.
The Gram-Schmidt scheme simplifies to:
1 1κ β
12 2 2proj κκ β β 2 2 1 1 β β e e
1 23 3 3 3proj proj κ κκ β β β 3 3 1 1 3 2 2 β β e e β e e
1
1j
k
k k kj
proj
κκ β β 1
1
k
k k j jj
β β e e
1 1 2 I e e β
1 1 2 2 2 I e e e e β
1
1
k
j j kj
I e e β
11
1
κ
eκ
22
2
κ
eκ
33
3
κ
eκ
kk
k
κ
eκ
Exercises 3.VI.2.
1. Perform the Gram-Schmidt process on this basis for 3,2 1 0
2 , 0 , 3
2 1 1
2. Show that the columns of an nn matrix form an orthonormal set if and only if the inverse of the matrix is its transpose. Produce such a matrix.
3.VI.3. Projection Into a Subspace
Definition 3.1:
For any direct sum V = M N and any v V such that
v = m + n with m M and n N
The projection of v into M along N is defined as
projM, N (v) = m
Reminder:
• M & N need not be orthogonal.
• There need not even be an inner product defined.
Example 3.2: The space 22 of 22 matrices is the direct sum of
,0 0
a bM a b
R0 0
,N c dc d
R
Task: Find projM , N (A), where 3 1
0 4
A
Solution:
Let the bases for M & N be1 0 0 1
,0 0 0 0M
B
→
0 0 0 0,
1 0 0 1N
B
M MB B Bº1 0 0 1 0 0 0 0
, , ,0 0 0 0 1 0 0 1
is a basis for 22 .
1 0 0 1 0 0 0 03 1 0 4
0 0 0 0 1 0 0 1
A∴
,
1 0 0 13 1
0 0 0 0M Nproj
A3 1
0 0
Example 3.3: Both subscripts on projM , N (v) are significant.
Consider 2 0
x
M y y z
z
with basis1 0
0 , 2
0 1M
B
0
0
1
N k k
R &and subspaces0
1
2
L k k
R
It’s straightforward to verify 3 M N M L R
where
2
2
5
vTask: Find projM , N (v) and projM , L (v)
Solution:
For1 0 0
0 , 2 , 0
0 1 1M N M N
B B Bº
2
1
4M N
v
B
→ ,
2
1
0M N
M Nproj
v
B
2
2
1
For1 0 0
0 , 2 , 1
0 1 2M L M L
B B Bº
2
2
5
v→
,
2
9 / 5
0M N
M Lproj
v
B
2
18 / 5
9 / 5
2 2
2 5
a b
a b
2
9 / 5
8 / 5M L
B
Note: ML is orthogonal but MN is not.
Definition 3.4: Orthogonal Complement
The orthogonal complement of a subspace M of n is
M = { v n | v is perpendicular to all vectors in M } ( read “M perp” ).
The orthogonal projection projM (v ) of a vector is its projection into M along M
.
Example 3.5:
In 3, find the orthogonal complement of the plane 3 2 0
x
P y x y z
z
Solution: Natural basis for P is 1 0
0 , 1
3 2
B ( parameter = z)
→1 1
2 2
3 3
1 0 3 0
0 1 2 0
v v
v v
v v
3
2
1
k k
R 1 20, 0P v v β v β
Lemma 3.7:
Let M be a subspace of n. Then M is also a subspace and n = M M .
Hence, v n, v projM (v) is perpendicular to all vectors in M.
Proof: Construct bases using G-S orthogonalization.
Theorem 3.8:
Let v be a vector in n and let M be a subspace of n with basis β1 , …, βk .
If A is the matrix whose columns are the β’s then
projM (v ) = c1β1 + …+ ck βk
where the coefficients ci are the entries of the vector (AT A) AT v. That is,
projM (v ) = A (AT A)1 AT v.
Proof: Mproj Mv
T TA A c A v
where c is a column vector
T 0 A v Ac
→ Mproj v A c
By lemma 3.7,
1T Tc A A A v→ →
Example 3.9:
To orthogonally project1
1
1
v 0
x
P y x z
z
1 0
0 1
1 0
A
into subspace
From1 0
0 1 ,
1 0
P x y x y
R we get
→
11 0
1/ 2 0 1 0 10 1
0 1 0 1 01 0
T T
A A A A
1 01 0 1
0 10 1 0
1 0
T
A A2 0
0 1
1 1/ 2 0
0 1T
A A
1 01/ 2 0 1/ 2
0 10 1 0
1 0
1/ 2 0 1/ 2
0 1 0
1/ 2 0 1/ 2
1/ 2 0 1/ 2 1
0 1 0 1
1/ 2 0 1/ 2 1Pproj
v
0
1
0
Exercises 3.VI.3.
1. Project v into M along N, where
3
0
1
v 0
x
M y x y
z
1
0N c c
R
2. Find M for 3 0
x
M y x y z
z
3. Define a projection to be a linear transformation t : V → V with the property
that repeating the projection does nothing more than does the projection alone:
( t t )(v) = t (v) for all v V.
(a) Show that for any such t there is a basis = β1 , …, βn for V such that
1, 2, ,
for1, 2, ,
ii
i rt
i r r n
ββ
0
where r is the rank of t.
(b) Conclude that every projection has a block partial-identity representation:
Rep t
I 0
0 0B B