Projection and Least Squares

Question: What is the projection a vector x onto a subspace V of , and how do we find this projection?

Note: We will discuss 2 methods this time, and the second of these is the easiest.

Method 1: Projecting a vector x onto a subspace V of when we have an orthogonal basis for V .

Example: Project the vector x onto span{v,w}.

1 1 2

2 , 1 , 1

1 1 3

x v w

Question: How can we easily find an orthogonal basis for a 2 dimensional subspace of ?

Example: Project the vector x onto span{v,w}.

1 1 2

2 , 1 , 1

1 1 3

x v w

Note: This method begs the question… How do we find an orthogonal basis for a subspace?

The Gram Schmidt Process (a great way to create an orthogonal basis)

Assumption: V is a subspace of Rn with basis {v1,…,vm}.

Goal: Find an orthogonal basis for V.

Example: Find an orthogonal basis for span , , . Then use this orthogonal basis to find the projection of x onto V.

1 2 3

1 0 1 1

1 2 3 1, , ,

1 2 0 1

2 1 1 1

x v v v

Using Least Squares to Find Projections (Least Squares and the System of Normal Equations)

Example: Find the projection of x onto span , , by using the system of normal equations.

1 2 3

1 0 1 1

1 2 3 1, , ,

1 2 0 1

2 1 1 1

x v v v

More General… Least Squares Solutions to Linear Systems of Equations

Example: Find the least squares solution to the system Ax = b.

1

1

3

2

1

,

021

111

112

210

121

bA

What is the projection of b onto Col A ?