Projection
Md. Sahidul Islam RiponDepartment of statistics
Rajshahi UniversityEmail: [email protected]
Content
Orthogonal vectorOrthonormal vectorProjectionGram-Schmidt orthogonalization
Orthogonal vector
In mathematics, two vectors are orthogonal if they are perpendicular, i.e., they form a right angle. The relation is clearly symmetric; that is, if x is orthogonal to y then then and so y is orthogonal to x.
Two vectors, x and y, in an inner product space, V, are orthogonal if their inner product is zero.
0, yx
Fig : The line segments AB and CD are orthogonal to each other
Orthogonal vector
x
y
x + y
Pythagoras,
This always not true. This only true when 0yx
222yxyx
0
02
0
)()(
yx
yx
xyyx
xyyxyyxxyyxx
yxyxyyxx
ExampleAre the vector (1,2,2)T and (2,3,-4)T are orthogonal ?
0
2
5
3
;
4
3
2
,
2
2
1
yx
yxyx
38yx
;29y;9x
Are the vector (4,2,3)T and (7,3,-4)T are orthogonal ?
22
12628
1
5
11
;
4
3
7
,
3
2
4
yx
yxyx
222yxyx
147yx
;74y;29x
Theorem 1: An orthogonal set of non zero vectors in a vector space is linearly independent.
Subspace S is orthogonal to subspace T
How???
row space is orthogonal to null space
Orthogonal Subspace
Orthonormal vector Defination: Two vector are said to be orthonormal if it is orthogonal and it has unit length.
Example: Two vector are said to be orthogonal
6
16
26
1
,
3
13
13
1
BA
Orthonormal vector
Theorem: Let { u1, …, un} be an orthonormal basis for a vector space V. Let v be a vector in V. v can be written as a linear combination of these vectors as follows.
nn uuvuuvuuvv ).(...).().( 2211
Proof: Since { u1, …, un} is a basis there exist scalars c1,…,cn such that
v= c1u1+…+cn un
We shall show that, c1 =v1 .u1,…,cn =vn .un
Projection
In linear algebra and functional analysis, a projection is a linear
transformation P from a vector space to itself such that P2 = P.
It leaves its image unchanged. Though abstract, this definition
of "projection" formalizes and generalizes the idea of graphical
projection. One can also consider the effect of a projection on a
geometrical object by examining the effect of the projection on
points in the object.
Projection
Orthogonal Projection
Oblique projection
Orthogonal projection
Let V be any inner-product space and let u V be any vector Let be defined by
The vector u is called the orthogonal projection of v onto u.if the vector u is unit. i,e.
Vvuu
uuvvPu ,
,
,:)(
VVPu :
1u
VvuvvPu ,:)(
x
y
x
y
x x
y
x
Projection Graph
Orthogonal projection
xxx
yx
x
y
)cos(Lx
The projection (or shadow) of a vector x on a vector y is=
Projection of x on y
yL
1
L
yxy
yy
yx
yy
cosθL
L
yx
LL
1
L
yx
x
y
yyy
Why projection?
Beause Ax=b may have no solution. That is when the system of equations are Inconsistent.
Example
Solution: Let be the projection vector. The magnitude of is,
With direction given by the unit vector
Then,
So that can be espressed as,
38.677
56.
X
YXY
Y
)46.0,68.0,57.0(
)4,6,5(78.8
1
X
X
73.077
56
.
.
)94.2,34.4,64.3(
)46.0,68.0,57.0(ˆ
XX
YXand
Y
Find the orthogonal projection of Y=(2,7,1) on to the vector X=(5,6,4)
Y
XY 73.0ˆ
Projection on to a plan
1x
Y
2x
Find the orthogonal projection of Y=(7,7,8) on to the plane spanned by vector X1=(5,6,4) and X2=(9,5,1).
)1,2,5(x1
(7,7,8)Y
)2,5,9(x2
,0.99)(7.59,7.34Y
Solution:
Since must lie in the plane spanned by X1 and X2
2211ˆ XXY
And forming inner products with X and X, we have the equations
and 2211 eXXY
eXXXXXXY
eXXXXXXY
.).().(.
.).().(.
22222112
11221111
339.0
107106106
106170127
1
21
21
and
)0.7,34.0,59.0(
)99.0,34.7,59.7(-(7,7,8)Y-Y e ,
)99.0,34.7,59.7(
0.665) 3.275, (5.895,0.339) 4.068, (1.695,
1) 3, (9, 0.6551) 12, (5, 0.339
ˆ221 1
Where
XXY
Application
1. Gram –schmidt orthogonalization 2. Curve fitting by ordinary least square method. 3. The area of a parallelogram
Gram Schmit Orthogonalization
The Gram-Schmidt orthogonalization process allows us to turn any set of linearly independent vectors into an orthonormal set of the same cardinality. In particular, this holds for a basis of an inner product space. If you feed the machine any basis for the space, the process cranks out an orthonormal basis.
Jorgen Pedersen Gram(1850 - 1916)
Erhard Schmidt(1876 - 1959)
Gram Schmit Orthogonalization
Let be a basis for vector space V. The set of vector defined as follows is orthogonal. To obtain a orthogonal basis for V, normalized each of the vector
},...,,{ 21 nvvv},...,,{ 21 nuuu
uuu
uvv
,
, Proj u
1
1iii
3333
222
11
vvu
vv v u
v v u
v u
21
1
i
ju
uu
u
jproj
projproj
proj
i
ii u
uq
u
uq
u
uq
u
uq
3
33
2
22
1
11
Fig: First two steps of Gram schmidt orthogonalization
Geometric Interpretation
Consider the following set of vectors in R2 (with the conventional inner product)
Now, perform Gram–Schmidt, to obtain an orthogonal set of vectors:
We check that the vectors u1 and u2 are indeed orthogonal:
noting that if the dot product of two vectors is 0 then they are orthogonal.We can then normalize the vectors by dividing out their sizes as shown above:
;
Example
ExampleThe set {(1,2,0,3), (4,0,5,8), (8,1,5,6)} is linearly independent in R4. The vectors form a basis for a three-dimensional subspace V of R4. Construct an orthonormal basis for V. Solution:Let v1=(1, 2, 0, 3), v2=(4, 0, 5, 8), v3=(8, 1, 5, 6). We now use the Gram- Schmidt process to consturect an orthogonal st {u, u, u} from these vectors.
222
231
11
1333333
111
122222
11
u .uu
.uvu
.uu
.uv -vvv v u Let,
u .uu
.uv -vv v u Let,
3) 0, 2, (1, v u Let,
21
1
uu
u
projproj
proj
When this process is implemented on a computer, the vectors uk are often
not quite orthogonal, due to rounding errors. For the Gram–Schmidt process as described above (sometimes referred to as "classical Gram–Schmidt") this loss of orthogonality is particularly bad; therefore, it is said that the (classical) Gram–Schmidt process is numerically unstable.The Gram–Schmidt process can be stabilized by a small modification; this version is sometimes referred to as modified Gram-Schmidt or MGS. This approach gives the same result as the original formula in exact arithmetic and introduces smaller errors in finite-precision arithmetic. Instead of computing the vector uk as it is computed as Each step finds a
vector orthogonal to . Thus is also orthogonalized against any errors introduced in computation of .
Modified Gram-Schmidt orthogonalization
Modified Gram-Schmidt orthogonalization
Compare classical and modified G-S for the vectorsTTT vvv )0,0,,1( ,)0,0,,1( ,)0,0,,1( 321
Making approximation
),0,,0(
)0,,0,1()0,,0,1).(0,,0,1(
)0,,0,1).(,0,0,1()0,0,,1(
)0,0,,1).(0,0,,1(
)0,0,,1).(,0,0,1(),0,0,1(
)0,,,0()0,0,,1()0,0,,1).(0,0,,1(
)0,0,,1).(0,,0,1()0,,0,1(
)0,0,,1(
3
2
1
u
u
u
21)/ 0, (0,1,
2/)0,1,1,0(
)0,0,,1(
3
33
2
22
1
11
u
uq
u
uq
u
uq
21
),0,,0(
)0,,0,1()0,,0,1).(0,,0,1(
)0,,0,1).(,0,0,1()0,0,,1(
)0,0,,1).(0,0,,1(
)0,0,,1).(,0,0,1(),0,0,1(
)0,,,0()0,0,,1()0,0,,1).(0,0,,1(
)0,0,,1).(0,,0,1()0,,0,1(
)0,0,,1(
3
2
1
u
u
u
21)/ 0, (0,1,
2/)0,1,1,0(
)0,0,,1(
3
33
2
22
1
11
u
uq
u
uq
u
uq
Classical Vs Modified
),0,,0(
)0,,0,1()0,,0,1).(0,,0,1(
)0,,0,1).(,0,0,1()0,0,,1(
)0,0,,1).(0,0,,1(
)0,0,,1).(,0,0,1(),0,0,1(
)0,,,0()0,0,,1()0,0,,1).(0,0,,1(
)0,0,,1).(0,,0,1()0,,0,1(
)0,0,,1(
3
2
1
k
k
k
u
u
u
61)/ 0, (0,1,
2/)0,1,1,0(
)0,0,,1(
3
33
2
22
1
11
u
uq
u
uq
u
uq
k
k
k
Classical Vs Modified
Classical Vs Modified
To check the orthogonality
0122)/ 1,- (0,-1,).0,1,1,0(
1/21)/2 0, (0,1,).0,1,1,0(
32
32
k
T
k
T
Curve fitting by ordinary least square method
One of the most widely used methods of curve fitting a straight line to data points is that of ordinary least squares, which makes use of orthogonal projections
eX
eYY
ˆ
).().(.
).().(.
XXXlXY
lXlllY
2
1
2
1
xnx
yxnyx
xy
n
ii
i
n
ii
Refference1. Applied linear algebra in the statistical sciences.
-by Alexander Basilevsky.2. Lecture on linear Algebra. (Gilbert Strange) Institute of MIT.3. Applied Multivariate analysis. -by R. A. Johnson, D. W. Wichern.4. Linear Algebra Theory and Application (2003) -by Ward Cheney and David Kincaid5. Linear Algebra with Application (2007)
- by Granth Williams
Thank You