Date post: | 29-Dec-2015 |
Category: |
Documents |
Upload: | bathsheba-owens |
View: | 214 times |
Download: | 1 times |
Ordinary Least-Squares
Emmanuel IarussiInria
2
Many graphics problems can be seen as finding the best set of parameters for a model, given some data
Surface reconstruction
3
Color diffusion
Many graphics problems can be seen as finding the best set of parameters for a model, given some data
4
Shape registration
Many graphics problems can be seen as finding the best set of parameters for a model, given some data
5
Temperature (a) Gas pressure (b)
Example: estimation of gas pressure for a given temperature sample
6
Assuming linear relationship
7
Assuming linear relationship
8
Assuming linear relationship
Temperature measurements (m)
Pressure measurements (m)
9
Assuming linear relationship
Temperature measurements (m)
Pressure measurements (m)
=
10
Assuming linear relationship
The relationship might not be exact
Temperature measurements (m)
Pressure measurements (m)
=
11
Assuming linear relationship
The relationship might not be exact
Temperature measurements (m)
Pressure measurements (m)
…
=
Match “as best as possible” the observations
12
Assuming linear relationship
The relationship might not be exact Match “as best as possible” the observations
Temperature measurements (m)
Pressure measurements (m)
=
13
Many of CG problems can be formulated as minimizing the sum of squares of the residuals
between some features in the model and the data.
or
14
Matrix notation (observations)
2
2
)(
)()(
)()(
ab
abab
xxe
xx
xabxe
T
iii
We can rewrite the residual function using linear algebra as:
15
Temperature
Gas pressure
Example: estimation of gas pressure for a given temperature and altitude samples
Altitude
16
Multidimensional linear regression, using a model with n parameters
17
Multidimensional linear regression, using a model with n parameters
before+ +…
18
Multidimensional linear regression, using a model with n parameters
Temperature measurements (m)
Pressure measurements (m)
Altitudemeasurements (m)
19
Multidimensional linear regression, using a model with n parameters
Temperature measurements (m)
Pressure measurements (m)
Altitudemeasurements (m)
=
20
=
21
=
22
=
23
Objective: find x subject to minimize:
24
Objective: find x subject to minimize:
25
Objective: find x subject to minimize:
Convex bowl function has minima when:
26
Objective: find x subject to minimize:
Convex bowl function has minima when:
gradient
27
Expanding square term
28
Expanding square term
29
Differentiating with respect to x (gradient)
Expanding square term
30
Expanding square term
Differentiating with respect to x (gradient)
31
Expanding square term
Normal equation
x = (A.transpose()*A).solve(A.transpose()*b);
Differentiating with respect to x (gradient)
32
Expanding square term
Normal equation
x = (A.transpose()*A).solve(A.transpose()*b);
Issue: matrix multiplication
Differentiating with respect to x (gradient)
33
Differentiating with respect to x
Expanding square term
Normal equation
x = (A.transpose()*A).solve(A.transpose()*b);
Issue: matrix multiplication Solution: find expression
34
Let's go back to gradient = 0
35
Let's go back to gradient = 0
Differentiating again with respect to x
36
Let's go back to gradient = 0
Differentiating again with respect to x
37
Let's go back to gradient = 0
Differentiating again with respect to x
hessian
38
Let's go back to gradient = 0
Differentiating again with respect to x
hessian
39
Let's go back to gradient = 0
Differentiating again with respect to x
x = (A.transpose()*A).solve(A.transpose()*b);
Normal equation:
40
Let's go back to gradient = 0
Differentiating again with respect to x
x = (A.transpose()*A).solve(A.transpose()*b);
Normal equation:
41
Let's go back to gradient = 0
Differentiating again with respect to x
x = (A.transpose()*A).solve(A.transpose()*b);
x = (H).solve(c);
Two alternatives for solving:
Constant terms in gradient
42
Example: Diffusion
43
Example: Diffusion
Minimize difference between neighbors
…
44
Example: Diffusion
Minimize difference between neighbors
…
45
Example: Diffusion
Minimize difference between neighbors
…
46
Example: Diffusion
Minimize difference between neighbors
…
47
Solving with normal equation:
…
1. Build A
48
Solving with normal equation:
…
1. Build A
49
Solving with normal equation:
…
1. Build A
50
Solving with normal equation:
…
1. Build A
51
Solving with normal equation:
…
1. Build A
52
Solving with normal equation:
…
1. Build A
53
Solving with normal equation:
2. Compute and
54
Solving with normal equation:
3. Solve using:• Cholesky decomposition• Conjugate Gradient• …
x = (A.transpose()*A).solve(A.transpose()*b);
in Eigen (i.e):
55
Solving with normal equation:
3. Solve using:• Cholesky decomposition• Conjugate Gradient• …
x = (A.transpose()*A).solve(A.transpose()*b);
in Eigen (i.e):
x wil have:
56
Solving with Hessian:
1. Gradient vector: first-order partial derivatives of energy terms
57
Solving with Hessian:
1. Gradient vector: first-order partial derivatives of energy terms
58
Solving with Hessian:
1. Gradient vector: first-order partial derivatives of energy terms
59
Solving with Hessian:
1. Gradient vector: first-order partial derivatives of energy terms
60
Solving with Hessian:
1. Gradient vector: first-order partial derivatives of energy terms
61
Solving with Hessian:
2. Hessian matrix: second-order partial derivatives of energy terms
62
Solving with Hessian:
2. Hessian matrix: second-order partial derivatives of energy terms
63
Solving with Hessian:
2. Hessian matrix: second-order partial derivatives of energy terms
64
Solving with Hessian:
2. Hessian matrix: second-order partial derivatives of energy terms
65
Solving with Hessian:
66
Solving with Hessian:
1. Gradient vector: first-order partial derivatives of energy terms
67
Solving with Hessian:
1. Gradient vector: first-order partial derivatives of energy terms
68
Solving with Hessian:
1. Gradient vector: first-order partial derivatives of energy terms
69
Solving with Hessian:
1. Gradient vector: first-order partial derivatives of energy terms
70
Solving with Hessian:
2. Hessian matrix: second-order partial derivatives of energy terms
71
Solving with Hessian:
2. Hessian matrix: second-order partial derivatives of energy terms
72
Solving with Hessian:
2. Hessian matrix: second-order partial derivatives of energy terms
73
Solving with Hessian:
2. Hessian matrix: second-order partial derivatives of energy terms
74
Solving with Hessian:
75
Solving with Hessian:
To recap:
76
Solving with Hessian:
3. Build system
…
77
Solving with Hessian:
3. Build system
…
78
Solving with Hessian:
3. Build system: add constraint
…
79
Solving with Hessian:
…
3. Build system: add constraint
80
Solving with Hessian:
…
3. Build system: add constraint
81
Solving with Hessian:
…
3. Build system: add constraint
82
Solving with Hessian:
…
3. Build system: add constraint
83
Solving with Hessian:
…
3. Build system: add constraint
84
Solving with Hessian:
…
3. Build system: add constraint
85
Solving with Hessian:
…
3. Build system: add constraint
86
Solving with Hessian:
…
3. Build system: add constraint
87
Solving with Hessian:Solving with Hessian:
4. Solve using:• Cholesky decomposition• Conjugate Gradient• …
88
Solving with Hessian:Solving with Hessian:
x = (H).solve(b`);
in Eigen (i.e):
x will have:
4. Solve using:• Cholesky decomposition• Conjugate Gradient• …
89
References:
• Solving Least Squares Problems. C. L. Lawson and R. J. Hanson. 1974.
• Practical Least Squares for Computer Graphics. Fred Pighin and J.P. Lewis. Course notes 2007.