Notes onNotes onWeighted Least Squares Straight Weighted Least Squares Straight
line Fitline FitPassing Through The OriginPassing Through The Origin
Amarjeet BhullarAmarjeet Bhullar
November 14, 2008November 14, 2008
Data SetData Set
• For given {xFor given {xii, y, yi i } find line through } find line through them;them;i.e., find a and b in y = a+bxi.e., find a and b in y = a+bx
(x(x11,y,y11))
(x(x22,y,y22))
(x(x33,y,y33))
(x(x44,y,y44))
(x(x55,y,y55))(x(x66,y,y66))
(x(x77,y,y77))
bxay
Least SquaresLeast Squares
• Universal formulation of fitting:Universal formulation of fitting:minimize squares of differences betweenminimize squares of differences betweendata and functiondata and function– Example: for fitting a line, minimizeExample: for fitting a line, minimize
Using appropriate a and bUsing appropriate a and b– General solution: take derivatives w.r.t. General solution: take derivatives w.r.t.
unknown variables, set equal to zerounknown variables, set equal to zero
iii
i
bxay 22
2 1
Linear Least Squares: Equal Linear Least Squares: Equal WeightingWeighting
iii
iii
i
bxaybxayaa
0)(212
22
2
iiii
iii
i
bxayxbxaybb
0)(212
22
2
i
ii
iii
i xbaNxbay ]1[
i
ii
ii
ii xbxayx 2
ba
xx
xN
yx
y
ii
ii
ii
iii
ii
2
iii
ii
ii
ii
ii
yx
y
xx
xN
ba
1
2
iii
ii
ii
ii
ii
yx
y
Nx
xx
ba
2
1
Linear Least Squares: Equal Linear Least Squares: Equal WeightingWeighting
i i i iiiiii yxxyxIntercepta 21
i i iiiii yxyxNSlopeb 1
22
ii
ii xxN
Data Reduction and Error Analysis for the Physical Sciences by Philip R Bevington (1969) Data Reduction and Error Analysis for the Physical Sciences by Philip R Bevington (1969)
Uncertainties or Estimation of Uncertainties or Estimation of Errors: In a & bErrors: In a & b
• Using the propagation of errors:Using the propagation of errors:
2
22
22
22
iyvuz y
zvz
uz
i
22
2
1
1
ii
ii
iij
j
i iiji
j
xxN
xNxyb
xxxya
Uncertainty or Estimation of Error: Uncertainty or Estimation of Error: In Calculated aIn Calculated a
ii
ii
ii
ii
i iii
ii
ii
ii
i i iijiij
ii
N
j
j ja
xxxNx
xxxxxN
xxxxxx
ya
222
222
2
222
222
2
2
222
22
12
2
2
22
2
2
The uncertainty in parameter aThe uncertainty in parameter a
Uncertainty or Estimation of Error: Uncertainty or Estimation of Error: In Calculated bIn Calculated b
The uncertainty in parameter bThe uncertainty in parameter b
222
2
2
2222
2
2
222
12
2
2
22
2
2
NxxNN
xNxNxN
xxNxxN
ya
ii
ii
ii
ii
ii
i iiijj
N
j
j jb
Uncertainties or Estimation of Uncertainties or Estimation of Errors: Errors:
In Calculated a & bIn Calculated a & bIntercept Uncertainty or ErrorIntercept Uncertainty or Error
iia x2
22
Slope Uncertainty or ErrorSlope Uncertainty or Error
2
2 Nb
22 )(2
1
i
ii bxayN
WhereWhere
&& 22
ii
ii xxN
bxay
?bxy
Linear Least Squares fit : Linear Least Squares fit :
• Linear least squares fitting and error of Linear least squares fitting and error of a straight line which MUST go through a straight line which MUST go through the origin (0, 0).the origin (0, 0).
• Partial derivative w. r. t. b is zeroPartial derivative w. r. t. b is zero
222 1
iii i
bxy
bxy
ii
iii
iii i
i
xbyx
bxyxb
2
2
2
02
ii
iii
x
yxSlopeb 2
bxy Uncertainty or Estimation of Error Uncertainty or Estimation of Error in b in b
22
2
2
22
22
2
22
)(
ii
ii
ji
i
jj
j jjb
x
x
xx
yb
ii
iii
x
yxb 2
ii
j
j xx
yb
2
ii
b x22
2
22 )(1
1
i
ii bxyN
WhereWhere
Weighted Least Squares Straight Weighted Least Squares Straight Line FittingLine Fitting
iii
iiii
i
bxaybxayaa
0)(1212
22
iii
i
i
iii
i
bxayxbxaybb
0)(212
22
2222
21
i
ii
i
i
i
i
i
i yxxyxa
2222
11
i
i
i
i
i
ii
i
yxyxb
2
22
2
2
1
i
i
i
i
i
xx
bxay
Uncertainties in a and b: Unequal Uncertainties in a and b: Unequal Weighting Weighting
Intercept Uncertainty or ErrorIntercept Uncertainty or Error
i i
ia
x2
22 1
Slope Uncertainty or ErrorSlope Uncertainty or Error
i ib 22 11
WhereWhere2
22
2
2
1
i
i
i
i
i
xx
bxay
?bxy
Weighted Least Squares Straight Weighted Least Squares Straight Line Fit:Line Fit:
bxy
iii
i
i
iii
i
bxyxbxybb
0)(212
22
2
i i
i
i i
ii
x
yx
b
2
2
2
i i
i
i i
ii xbyx2
2
2
Eq (6) in draft should beEq (6) in draft should be
0
850
5202 log11
IIbADC i
ii i
WhereWhere2
850
5202
1i
i i
b
Eq (7) in draft should beEq (7) in draft should be
Uncertainty in b: Unequal Uncertainty in b: Unequal WeightingWeighting
2
2
2
2
1
,1
j
j
j
i i
i
i i
ii
xyb
xwithyxb
i i
i
i
i
i
j j
jj
j jjb
xx
x
yb
2
2
22
2
2
4
2
22
2
22
11
1
i i
ii i
ib x
x
2
22
2
22 11
Eq (8) in draft should beEq (8) in draft should be
850
5202
22 1
i i
iADC b
ConclusionConclusion
0
850
5202 log11
IIbADC i
ii i
2850
5202
1i
i i
b
850
5202
22 1
i i
iADC b
Eq (6) Eq (6)
Eq (7) Eq (7)
Eq (8) Eq (8)