12/8/2011
1
Titus Beu 2011
Titus BeuUniversity “Babes-Bolyai”Department of Theoretical and Computational PhysicsCluj-Napoca, Romania
10. Approximation of tabulated functions10. Approximation of tabulated functions
Titus Beu 2011
Bibliography
Introduction
Interpolation and regression
Lagrange interpolating polynomial
Spline interpolation
Linear regression
12/8/2011
2
Titus Beu 2011
D.W. Marquardt, Journal of the Society for Industrial and Applied Mathematics, vol. 11, p. 431-441 (1963).
C.L. Lawson şi R. Hanson, Solving Least Squares Problems (Prentice-Hall, Englewood Cliffs, NJ, 1974).
B.P. Demidovich şi I.A. Maron, Computational Mathematics (MIR Publishers, Moskow, 1981).
C. Iacob, D. Homentcovschi, N. Marcov şi A. Nicolau, Matematici clasice şi moderne, vol. 4 (Editura tehnică, Bucureşti, 1983).
R.L. Burden şi J.D. Faires, Numerical Analysis, Third Edition (Prindle, Weber & Schmidt, Boston, 1985).
W.H. Press, S.A. Teukolsky, W.T. Vetterling şi B.P. Flannery, Numerical Recipes in C: The Art of Scientific Computing, Second Edition (Cambridge University Press, Cambridge, 1992).
Beu, T. A., Numerical Calculus in C, Third Edition(MicroInformatica Publishing House, Cluj-Napoca, 2004).
Titus Beu 2011
The problem of approximating tabulated functions
Let f be a real function of a real variable:
specified by its values on a mesh of points, xi ∈ [α,β]:
Main goal: constructing new data points within the range of the known data points
Dependence typically represented by a continuous model function depending on a finite number of parameters:
Model function F and parameters aj are chosen such as to “optimally” approximate the original function f.
: [ , ] , [ , ]f α β α β→ ℜ ⊂ ℜ
( ) , 1,2, ,i if x y i n= = …
1( ; , , )mF F x a a= …
12/8/2011
3
Titus Beu 2011
Choice of the model function:
From an analytically convenient class of functions (polynomials, Gaussians etc.), which can be easily manipulated
From a theory in which the parameters have a well-established significance
From a compromise between the constraints and advantages of the first options
Distance between the original and the model function:
Can be used to characterize the quality of the approximation
Root mean square (RMS) or quadratic mean approximation – based on distance
Method of least squares – used in interpolation and regression
a
1/2
2
1
( , ) ( ) ( ; )
n
i i
i
d f F f x F x
=
= − ∑
Titus Beu 2011
Intepolation
Values yi are considered to be exact
Model function must satisfy the interpolation conditions:
F(x;a) – interpolant
xi – interpolation points
Model parameters aj – solving the system of interpolation conditions
Popular polynomial interpolants: Lagrange, spline, Akima
( , ) 0d f F =
a( ; ) , 1,2, ,i iF x y i n= = …
12/8/2011
4
Titus Beu 2011
Regression
Tabulated values yi originate from observations (measurements)
Associated measurement errors –
standard deviations σi
Merit functions used practically –Sum of squared residuals:
Chi-Square:
a
22
21
1( ; )
n
i i
i i
y F xχσ=
= − ∑
( , ) minja
d f F =
a
2
1
( ; )
n
i i
i
S y F x
=
= − ∑
Titus Beu 2011
Regression
Minimum of χ2 with respect to the model parameters:
Model parameters – solution of system
Approach – regression or curve fittingby the method of least squares
2
0, 1,2, ,j
j ma
χ∂= =
∂…
12/8/2011
5
Titus Beu 2011
One of the oldest and most general interpolation schemes
Basis for classes of numerical integration and differentiation formulas
Assume the known values:
Define a polynomial satisfying the interpolation conditions:
Family of auxiliary polynomials vanishing at all tabulation points except for xi:
( ) , 1,2, ,i if x y i n= = …
( ) , 1,2, ,m i iP x y i n= = …
( )
( ) ( )( )
jj i
i j ij ii j
j i
x x
p x p xx x
δ≠
≠
∏ −
= → =∏ −
Titus Beu 2011
Polynomial
satisfies the interpolation conditions:
Lagrange interpolating polynomial:
1
1
( ) ( )
n
n i i
i
P x p x y−
=
= ∑
1
1 1
( ) ( ) , 1,2, ,
n n
n j i j i ij i j
i i
P x p x y y y j nδ−
= =
= = = =∑ ∑ …
1
1
( )
( )
( )
n
n jj i
n ini
i jj i
x x
P x y
x x
≠−
=
≠
∏ −
=
∏ −
∑
12/8/2011
6
Titus Beu 2011
Uniqueness
Assume there exists another polynomial of order n − 1 satisfying the same interpolation conditions:
At the tabulation points:
Being a polynomial of order n having n zeroes it must be identically null:
1( ) , 1,2, ,n i iP x y i n− = =� …
1 1( ) ( ) 0, 1,2, ,n i n iP x P x i n− −− = =� …
1 1( ) ( )n nP x P x− −≡�
Titus Beu 2011
Particular cases
Linear interpolation, n = 2:
Parabolic interpolation, n = 3:
2 11 1 2
1 2 2 1
( )x x x x
P x y yx x x x
− −= +
− −
2 3 1 3 1 22 1 2 3
1 2 1 3 2 1 2 3 3 1 3 2
( )( ) ( )( ) ( )( )( )
( )( ) ( )( ) ( )( )
x x x x x x x x x x x xP x y y y
x x x x x x x x x x x x
− − − − − −= + +
− − − − − −
12/8/2011
7
Titus Beu 2011
//===========================================================================
float Lagrange(float xi[], float yi[], int ni, float x)//---------------------------------------------------------------------------
// Evaluates the Lagrange interpolating polynomial at point x//
// xi[] - abscissas of interpolation points// yi[] - ordinates of interpolation points
// ni - no. of interpolation points// x - argument
//---------------------------------------------------------------------------{
float p, y;int i, j;
y = 0.0;
for (i=1; i<=ni; i++) {p = 1.0;
for (j=1; j<=ni; j++)if (j != i) p *= (x-xi[j])/(xi[i]-xi[j]);
y += p*yi[i];}
return y;}
//===========================================================================
float Lagrange(float xi[], float yi[], int ni, float x)//---------------------------------------------------------------------------
// Evaluates the Lagrange interpolating polynomial at point x//
// xi[] - abscissas of interpolation points// yi[] - ordinates of interpolation points
// ni - no. of interpolation points// x - argument
//---------------------------------------------------------------------------{
float p, y;int i, j;
y = 0.0;
for (i=1; i<=ni; i++) {p = 1.0;
for (j=1; j<=ni; j++)if (j != i) p *= (x-xi[j])/(xi[i]-xi[j]);
y += p*yi[i];}
return y;}
Titus Beu 2011
Inverse interpolation
Finding the argument for which the
Lagrange interpolant has given value y:
Application – approximate solutions of algebraic equations f(x) = 0
Calculate yi for n arguments xi close to the solution and take y = 0
Pitfalls of Lagrange interpolation
Oscillations for large no. of tabulation points due to large order of polynomial
1
( )
( )
n jj i
ii ji
j i
y y
x xy y
≠
=≠
∏ −
=∏ −∑
12/8/2011
8
Titus Beu 2011
Spline – instrument used in old times to draw smooth curves through given points
Division of interval [α,β]:
Function f : [α,β] → ℜ satisfying the interpolation conditions:
Spline function of order m relative to division ∆ of interval [α,β]:function S : [α,β] → ℜ of class Cm−1[α,β], whose restrictions Si(x) on subintervals [xi,xi+1] are polynomials of order m
Piece-wise smooth function – first (m−1) derivatives are continuous on [α,β]
Cubic spline function ∈ C2[α,β] – restrictions Si(x) are 3rd order polynomials
1 2: nx x xα β∆ = < < < =…
( ), 1,2, ,i iy f x i n= = …
3 21( ) [ , ], 1,2, , 1i i i i i i iS x Ax B x C x D x x x i n+= + + + = −…∈
Titus Beu 2011
2nd order derivative – linear function ( – 2nd order derivatives in mesh points):
or:
By integration:
1
1
( ) i ii i
i i i
D DS x D
x x x x
′′+
+
−−=
− −
� ��
1 11
( ) ( )( ) , , 1,2, , 1i i i ii i i i
i
D x x D x xS x h x x i n
h
+ +′′+
− + −= = − = −� �
…
2 21 1
3 31 1
( ) ( )( )
2
( ) ( )( )
6
i i i ii i
i
i i i ii i i
i
D x x D x xS x C
h
D x x D x xS x C x C
h
+ +′ ′
+ + ′ ′′
− − −= +
− + −= + +
� �
� �
iD�
12/8/2011
9
Titus Beu 2011
Interpolation conditions – integration constants:
Finally:
2
21 1 1
1 1
( ) 6( )
6
i ii i i ii i i
i i i i ii i i i
D hC x C yS x y
S x y D hC x C y
′ ′′
+ + + ′ ′′+ +
+ + = = ⇒ = + + =
�
�
1 1
1 1 1 1
( )
6
( )
6
i i i i ii
i
i i i i i i i i ii
i
y y D D hC
h
x y x y x D x D hC
h
+ +′
+ + + +′′
− −= −
− −= +
� �
� �
Titus Beu 2011
Restriction coefficients:
1
1 1
2 21 1 1 2
3 3 21 1 1 1
6
2
2
6 3
i ii
i
i i i ii
i
i i i i i ii i i
i i
i i i i i i i i i ii
i i
D DA
h
D x D xB
h
D x D x y yC Ah
h h
D x D x y x y x B hD
h h
+
+ +
+ + +
+ + + +
− = − = − − = + − − − = + −
� �
� �
� �
� �
12/8/2011
10
Titus Beu 2011
Continuity of 1st order derivatives:
System for 2nd order derivatives:
Supplementary conditions for end-points x1 and xn:
Hence:
1( ) ( ), 2, 3, , 1i i i iS x S x i n′ ′− = = −…
11 1 11 1
16 3 6
2,3, , 1
i ii i i i i ii i i
i i
y yh h h h y yD D D
h h
i n
+− − −− +
−
−+ −+ + = −
= −
� � �
…
1 1 1 1( ) , ( )n n nS x y S x y′ ′ ′ ′−= =
1 1 2 11 2 1
1
1 1 11
1
3 6
6 3n n n n
n n nn
h h y yD D y
h
h h y yD D y
h
′
′− − −−
−
−+ = −
−+ = −
� �
� �
Titus Beu 2011
Second order derivatives in mesh points from tridiagonal system:
Coefficients:
1 1 1 2 1
1 1
1
, 2, 3, , 1i i i i i i i
n n n n n
b D c D d
a D b D c D d i n
a D b D d
− +
−
+ = + + = = − + =
� �
� � � …
� �
2 11 1 1 1 1 1 1
1
1 11 1
1
11 1
1
0, 2 , , 6
, 2( ), , 6
, 2 , 0, 6
i i i ii i i i i i i i
i i
n nn n n n n n n
n
y ya b h c h d y
h
y y y ya h b h h c h d
h h
y ya h b h c d y
h
′
+ −− −
−
′ −− −
−
− = = = = − − − = = + = = − − = = = = −
12/8/2011
11
Titus Beu 2011
Calculation steps:
1. Second order derivatives in mesh points from tridiagonal system
2. Restriction coefficients Ai, Bi, Ci, Di
3. Evaluation of the intepolant for particular arguments
Advantages:
Reduced oscillations – better approximation than the Lagrange polynomial
Order of the restrictions is independent on the no. of interpolation points.
Pittfall – too few tabulation points and large variations of observations
iD�
Titus Beu 2011
//===========================================================================
void Spline(float xi[], float yi[], int ni, float x[], float y[], int n,float a[], float b[], float c[], float d[],
float d1, float dn, int iopt)//---------------------------------------------------------------------------
// Calculates the coefficients of the restrictions of he cubic spline// function relative to an interpolation point mesh and evaluates the
// polynomial on an evaluation point mesh//
// xi[] - abscissas of interpolation points// yi[] - ordinates of interpolation points
// ni - no. of interpolation points// x[] - abscissas of the evaluation points
// y[] - evaluated polynomial values (output)// n - no. of evaluation points
// a..d[]- coefficients of restrictions// d1,dn - 1st derivatives for end-points
// iopt - iopt == 0, calculates the 1st order derivatives for end-points// iopt /= 0, takes input values d1 and dn
//---------------------------------------------------------------------------{
float di, dm, hi, hm, xx, xp;int i, ip;
//===========================================================================
void Spline(float xi[], float yi[], int ni, float x[], float y[], int n,float a[], float b[], float c[], float d[],
float d1, float dn, int iopt)//---------------------------------------------------------------------------
// Calculates the coefficients of the restrictions of he cubic spline// function relative to an interpolation point mesh and evaluates the
// polynomial on an evaluation point mesh//
// xi[] - abscissas of interpolation points// yi[] - ordinates of interpolation points
// ni - no. of interpolation points// x[] - abscissas of the evaluation points
// y[] - evaluated polynomial values (output)// n - no. of evaluation points
// a..d[]- coefficients of restrictions// d1,dn - 1st derivatives for end-points
// iopt - iopt == 0, calculates the 1st order derivatives for end-points// iopt /= 0, takes input values d1 and dn
//---------------------------------------------------------------------------{
float di, dm, hi, hm, xx, xp;int i, ip;
12/8/2011
12
Titus Beu 2011
// parabolic approximation for 1st derivative at end-points
if (iopt==0) {d1 = (yi[2] - yi[1])/(xi[2] - xi[1])
- (yi[3] - yi[2])/(xi[3] - xi[2])+ (yi[3] - yi[1])/(xi[3] - xi[1]);
dn = (yi[ni ] - yi[ni-1])/(xi[ni ] - xi[ni-1])- (yi[ni-1] - yi[ni-2])/(xi[ni-1] - xi[ni-2])
+ (yi[ni ] - yi[ni-2])/(xi[ni ] - xi[ni-2]);}
// coefficients of the system of 2nd order derivativehi = 0.0; di = d1;
for (i=1; i<=(ni-1); i++) {hm = hi; hi = xi[i+1] - xi[i];
dm = di; di = (yi[i+1] - yi[i])/hi;a[i] = hm; b[i] = 2*(hm + hi); c[i] = hi; d[i] = 6*(di - dm);
}a[ni] = hi; b[ni] = 2*hi; c[ni] = 0.0; d[ni] = 6*(dn - di);
TriDiag(a,b,c,d,ni); // system solution => 2nd derivatives in d
for (i=1; i<=(ni-1); i++) { // coefficients of restrictions
ip = i+1;xx = xi[i]; xp = xi[ip]; hi = xp - xx;
a[i] = (d[ip] - d[i])/(6*hi);b[i] = (d[i]*xp - d[ip]*xx)/(2*hi);
c[i] = (d[ip]*xx*xx - d[i]*xp*xp)/(2*hi)+ (yi[ip] - yi[i])/hi - a[i]*hi*hi;
d[i] = (d[i]*xp*xp*xp - d[ip]*xx*xx*xx)/(6*hi)+ (yi[i]*xp - yi[ip]*xx)/hi - b[i]*hi*hi/3;
}
// parabolic approximation for 1st derivative at end-points
if (iopt==0) {d1 = (yi[2] - yi[1])/(xi[2] - xi[1])
- (yi[3] - yi[2])/(xi[3] - xi[2])+ (yi[3] - yi[1])/(xi[3] - xi[1]);
dn = (yi[ni ] - yi[ni-1])/(xi[ni ] - xi[ni-1])- (yi[ni-1] - yi[ni-2])/(xi[ni-1] - xi[ni-2])
+ (yi[ni ] - yi[ni-2])/(xi[ni ] - xi[ni-2]);}
// coefficients of the system of 2nd order derivativehi = 0.0; di = d1;
for (i=1; i<=(ni-1); i++) {hm = hi; hi = xi[i+1] - xi[i];
dm = di; di = (yi[i+1] - yi[i])/hi;a[i] = hm; b[i] = 2*(hm + hi); c[i] = hi; d[i] = 6*(di - dm);
}a[ni] = hi; b[ni] = 2*hi; c[ni] = 0.0; d[ni] = 6*(dn - di);
TriDiag(a,b,c,d,ni); // system solution => 2nd derivatives in d
for (i=1; i<=(ni-1); i++) { // coefficients of restrictions
ip = i+1;xx = xi[i]; xp = xi[ip]; hi = xp - xx;
a[i] = (d[ip] - d[i])/(6*hi);b[i] = (d[i]*xp - d[ip]*xx)/(2*hi);
c[i] = (d[ip]*xx*xx - d[i]*xp*xp)/(2*hi)+ (yi[ip] - yi[i])/hi - a[i]*hi*hi;
d[i] = (d[i]*xp*xp*xp - d[ip]*xx*xx*xx)/(6*hi)+ (yi[i]*xp - yi[ip]*xx)/hi - b[i]*hi*hi/3;
}
Titus Beu 2011
// evaluates interpolant
for (i=1; i<=n; i++) { // loop over argumentsxx = x[i];
ip = 1; while (xx > xi[ip+1]) ip++; // index of restrictiony[i] = ((a[ip]*xx + b[ip])*xx + c[ip])*xx + d[ip];
}}
// evaluates interpolant
for (i=1; i<=n; i++) { // loop over argumentsxx = x[i];
ip = 1; while (xx > xi[ip+1]) ip++; // index of restrictiony[i] = ((a[ip]*xx + b[ip])*xx + c[ip])*xx + d[ip];
}}
12/8/2011
13
Titus Beu 2011
Simplest curve fitting by the method of least squares to a straight line
Model function:
Merit function:
Minimum condition:
( ; , )F x a b ax b= +
2 2
21
1( , ) ( )
n
i i
i i
a b y ax bχσ=
= − −∑
2
21
2
21
12 ( ) 0
12 ( ) 0
n
i i i
i in
i i
i i
y ax b xa
y ax bb
χ
σ
χ
σ
=
=
∂ ≡ − − − = ∂∂ ≡ − − − = ∂
∑
∑
Titus Beu 2011
Defining
Linear system for model parameters a and b:
Optimized model parameters:
2
2 2 21 1 1
21 1
1, ,
,
n n n
i ix xx
i i ii i in n
iy xy i i
i ii
x xs s s
ys s x y
σ σ σ
σ
= = =
= =
= = = = =
∑ ∑ ∑
∑ ∑
xx x xy
x y
s a s b s
s a s b s
+ = + =
2
,
,
xy x y
y xx x xy
xx x
s s s sa
s s s sb s s s
−=
∆−
= ∆ = −∆
12/8/2011
14
Titus Beu 2011
Variance (probable uncertainty) of the model parameters (assuming independent observations):
Derivatives of model parameters:
In particular:
Probable uncertainties (variances) of model parameters:
2 2
2 2 2 2
1 1
,
n n
a i b ii ii i
a b
y yσ σ σ σ
= =
∂ ∂ = = ∂ ∂ ∑ ∑
2 2,
i x xx x i
i ii i
s x s s s xa b
y yσ σ
− −∂ ∂= =
∂ ∂∆ ∆
2 2 2 2 2 2 2
2
2 2 2 21
2 2 ( )ni i x x xx x x xx x
a
i i
s x s x s s s s s s s s s s s s sσ
σ=
− + − + −= = = =
∆∆ ∆ ∆∑
, xxa b
ssσ σ= =
∆ ∆
Titus Beu 2011
Example of linear fitting for exponential dependence:
By taking the logarithm:
a and b′ are adjusted relative to the data set (xi,ln yi)
Merit functional S:a = 0.9100, b = 0.0900
Merit functional χ2 :a = 0.9983, b = −0.1681
fitted line goes through all error bars!
axy be=
ln
ln
y ax b
y y
b b
′ ′= +
′ =
′ =
12/8/2011
15
Titus Beu 2011
//===========================================================================
void LinFit(float x[], float y[], float sigmy[], int ndat, int iopt,float &a, float &b, float &sigma, float &sigmb, float &chi2)
//---------------------------------------------------------------------------// Determines by Chi-square regression the model coefficients a and b and the
// associated probable uncertainties, sigma and sigmb, for a linear model,// F(x;a,b) = a x + b, relative to a set of observed data
// x[] - abscissas of observed data// y[] - ordinates of observed data
// sigmy[] - standard deviations associated with y[]// ndat - no. of observations
// iopt - iopt == 0 Sum of squares criterion is used// iopt /= 0 Chi-square criterion is used
// a, b - model parameters (output)// sigma - probable uncertainties of model parameters (output)
// sigmb// chi2 - Chi-square sum for optimized model parameters (output)
//---------------------------------------------------------------------------{
float f, s, sx, sy, sxx, sxy;int i;
if (iopt == 0) for (i=1; i<=ndat; i++) sigmy[i] = 1.0;
s = sx = sy = sxx = sxy = 0.0;
for (i=1; i<=ndat; i++) {f = 1.0/(sigmy[i]*sigmy[i]);
s += f;sx += x[i]*f; sxx += x[i]*x[i]*f;
sy += y[i]*f; sxy += x[i]*y[i]*f; }
f = 1.0/(s*sxx - sx*sx);
a = (s *sxy - sx*sy )*f; sigma = sqrt(s*f);b = (sy*sxx - sx*sxy)*f; sigmb = sqrt(sxx*f);
chi2 = 0.0;
for (i=1; i<=ndat; i++)chi2 += pow((y[i] - a*x[i] - b)/sigmy[i],2);
}
//===========================================================================
void LinFit(float x[], float y[], float sigmy[], int ndat, int iopt,float &a, float &b, float &sigma, float &sigmb, float &chi2)
//---------------------------------------------------------------------------// Determines by Chi-square regression the model coefficients a and b and the
// associated probable uncertainties, sigma and sigmb, for a linear model,// F(x;a,b) = a x + b, relative to a set of observed data
// x[] - abscissas of observed data// y[] - ordinates of observed data
// sigmy[] - standard deviations associated with y[]// ndat - no. of observations
// iopt - iopt == 0 Sum of squares criterion is used// iopt /= 0 Chi-square criterion is used
// a, b - model parameters (output)// sigma - probable uncertainties of model parameters (output)
// sigmb// chi2 - Chi-square sum for optimized model parameters (output)
//---------------------------------------------------------------------------{
float f, s, sx, sy, sxx, sxy;int i;
if (iopt == 0) for (i=1; i<=ndat; i++) sigmy[i] = 1.0;
s = sx = sy = sxx = sxy = 0.0;
for (i=1; i<=ndat; i++) {f = 1.0/(sigmy[i]*sigmy[i]);
s += f;sx += x[i]*f; sxx += x[i]*x[i]*f;
sy += y[i]*f; sxy += x[i]*y[i]*f; }
f = 1.0/(s*sxx - sx*sx);
a = (s *sxy - sx*sy )*f; sigma = sqrt(s*f);b = (sy*sxx - sx*sxy)*f; sigmb = sqrt(sxx*f);
chi2 = 0.0;
for (i=1; i<=ndat; i++)chi2 += pow((y[i] - a*x[i] - b)/sigmy[i],2);
}