Numerical(Lecture Notes)

Numerical Methods

and Analysis

Part I – Roots of Equations

The root of an equation is the value of x that makes f(x)=0. Roots are sometimes

called the zeros of the equation.

Chapter 1 – Bracketing Methods

These methods exploit the fact that a function typically changes sign in the vicinity of

a root. They are called bracketing methods because two initial guesses for the root are

required.

Bisection Method

Also known as Binary Chopping, this incremental search method locates the interval

where the function changes sign by dividing the interval into two sub-intervals. Each of these

sub-intervals is searched to locate the sign change. The method uses an initial bracket [a b]

that contains the root x. By continuously bisecting the bracket, the midpoint m of the

bisection approximates the root x of the function.

Shown below is the graph of the function f(x) = x2 – 3x + 1. From

inspection, we find that the roots of the function (i.e., the values of x for which f(x) is 0) lie

somewhere around x=0.5 and x=2.5, respectively.

Using Bisection Method to estimate the first root (around x=0.5), the following are

the steps:

1. Make an initial guess of an interval [a b] that necessarily contains the first root,

say, [0 1] (we cannot use [0 3] since it also covers the second root).

2. Bisect the interval and locate the midpoint, m.

3. Solve for f(a), f(b) and f(m):

a m b f(a) f(m) f(b)

0 0.5000 1.0000 1.0000 -0.2500 -1.0000

4. We must choose the interval for the next iteration from between the two sub-

intervals [a m] and [m b]. Notice that the sub-interval [a m] does contain the zero-

crossing since the function changes sign from f(a) to f(m), i.e., from positive to

negative, passing through zero (if neither or both sub-intervals contain a zero-

crossing, then the initial interval does not contain any root or does contain more

than one root and we must look for another initial interval). Hence, the root is

found in the sub-interval [a m] rather than in [m b] which does not contain the

zero-crossing (the function does not change sign from f(m) to f(b)). Therefore our

interval for the next iteration is [a m], i.e., the old [a m] becomes the new [a b]:


0 0.5000 1.0000 1.0000 -0.2500 -1.0000

0 0.5000

5. Solve for the new midpoint and complete the table. After 5 iterations we will get:


0 0.5000 1.0000 1.0000 -0.2500 -1.0000

0 0.2500 0.5000 1.0000 0.3125 -0.2500

0.2500 0.3750 0.5000 0.3125 0.0156 -0.2500

0.3750 0.4375 0.5000 0.0156 -0.1211 -0.2500

0.3750 0.4063 0.4375 0.0156 -0.0537 -0.1211

6. Our answer so far, therefore, is x=0.4063.

7. If we continue with more iterations, we will reach m=0.3820, which is the true

value of the first root. Therefore, our first root is x=0.3820.

To get the second root (around x=2.5), we can use the initial interval [1 3] (we

cannot use [0 3] since it also covers the first root). Here are the results:


1.0000 2.0000 3.0000 -1.0000 -1.0000 1.0000

2.0000 2.5000 3.0000 -1.0000 -0.2500 1.0000

2.5000 2.7500 3.0000 -0.2500 0.3125 1.0000

2.5000 2.6250 2.7500 -0.2500 0.0156 0.3125

2.5000 2.5625 2.6250 -0.2500 -0.1211 0.0156

Our answer so far is x=2.5625. If we continue, we will reach m=2.6180, which

is the true value of the second root. Hence, our second root is x=2.6180.

Using 4 decimal places, the true value of the first root appeared after 13 iterations,

while the second appeared after 14 iterations. For other functions the true values of

the roots may appear after 100 iterations or more. There are various criteria to stop

the computation after coming up with a fairly acceptable estimate of the root. The

criterion will be specifically provided for in the problem. For example, if the problem

requires you to stop the computation when: |f(x)|<0.1, then, since the root is

estimated by the midpoint, you stop when |f(m)|<0.1. Therefore, our answers are

x=0.3750 (3rd

iteration) and x=2.6250 (4th

iteration), respectively.

Drill: Solve for the roots of f(x) = 2x2 - 2x - 3. Assume that the roots are

bounded by ± [0 2]. Stopping criterion: |f(x)|<0.1. Answer: x=1.8125 (5th

iteration)

and x=-0.8125 (5th

iteration)

False Position (Regula Falsi) Method

Also known as Linear Interpolation, this method proceeds the same way as the

Bisection Method, except that the interval is not bisected and, therefore, there is no midpoint

that approximates the root. The graph given below is the same as above except that only the

first root is shown. Our initial interval [a b] is still [0 1]. The point (c,0) lies along the line

connecting the points (a,f(a)) and (b,f(b)), intersecting the x-axis. This intersection

represents an improved estimate of the root. The name originates from the fact that the

replacement of the curve by a straight line gives a “false position” of the root.

We choose the interval for the next iteration from between the two sub-intervals [a c]

and [c b]. As the interval gets smaller, c approximates the root of the function. Using similar

triangles, we can solve for c.

For the first root, the results for 5 iterations are given below:

a c b f(a) f(c) f(b)

0 0.5000 1.0000 1.0000 -0.2500 -1.0000

0 0.4000 0.5000 1.0000 -0.0400 -0.2500

0 0.3846 0.4000 1.0000 -0.0059 -0.0400

0 0.3824 0.3846 1.0000 -0.0009 -0.0059

0 0.3820 0.3824 1.0000 -0.0001 -0.0009

For the second root, the results for 6 iterations are given below:

a c b f(a) f(c) f(b)

1.0000 2.0000 3.0000 -1.0000 -1.0000 1.0000

2.0000 2.5000 3.0000 -1.0000 -0.2500 1.0000

2.5000 2.6000 3.0000 -0.2500 -0.0400 1.0000

2.6000 2.6154 3.0000 -0.0400 -0.0059 1.0000

2.6154 2.6176 3.0000 -0.0059 -0.0009 1.0000

2.6176 2.6180 3.0000 -0.0009 -0.0001 1.0000

Using the same stopping criterion given earlier, our answers are x=0.4000 (2nd

iteration) and x=2.6000 (3rd

iteration), respectively. If we compare this result with the

number of iterations required in the Bisection Method to satisfy the stopping criterion (3 and

4 iterations, respectively), we observe that the False Position Method gives a faster

convergence in the computation for the roots. Observe that the False Position Method

requires 5 and 6 iterations, respectively, to get to the true values of the roots. The Bisection

Method requires 13 and 14 iterations, respectively. The difference can be explained by the

fact that the Bisection Method approximates the root roughly as the midpoint of the bisection

of the bracket while the False Position Method uses a more sophisticated approximation.

Drill: Solve for the roots of f(x) = 2x2 - 2x - 3. Assume that the roots are

bounded by ± [0 2]. Stopping criterion: |f(x)|<0.1. Answer: x=1.8214 (3rd

iteration) and x=-0.8122 (4th

iteration)

Chapter 2 – Open Methods

In bracketing methods, the root is located within an interval. Repeated application of

these methods always results in closer estimates of the true value of the root. Such methods

are said to converge because they move closer to the truth as the computation progresses. In

contrast, open methods are based on formulas that require a single starting estimate of the

root or two starting estimates that do not necessarily bracket the root. As such, they

sometimes diverge or move away from the true roots. However, when open methods

converge, they usually do so much more quickly than the bracketing methods.

Simple One-Point Iteration Method

This method employs a formula by rearranging the function f(x)=0 so that x is on

the left-hand side of the equation. This transformation can be accomplished either by

algebraic manipulation or by simply adding x to both sides of the original equation. For

example, f(x) = x2 – 3x + 1 can be transformed by transposing -3x to the left and

dividing the equation by 3. With an initial estimate of x=0, we can replace this value to the

transformed equation to obtain a better estimate of the root. The first 5 results are:

x f(x)

0 1.0000

0.3333 0.1111

0.3704 0.0261

0.3791 0.0065

0.3812 0.0017

Using the stopping criterion given earlier, the root is x=0.3704.

Observe how fast the function approaches zero. Using the stopping criterion given

earlier, we stop computing at the 3rd

iteration. We can also transform the original equation by

transposing x2 to the left, multiplying the equation by -1 and getting the square root of both

sides. With an initial estimate of x=5, the first 5 results give us an estimate of the second

root.

x f(x)

5.0000 11.0000

3.7417 03.7750

3.1977 01.6320

2.9314 00.7988

2.7918 00.4187

Using the same stopping criterion, we do not have an answer given only 5 iterations.

Drill: Solve for the roots of f(x) = x2 – 3x + 1 using one-point iteration

method by adding x to both sides of the equation. Use an initial estimate of x=0.

Answer: Divergent! x f(x)

0 +1

1 -1

0 +1

1 -1

0 +1

Newton-Raphson Method

Shown below is the graph of the function f(x) = x2 – 3x + 1. If the initial

estimate of the root is x=a, we define a point along the curve (a,f(a)) as a point of

tangency and draw a line tangent to the curve at the said point, intersecting the x-axis at x=b.

Using the relationship between the curve and the slope of the line tangent to it, we can solve

for b which gives a better estimate of the root. Repeating the computation will move b closer

each time to the true value of the root.

Using an initial estimate of x=5, the first 5 results are:

x f(x)

5.0000 11.0000

3.4286 02.4694

2.7884 00.4099

2.6293 00.0253

2.6181 00.0001

Using the stopping criterion given earlier, the root is x=2.6293. For the second

root, with initial estimate x=0, the first 5 estimates are:

x f(x)

0 1.0000

0.3333 0.1111

0.3810 0.0023

0.3820 0.0000

0.3820 0.0000

Using the same stopping criterion, the root is x=0.3810.

Drill: Solve for a root of f(x) = 2x2 - 2x - 3 using an initial estimate of

x=0 and the same stopping criterion. Answer: x=-0.8274 (4th

iteration)

Secant Method

This method does away with derivatives but requires two initial estimates of the root.

However, these estimates do not have to form a bracket that contains a root. Shown below is

the graph of the function f(x) = x2 – 3x + 1. If the initial estimates of the root are

x=a and x=b, respectively, we define two points along the curve, (a,f(a))

and(b,f(b)), and draw a line touching the curve at these points, intersecting the x-axis at

x=c. Using similar triangles we can solve for c which gives a better estimate of the root.

For the next iteration do the following substitutions: a=b and b=c. Repeating the

computation will move c closer each time to the true value of the root.

Using initial estimates a=0 and b=1, the first 5 results are:

x f(x)

0.5000 -0.2500

0.3333 +0.1111

0.3846 -0.0059

0.3820 -0.0001

0.3820 +0.0000

Using the stopping criterion given earlier, the root is x=0.3846. For the second

root, with initial estimates a=4 and b=5, the first 5 estimates are:

x f(x)

3.1667 1.5278

2.8710 0.6296

2.6637 0.1042

2.6226 0.0102

2.6181 0.0002 Using the same stopping criterion,

the root is x=2.6226.

Drill: Solve for a root of f(x) = 2x2 - 2x - 3 using initial estimates of x=1

and x=2 and the same stopping criterion. Answer: x=1.8182 (2nd

iteration)

Newton Method

This method is a modification of the Newton-Raphson Method and is applicable only

to solving the square root of a non-negative real number. The formula for the estimates can

be derived from the Newton-Raphson formula. If a is a real number, then we can estimate its

square root as xo. If we divide a by such estimated square root, the quotient, i.e., a/xo, is

another estimated square root of the number. Getting the average of the two estimates gives

us an improved estimate of the square root. Solving for the square root of 3 with an initial

estimate of x=1, the first 5 results are as follows:

xo f(xo)

1.0000 -2.0000

2.0000 +1.0000

1.7500 +0.0625

1.7321 +0.0003

1.7321 +0.0000

Using the stopping criterion given earlier, the answer is x=1.7500 which is one

iteration away from the true value x=1.7321.

Drill: Solve for a square root of π using initial estimate x=1 and the same stopping

criterion. Answer: x=1.7939 (3rd

iteration)

Drill: Derive the Newton Formula.

Part II – Systems of Linear

Algebraic Equations

A system of linear algebraic equations can be expressed as a single equation in matrix

form:

Ax = b

where A is the coefficient matrix, x the unknown matrix and b the right-hand matrix. Thus,

the system:

-12a + b - 8c = -80

a - 6b + 4c = 13

-2a - b + 10c = 90

can be expressed as:

where A is x is and b is

Chapter 1 – Determinants

Cramer’s Rule

This rule states that the its ith

unknown in a system of linear algebraic equations may

be expressed as a fraction of two determinants with denominator D and numerator Di, where

D is the determinant of the coefficient matrix and Di the determinant of the coefficient matrix

with its ith

column replaced with the right-hand matrix. Hence, given the system described

above:

D = 758

D1 = 494

D2 = 3,272

D3 = 7,248

Therefore: a = 0.6517 b = 4.3166 c = 9.5620

Drill: Solve the following system of linear algebraic equations:

2a - 5b + c = 12

-a + 3b - c = -8

3a - 4b + 2c = 16

Answer:

D = 4

D1 = 8

D2 = -4

D3 = 12

a = 2

b = -1

c = 3

Chapter 2 – Gauss Elimination Method

Naive Gauss Elimination Method

This method starts with appending the right-hand matrix to the coefficient matrix to

form an n by n+1 matrix, where n is the number of unknowns. Hence, given the same

system, we have the matrix:

The method proceeds with forward elimination by eliminating the elements below the

diagonal using the elementary row operations. The following are the elementary row

operations:

1. multiplying with or dividing an equation by any constant;

2. adding to or subtracting one equation from another; and

3. interchanging any two equations.

These operations preserve the system. The elements along the diagonal are indicated in red

color as shown below. Hence, the element to be eliminated in the second row is 1, while the

elements in the third row are -2 and -1.

Using the elementary row operations between row1 and row2 and applying the result to

row2, we have:


row3, we have:


row3, we have:

The final step is backward substitution, starting with the last row to solve for the last

unknown:

Therefore: c = 9.5620 b = 4.3166 a = 0.6517

Drill: Solve the same system of linear algebraic equations.

Answer:

2.0000 -5.0000 1.0000 12.0000

0 1.0000 -1.0000 -4.0000

0 0 -1.1429 -3.4286

a = 2 b = -1 c = 3

Chapter 3 – Gauss-Jordan and the

Matrix Inverse Methods

Gauss-Jordan Method

This method starts with appending the right-hand matrix to the coefficient matrix to

form an n by n+1 matrix, where n is the number of unknowns. Hence, given the same

system, we have the matrix:

It proceeds by converting the original coefficient matrix into an identity matrix using the

elementary row operations. We can take the course via Gauss Elimination and using the

result above we have:

Then use the elementary row operations to convert it to:

And eventually to:

The last column of the resulting matrix gives us the solution. Therefore,

a = 0.6517

b = 4.3166

c = 9.5620


Answer:

1.0000 -2.5000 0.5000 6.0000

0 1.0000 -1.0000 -4.0000

0 0 1.0000 3.0000

a = 2 b = -1 c = 3

Matrix Inverse Method

As discussed above, a system of linear algebraic equations can be expressed as a

single equation in matrix form:

Ax = b

To solve for the solution matrix x, we pre-multiply the equation by the inverse of A:

A-1 (Ax) = A-1 (b)

(A-1A) x = A

-1 (b)

(I) x = A-1 (b)

x = A-1 (b)

Hence, the solution matrix is the product of the inverse coefficient matrix and the right-hand

matrix. We can use the Gauss-Jordan Method to solve for the inverse coefficient matrix by

appending an identity matrix to the coefficient matrix to form an n by 2n matrix, where n is

the number of unknowns. Hence, given the same system, we have the matrix:

We then use the elementary row operations to convert the coefficient matrix into an identity

matrix. Hence, we have:

The right half of the resulting matrix gives us the inverse coefficient matrix which we

multiply with the right-hand matrix. Thus:

We multiply the first row of the first matrix by the first column of the second matrix:

(-0.0739)(-80)+(-0.0026)(13)+(-0.0580)(90) = 0.6517

and so on and so forth. Therefore,

a = 0.6517

b = 4.3166

c = 9.5620


Answer:

A-1 =

0.5000 1.5000 0.5000

-0.2500 0.2500 0.2500

-1.2500 -1.7500 0.2500

a = 2

b = -1

c = 3

Chapter 4 – Gauss-Seidel and Jacobi Methods

Gauss-Seidel Method

This method is an extension of the One-Point Iteration Method discussed earlier.

Hence, using the system described above, we have the following equations:

a = (80 + b – 8c)/12

b = (-13 + a + 4c)/6

c = (90 + 2a + b)/10

We may use initial estimates of 0, 0 and 0, respectively. We compute for the next estimate

of a using the latest values of b and c, which are 0 and 0, respectively. Hence, we have:

a b c

0 0 0

6.6667

We compute for the next estimate of b using the latest values of a and c, which are 6.6667

and 0, respectively. Hence, we have:

a b c

0 0 0

6.6667 -1.0556

We compute for the next estimate of c using the latest values of a and b, which are 6.6667

and -1.0556, respectively.

Hence, we have:

a b c

0 0 0

6.6667 -1.0556 10.2278

We continue to the next row by using the latest values of each of the variables. The

following are the first 10 results:

a b c

0 0 0

6.6667 -1.0556 10.2278

-0.2398 4.6119 9.4132

0.7755 4.2381 9.5789

0.6339 4.3249 9.5593

0.6542 4.3152 9.5624

0.6514 4.3168 9.5620

0.6518 4.3166 9.5620

0.6517 4.3166 9.5620

0.6517 4.3166 9.5620

We observe that all the estimates no longer change in the 10th

iteration. Therefore,

a = 0.6517

b = 4.3166

c = 9.5620


Answer: a b c

0 0 0

6.0000 -0.6667 -2.3333

5.5000 -1.6111 -3.4722

3.7083 -2.5880 -2.7384

0.8993 -3.2797 0.0916

-2.2451 -3.3845 4.5987

-4.7605 -2.7206 9.6995

-5.6513 -1.3173 13.8425

-4.2144 0.5427 15.4070

-0.3468 2.3534 13.2270

Observe that the solution does not seem to converge after 10 iterations. In

fact, it does not converge after 100 iterations. Even if we interchange any two

equations, the solution is still divergent.

Jacobi Method

This method is a slight modification of the Gauss-Seidel Method. The only difference

lies in the values used in each computation. While the Gauss-Seidel Method uses the latest

values of each of the variables, the Jacobi method uses the values in the immediately

preceding row, regardless of whether or not they are the latest values. Hence, we compute

for the next estimate of a using the values of b and c found in the first row which are 0 and

0, respectively. We have:

a b c

0 0 0

6.6667

We compute for the next estimate of b using the values of a and c found in the first row

which are 0 and 0, respectively. Hence, we have:

a b c

0 0 0

6.6667 -2.1667

We compute for the next estimate of c using the values of a and b found in the first row

which are 0 and 0, respectively. Hence, we have:

a b c

0 0 0

6.6667 -2.1667 9.0000

We continue to the third row by using the values of each of the variables in the second row.

The following are the first 15 results:

a b c

0 0 0

6.6667 -2.1667 9.0000

0.4861 4.9444 10.1167

0.3343 4.6588 9.5917

0.6605 4.2835 9.5327

0.6685 4.2986 9.5604

0.6513 4.3184 9.5636

0.6508 4.3176 9.5621

0.6517 4.3165 9.5619

0.6518 4.3166 9.5620

0.6517 4.3166 9.5620

0.6517 4.3166 9.5620

0.6517 4.3166 9.5620

0.6517 4.3166 9.5620

0.6517 4.3166 9.5620

We observe that all the estimates no longer change in the 12th

iteration. Therefore,

a = 0.6517 b = 4.3166 c = 9.5620


Answer:

a b c

0 0 0

6.0000 -2.6667 8.0000

-4.6667 2.0000 -6.3333

14.1667 -6.3333 19.0000

-19.3333 8.3889 -25.9167

Observe that the solution does not seem to converge after 5 iterations. The

solution is divergent.

Chapter 5 – LU Decomposition

Crout LU Decomposition Method

This method decomposes the coefficient matrix A into the product of a lower

triangular matrix L and an upper triangular matrix U. Hence:

A = LU

where A, L and U are expressed as:

A = A11 A12 A13 L = L11 0 0 U = 1 U12 U13

A21 A22 A23 L21 L22 0 0 1 U23

A31 A32 A33 L31 L32 L33 0 0 1

Using our original equation:

Ax = b

We substitute: (LU) x = b

L-1 (LU)x = L

-1 b

(L-1 L) Ux = L-1 b

(I) Ux = L-1 b

Ux = L-1

b

U-1 (Ux) = U-1 (L-1 b)

(U-1

U)x = U-1

(L-1

b)

(I)x = U-1

(L-1

b)

x = U-1

(L-1

b) eqn. 1

We let:

d = L-1 b

L(d) = L(L-1 b)

Ld = (LL-1)b

Ld = (I) b

Ld = b eqn. 2

Then from eqn. 1 :

x = U-1 d

(U)x = (U) U-1 d

(U)x = (UU-1)d

(U)x = (I) d

Ux = d eqn. 3

Now, using the same system of linear algebraic equations given earlier, we have:

A =

b =

Generally, if we multiply L by U we will get A. If we multiply the first row (r=1) of L by the

first column (c=1) of U, we get A11 ( i.e., Arc ) :

A11 = L11(1) + 0(0) + 0(0)

Hence, L11 = -12

If we continue, we will get the following results:

L =

-12.0000 0 0

1.0000 -5.9167 0

-2.0000 -1.1667 10.6761

U =

1 -0.0833 0.6667

0 1 -0.5634

0 0 1

Using eqn. 2, Ld = b, we have:

L d = b

-12.0000 0 0 d1 -80

+01.0000 -5.9167 0 d2 +13

-02.0000 -1.1667 10.6761 d3 +90

If we do the multiplication, we will get the following results:

d = +6.6667

-1.0704

+9.5620

Using eqn. 3, Ux = d, we have:

U x = d

1 -0.0833 0.6667 x1 +6.6667

0 1 -0.5634 x2 -1.0704

0 0 1 x3 +9.5620

If we do the multiplication, we will get the following results:

x = 0.6517

4.3166

9.5620

Therefore, a = 0.6517

b = 4.3166

c = 9.5620


Answer:

L =

2.0000 0 0

-1.0000 0.5000 0

3.0000 3.5000 4.0000

U =

1.0000 -2.5000 0.5000

0 1.0000 -1.0000

0 0 1.0000

d = 6 a = 2

-4 b = -1

3 c = 3

Matrix Inverse by LU Decomposition Method

The LU Decomposition can also be used to compute the inverse of the coefficient

matrix. Using the same system given earlier we have:

L =

-12.0000 0 0

1.0000 -5.9167 0

-2.0000 -1.1667 10.6761

U =

1.0000 -0.0833 0.6667

0 1.0000 -0.5634

0 0 1.0000

We now solve for the first column of the inverse coefficient matrix:

x1

x2

x3

The formula is:

xn = dn eqn. 1

xi = di - ∑( Uij xj ) for i = n-1, n-2,. . ., 1 eqn. 2

where n is the number of unknowns in the system and the summation index j takes

values from i+1 to n.

The matrix d is solved from:

L (d ) = 1

0

0

L d1 = 1

d2 0

d3 0

If we do the multiplication, we will get the results:

d1 = -0.0833

d2 = -0.0141

d3 = -0.0172

Applying eqn. 1, we have:

x3 = -0.0172

Applying eqn. 2, we have:

x2 = -0.0237 x1 = -0.0739

Therefore, the first column of the inverse coefficient matrix is:

-0.0739

-0.0237

-0.0172

We can solve for the second column of the inverse coefficient matrix using:

L (d ) = 0

1

0

If we do the multiplication, we will get the results:

d1 = 0

d2 = -0.1690

d3 = -0.0185

Further, we have the results:

x3 = -0.0185

x2 = -0.1794

x1 = -0.0026

Therefore, the second column of the inverse coefficient matrix is:

-0.0026 -0.1794

-0.0185

We can solve for the third column of the inverse coefficient matrix using:

0

0

1

Then we will get the results:

d1 = 0

d2 = 0

d3 = 0.0937

Further, we have the results:

x3 = 0.0937

x2 = 0.0528

x1 = -0.0580

Therefore, the third column of the inverse coefficient matrix is:

-0.0580

0.0528

0.0937

Therefore, the inverse coefficient matrix is:

A-1 =

-0.0739 -0.0026 -0.0580

-0.0237 -0.1794 0.0528

-0.0172 -0.0185 0.0937

Since:

x = A-1 (b)

x =

0.6517

4.3166

9.5620

Therefore:

a = 0.6517

b = 4.3166

c = 9.5620


Answer:

d = 0.5000

1.0000

-1.2500

d = 0

2.0000

-1.7500

d = 0

0

0.2500

A-1 =

0.5000 1.5000 0.5000

-0.2500 0.2500 0.2500

-1.2500 -1.7500 0.2500

a = 2 b = -1 c = 3

Part III – Curve Fitting

Data is often given for discrete values along a continuum. However, you may require

estimates at points between the discrete values. This chapter describes techniques to fit

curves to such data in order to obtain intermediate estimates.

Chapter 1 – Least-Squares Regression

Where the data exhibits a significant degree of error or “noise,” the strategy is to

derive a single curve that represents the general trend of the data. Because any individual

data point may be incorrect, we make no effort to intersect every point. Rather, the curve is

designed to follow the pattern of the points taken as a group.

Linear Regression

The simplest example of a least-squares approximation is fitting a straight line to a set

of paired observations. Suppose we have m observations (xi,yi), the following are the

governing equations:

a0 ∑(x0) + a1 ∑(x

1) + a2 ∑(x2) + . . + an ∑(x

n+0) = ∑(x0 y)

a0 ∑(x1) + a1 ∑(x

2) + a2 ∑(x3) + . . + an ∑(x

n+1) = ∑(x1 y)

a0 ∑(x2) + a1 ∑(x

3) + a2 ∑(x4) + . . + an ∑(x

n+2) = ∑(x2 y)

. . .

. . .

. . . eqn. 1

a0 ∑(xn) + a1 ∑(x

n+1) + a2 ∑(xn+2) + . . + an ∑(x

2n) = ∑(xn y)

y = a0x0 + a1 x

1 + a2 x2 + . . . + an x

n eqn. 2

where n is the order of equation, i.e., n=1 for a straight line, n=2 for a parabola, etc.

Now, suppose we have the following observations of the water temperature as a

function of time:

1:00am 0.5°C

2:00am 2.5°C

3:00am 2.0°C

4:00am 4.0°C

5:00am 3.5°C

6:00am 6.0°C

7:00am 5.5°C

Shown below is the plot of the observations.

We observe that the data has a general trend with the shape of a straight line.

We now fit a straight line to the data using regression. From eqn. 1 we have:

a0 ∑(x

0) + a1 ∑x = ∑y

a0 ∑x + a1 ∑(x2) = ∑(xy)

Solving the equations, we have:

a0 = 0.0714 a1 = 0.8393

Substituting into eqn. 2, we have the equation of the straight line:

y = 0.0714 + 0.8393x

Shown below is the straight line plotted over the data.

According to the fitted straight line, the estimated temperature at 5:30am is

4.6875°C. The freezing point was reached at around x = -0.0851 or shortly before

midnight at 11:54:54pm.

The standard error of the estimates is the square root of the quotient:

Sr /(m-(n+1))

where Sr is the sum of the squares of the difference between the estimates and the

observations:

Observations Estimates

0.5°C 0.9107°C

2.5°C 1.7500°C

2.0°C 2.5893°C

4.0°C 3.4286°C

3.5°C 4.2679°C

6.0°C 5.1071°C

5.5°C 5.9464°C

The standard deviation of the observations about the mean can be computed as the

square root of the quotient:

St /(m-1)

where St is the sum of the squares of the difference between the observations and

their mean.

The standard error of the estimates is 0.7734 and the standard deviation of the

observations is 1.9457. Now, because the standard error is less than the standard

deviation, the linear regression model has merit.

The extent of the improvement of our model is quantified by the coefficient of

determination:

(St-Sr)/St

The coefficient of determination is 0.8683. This means that 86.83% of the

original uncertainty has been explained by our linear regression model.

Drill: What if the data does not have a general trend with the shape of a straight line?

What would be the results if, despite this fact, we try to fit a straight line using least-

squares regression?

0000hrs 2.1°C

0001hrs 7.7°C

0002hrs 13.6°C

0003hrs 27.2°C

0004hrs 40.9°C

0005hrs 61.1°C

Answer:

y = -3.7238 + 11.6629x


60.4219°C. The freezing point was reached at around x = 0.3193 or shortly after

midnight at 12:19:09am.

The standard error of the estimates is quite high at 5.7664 and the standard

deviation of the observations is 22.4205. Now, because the standard error is less than the

standard deviation, the linear regression model has merit.


original uncertainty has been explained by our linear regression model.

Polynomial Regression

We can fit a parabola (n=2) into the data using the same equations eqn. 1 and eqn. 2:

a0 ∑(x0) + a1 ∑(x

1) + a2 ∑(x2) = ∑(x0 y)

a0 ∑(x1) + a1 ∑(x

2) + a2 ∑(x3) = ∑(x1 y)

a0 ∑(x2) + a1 ∑(x

3) + a2 ∑(x4) = ∑(x2 y)

y = a0x

0 + a1 x

1 + a2 x2

In solving for the constants a0, a1 and a2 you may use any method learned in Part II

or you may use a programmable calculator, if you have one. Most programmable calculators

can only handle up to three (3) unknowns. Thus, for n=3, where there are four (4) unknowns,

you will have to use the methods learned in Part II.

We use the same data as reproduced below:

1:00am 0.5°C

2:00am 2.5°C

3:00am 2.0°C

4:00am 4.0°C

5:00am 3.5°C

6:00am 6.0°C

7:00am 5.5°C

The parabola is given by:

y = -0.2857 + 1.0774x - 0.0298x

2

Shown below is the graph of the parabola plotted over the data.


4.7396°C. The freezing point was reached at around x = 0.2672 or shortly after

midnight at 12:16:02am.


observations as computed earlier is 1.9457. Now, because the standard error is less than

the standard deviation, the parabolic regression model has merit.


original uncertainty has been explained by our parabolic regression model.

Based on the error of estimates, our linear regression model (0.7734) is the better fit

as compared to our parabolic regression model (0.8539).

Based on the coefficient of determination, our parabolic regression model (87.16%)

is still the better fit as compared to our linear regression model (86.83%).

Shown below are the graphs of both the linear (red) and parabolic (blue) fits plotted

over the data.

Through visual inspection we find that the two models are very much alike within the

range of the observations. We also find that they have two common estimates and these are

computed as: 1.7500°C at 2:00am and 5.1071°C at 6:00am.

Drill: Using the data from the previous drill, fit a parabola using least-squares

regression. Compare the results with that of the linear model of the previous drill.

Answer:

y = 2.4786 + 2.3593x + 1.8607x2

According to the fitted parabola, the estimated temperature at 5:30am is

71.7413°C. The parabola does not touch the x-axis.


observations as computed earlier is 22.4205. Now, because the standard error is less than

the standard deviation, the parabolic regression model has merit. The coefficient of

determination is 0.9985.

Based on the error of estimates, our parabolic regression model (1.1175) is the

better fit as compared to our linear regression model (5.7664). Based on the coefficient of

determination, our parabolic regression model (99.85%) is still the better fit as compared to

our linear regression model (94.71%). Below are graphs of the linear (red) and parabolic

(blue) fits plotted over the data.

The parabola is a better fit than the linear model. They have two common estimates

and these are computed as: 45.3515°C at 4:12:28am and 5.5152°C at 12:47:32am.

Chapter 2 – Nonlinear Models

Linear regression provides a powerful technique for fitting a “best” line to a data.

However, it is predicated on the fact that the relationship between the dependent and

independent variables is linear. This is not always the case and so transformations can be

used to express the data in a form that is compatible with linear regression.

Exponential Model

This model is used in many fields of engineering to characterize quantities that

increase (b>0) or decrease (b<0) at a rate that is directly proportional to their own

magnitude. This model is expressed by the equation:

y = a ebx eqn. 1

where a and b are constants.

We use the data in the previous drill as reproduced below:

0000hrs 2.1°C

0001hrs 7.7°C

0002hrs 13.6°C

0003hrs 27.2°C

0004hrs 40.9°C

0005hrs 61.1°C

The transformation of eqn. 1 is done by getting its natural logarithm. Then we change

variables by letting:

y1 = ln(y) x1 = x

Then we have:

y1 = a0 + a1x1

If we do linear regression on data (x1,y1 ) we have:

a0 = 1.1422

a1 = 0.6444

Reverting to our original variables, we have:

a = 3.1337

b = 0.6444

Hence, our exponential model is:

y = 3.1337 e0.6444x

Shown below is the graph of the exponential model plotted against the data.

Shown below is the same graph (red) but this time the parabolic fit (blue), as

computed earlier in the previous drill, is superimposed. How do the exponential and

parabolic fits compare?

According to the exponential model, the estimated temperature at 5:30am is

108.4943°C. The boiling point is reached at around x = 5.3735 or at 05:22:25am.

The first point (0,2.1) is located below both curves since:

fexp(0) = 3.1337

fpar(0) = 2.4786

The sum of the squares of the discrepancies can be computed as:

Sr = ∑(yi - ye)2

where yi is any of the observations and ye is the corresponding estimate. The error

Sr of the exponential fit is 346.3562 while that of the parabolic fit is only 3.7466.

Clearly, the parabolic fit is better.

Drill: Using the data given below fit an exponential curve and a parabola. Compare.

1 0.5

2 1.7

3 3.4

4 5.7

5 8.4

Answer:

y = 0.3431 e0.6853x y = -0.2000 + 0.4371x + 0.2571x2

The table below shows that only the first 3 points are located between the curves.

x y y_exp y_par

1 0.5000 0.6808 0.4943

2 1.7000 1.3508 1.7029

3 3.4000 2.6805 3.4257

4 5.7000 5.3188 5.6629

5 8.4000 10.5541 8.4143

The error Sr of the exponential fit is 5.4576 while that of the parabolic fit is only

0.0023. Clearly, the parabolic fit is better.

Power Model

This model has wide applicability in all fields of engineering and is expressed by the

equation:

y = a xb eqn. 2


We use the same data in the previous drill as shown below:

1 0.5

2 1.7

3 3.4

4 5.7

5 8.4

The transformation of eqn. 2 is done by getting its base-10 logarithm. Then we

change variables by letting:

y1 = log(y) x1 = log(x)

Then we have:

y1 = a0 + a1x1


a0 = -0.3002

a1 = 1.7517


a = 0.5009

b = 1.7517

Hence, our power model is:

y = 0.5009 x1.7517

Shown below is the graph of the power model plotted against the data.

We observe from the graph that our power model fits our data as accurately as the

parabolic model that was computed in the previous drill. However, the error Sr of the power

fit is 0.0016 while that of the parabolic fit is, as previously computed, 0.0023. Clearly,

the power fit is better.

Drill: Compute the power model of the first 9 terms of the Fibonacci sequence.

Answer: y = 0.4768 x1.6624

Saturation-Growth-Rate Model

This model is particularly well-suited for characterizing population growth rate under

limiting conditions. It represents a non-linear relationship between y and x that levels off or

saturates as x increases and is expressed by the equation:

y = ax / (x+b) eqn. 3


We use the data shown below.

7 0.29

9 0.37

15 0.48

25 0.65

40 0.80

75 0.97

100 0.99

150 1.07

The linearization of eqn. 3 is done by inverting it. We then change variables by

letting:

y1 = 1/y

x1 = 1/x

Then we have:

y1 = a0 + a1x1


a0 = 0.8127

a1 = 18.0365


a = 1.2304

b = 22.1927

Hence, our saturation-growth-rate model is:

y = 1.2304x / (x+22.1927)

Shown below is the graph of the model plotted against the data.

We observe that as x increases the graph tends to saturate at the value of 1. In fact,

if we substitute x=∞ to our model we get the exact value of this saturation point:

x = 1.2304

The error Sr of our model is 0.0013.

Drill: Compute the saturation-growth-rate and parabolic models of the data shown

below. Which model is better?

0.7 0.29

0.9 0.37

1.5 0.48

2.5 0.65

4.0 0.75

7.5 0.97

10.0 1.09

15.0 1.20

Answer:

y = 1.2803x / (x+2.3531)

y = 0.2678 + 0.1337x - 0.0048x2

The error Sr of the saturation model is 0.0154 while that of the parabolic model is

0.0142.

Clearly, the parabolic model is better.

Chapter 3 – Multiple Regression

Multiple Linear Regression

A useful extension of linear regression is the case where y is a linear function of two

or more variables. Such an equation is particularly useful when fitting experimental data

where the variable being studied is often a function of two other variables as in:

y = a0 + a1 x1 + a2 x2 + . . . + an xn eqn. 1

The coefficients can be solved using the system: eqn. 2

ma0 + a1 ∑x1 + a2 ∑x2 +…+ an ∑xn = ∑y

a0 ∑x1 + a1 ∑(x1x1) + a2 ∑(x2x1) +…+ an ∑(xnx1) = ∑(x1y)

a0 ∑x2 + a1 ∑(x1x2) + a2 ∑(x2x2) +…+ an ∑(xnx2) = ∑(x2y)

. . . . .

. . . . .

. . . . .

a0 ∑xn + a1 ∑(x1xn) + a2 ∑(x2xn) +…+ an ∑(xnxn) = ∑(xny)

where m is the number of observations and n is the number of independent variables.

We use the data shown below:

x1

x2 y

0 0 5

2 1 10

2.5 2 9

1 3 1

4 6 3

7 2 27

From eqn. 1 we have:

y = a0 + a1 x1 + a2 x2

From eqn. 2 we have:

6a0 + a1 ∑x1 + a2 ∑x2 = ∑y

a0 ∑x1 + a1 ∑(x1x1) + a2 ∑(x2x1) = ∑(x1y)

a0 ∑x2 + a1 ∑(x1x2) + a2 ∑(x2x2) = ∑(x2y)

Solving the system of equations, we have:

a0 = 5.2252

a1 = 3.9232

a2 = -2.9345

Hence, our multiple linear regression model is:

y = 5.2252 + 3.9232x1 - 2.9345x2


Drill: Compute the saturation-growth-rate model of the data shown below.

x1

x2 x3 y

0 0 0 5

2 1 1 10

2.5 2 4 9

1 3 9 1

4 6 16 3

7 2 25 27

Answer:

y = 5.3697 + 3.7093x1 - 3.0082x2 + 0.0672x3


Chapter 4 – Interpolation

Where the data is known to be very precise, the basic approach is to fit a curve or a

series of curves that pass directly through each of the points. Such data usually originates

from tables. Polynomial interpolation consists of determining the unique nth-order

polynomial that fits n+1 data points. This polynomial then provides a formula to compute

intermediate values.

Newton’s Divided-Difference Interpolating Polynomials

Consider the graph shown below:

If we interpolate at x between two known values f(x0) and f(x1), we have:

f(x) = f(x0) + f(x1)-f(x0) (x-x0) eqn. 1

x1-x0

Let:

b0 = f(x0) eqn. 2

b1 = f(x1)-f(x0) eqn. 3

x1-x0

We then have the 1st -order Newton’s interpolating polynomial:

f(x) = b0 + b1 (x-x0)

Now, let us estimate the natural logarithm of 2 using Newton’s linear interpolation

between the values 1 and 6, decreasing the upper value by 0.5 until it reaches 4. Results:

x0 x1 f(x) error(%)

1.0000 6.0000 0.3584 48.3007

1.0000 5.5000 0.3788 45.3460

1.0000 5.0000 0.4024 41.9518

1.0000 4.5000 0.4297 38.0021

1.0000 4.0000 0.4621 33.3333

Last estimate is 0.4621. As against the actual value, the error is 33.3333%.

Drill: Estimate sin(π/4) using Newton’s linear interpolation between the values

0 and π/2, decreasing the upper value by 0.1 until the error is below 10%. Answer:

0.6437 @ 8.9697%

Drill: Derive equation #1.

Previously we derived the 1st -order Newton’s interpolating polynomial:

f(x) = b0 + b1 (x-x0)

However, if three data points are available, then we can expand this equation to get

the 2nd

-order Newton’s interpolating polynomial:

f(x) = b0 + b1 (x-x0) + b2 (x-x0)(x-x1)

If we let x=x2 and using eqn. 2 and eqn. 3, we will get the equation for b2 :

f(x2) – f(x1) f(x1) – f(x0)

-

x2 - x1 x1 – x0

b2 = eqn. 4

x2 - x0

Let us estimate the natural logarithm of 2 using Newton’s quadratic interpolation

across 1, 4 and 6, decreasing the upper values by 0.2 until error < 10%. The results are:

x0 x1 x2 f(x) error(%)

1.0000 4.0000 6.0000 0.5658 18.3659

1.0000 3.8000 5.8000 0.5763 16.8582

1.0000 3.6000 5.6000 0.5872 15.2866

1.0000 3.4000 5.4000 0.5986 13.6475

1.0000 3.2000 5.2000 0.6104 11.9371

1.0000 3.0000 5.0000 0.6228 10.1519

1.0000 2.8000 4.8000 0.6357 8.2882

Our last estimate is 0.6357 with an error of 8.2882%.

Drill: Estimate sin(π/4) using Newton’s quadratic interpolation across the values

0, π/3 and π/2, decreasing the two upper values by 0.1 until the error is below

1%. Answer: 0.7105 @ 0.4778%

Drill: Derive equation #4.

From our two equations:

f(x) = b0 + b1 (x-x0) linear

f(x) = b0 + b1 (x-x0) + b2 (x-x0)(x-x1) quadratic

we can write the general form of Newton’s interpolating polynomial as:

fn(x) = b0 + b1(x-x0) + . . + bn(x-x0)(x-x1). .(x-xn-1)

where n is the order of the interpolating polynomial.

Hence, for the 3rd

-order Newton’s interpolating polynomial, we have:

If we let x=x3 and using equations 2, 3 and 4, we can derive the equation for b3.

However, this derivation is very long and is prone to errors. Hence, a short-cut is in order.

This short-cut is in the form of a finite divided difference. Our first finite divided

difference is equation #3 which involves x0 and x1, and which we designate as F[0,1]:

b1 = F[1,0] = f(x1)-f(x0) eqn. 5

x1-x0

If we analyze equation #4, it looks like:

b2 = F[2,1,0] = F[2,1] - F[1,0] eqn. 6

x2 - x0

If we follow the trend, we have:

b3 = F[3,2,1,0] = F[3,2,1] - F[2,1,0]

x3 - x0

From equation #6, we have:

F[3,2,1] = F[3,2] - F[2,1]

x3 – x1

f3(x) = b0 + b1(x-x0) + b2 (x-x0)(x-x1) + b3(x-x0)(x-x1)(x-x2)

From equation #5, we have:

F[3,2] = f(x3)-f(x2)

x3-x2

Hence we can now have an equation of b3 using a finite divided difference.

Now, let us estimate the natural logarithm of 2 using 3rd

-order Newton’s interpolation

across the values 1, 4, 5 and 6, decreasing all upper values by 0.2 until the error is below

10%. We have the results:

x0 x1 x2 x3 f(x) error(%)

1.0000 4.0000 5.0000 6.0000 0.6288 9.2879

1.0000 3.8000 4.8000 5.8000 0.6359 8.2635

The answer is 0.6288 with an error of 9.2879%.

Drill: Estimate sin(π/4) using 3rd

-order Newton’s interpolation across the values

0, π/3, π/2 and π, decreasing all upper values by 0.1 until the error is below 1%.

Answer: 0.7093 @ 0.3086%

Lagrange Interpolating Polynomials

The Lagrange interpolating polynomial is simply a reformulation of the Newton

polynomial that avoids the computation of divided differences. The 1st-order Newton

interpolating polynomial, as given by equation #1 above, is reproduced below:

f1(x) = f(x0) + f(x1) - f(x0) (x-x0)

x1-x0

If we try to get rid of the divided difference:

f(x1) - f(x0)

x1-x0

We can transform it, hence:

f1(x) = f(x0) + f(x1)(x-x0) - f(x0)(x-x0)

x1-x0 x1-x0

f1(x) = f(x0) + (x-x0) f(x1) + (x-x0) f(x0)

x1-x0 x0-x1

f1(x) = f(x0)[ 1 + x-x0 ] + (x-x0) f(x1)

x0-x1 x1-x0

f1(x) = (x-x1) f(x0) + (x-x0) f(x1) eqn. 7

(x0-x1) (x1-x0)

The 2nd

-order Newton’s interpolating polynomial, previously derived above, is

reproduced below:

f2(x) = b0 + b1 (x-x0) + b2 (x-x0)(x-x1)

Using equations #2 and #3, we have:

f2(x) = f(x0) + f(x1) - f(x0) (x-x0) + b2 (x-x0)(x-x1)

x1-x0

Using equation #4 and after some manipulations, we will have:

f2(x) = (x-x1)(x-x2) f(x0) + (x-x0)(x-x2) f(x1) +

(x0-x1)(x0-x2) (x1-x0)(x1-x2)

(x-x0)(x-x1) f(x2)

(x2-x0)(x2-x1) eqn. 8

Drill: Give the complete and detailed derivation of equation #8.

We can generalize our results from equations #7 and #8 to derive a concise

representation of the nth-order Lagrange interpolating polynomial:

fn(x) = ∑[Li(x)f(xi)] eqn. 9

where n is the order of the polynomial and the index of the summation (i) takes values from

0 to n and where:

Li(x) = Π (x-xj)/(xi-xj) eqn. 10

where the index of the product (j) takes values from 0 to n and j≠i. Notice the divisor in

equation #10. Always bear in mind that where j is equal to i, you must skip this factor, else

you will have a division by zero.

Drill: Based on equations #9 and #10, write the 3rd

-order Lagrange interpolating

polynomial.

Now, let us do the interpolation exercises again, this time using Lagrange

interpolating polynomials.

Let us estimate the natural logarithm of 2 using Lagrange linear interpolation between

the values 1 and 6, decreasing the upper value by 0.5 until it reaches 4. We have the same

results as obtained above using Newton’s linear interpolating polynomial:

x0 x1 f(x) error(%) L0 L1

1.0 6.0 0.3584 48.3007 0.8000 0.2000

1.0 5.5 0.3788 45.3460 0.7778 0.2222

1.0 5.0 0.4024 41.9518 0.7500 0.2500

1.0 4.5 0.4297 38.0021 0.7143 0.2857

1.0 4.0 0.4621 33.3333 0.6667 0.3333

Our last estimate is 0.4621 with an error of 33.3333% with L0=0.6667

L1=0.3333

Drill: Estimate sin(π/4) using Lagrange linear interpolation between the values 0

and π/2, decreasing the upper value by 0.1 until the error is below 10%. Answer:

0.6437 @ 8.9697% with L0=0.2665 L1=0.7335

Now, let us estimate the natural logarithm of 2 using Lagrange quadratic interpolation

across the values 1, 4 and 6, decreasing the two upper values by 0.2 until the error is below

10%. We have the same results as obtained above using Newton’s quadratic interpolating

polynomial:

x0 x1 x2 f(x) error(%) L0 L1 L2

1.0 4.0 6.0 0.5658 18.3659 0.5333 0.6667 -0.2000

1.0 3.8 5.8 0.5763 16.8582 0.5089 0.6786 -0.1875

1.0 3.6 5.6 0.5872 15.2866 0.4816 0.6923 -0.1739

1.0 3.4 5.4 0.5986 13.6475 0.4508 0.7083 -0.1591

1.0 3.2 5.2 0.6104 11.9371 0.4156 0.7273 -0.1429

1.0 3.0 5.0 0.6228 10.1519 0.3750 0.7500 -0.1250

1.0 2.8 4.8 0.6357 8.2882 0.3275 0.7778 -0.1053

Our last estimate is 0.6357 with an error of 8.2882% L0=0.3275 L1=0.7778

L2=-0.1053

Drill: Estimate sin(π/4) using Lagrange quadratic interpolation across the values

0, π/3 and π/2, decreasing the two upper values by 0.1 until the error is below 1%.

Answer: 0.7105 @ 0.4778% L0=0.0312 L1=1.0365 L2=-0.0676

Now, let us estimate the natural logarithm of 2 using 3rd

-order Lagrange interpolation

across the values 1, 4, 5 and 6, decreasing all upper values by 0.2 until the error is below

10%. We have the results:

x1 x2 x3 f(x) error L0 L1 L2 L3

4.0 5.0 6.0 0.6288 9.2879 0.4000 2.0000 -2.0000 0.6000

3.8 4.8 5.8 0.6359 8.2635 0.3750 1.9000 -1.8000 0.5250

The answer is 0.6288 with an error of 9.2879% with L0=0.4000 L1=2.0000

L2=-2.0000 L3=0.6000

Drill: Estimate sin(π/4) using 3rd

-order Lagrange interpolation across the values

0, π/3, π/2 and π, decreasing all upper values by 0.1 until the error is below 1%.

Answer: 0.7093 @ 0.3086% with L0 = 0.0228 L1 = 1.0671 L2 = -0.0928 L3 = 0.0029

Spline Interpolation

In the previous sections, nth-order polynomials were used to interpolate between n+1

data points. However, there are cases where these functions can lead to erroneous results.

An alternative approach is to apply lower-order polynomials to subsets of data points. Such

connecting polynomials are called spline functions.

The simplest connection between two points is a straight line. The 1st-order splines

for a group of ordered data points can be defined as a set of linear functions, one linear

function per interval:

f1(x) = m1x + b1 x0 ≤ x ≤ x1

f2(x) = m2x + b2 x1 ≤ x ≤ x2 .

.

.

fn(x) = mnx + bn xn-1 ≤ x ≤ xn

where mi is the slope of the straight line connecting the points (xi,yi) and

(xi-1,yi-1) and bi is the corresponding y-intercept.

These equations can be used to evaluate the functions at any point between x0 and xn by first locating the interval within which the point lies. Then the appropriate equation is

used to determine the function value within the interval.

Suppose we fit the data shown below with 1st-order splines.

x f(x)

3.0 2.5

4.5 1.0

7.0 2.5

9.0 0.5

We have the following linear splines:

f1(x) = -x + 5.5 3.0 ≤ x ≤ 4.5

f2(x) = 0.6x – 1.7 4.5 ≤ x ≤ 7.0

f3(x) = -x + 9.5 7.0 ≤ x ≤ 9.0

The graph of the three linear splines is shown below.

To estimate the value at x=5, we use the second linear spline since its interval

contains x=5. Hence f(x=5) = 1.3000.

Drill: Fit the data shown below with 1st-order splines and estimate the value of the

function at x=5.

x f(x)

3.0 1.0

4.5 2.5

7.0 0.5

9.0 2.5

Answer: f1(x) = x - 2.0 3.0 ≤ x ≤ 4.5

f2(x) = -0.8x + 6.1 4.5 ≤ x ≤ 7.0

f3(x) = x - 6.5 7.0 ≤ x ≤ 9.0

f(x=5) = 2.1000

A visual inspection of the two foregoing graphs indicates that the primary

disadvantage of linear or 1st -order splines is that they are not smooth. In essence, at the data

points where two splines meet (called the knot), the slope changes abruptly. In formal terms,

the first derivate of the function is discontinuous at these points. This deficiency is overcome

by using higher-order polynomial splines that ensure smoothness at the knots. This

smoothness is achieved by equating derivatives at the knots.

Let us now learn how to formulate 2nd

-order or quadratic splines. These splines have

continuous first derivates at the knots. The objective is to derive a 2nd

-order polynomial for

each interval between data points:

f1(x) = a1x

2 + b1x + c1 x0 ≤ x ≤ x1

f2(x) = a2x2 + b2x + c2 x1 ≤ x ≤ x2

.

.

.

fn(x) = anx2 + bnx + cn xn-1 ≤ x ≤ xn

For n+1 data points there are n intervals and, consequently, 3n unknown constants to

evaluate. Therefore, 3n equations or conditions are required to evaluate the unknowns.

These are:

1. The function values must be equal at the interior knots.

ai-1 xi-1

2 + bi-1 xi-1 + ci-1 = f(xi-1)

ai xi-1 2 + bi xi-1 + ci = f(xi-1)

for i = 2 (first interior knot) to n (last interior knot)

Each equation provides n-1 conditions, for a total of 2n-2 conditions.

2. The first and last functions must pass through the end points.

a1 x02 + b1 x0 + c1 = f(x0) first function through first point

an xn2 + bn xn + cn = f(xn) last function through last point

This adds two conditions for a total of 2n-2+2 = 2n conditions.

3. The first derivatives at the interior knots must be equal. This ensures that the

transition of the splines through the interior knots will be smooth.

ai-1 xi-1 2 + bi-1 xi-1 + ci-1 left side of interior knot

ai xi-1 2 + bi xi-1 + ci right side of interior knot

2ai-1 xi-1 + bi-1 first derivative of left side

2ai xi-1 + bi first derivative of right side

Hence: 2ai-1 xi-1 + bi-1 = 2ai xi-1 + bi


This equation provides n-1 conditions, for a total of 2n+(n-1) = 3n-1

conditions.

4. Assume that the second derivative is zero at the first point.

aix 2 + bi x + ci original function

2aix + bi first derivative 2ai second derivative 2a1 = 0 zero at the first point

Hence: a1 = 0

This brings our number of conditions to a total of 3n-1+1= 3n.

Let us now fit the data with quadratic splines by going through each of the four sets of

conditions:


a1x12 + b1x1 + c1 = f(x1)

a2x12 + b2x1 + c2 = f(x1)

a2x22 + b2x2 + c2 = f(x2)

a3x22 + b3x2 + c3 = f(x2)

(eqn. 1) 20.25a1 + 4.5b1 + c1 = 1

(eqn. 2) 20.25a2 + 4.5b2 + c2 = 1

(eqn. 3) 49a2 + 7b2 + c2 = 2.5

(eqn. 4) 49a3 + 7b3 + c3 = 2.5


a1 x02 + b1 x0 + c1 = f(x0)

a3 x32 + b3 x3 + c3 = f(x3)

(eqn. 5) 9a1 + 3b1 + c1 = 2.5

(eqn. 6) 81a3 + 9b3 + c3 = 0.5

3. The first derivatives at the interior knots must be equal.

2a1 x1 + b1 = 2a2 x1 + b2

2a2 x2 + b2 = 2a3 x2 + b3

(eqn. 7) 9a1 + b1 = 9a2 + b2

(eqn. 8) 14a2 + b2 = 14a3 + b3

4. Assume that the second derivative is zero at the first point.

(eqn. 9) a1 = 0

Collecting all the 3n = 9 equations we have the table below.

a1

b1 c1 a2 b2 c2 a3 b3 c3

20.25 4.5 1 0 0 0 0 0 0 = 1

0 0 0 20.25 4.5 1 0 0 0 = 1

0 0 0 49 7 1 0 0 0 = 2.5

0 0 0 0 0 0 49 7 1 = 2.5

9 3 1 0 0 0 0 0 0 = 2.5

0 0 0 0 0 0 81 9 1 = 0.5

9 1 0 -9 -1 0 0 0 0 = 0

0 0 0 14 1 0 -14 -1 0 = 0

1 0 0 0 0 0 0 0 0 = 0

To solve for the 9 unknowns we can use any of the methods learned in Part II (or we

may opt for the fast and dirty solution). We have the following results:

a1 = 0 b1 = -1.0000 c1 = 5.5000

a2 = 0.6400 b2 = -6.7600 c2 = 18.4600

a3 = -1.6000 b3 = 24.6000 c3 = -91.3000

Therefore, our quadratic splines are:

f1(x) = -x + 5.5 3.0 ≤ x ≤ 4.5

f2(x) = 0.64x2 – 6.7600x + 18.46 4.5 ≤ x ≤ 7.0

f3(x) = -1.6000x2 + 24.6000x – 91.3000 7.0 ≤ x ≤ 9.0

f(x=5) = 0.6600

Notice that our first spline is not really a 2nd

- order polynomial.

The graph of the three splines is shown below on the right side.

Compare our new graph (quadratic splines) with the old one (linear splines

reproduced above on the left side). Notice the smooth function transition between the interior

knots. However, notice also that there are two shortcomings that detract from the new fit:

1. The straight line connecting the first two points.

2. The spline for the last interval seems to swing too high.

Cubic or 3rd

-order splines do not exhibit these shortcomings and as a consequence are

usually better methods for spline interpolation.

Drill: Fit the data from the last drill with 2nd

-order splines and estimate the value of

the function at x=5.

Answer:

f1(x) = x – 2 3.0 ≤ x ≤ 4.5

f2(x) = -0.7200x2 + 7.4800x – 16.5800 4.5 ≤ x ≤ 7.0

f3(x) = 1.8000x2 - 27.8000x + 106.9000 7.0 ≤ x ≤ 9.0

f(x=5) = 2.8200

Let us now design cubic splines by deriving a 3rd

–order polynomial for each interval:

f1(x) = a1x

3 + b1x2 + c1x + d1 x0 ≤ x ≤ x1

f2(x) = a2x3 + b2x

2 + c2x + d2 x1 ≤ x ≤ x2 .

.

.

fn(x) = anx3 + bnx

2 + cnx + dn xn-1 ≤ x ≤ xn

For n+1 data points there are n intervals and, consequently, 4n unknown constants to

evaluate. Therefore, 4n equations or conditions are required to evaluate the unknowns.

These are (please note that #1, #2 and #3 are identical to those from quadratic splines, except

for the order of the function):


ai-1 xi-1

3 + bi-1 xi-1 2 + ci-1 xi-1 + di-1 = f(xi-1)

ai xi-1 3 + bi xi-1

2 + ci xi-1 + di = f(xi-1)

For i = 2 (first interior knot) to n (last interior knot). Each equation provides

n-1 conditions, for a total of 2n-2 conditions.


a1 x03 + b1 x0

2 + c1 x0 + d1 = f(x0) first function through first point

an xn3 + bn xn

2 + cn xn + dn = f(xn) first function through last point

This adds two conditions for a total of 2n-2+2 = 2n conditions.

3. The first derivatives at the interior knots must be equal. This ensures that the


ai-1 xi-1 3 + bi-1 xi-1

2 + ci-1 xi-1 + di-1 left side of interior knot

ai xi-1 3 + bi xi-1

2 + ci xi-1 + di right side of interior knot

3ai-1 xi-1 2 + 2bi-1 xi-1 + ci-1 first derivative of left side

3ai xi-1 2 + 2bi xi-1 + ci first derivative of right side

Hence:

3ai-1 xi-1

2 + 2bi-1 xi-1 + ci-1 = 3ai xi-1 2 + 2bi xi-1 + ci


This equation provides n-1 conditions, for a total of 2n+(n-1) = 3n-1.

4. The second derivatives at the interior knots must be equal. This ensures that the


Using the results from #3 above, we have:

6ai-1 xi-1 + 2bi-1 second derivative of left side

6ai xi-1 + 2bi second derivative of right side

Hence: 6ai-1 xi-1 + 2bi-1 = 6ai xi-1 + 2bi


This equation provides n-1 conditions, for a total of 3n-1+n-1) = 4n-2.

5. The second derivatives at the end knots are zero.

Using the second derivative results from #4, we have:

6ai-1 xi-1 + 2bi-1 second derivative of left side

6ai xi-1 + 2bi second derivative of right side

At the end knots we have:

6a1 x0 + 2b1 second derivative at first knot

6an xn + 2bn second derivative at last knot

Hence, we have:

6a1 x0 + 2b1 = 0

6an xn + 2bn = 0

This provides two additional conditions, bringing our total to 4n-2+2 = 4n.

Let us now fit the data with cubic splines by going through each of the five sets of

conditions:


a1 x1

3 + b1 x1 2 + c1 x1 + d1 = f(x1)

a2 x1 3 + b2 x1

2 + c2 x1 + d2 = f(x1)

a2 x2 3 + b2 x2

2 + c2 x2 + d2 = f(x2)

a3 x2 3 + b3 x2

2 + c3 x2 + d3 = f(x2)

(eqn. 01) 91.1250a1 + 20.25b1 + 4.5c1 + d1 = 1

(eqn. 02) 91.1250a2 + 20.25b2 + 4.5c2 + d2 = 1

(eqn. 03) 343a2 + 49b2 + 7c2 + d2 = 2.5

(eqn. 04) 343a3 + 49b3 + 7c3 + d3 = 2.5


a1 x03 + b1 x0

2 + c1 x0 + d1 = f(x0)

a3 x33 + b3 x3

2 + c3 x3 + d3 = f(x3)

(eqn. 05) 27a1 + 9b1 + 3c1 + d1 = 2.5

(eqn. 06) 729a3 + 81b3 + 9c3 + d3 = 0.5


3a1 x1

2 + 2b1 x1 + c1 = 3a2 x1 2 + 2b2 x1 + c2

3a2 x2 2 + 2b2 x2 + c2 = 3a3 x2

2 + 2b3 x2 + c3

(eqn. 07) 60.75a1 + 9b1 + c1 = 60.75a2 + 9b2 + c2

(eqn. 08) 147a2 + 14b2 + c2 = 147a3 + 14b3 + c3

4. The second derivatives at the interior knots must be equal.

6a1 x1 + 2b1 = 6a2 x1 + 2b2

6a2 x2 + 2b2 = 6a3 x2 + 2b3

(eqn. 09) 27a1 + 2b1 = 27a2 + 2b2

(eqn. 10) 42a2 + 2b2 = 42a3 + 2b3


6a1 x0 + 2b1 = 0

6a3 x3 + 2b3 = 0

(eqn. 11) 18a1 + 2b1 = 0

(eqn. 12) 54a3 + 2b3 = 0

We have the following results:

a1 = 0.1866 b1 = -1.6791 c1 = 3.6175 d1 = 1.7221

a2 = -0.2141 b2 = 3.7305 c2 = -20.7256 d2 = 38.2367

a3 = 0.1278 b3 = -3.4494 c3 = 29.5338 d3 = -79.0354

Therefore, our cubic splines are:

f1(x) = 0.1866x3 – 1.6791x2 + 3.6175x + 1.7221

3.0 ≤ x ≤ 4.5

f2(x) = -0.2141x3 + 3.7305x2 – 20.7256x + 38.2367

4.5 ≤ x ≤ 7.0

f3(x) = 0.1278x3 - 3.4494x2 + 29.5338x - 79.0354

7.0 ≤ x ≤ 9.0

f(x=5) = 1.1029

Notice that all of our functions are 3rd

-order polynomials. Their graph is shown

below on the right side.

Compare our new graph (cubic splines) with the old one (quadratic splines reproduced

above on the left side). Notice the disappearance of the straight line connecting the first two

points and of the high-swinging spline in the last interval.

Drill: Fit the data shown below with cubic splines.

x f(x)

3.0 1.0

4.5 2.5

6.0 2.0

7.0 0.5

9.0 2.5

Answer:

This time, there are 4 intervals. Hence, n=4. Also, there will be 4n=16

unknowns to evaluate.


a1 x1

3 + b1 x1 2 + c1 x1 + d1 = f(x1)

a2 x1 3 + b2 x1

2 + c2 x1 + d2 = f(x1)

a2 x2 3 + b2 x2

2 + c2 x2 + d2 = f(x2)

a3 x2 3 + b3 x2

2 + c3 x2 + d3 = f(x2)

a3 x3 3 + b3 x3

2 + c3 x3 + d3 = f(x3)

a4 x3 3 + b4 x3

2 + c4 x3 + d4 = f(x3)

(eqn. 01) 91.1250a1 + 20.25b1 + 4.5c1 + d1 = 2.5

(eqn. 02) 91.1250a2 + 20.25b2 + 4.5c2 + d2 = 2.5

(eqn. 03) 216a2 + 36b2 + 6c2 + d2 = 2.0

(eqn. 04) 216a3 + 36b3 + 6c3 + d3 = 2.0

(eqn. 05) 343a3 + 49b3 + 7c3 + d3 = 0.5

(eqn. 06) 343a4 + 49b4 + 7c4 + d4 = 0.5


a1 x03 + b1 x0

2 + c1 x0 + d1 = f(x0)

a4 x43 + b4 x4

2 + c4 x4 + d4 = f(x4)

(eqn. 07) 27a1 + 9b1 + 3c1 + d1 = 1

(eqn. 08) 729a4 + 81b4 + 9c4 + d4 = 2.5


3a1 x1

2 + 2b1 x1 + c1 = 3a2 x1 2 + 2b2 x1 + c2

3a2 x2 2 + 2b2 x2 + c2 = 3a3 x2

2 + 2b3 x2 + c3 3a3 x3

2 + 2b3 x3 + c3 = 3a4 x3 2 + 2b4 x3 + c4

(eqn. 09) 60.75a1 + 9b1 + c1 = 60.75a2 + 9b2 + c2

(eqn. 10) 108a2 + 2b2 + c2 = 108a3 + 12b3 + c3

(eqn. 11) 147a3 + 14b3 + c3 = 147a4 + 14b4 + c4

4. The second derivatives at the interior knots must be equal.

6a1 x1 + 2b1 = 6a2 x1 + 2b2 6a2 x2 + 2b2 = 6a3 x2 + 2b3

6a3 x3 + 2b3 = 6a4 x3 + 2b4

(eqn. 12) 27a1 + 2b1 = 27a2 + 2b2

(eqn. 13) 36a2 + 2b2 = 36a3 + 2b3

(eqn. 14) 42a3 + 2b3 = 42a4 + 2b4


6a1 x0 + 2b1 = 0

6a4 x4 + 2b4 = 0

(eqn. 15) 18a1 + 2b1 = 0

(eqn. 16) 54a4 + 2b4 = 0

f1(x) = -0.1108x3 + 0.9969x2 - 1.7414x + 0.2430

for 3.0 ≤ x ≤ 4.5

f2(x) = -0.0388x3 + 0.0250x2 + 2.6320x – 6.3172

for 4.5 ≤ x ≤ 6.0

f3(x) = 0.6784x3 - 12.8833x2 + 79.8320x - 159.7173

for 6.0 ≤ x ≤ 7.0

f4(x) = -0.2270x3 + 6.1297x2 – 53.2591x + 150.8286

for 7.0 ≤ x ≤ 9.0

Chapter 5 – Fourier Approximation

Engineers often deal with systems that oscillate or vibrate. As might be expected,

trigonometric functions play a fundamental role in modeling such problem contexts. Fourier

approximation represents a systematic framework for using trigonometric series for this

purpose.

Sinusoidal Functions

A sinusoid is any waveform that can be described as a sine or cosine. There is no

clear-cut convention for choosing either function, and in any case the results will be identical.

A sinusoid f(t) can be represented generally as a function of time as:

f(t) = A + A0 cos(ω0t + θ) eqn. 1

The mean value A sets the average height above the abscissa. The amplitude C

specifies the height of the oscillation. The angular frequency ω0 characterizes how often the

cycles occur. Finally, the phase angle or phase shift θ parameterizes the extent to which the

sinusoid is shifted horizontally.

Although equation #1 is an adequate mathematical characterization of a sinusoid, it is

awkward to work with from the standpoint of curve fitting because the phase shift is included

in the argument of the cosine function. This deficiency can be overcome by invoking a

trigonometric identity. After substituting it into the equation and collecting terms we have:

f(t) = A + B cos(ω0t) + C sin(ω0t) eqn. 2

where: B = A0 cos(θ) C = -A0 sin(θ)

Drill: Give the derivation of equation #2.

We can treat equation #2 as another example of the general model from the multiple

linear regression discussion of Part III Chapter 3:

y = a0 + a1 x1 + a2 x2 + . . . + an xn

for n=2, we have:

y = a0 + a1 x1 + a2 x2 eqn. 3

Equating equations#2 and #3, we have:

A = a0 B = a1 C = a2

x1 = cos(ω0t) x2 = sin(ω0t)

We then use multiple linear regression to solve for the coefficients.

Let us now fit a sinusoid with ω0=4.1890 to the data given below:

t f(t)

0.00 2.2000

0.15 1.5950

0.30 1.0310

0.45 0.7220

0.60 0.7860

0.75 1.2000

0.90 1.8050

1.05 2.3690

1.20 2.6780

1.35 2.6140

After multiple linear regression: A = 1.7000 B = 0.5001 C = -0.8661

Hence, our sinusoidal fit is:

f(t) = 1.7000 + 0.5001 cos(4.1890t) – 0.8661 sin(4.1890t)

or alternatively from equation #1:

f(t) = 1.7000 + 1.0001 cos(4.1890t + 1.0471)

or: f(t) = 1.7000 + 1.0001 sin(4.1890t + 2.6179)

The error Sr of our sinusoidal model is 0.

Shown below is the graph of the sinusoid plotted against the data.

Drill: Fit a sinusoid (ω0=4) to the data given below:

t f(t)

0.45 0.7000

0.60 0.8911

0.75 1.3911

0.90 2.0092

1.05 2.5092

1.20 2.7000

1.35 2.5088

1.50 2.0087

1.65 1.3906

1.80 0.8908

f(t) = 1.6533 + 0.0876 cos(4t) – 1.0189 sin(4t)

f(t) = 1.6533 + 1.0227 cos(4t + 1.4850)

f(t) = 1.6533 + 1.0227 sin(4t + 3.0558)

The error Sr of our sinusoidal model is 0.0067.

Continuous Fourier Series

Fourier showed that an arbitrary periodic function can be represented by an infinite

series of sinusoids of harmonically related frequencies. For a function with period T, a

continuous Fourier Series can be written as:

f(t) = a0 + a1 cos(ω0t) + b1 sin(ω0t) + a2 cos(2ω0t) +

b2 sin(2ω0t) + . . .

or more concisely:

f(t) = a0 + ∑[ak cos(kω0t) + bk sin(kω0t)]

where ω0 = 2π/T is called the fundamental frequency, its constant multiples 2ω0, 3ω0,

etc., are called harmonics and k = 1, 2, 3, . . .

The coefficients can be computed via:

ak = 2 f(t) cos(kω0t) dt

T and

bk = 2 f(t) sin(kω0t) dt

T and

a0 = 1 f(t) dt

T

where k = 1, 2, 3, . . . and the integration is over the period T.

Use the continuous Fourier Series to approximate the rectangular wave function

shown below.

∫

∫

∫

Analyzing the rectangular wave, we have:

f(t) = -1 , -T/2 < t < -T/4

1 , -T/4 < t < T/4

-1 , T/4 < t < T/2

We have:

ak = (2/T)[-(1/kω0)sin(kω0t) + (1/kω0)sin(kω0t) –

(1/kω0)sin(kω0t)]

Since ω0 = 2π/T:

ak = -(1/kπ)sin(kω0t) + (1/kπ)sin(kω0t) – (1/kπ)sin(kω0t)

ak = -(1/kπ)[sin(-kπ/2) - sin(-kπ)]

+(1/kπ)[sin(kπ/2) - sin(-kπ/2)]

-(1/kπ)[sin(kπ) - sin(kπ/2)]

ak = -(1/kπ)[-sin(kπ/2) + sin(kπ)]

+(1/kπ)[sin(kπ/2) + sin(kπ/2)]

-(1/kπ)[sin(kπ) - sin(kπ/2)]

ak = (1/kπ)sin(kπ/2) - (1/kπ)sin(kπ)

(1/kπ)sin(kπ/2) + (1/kπ)sin(kπ/2)

(-1/kπ)sin(kπ) + (1/kπ)sin(kπ/2)

ak = (4/kπ)sin(kπ/2) - (2/kπ)sin(kπ)

a1 = (4/π)sin(π/2) - (2/π)sin(π) = 4/π

a2 = (2/π)sin(π) - (1/π)sin(2π) = 0

a3 = (4/3π)sin(3π/2) - (2/3π)sin(3π) = -4/3π

a4 = (1/π)sin(2π) - (1/2π)sin(4π) = 0

a5 = (4/5π)sin(5π/2) - (2/5π)sin(5π) = 4/5π

Hence:

ak = 4/kπ for k = 1, 5, 9, . . .

-4/kπ for k = 3, 7, 11, . . .

0 for k = 2, 4, 6, . . .

We have:

bk = (2/T)[(1/kω0)cos(kω0t) - (1/kω0)cos(kω0t) +

(1/kω0)cos(kω0t)]

Since ω0 = 2π/T:

bk = (1/kπ)cos(kω0t) - (1/kπ)cos(kω0t) + (1/kπ)cos(kω0t)

bk = (1/kπ)[cos(-kπ/2) - cos(-kπ)]

-(1/kπ)[cos(kπ/2) - cos(-kπ/2)]

+(1/kπ)[cos(kπ) - cos(kπ/2)]

bk = (1/kπ)[cos(kπ/2) - cos(kπ)]

-(1/kπ)[cos(kπ/2) - cos(kπ/2)]

+(1/kπ)[cos(kπ) - cos(kπ/2)]

bk = (1/kπ)cos(kπ/2) - (1/kπ)cos(kπ)

-(1/kπ)cos(kπ/2) + (1/kπ)cos(kπ/2)

(1/kπ)cos(kπ) - (1/kπ)cos(kπ/2)

We have:

a0 = -(1/T)[(-T/4) – (-T/2)]

+(1/T)[(T/4) – (-T/4)]

-(1/T)[(T/2) – (T/4)]

a0 = -(1/T)[(-T/4) + (T/2)]

+(1/T)[(T/4) + (T/4)]

-(1/T)[(T/2) – (T/4)]

a0 = -(1/T)(T/4) a0 = -1/4 + 1/2 – 1/4

+(1/T)(T/2)

-(1/T)(T/4)

bk = 0

a0 = 0

Therefore, the Fourier Series approximation is:

f(t) = 4 cos(ω0t) – 4 cos(3ω0t) + 4 cos(5ω0t)

π 3π 5π

- 4 cos(7ω0t) + . . .

7π

Shown below is the graph of the Fourier Series approximation (up to the fourth term)

plotted against the rectangular wave.

Drill: Use a continuous Fourier Series to approximate the saw-tooth wave (T=2) shown

below.

Answer:

Analyzing the wave, we have f(t) = 0.5t , 0 < t < 2

Hence,

ak = (2/T) 0.5t cos(kω0t) dt

∫

ak = 0.5 t cos(kω0t) dt

Since:

ak = 0.5 cos(kω0t) + 0.5t sin(kω0t)

(kω0)2 kω0

Since ω0 = 2π/T = π:

ak = 0.5 cos(kπt) + 0.5t sin(kπt)

(kπ)2 kπ

ak = 0.5 cos(k2π) + sin(k2π) – 0.5 cos(0)

(kπ)2 kπ (kπ)2

ak = 0.5 + sin(k2π) – 0.5

(kπ)2 kπ (kπ)2

ak = sin(k2π)

kπ

Next, we have:

bk = 2 0.5t sin(kω0t) dt

T

bk = 0.52 t sin(kω0t) dt

Since:

bk = 0.5 sin(kω0t) - 0.5t cos(kω0t)

(kω0)2 kω0

Since ω0 = 2π/T = π:

bk = 0.5 sin(kπt) - 0.5t cos(kπt)

(kπ)2 kπ

∫

ak = 0

∫

∫

bk = 0.5 sin(k2π) - cos(k2π) – 0.5 sin(0)

(kπ)2 kπ (kπ)2

bk = 0 - cos(k2π) – 0

kπ

bk = - cos(k2π)

kπ Then we have:

a0 = 1 0.5t dt

T

a0 = 0.25 t2/2

a0 = 0.125 t2 = 0.125(4-0)

Therefore, the Fourier Series approximation is:

f(t) = 0.5 – 1 sin(ω0t) – 1 sin(2ω0t)

π 2π

- 1 sin(3ω0t) – 1 sin(4ω0t) - . . .

3π 4π

Shown below is the graph of the Fourier Series approximation (up to the fifth term)

plotted against the saw-tooth wave.

bk = -1/(kπ)

∫

a0 = 0.5

Part IV – Numerical Differentiation

and Integration

Calculus is the mathematics of change. Because engineers must continuously deal

with systems and processes that change, calculus is an essential tool of our profession.

Standing at the heart of calculus are the related mathematical concepts of differentiation and

integration.

Chapter 1 – Newton-Cotes Integration Formulas

The Newton-Cotes formulas are the most common numerical integration schemes.

They are based on the strategy of replacing a complicated function or tabulated data with

some approximating function (a polynomial) that is easy to integrate. For example, in Figure

1-a, a first-order polynomial (trapezoidal rule) is used as an approximation. In Figure 1-b, a

second-order polynomial (Simpson’s rules) is used for the same purpose. The integral can

also be approximated using a series of polynomials applied piece-wise (i.e., multiple

application) to the function or data over segments of constant length. In Figure 2, three

straight lines are used to approximate the integral. Higher-order polynomials can be utilized

for the same purpose.

Figure 1

Figure 2

∫

Closed and open forms of the Newton-Cotes formulas are available. The closed forms

are those where the data points at the beginning and end of the limits of integration are

known. The open forms have integration limits that extend beyond the range of the data.

The Open Newton-cotes formulas are not generally used for definite integration.

However, they are utilized for evaluating improper integrals and for the solution of ordinary

differential equations.

Trapezoidal Rule

The trapezoidal rule is the first of the Newton-Cotes closed integration formulas. It

corresponds to the case where the approximating polynomial used is first-order as shown in

Figure 3.

Figure 3

Recall that in Part III, Chapter 4 equation #1, the line can be represented as:

f1(x) = f(x0) + f(x1)-f(x0) (x-x0)

x1-x0

The area under this straight line (i.e., the integral of f1(x)) is an estimate of the

integral of f(x) between the limits x0 and x1 :

I ≅ f1(x) eqn. 1

Drill: Derive the trapezoidal rule formula from equation #1.

Let us now use the trapezoidal rule to numerically integrate:

f(x) = 400x5 - 900x4 + 675x3 - 200x2 + 25x + 0.2

from 0 to 0.8.

The exact value of the integral is I = 1.6405. Using equation #1, the estimate of

the integral is I ≅ 0.1728, with an error of 89.4668 %. The large error is evident from

the fact that the trapezoid covers only a very small area under the curve, neglecting a

significant area above the line, as shown in Figure 4.

Figure 4

Drill: Use the trapezoidal rule to numerically integrate f(x) = 8 + 5sin(x)

from 0 to π.

Answer: I ≅ 25.1327 error = 28.4635%

Multiple-Application of Trapezoidal Rule

One way to improve the accuracy of the trapezoidal rule is to divide the integration

interval into a number of segments and apply the method to each segment. The areas of

individual segments can then be added to yield the integral for the entire interval.

Let us now use multiple application of the trapezoidal rule to numerically integrate:

f(x) = 400x5 - 900x4 + 675x3 - 200x2 + 25x + 0.2

from 0 to 0.8. Let us use the intervals [0 0.4] and [0.4 0.8].

The exact value of the integral for the first interval is I1 = 0.5632. The estimate

of this integral is I1 ≅ 0.5312. The exact value of the integral for the second interval is

I2 = 1.0773. The estimate of this integral is I2 ≅ 0.5376. Combining the two

estimates we have I ≅ 1.0688, with an error of 34.85%. Compare this error with that of

the single application method which is 89.4668 %.

Drill: Use multiple application of the trapezoidal rule to numerically integrate

f(x) = 8 + 5sin(x) from 0 to π. Use the intervals [0 π/3] and

[π/3 π].

Answer: The exact value of the integral for the first interval is

I1 = 10.8776. The estimate of this integral is I1 ≅ 10.6448. The exact value

of the integral for the second interval is I2 = 24.2552. The estimate of this

integral is I2 ≅ 21.2897. Combining the two estimates we have I ≅ 31.9345,

with an error of 9.10%. Compare this error with that of the single application

method which is 28.4635%.

∫

Aside from multiple application of the trapezoidal rule, another way to obtain a more

accurate estimate of an integral is to use higher-order polynomials to connect the points. The

formulas that result from taking the integrals under these polynomials are called Simpson’s

Rules.

Simpson’s 1/3 Rule

This rule results when a second-order interpolating polynomial is used to approximate

an integral. Recall that in Part III, Chapter 4 equation #8, a second-order Lagrange

Interpolating polynomial is given by:

f2(x) = (x-x1)(x-x2) f(x0) + (x-x0)(x-x2) f(x1) +

(x0-x1)(x0-x2) (x1-x0)(x1-x2)

(x-x0)(x-x1) f(x2)

(x2-x0)(x2-x1)

The integral of this second-order interpolating polynomial, evaluated from x0 to x2,

gives what is called Simpson’s 1/3 Rule: (where x1 is midway between x0 and x2)

I ≅ f2(x) eqn. 2

Drill: Derive Simpson’s 1/3 Rule formula from equation #2.

Let us now use Simpson’s 1/3 Rule to numerically integrate the same function from 0

to 0.8.


the integral is I ≅ 1.3675, with an error of 16.6450%.

Drill: Use Simpson’s 1/3 rule to numerically integrate f(x) = 8 + 5sin(x)

from 0 to π.

Answer: I ≅ 35.6047 error = 1.3434%

Multiple-Application of Simpson’s 1/3 Rule

Simpson’s 1/3 rule can be improved by dividing the integration interval into a number

of segments of equal width. Let us now use multiple application of Simpson’s 1/3 rule to

numerically integrate the same function from 0 to 0.8. Let us use the intervals [0 0.4]

and [0.4 0.8].

The estimate of the integral for the first interval is I1 ≅ 0.5205. For the second

interval the estimate is I2 ≅ 1.1029. Combining the two estimates we have

I ≅ 1.6235, with an error of 1.04%.

∫

Drill: Use multiple application of Simpson’s 1/3 rule to numerically integrate

f(x) = 8 + 5sin(x) from 0 to π. Use the intervals [0 π/2] and

[π/2 π].

Answer: The estimate of the integral for the first interval is I1 ≅ 17.5778.

For the second interval the estimate is I2 ≅ 17.5778. Combining the two

estimates we have I ≅ 35.1555, with an error of 0.065%.


This rule results when a third-order interpolating polynomial is used to approximate

an integral. Recall that in Part III, Chapter 4 equations #9 and #10, a third-order Lagrange

interpolating polynomial is given by:

f3(x) = (x-x1)(x-x2)(x-x3)f(x0) + (x-x0)(x-x2)(x-x3) f(x1) +

(x0-x1)(x0-x2)(x0-x3) (x1-x0)(x1-x2)(x1-x3)

(x-x0)(x-x1)(x-x3)f(x2) + (x-x0)(x-x1)(x-x2) f(x3)

(x2-x0)(x2-x1)(x2-x3) (x3-x0)(x3-x1)(x3-x2)

The integral of this third-order interpolating polynomial, evaluated from x0 to x3,

gives what is called Simpson’s 3/8 Rule: (where x1 and x2 are equally spaced)

I ≅ f3(x) eqn. 3

Drill: Derive Simpson’s 3/8 Rule formula from equation #3.

Let us now use Simpson’s 3/8 Rule to numerically integrate the same function from 0

to 0.8.


the integral is I ≅ 1.5192 with an error of 7.3978%.

Drill: Use Simpson’s 3/8 rule to numerically integrate f(x) = 8 + 5sin(x)

from 0 to π.

Answer: I ≅ 35.3354 error = 0.5767%

Combined Application of Simpson’s 1/3 and 3/8 Rules

The combination of Simpson’s 1/3 and 3/8 rules require application to five (5)

segments. The first two segments are for the 1/3 rule and the last three segments are for the

3/8 rule. Let us now use combined application of Simpson’s 1/3 and 3/8 rules to numerically

integrate the same function from 0 to 0.8. Let us divide the interval into five equally-spaced

sub-intervals.

The estimate of the integral for the first two sub-intervals is I1 ≅ 0.3803. For the

last three sub-intervals the estimate is I2 ≅ 1.2648. Combining the two estimates we have

I ≅ 1.6451, with an error of 0.28%.

Drill: Use a combination of Simpson’s 1/3 and 3/8 rules to numerically integrate

f(x) = 8 + 5sin(x) from 0 to π. Divide the interval into five equally-

spaced sub-intervals.

Answer: The estimate of the integral for the first two sub-intervals is

I1 ≅ 13.5111. For the last three sub-intervals the estimate is I2 ≅ 21.6388.

Combining the two estimates we have I ≅ 35.1499, with an error of 0.05%.

It must be stressed that, in practice, higher-order formulas are rarely used. Simpson’s

rules are sufficient for most applications. Accuracy can be improved by using the multiple-

application version rather than opting for the higher-point formulas.

The tables below summarize the results and compare the errors from the methods

discussed above.

Table1 – Comparison of Errors

error %

method

example function

drill function

Trapezoidal Rule

89.47

28.46

Multiple Trapezoidal Rule

34.85

9.10


16.65

1.34

Multiple Simpson’s 1/3 Rule

1.04

0.07


7.40

0.58

Combined Simpson’s 1/3 & 3/8 Rules

0.28

0.05

Table 2 – Estimates of the Integrals

example function

drill function

method

h*

I

h*

I

Trapezoidal Rule

0.8

0.1728

π

25.1327

Multiple Trapezoidal Rule

0.4

1.0688 π/3

2π/3

31.9345


0.4

1.3675

π/2

35.6047

Multiple Simpson’s 1/3 Rule

0.2

1.6235

π/4

35.1555


4/15

1.5192

π/3

35.3354

Combined Simpson’s 1/3 & 3/8 Rules

0.16

1.6451

π/5

35.1499

*h is the length of segment

Integration with Unequal Segments

All the formulas discussed above are based on equally spaced data points. However,

there are many situations where this assumption does not hold and we must deal with

unequal-sized segments. For these cases, one method is to apply the trapezoidal rule to each

segment and sum the results. Note that this is the same approach used for the multiple-

application trapezoidal rule. In fact, the drill problem under multiple-application trapezoidal

rule uses segments of unequal lengths, i.e., [0 π/3] and [π/3 π].

Use the trapezoidal rule to determine the integral of f(x) evaluated from 0 to 2/5.

x

f(x)

0.00 0.2000

0.12 1.3097

0.22 1.3052

0.32 1.7434

0.36 2.0749

0.40 2.4560

There are five segments and the integral for each is listed in the table below:

segment

integral

0.00 – 0.12 0.0906

0.12 – 0.22 0.1307

0.22 – 0.32 0.1524

0.32 – 0.36 0.0764

0.36 – 0.40 0.0906

total

0.5407

Drill: Use the trapezoidal rule to determine the integral of f(x) evaluated from

2/5 to 4/5.

Answer:

x

f(x)

0.40 2.4560

0.44 2.8430

0.54 3.5073

0.64 3.1819

0.70 2.3630

0.80 0.2320

There are five segments and the integral for each is listed in the table below:

segment

integral

0.40 – 0.44 0.1060

0.44 – 0.54 0.3175

0.54 – 0.64 0.3345

0.64 – 0.70 0.1663

0.70 – 0.80 0.1298

total

1.0541

Using the data given below, let us compute the integral of f(x) using Simpson’s

rules for those segments where they are appropriate.

x

f(x)

0.00 0.2000

0.12 1.3097

0.22 1.3052

0.32 1.7434

0.36 2.0749

0.40 2.4560

0.44 2.8430

The segments and the methods to be used are listed below.

segment

length

method

1 0.00 – 0.12 0.12 trapezoid

2 0.12 – 0.22 0.10

3 0.22 – 0.32 0.10

Simpson’s 1/3

4 0.32 – 0.36 0.04

5 0.36 – 0.40 0.04

6 0.40 – 0.44 0.04

Simpson’s 3/8

Segments 1 and 2 cannot be solved using a Simpson’s rule because they are of

unequal lengths. Hence, segment 1 will be solved using trapezoidal rule. Segments 2 and 3

are of equal lengths and, hence, can be solved using Simpson’s 1/3 rule. Segments 4 and 5

are of equal lengths and, hence, can also be solved using Simpson’s 1/3 rule. However, since

segment 6 is also of equal length as segments 4 and 5, and since Simpson’s 3/8 rule results to

less errors, we will group segments 4, 5 and 6 and solve them using Simpson’s 3/8 rule. The

results are tabulated below.

segment

integral

1 0.00 – 0.12 0.0906

2 0.12 – 0.22

3 0.22 – 0.32

0.2758

4 0.32 – 0.36

5 0.36 – 0.40

6 0.40 – 0.44

0.2727

total

0.6391

Drill: Using the data given below, compute the integral of f(x) using Simpson’s

rules for those segments where they are appropriate.

x

f(x)

0.44 2.8430

0.54 3.5073

0.64 3.1819

0.70 2.3630

0.80 0.2320

The segments and the methods to be used are listed below.

segment

length

method

1 0.44 – 0.54 0.10

2 0.54 – 0.64 0.10

Simpson’s 1/3

3 0.64 – 0.70 0.06 trapezoid

4 0.70 – 0.80 0.10 trapezoid

segment

integral

1 0.44 – 0.54

2 0.54 – 0.64

0.6685

3 0.64 – 0.70 0.1663

4 0.70 – 0.80 0.1298

total

0.9646

Chapter 2 – Integration of Equations

Functions to be integrated numerically will typically be of two forms: a table of

values or an equation. There are two techniques designed to analyze cases where the function

equation is given. The first is based on Richardson’s Extrapolation, which is a method for

combining two numerical integral estimates in order to obtain a third, more accurate value.

The second method is called Gauss Quadrature.

Richardson’s Extrapolation (Romberg Integration)

Richardson’s extrapolation is designed to obviate the shortcomings of multiple-

application trapezoidal rule. It is quite similar to the techniques of Chapter 1 in the sense that

it is based on successive application of the trapezoidal rule. However, through mathematical

manipulations, superior results are attained for less effort. Romberg integration is a

computational algorithm for implementing Richardson’s extrapolation in a highly efficient

manner.

The estimate and error associated with a multiple-application trapezoidal rule can be

represented generally as:

I = I(h) + E(h) eqn. 1

where I is the exact value of the integral, I(h) is the approximation from an n-segment

application of the trapezoidal rule with segment length h, and E(h) is the truncation error. If

we make two separate estimates using segment lengths h1 and h2 and have exact values for

the error:

I(h1) + E(h1) = I(h2) + E(h2) eqn. 2

The ratio of the two errors can be approximated by:

E(h1) ≅ h12

E(h2) h2

2

Hence,

E(h1) ≅ E(h2) h12 / h2

2

Substituting into equation #2, we have:

I(h1) + [E(h2) h12 / h2

2)] = I(h2) + E(h2)

Hence,

E(h2)[(h1

2/h22) – 1] = I(h2) - I(h1)

E(h2) = I(h2) - I(h1)

[(h12/h2

2) – 1]

Substituting into equation #1, we have Richardson’s extrapolation formula:

I ≅ I(h2) + I(h2) - I(h1) eqn. 3

[(h1/h2)2 – 1]

Let us now numerically compute the integral of the function f(x) = 400x5 -

900x4 + 675x3 - 200x2 + 25x + 0.2, evaluated from 0 to 0.8, using multiple-

application trapezoidal rule of 1, 2, 4 and 8 segments. The answers are summarized in the

table below.

Table 3

segments

h

I

error%

1 0.8 0.1728 89.5

2 0.4 1.0688 34.9

4 0.2 1.4848 09.5

8 0.1 1.6008 02.4

Using Table 3 and Richardson’s extrapolation we can have improved estimates of the

integral of the function f(x) = 400x5 - 900x4 + 675x3 - 200x2 + 25x +

0.2, evaluated from 0 to 0.8. The results are summarized in the table below.

Table 4

h1

h2

I

error%

0.8 0.4 1.3675 16.64

0.4 0.2 1.6235 01.04

0.2 0.1 1.6395 00.07

Drill: Use Table 4 to do another iteration of Richardson’s extrapolation to get more

improved estimates of the integral.

Answer: The results are summarized in the table below.

h1

h2

I

error%

0.8 0.2 1.6406 0.00

0.4 0.1 1.6406 0.00

Drill: Do a final iteration of Richardson’s extrapolation to get the best estimate of

the integral.

Answer: The results are summarized in the table below.

h1

h2

I

error%

0.8 0.1 1.6406 0.00

The formulations of Richardson’s extrapolation can be expressed in a general form

that is well-suited for computer implementation. This is called the Romberg Integration:

Ij,k = 4k-1

Ij+1,k-1 – Ij,k-1 eqn. 4

4k-1

- 1

where Ij+1,k-1 and Ij,k-1 are the more and less accurate integrals, respectively, Ij,k is the

improved integral and k signifies the level of integration. Hence, k=1 corresponds to the

original trapezoidal rule estimates, k=2 corresponds to the estimates using the first iteration

of the Romberg Integral, k=3 corresponds to the estimates using the second iteration of the

Romberg Integral, and so on.

Drill: Use Romberg Integration to numerically compute the integral of the function

evaluated from 0 to 3.

f(x) = ex sin(x) dx

1 + x2

Answer:

segments

h

I

1 3.000 0.4252

2 1.500 2.2759

4 0.750 2.7438

8 0.375 2.8478

h1

h2

I

3.00 1.500 2.8928

1.50 0.750 2.8998

0.75 0.375 2.8825

∫

∫

h1

h2

I

3.00 0.750 2.9003

1.50 0.375 2.8813

h1

h2

I

3.0 0.375 2.8810

Gauss Quadrature (Gauss-Legendre Formulas)

A characteristic of most of the formulas in Chapter 1 is that the integral estimates are

based on evenly spaced function values. Consequently, the location of the base points used

in these equations is predetermined or fixed. Suppose that the constraint of fixed base points

is removed and we are free to evaluate the area under a straight line joining any two points on

the curve. By positioning these points wisely, we could define a straight line that would

balance the positive and negative errors. Hence, we would arrive at an improved estimate of

the integral.

The Gauss quadrature is one technique to implement such a strategy. The particular

Gauss quadrature formulas described in this section are called Gauss-Legendre formulas.

The object of the Gauss quadrature is to determine the coefficients of an equation of

the form:

I ≅ c0f(x0) + c1f(x1) eqn. 1

where c0 and c1 are the unknown coefficients. Also, since the function arguments x0 and x1

are not fixed at the endpoints, but are unknowns, we have a total of four unknowns that must

be evaluated. Consequently, we require four conditions to determine them exactly.

We can obtain two of these conditions by assuming that equation #1 fits the integral

of a constant (y=1) and a linear (y=x) function exactly:

c0f(x0) + c1f(x1) = (1) dx

c0f(x0) + c1f(x1) = (x) dx

To arrive at the two other conditions, we merely extend this reasoning by assuming

that it also fits the integral of a parabolic (y=x2) and a cubic (y=x3) function.

∫

∫ c0f(x0) + c1f(x1) = (x2) dx

c0f(x0) + c1f(x1) = (x3) dx

Assuming the limits of integration are -1 and 1, we have:

c0f(x0) + c1f(x1) = dx = 2

c0f(x0) + c1f(x1) = x dx = 0

c0f(x0) + c1f(x1) = x2 dx = 2/3

c0f(x0) + c1f(x1) = x3 dx = 0

or, we have:

c0f(x0) + c1f(x1) = 2 eqn. 2

c0f(x0) + c1f(x1) = 0 eqn. 3

c0f(x0) + c1f(x1) = 2/3 eqn. 4

c0f(x0) + c1f(x1) = 0 eqn. 5

Inspecting equation #2, since the function is a constant function y=1, we have:

f(x0) = f(x1) = 1

Hence,

c0 + c1 = 2 eqn. 6

Inspecting equation #3, since the function is a linear function y=x, we have:

f(x0) = x0 f(x1) = x1

Hence,

c0x0 + c1x1 = 0 eqn. 7

Inspecting equation #4, since the function is a parabolic function y=x2, we have:

f(x0) = (x0)2 f(x1) = (x1)

2

Hence,

c0(x0)2 + c1(x1)

2 = 2/3 eqn. 8

Inspecting equation #5, since the function is a parabolic function y=x3, we have:

f(x0) = (x0)3 f(x1) = (x1)

3

Hence,

c0(x0)3 + c1(x1)

3 = 0 eqn. 9

In summary we have:

c0 + c1 = 2 eqn. 10

c0x0 + c1x1 = 0 eqn. 11

c0(x0)2 + c1(x1)

2 = 2/3 eqn. 12

c0(x0)3 + c1(x1)

3 = 0 eqn. 13

Combining equations #10 and #11, we have:

x0 = -(2-c0)x1/c0 eqn. 14

Substituting equations #14 and #10 into equation #12, we have:

(x1)2 = -c0 /[3(c0-2)] eqn. 15

Substituting equations #14 and #10 into equation #13, we have:

c0 = 1

Hence, from equation #10:

c1 = 1

From equation #15:

x1 = (1/3)1/2

From equation #14:

x0 = -(1/3)1/2

Finally, from equation #1, we have the Two-Point Gauss-Legendre Formula:

I ≅≅≅≅ f[-(1/3)1/2] + f[(1/3)1/2] eqn. 16

Notice that the integration limits we used are from -1 to 1. This was done to

simplify the mathematics and to make the formulation as general as possible. A simple

change of variable can be used to translate other limits of i9ntegration into this form. This is

accomplished by assuming that a new variable z is related to the original variable x in a

linear fashion:

x = a0 + a1 z eqn. 17

If the lower limit, x=a, corresponds to z=-1, these values can be substituted into

equation #17 to yield:

a = a0 - a1 eqn. 18

Similarly, if the upper limit, x=b, corresponds to z=1, then:

b = a0 + a1 eqn. 19

Adding equations #18 and #19 yields:

a0 = (b + a)/2 eqn. 20

Subtracting equation #19 from #18 yields:

a1 = (b - a)/2 eqn. 21

Substituting equations #20 and #21 into #17 yields:

x = (b+a) + (b-a)z eqn. 22

2

Let us now use the two-point Gauss-Legendre formula to numerically calculate the

integral of the function f(x) = 400x5 - 900x4 + 675x3 - 200x2 + 25x +

0.2, evaluated from 0 to 0.8.

Before integrating the function, we must perform a change of variable so that the

limits are from -1 to 1. Using equation #22, where a=0 and b=0.8, we have:

x = 0.4 + 0.4z dx = 0.4dz

We then substitute these equations into the original function to give the transformation:

f(z) = [400(0.4+0.4z)5 – 900(0.4+0.4z)4 + 675(0.4+0.4z)3 –

200(0.4+0.4z)2 + 25(0.4+0.4z) + 0.2] [0.4dz]

Evaluating this function using equation #16, we have:

f[-(1/3)1/2] = 0.5167 f[(1/3)1/2] = 1.3058

Hence:

I ≅ 1.8226 error = 11.10%

Beyond the two-point formula, higher-point versions can be developed in the general

form:

I ≅ c0f(x0) + c1f(x1) + . . . + cn-1f(xn-1)

where n is the number of points. The following table shows the weighing factors c and

function arguments x used in Gauss-Legendre formulas.

Table 5

points

weighing factors

function arguments

2

c0 = 1

c1 = 1

x0 = -0.57735

x1 = 0.57735

3

c0 = 0.55556

c1 = 0.88889

c2 = 0.55556

x0 = -0.77460

x1 = 0

x2 = 0.77460

4

c0 = 0.34785

c1 = 0.65215

c2 = 0.65215

c3 = 0.34785

x0 = -0.86114

x1 = -0.33998

x2 = 0.33998

x3 = 0.86114

5

c0 = 0.23693

c1 = 0.47863

c2 = 0.56889

c3 = 0.47863

c4 = 0.23693

x0 = -0.90618

x1 = -0.53847

x2 = 0

x3 = 0.53847

x4 = 0.90618

6

c0 = 0.17132

c1 = 0.36076

c2 = 0.46791

c3 = 0.46791

c4 = 0.36076

c5 = 0.17132

x0 = -0.93247

x1 = -0.66121

x2 = -0.23862

x3 = 0.23862

x4 = 0.66121

x5 = 0.93247

Drill: Use the three-point Gauss-Legendre formula to numerically calculate the

integral of the same function, evaluated from 0 to 0.8.

Answer: I ≅ 0.2813 + 0.8732 + 0.4860 ≅ 1.6405

error = 0 %

∫

Drill: Use Gauss-Legendre formulas up to six points to numerically calculate the

integral of the function f(x), evaluated from 0 to 10, where g=9.8,

c=12.5, and m=68.1.

f(x) = (gm/c)[1 – e-cx/m

]

Answer: The table below summarizes the results.

Improper Integrals (Extended Midpoint Rule)

In this section we will focus on one type of improper integral – that is, one with a

lower limit of –∞ or an upper limit of +∞. Such integrals usually can be evaluated by

making a change of variable that transforms the infinite range to one that is finite. The

following identity serves this purpose and works for any function that decreases towards zero

at least as fast as 1/x2 as x approaches infinity:

eqn. 22

for ab>0. Therefore, it can only be used when a is positive and b is ∞ or when a is -∞ and

b is negative. For cases where the limits are from -∞ to a positive value or from a negative

value to ∞, the integral can be implemented in two steps. Fore example:

eqn. 23

where -A is chosen as a sufficiently large negative value so that the function has begun to

approach zero asymptotically at least as fast as 1/x2. After the integral has been divided

into two parts, the first can be evaluated with equation #22 and the second with a Newton-

Cotes closed formula such as Simpson’s 1/3 rule.

points

I

error %

2

290.0145

0.20

3

289.4404

0.00

4

289.4353

0.00

5

289.4364

0.00

6

289.4323

0.00

∫

Problem – Using equations #22 and #23 and A=-2, solve for N(1).

Using equation #23, we can express N(1) as:

The first integral can be evaluated by applying equation #22 to give:

This integral can be solved using the extended midpoint rule:

f(x) dx = h[f(m0)+f(m1)+. . .]

evaluated from q to r, where m0, m1, etc. are the midpoints between the interval [q r].

The first midpoint is h/2 away from q, the second is 3h/2 away, the next is 5h/2

away and so on towards the direction of r. Hence, if we choose h=1/8, then:

midpoint

f(m)

-0.4375

0.3833

-0.3125

0.0612

-0.1875

0.0000

-0.0625

0.0000

total

0.4445

Therefore, the first integral is evaluated as:

I1 ≅ [1/(2π)1/2] * h * 0.4445 ≅ 0.0222

The second integral can be evaluated using Simpson’s 1/3 rule (h=0.5) to give:

segment

I

-2.0000

-1.5000

-1.0000

0.3401

-1.0000

-0.5000

0

0.8561

0

0.5000

1.0000

0.8561

total

2.0523

I2 ≅ [1/(2π)1/2] * 2.0523 ≅ 0.8187

Hence,

N(1) ≅ 0.0222 + 0.8187 ≅ 0.8409

Drill: Solve for N(1) using h=1/16 for the first integral and h=0.25 for the

second.

Answer:

midpoint

f(m)

-0.4688

0.4676

-0.4063

0.2929

-0.3438

0.1230

-0.2813

0.0227

-0.2188

0.0006

-0.1563

0.0000

-0.0938

0.0000

-0.0313

0.0000

total

0.9068

I1 ≅ 0.0226

segment

I

-2.0000

-1.7500

-1.5000

0.1104

-1.5000

-1.2500

-1.0000

0.2302

-1.0000

-0.7500

-0.5000

0.3757

-0.5000

-0.2500

0.0000

0.4800

0

0.2500

0.5000

0.4800

0.5000

0.7500

1.0000

0.3757

total

2.0519

I2 ≅ 0.8186

N(1) ≅ 0.8412

Chapter 3 – Numerical Differentiation

Taylor’s Theorem states that if the function f and its first n+1 derivatives are

continuous on an interval containing a and x, then the value of the function at x is given by:

f(x) = f(a) (x-a)0 + f’(a) (x-a)1 + f’’(a) (x-a)2 + . . .

0! 1! 2!

+ f(n)(a) (x-a)n + Rn

n!

The Taylor’s Series expansion is obtained by replacing x with xi+1 and a with xi:

f(xi+1) = f(xi)(xi+1 - xi)0 + f’(xi)(xi+1 - xi)

1 + . . .

0! 1!

+ f(n)(xi) (xi+1 - xi)

n + Rn

n!

This theorem and its associated formula, the Taylor’s Series, is of great value in the

study of numerical methods. In essence, the Taylor series provides a means to predict a

function value at one point in terms of the function value and its derivatives at another point.

First Forward, Backward and Central Divided Difference Approximations

It is convenient to simplify the series by defining a step size h = xi+1 - xi. Then

the Taylor’s Series expansion will become:

f(xi+1) = f(xi) h

0 + f’(xi) h1 + . . . + f(n)(xi) h

n + Rn

0! 1! n!

Let us truncate the series after the first derivative term (i.e., n=1):

f(xi+1) = f(xi) h

0 + f’(xi) h1 + R1

0! 1!

f(xi+1) = f(xi) + f’(xi) h + R1

This can now be solved for the derivative in the form of a finite divided difference:

f’(xi) = f(xi+1) - f(xi) - R1

h h

The first part of the equation gives us the formula to approximate the first derivative

while the second part gives us the formula to compute the truncation error associated with the

approximation of the first derivative. Hence, the first forward divided difference

approximation is given by:

f’(xi) ≅ f(xi+1) - f(xi) eqn. 1

h

Similarly, the first backward divided difference approximation is given by:

f’(xi) ≅ f(xi) - f(xi-1) eqn. 2

h

Lastly, the first central divided difference approximation is given by:

f’(xi) ≅ f(xi+1) - f(xi-1) eqn. 3

2h

Let us now use the first forward, backward and central divided difference equations to

estimate the first derivative of:

f(x) = -0.1x4 - 0.15x3 - 0.5x2 - 0.25x + 1.2

at x=0.5 using a step size h=0.5. The results are tabulated below:

derivative

error %

forward

-1.4500

58.90

backward

-0.5500

39.73

central

-1.0000

9.59

Drill: What are the effects on the results if the step size is halved?

Answer: The results are tabulated below:

derivative

error %

forward

-1.1547

26.54 (halved)

backward

-0.7141

21.75 (halved)

central

-0.9344

2.40 (quartered)

Second Forward, Backward and Central Divided Difference Approximations

The Taylor’s Series expansion can be written up to the second derivative:

f(xi+1) = f(xi) h0 + f’(xi) h

1 + f’’(xi) h2 + R2

0! 1! 2!

f(xi+1) = f(xi) + f’(xi) h + 0.5 f’’(xi) h2 eqn. 4

Also, it can be written for f(xi+2) in terms of f(xi) up to the second derivative:

f(xi+2) = f(xi) + f’(xi) 2h + f’’(xi) (2h)2 eqn. 5

2

Equation #4 can be substituted into equation #5 to yield the second forward divided

difference approximation:

f’’(xi) = f(xi) - 2f(xi+1) + f(xi+2) eqn. 6

h2

Similar manipulation can be employed to derive the second backward divided

difference approximation:

f’’(xi) = f(xi) - 2f(xi-1) + f(xi-2) eqn. 7

h2

Lastly, we have the second central divided difference approximation:

f’’(xi) = f(xi+1) - 2f(xi) + f(xi-1) eqn. 8

h2

Let us now use the second forward, backward and central divided difference

equations to estimate the second derivative of:

f(x) = -0.1x4 - 0.15x3 - 0.5x2 - 0.25x + 1.2

at x=0.5 using a step size h=0.5. The results are tabulated below:

derivative

error %

forward

-3.1500

80.00

backward

-1.0500

40.00

central

-1.8000

2.86

Drill: What are the effects on the results if the step size is halved?

Answer: The results are tabulated below:

derivative

error %

forward

-2.3625

35.00 (halved)

backward

-1.3125

25.00 (halved)

central

-1.7625

0.71 (quartered)

Richardson Extrapolation

Richardson extrapolation is another way to improve derivative estimates when

employing finite divided differences aside from decreasing the step size. The method uses

two derivative estimates to compute a third, more accurate approximation. From Chapter 2

(Integration of Equations) above, equation #3, the Richardson extrapolation formula for

integration, is reproduced below:

I ≅ I(h2) + I(h2) - I(h1)

[(h1/h2)2 – 1]

In a similar fashion, we can write the Richardson extrapolation formula for

derivatives as:

D ≅ D(h2) + D(h2) - D(h1) eqn. 9

[(h1/h2)2 – 1]

Let us now use Richardson extrapolation to estimate the first derivative of:

f(x) = -0.1x4 - 0.15x3 - 0.5x2 - 0.25x + 1.2

at x=0.5 using step sizes h1=0.5 and h2=0.25. Recall from previous results that:

h1

h2

D(h1)

error %

D(h2)

error %

forward

-1.4500

58.90

-1.1547

26.54

backward

-0.5500

39.73

-0.7141

21.75

central

-1.0000

9.59

-0.9344

2.40

Using equation #9, we have:

D

error %

forward

-1.0562

15.75

backward

-0.7688

15.75

central

-0.9125

0

Observe that in the present case Richardson extrapolation on the central divided

difference approximations yields a perfect estimate.

Drill: Use Richardson extrapolation to estimate the second derivative of the same

function at x=0.5 using step sizes h1=0.5 and h2=0.25.

Answer: The results are tabulated below.

Recall from previous results that:

h1

h2

D(h1)

error %

D(h2)

error %

forward

-3.1500

80.00

-2.3625

35.00

backward

-1.0500

40.00

-1.3125

25.00

central

-1.8000

2.86

-1.7625

0.71

And so we have:

D

error %

forward

-2.1000

20.00

backward

-1.4000

20.00

central

-1.7500

0

Romberg Algorithm

As with integration, Chapter 2 (Integration of Equations) above, the Richardson

extrapolation can be applied iteratively using the Romberg algorithm until the result falls

below an acceptable error criterion.

Let us now use the Romberg algorithm to estimate the first derivative of the same

function at x=0.5 using step sizes 1.0, 0.5 and 0.25. The results are tabulated below:

forward backward central

h D error D error D error

1.00 -2.2375 145.2 -0.2875 68.5 -1.2625 38.4

0.50 -1.4500 58.9 -0.5500 39.7 -1.0000 9.6

0.25 -1.1547 26.5 -0.7141 21.8 -0.9344 2.4

1.00

0.50

-1.1875

30.1

-0.6375

30.1

-0.9125

0

0.50

0.25

-1.0562

15.8

-0.7688

15.8

-0.9125

0

1.00

0.25

-1.0475

14.8

-0.7775

14.8

-0.9125

0

Drill: Use the Romberg algorithm to estimate the second derivative of the same

function at x=0.5 using step sizes 1.0, 0.5 and 0.25.

Answer:

forward backward central

h D error D error D error

1.00 -5.2500 200.0 -1.0500 40.0 -1.9500 11.4

0.50 -3.1500 80.0 -1.0500 40.0 -1.8000 2.9

0.25 -2.3625 35.0 -1.3125 25.0 -1.7625 0.7

1.00

0.50

-2.4500

40.0

-1.0500

40.0

-1.7500

0

0.50

0.25

-2.1000

20.0

-1.4000

20.0

-1.7500

0

1.00

0.25

-2.0767

18.7

-1.4233

18.7

-1.7500

0

Part V – Ordinary Differential

Equations

A solution of an ordinary differential equation is a specific function of the

independent variable and parameters that satisfies the original differential equation.

Chapter 1 – Euler’s Method

This chapter is devoted to solving ordinary differential equations of the form:

dy/dx = f(x,y)

An example of a differential equation is:

dv/dt = g -(cv/m) eqn. 1

which is a model that relates the acceleration of a falling object, such as a parachutist, to the

forces acting on it. The parameter g is the gravitational constant, c is the drag coefficient and

m is the mass. Equation #1 is a differential equation because it is written in terms of the

differential rate of change (dv/dt) of the variable that we are interested in predicting.

However, the exact solution of this equation for the velocity of the falling parachutist cannot

be obtained using simple algebraic manipulation. Rather, more advanced techniques must be

applied to obtain an exact or analytical solution. For example, if the falling parachutist is

initially at rest (v=0 at t=0), calculus can be used to solve the equation for:

v(t) = (gm/c) [1 – e-ct/m

] eqn. 2

Equation #2 is called an analytical or exact solution because it exactly satisfies the

original differential equation. Unfortunately, there are many mathematical models that

cannot be solved exactly. In many cases, the only alternative is to develop a numerical

solution that approximates the exact solution.

Numerical methods are those in which the mathematical problem is reformulated so it

can be solved by arithmetic operations. This can be illustrated for Newton’s second law by

realizing that the time rate of change of velocity can be approximated by:

dv/dt = ∆v/∆t = v(ti+1) – v(ti) eqn. 3

ti+1 – ti

Equation #3 is called a finite divided difference approximation of the derivative which

can be substituted into equation #1 to give:

v(ti+1) – v(ti) = g – c v(ti)

ti+1 – ti m

This equation can then be rearranged to yield:

v(ti+1) = v(ti) + [g–(c/m)v(ti)] (ti+1–ti) eqn. 4

Euler’s Method

This method uses the differential equation to estimate the slope in the form of the first

derivative. It uses the slope at the beginning of the interval as an approximation of the

average slope over the whole interval.

Notice in equation #4 that the term in square brackets, which is the slope, is the right-

hand side of the differential equation itself (equation #1). If we are given an initial value for

velocity at some time ti, we can easily compute velocity at a later time ti+1. This new

value of velocity can in turn be employed to extend the computation to velocity at some other

time.

Equation #4 can be expressed in general terms as:

yi+1 = yi + f(xi,yi) h eqn. 5

This formula is referred to as Euler’s method. A new value of y is predicted using the

slope f(xi,yi) (equal to the first derivative at the original value of x) to extrapolate linearly

over the step size h.

Let us now compute the velocity of the falling parachutist prior to opening the chute.

Suppose that the parachutist has a mass of 68.1 kg and that the drag coefficient is 12.5

kg/s. Let us employ a step size of 2 s. Some of the results are tabulated below.

t

v

00 00.0000

02 19.6000

04 32.0047

06 39.8555

08 44.8243

10 47.9690

12 49.9592

Drill: After a sufficiently long time, a constant velocity, called the terminal velocity,

is reached. This velocity is constant because, eventually, the force of gravity will be in

balance with the air resistance. Thus, the net force is zero and acceleration has ceased.

Compute this constant.

Answer: 53.3904 m/s

Drill: Use Euler’s method (equation #5) to numerically integrate the equation given

below from 0 to 4 with a step size of 0.5. The initial condition at x=0 is

y=1.

dy/dx = -2x3 + 12x2 - 20x + 8.5

Answer: The analytical solution is y = -0.5x4 + 4x3 - 10x2 + 8.5x + 1

x y euler y true error %

0.0 1.0000 1.0000 000.0

0.5 3.2188

slope = f(0,1) = [-2(0)3 + 12(0)2 – 20(0) + 8.5]

slope = f(0,1) = 8.5

y(0.5) = y(0) + f(0,1)0.5 = 1 + (8.5)0.5 = 5.25


0.0 1.0000 1.0000 000.0

0.5 5.2500 3.2188 063.1


0.0 1.0000 1.0000 000.0

0.5 5.2500 3.2188 063.1

1.0 5.8750 3.0000 095.8

1.5 5.1250 2.2188 131.0

2.0 4.5000 2.0000 125.0

2.5 4.7500 2.7188 074.7

3.0 5.8750 4.0000 046.9

3.5 7.1250 4.7188 051.0

4.0 7.0000 3.0000 133.3

Chapter 2 – Runge-Kutta Methods

A fundamental source of error in Euler’s method is that the derivative at the beginning

of the interval is assumed to apply across the entire interval. The Runge-Kutta methods are

available to help circumvent this shortcoming. These methods achieve the accuracy of a

Taylor series approach without requiring the calculation of higher derivatives.

Heun’s Method

This method is designed to improve the estimate of the slope. It involves the

determination of two derivatives for the interval – one at the initial point and another at the

end point. The two derivatives are then averaged to obtain an improved estimate of the slope

for the entire interval.

Recall that in Euler’s method, the slope at the beginning of an interval:

y’i = f(xi,yi) eqn. 6

is used to extrapolate linearly to yi+1:

y0i+1 = yi + f(xi,yi) h eqn. 7

For the standard Euler’s method we would stop at this point. However, in Heun’s method,

equation #7 gives us not the final answer but only an intermediate prediction. This is why we

have distinguished it with a superscripted 0. Equation #7 is called a predictor equation. It

provides an estimate of yi+1 that allows the calculation of an estimated slope at the end of the

interval:

y’i+1 = f(xi+1,y0i+1) eqn. 8

Thus, the two slopes, equations #6 and #8, can be combined to obtain an average

slope m for the interval. This average slope is then used to extrapolate linearly from yi to

yi+1 using Euler’s method:

yi+1 = yi + m h eqn. 9

Equation #9 is called a corrector equation. Hence, Heun’s method is a predictor-corrector

approach.

Let us now use Heun’s method to integrate the equation given below from 0 to 4 with

a step size of 1. The initial condition at x=0 is y=2.

y’ = 4e0.8x

– 0.5y

Before solving the problem numerically, calculus can be used to determine the

following analytical solution (you should be able to derive this):

y = (4/1.3)[ e0.8x

- e-0.5x

] + 2e-0.5x

Now, the numerical solution:

x y heun y true error %

0 2.0000 2.0000 00.0

1 6.1946

From equation #7:

y01 = y0 + f(x0,y0) h = 2 + f(0,2) (1)

y01 = 2 + [4e0.8(0)

– 0.5(2)](1) = 5

From equation #6:

y’0 = f(x0,y0) = f(0,2) = 4e0.8(0)

– 0.5(2) = 3

From equation #8:

y’1 = f(x1,y01)= f(1,5) = 4e

0.8(1) – 0.5(5) = 6.4022

Hence:

m = (3+6.4022)/2 = 4.7011

Finally, from equation #9:

y1 = y0 + m h = 2 + (4.7011)(1) = 6.7011


0 02.0000 02.0000 00.0

1 06.7011 06.1946 08.2


0 02.0000 02.0000 00.0

1 06.7011 06.1946 08.2

2 16.3198 14.8439 09.9

3 37.1992 33.6772 10.5

4 83.3378 75.3390 10.6

Notice how the error sometimes grows as the iterations proceed. Such increase can

occur, especially for large step sizes, and they prevent us from drawing the general

conclusion that an additional iteration will always improve the result. However, for a

sufficiently small step size, the iterations should eventually converge on a single value.

Drill: Use Heun’s method to solve the previous drill.

Answer:


0.0 1.0000 1.0000 00.0

0.5 3.4375 3.2188 06.8

1.0 3.3750 3.0000 12.5

1.5 2.6875 2.2188 21.1

2.0 2.5000 2.0000 25.0

2.5 3.1875 2.7188 17.2

3.0 4.3750 4.0000 09.4

3.5 4.9375 4.7188 04.6

4.0 3.0000 3.0000 00.0

Improved Polygon Method

This technique uses Euler’s method to predict a value of y at the midpoint of the

interval:

yi+1/2 = yi + f(xi,yi) h/2 eqn. 10

Then this predicted value is used to estimate a slope at the midpoint:

y’i+1/2 = f(xi+1/2,yi+1/2) eqn. 11

which is assumed to represent a valid approximation of the average slope for the entire

interval. This slope is then used to extrapolate linearly from xi to xi+1 using Euler’s method:

yi+1 = yi + f(xi+1/2,yi+1/2) h eqn. 12

Notice that because yi+1 is not on both sides, the corrector, equation #12, cannot be applied

iteratively to improve the solution. This method is superior to Euler’s method because it

utilizes a slope estimate at the midpoint of the prediction interval.

Let us now use the improved polygon method to solve the previous drill.

x y IPM y true error %

0.0 1.0000 1.0000 00.0

0.5 3.2188

Using equation #10:

yi+1/2 = y0 + f(x0,y0) h/2 = 1 + f(0,1) h/2

= 1 + [-2(0)3 + 12(0)2 – 20(0) + 8.5] (0.25)

= 3.1250

Using equation #11:

y’i+1/2 = f(xi+1/2,yi+1/2) = f(0.25,3.1250)

= -2(0.25)3 + 12(0.25)2 – 20(0.25) + 8.5

= 4.2188

Using equation #12:

yi+1 = 1 + (4.2188) 0.5 = 3.1094


0.0 1.0000 1.0000 00.0

0.5 3.1094 3.2188 03.4


0.0 1.0000 1.0000 00.0

0.5 3.1094 3.2188 03.4

1.0 2.8125 3.0000 06.3

1.5 1.9844 2.2188 10.6

2.0 1.7500 2.0000 12.5

2.5 2.4844 2.7188 08.6

3.0 3.8125 4.0000 04.7

3.5 4.6094 4.7188 02.3

4.0 3.0000 3.0000 00.0

Ralston’s Method

All the methods discussed above and many other variations of the Runge-Kutta

methods can be cast in the generalized form of equation #5:

yi+1 = yi + φ(xi,yi,h) h eqn. 13

where φ(xi,yi,h) is called an increment function which can be interpreted as a

representative slope over the interval. The increment function can be written in general form

as:

φ = a1k1 + a2k2 + . . . + ankn eqn. 14

where n is the order, the a’s are constants and the k’s are:

k1 = f(xi,yi)

k2 = f(xi+p1h,yi+q11k1h)

k3 = f(xi+p2h,yi+q21k1h+q22k2h) . . . and so on.

Notice that the k’s are recurrence relationships. That is, k1 appears in the equation

for k2, which appears in the equation for k3, and so forth. This recurrence makes Runge-

Kutta methods efficient for computer calculations.

Ralston determined that choosing a2=2/3 provides a minimum bound on the

truncation error for the second-order Runge-Kutta algorithms. For this version, a1=1/3 and

p1=q11=3/4. Hence, from equations #13 and #14, we have:

yi+1 = yi + [(1/3)k1 + (2/3)k2] h eqn. 15

k1 = f[xi,yi] eqn. 16

k2 = f[xi+(3/4)h,yi+(3/4)hk1] eqn. 17

Let us now use Ralston’s method to solve the previous drill.

x y ralston y true error %

0.0 1.0000 1.0000 0.00

0.5 3.2188

Using equation #16:

k1 = f(x0,y0) = -2(0)3 + 12(0)2 – 20(0) + 8.5 = 8.5

Using equation #17:

k2 = f(0.3750,4.1875)

= -2(0.3750)3 + 12(0.3750)2 – 20(0.3750) + 8.5 = 2.5820

Using equation #15:

yi+1 = 1 + [2.8333 + 1.7213] (0.5) = 3.2773


0.0 1.0000 1.0000 0.00

0.5 3.2773 3.2188 1.82


0.0 1.0000 1.0000 0.00

0.5 3.2773 3.2188 1.82

1.0 3.1016 3.0000 3.39

1.5 2.3477 2.2188 5.81

2.0 2.1406 2.0000 7.03

2.5 2.8555 2.7188 5.03

3.0 4.1172 4.0000 2.93

3.5 4.8008 4.7188 1.74

4.0 3.0313 3.0000 1.04

Notice how all the second-order Runge-Kutta methods are superior to Euler’s method,

which is actually a first-order Runge-Kutta method.

Drill: Use the following methods to integrate the equation given below from 0 to 4

with a step size of 1. The initial condition at x=0 is y=2.

y’ = 4e0.8x

– 0.5y

1. Euler’s method

2. Improved Polygon method

3. Ralston’s method

Answer: The results are summarized below. The previous results on Heun’s method

are reproduced for purpose of comparison.

Systems of Equations

Many practical problems in engineering and science require the solution of a system

of simultaneous differential equations rather than a single equation. Such system may be

represented generally as:

dy1/dx = f1(x,y1,y2,...,yn)

dy2/dx = f2(x,y1,y2,...,yn)

.

.

.

dyn/dx = fn(x,y1,y2,...,yn)

The solution of such a system requires that n initial conditions be known at the

starting value of x. The procedure for solving a system simply involves applying the one-

step technique for every equation at each step before proceeding to the next step.

Let us solve the following set of differential equations using Euler’s method assuming

that at x=0, y1=4 and y2=6. Integrate to x=2 with a step size of 0.5.

dy1/dx = -0.5y1

dy2/dx = 4 - 0.3y2 – 0.1y1

x euler e heun e IPM e ralston e

0 02.0000 00 02.0000 00 02.0000 0.0 02.0000 0.0

1 05.0000 19 06.7011 08 06.2173 0.4 06.4423 4.0

2 11.4022 23 16.3198 10 14.9407 0.7 15.5822 5.0

3 25.5132 24 37.1992 11 33.9412 0.8 35.4566 5.3

4 56.8493 25 83.3378 11 75.9686 0.8 79.3962 5.4

Using equation #5:

y1(0.5) = 4 + f1(0,4)(0.5) = 4 + [(-0.5)(4)](0.5)

y1(0.5) = 3

y2(0.5) = 6 + f2(0,6)(0.5) = 6 + [4-0.3(6)–0.1(4)]0.5

y2(0.5) = 6.9000

Notice that in solving for the new value of y2, the old values of y1 and y2 are used.

All the methods previously discussed in this Part for single equations can be extended

to the system shown above.

Drill: Use Heun’s, improved polygon and Ralston’s methods to solve the same set of

differential equations:

Answer:

Notice that, in this particular case, the results under the three methods are identical.

<end>

Reference: Numerical Methods for Engineers

2nd

ed., S. Chapra and R. Canale

McGraw-Hill Book Company 1990

x y1 y2

0.0 4.0000 6.0000

0.5

x y1 y2

0.0 4.0000 6.0000

0.5 3.0000 6.9000

x y1 y2

0.0 4.0000 6.0000

0.5 3.0000 6.9000

1.0 2.2500 7.7150

1.5 1.6875 8.4453

2.0 1.2656 9.0941

heun IPM ralston

x y1 y2 y1 y2 y1 y2

0.0 4.0000 6.0000 4.0000 6.0000 4.0000 6.0000

0.5 3.1250 6.8325 3.1250 6.8325 3.1250 6.8325

1.0 2.4414 7.5900 2.4414 7.5900 2.4414 7.5900

1.5 1.9073 8.2739 1.9073 8.2739 1.9073 8.2739

2.0 1.4901 8.8877 1.4901 8.8877 1.4901 8.8877

Date post:	07-Oct-2014
Category:	Documents
Upload:	jeremy-borres
View:	77 times
Download:	3 times

Numerical(Lecture Notes)

Documents