SE301: Numerical MethodsTopic 1:

Introduction to Numerical methods and Taylor Series

Lectures 1-4:

SE301_Topic1 (c) AL-AMER2006 ١

Dr. Samir Al-Amer(Term 071)

Lecture 1Introduction to Numerical Methods

SE301_Topic1 (c) AL-AMER2006 ٢

What are NUMERICAL METHODS?Why do we need them?Topics covered in SE301.

Reading Assignment: pages 3-10 of text book

SE301_Topic1 (c) AL-AMER2006 ٣

Numerical MethodsNumerical Methods:

Algorithms that are used to obtain numerical solutions of a mathematical problem.

Why do we need them?1. No analytical solution exists,2. An analytical solution is difficult to obtain

or not practical.

SE301_Topic1 (c) AL-AMER2006 ٤

What do we needBasic Needs in the Numerical Methods:

Practical: can be computed in a reasonable amount of time.

Accurate: Good approximate to the true valueInformation about the approximation error (Bounds, error order,… )

SE301_Topic1 (c) AL-AMER2006 ٥

Outlines of the CourseTaylor TheoremNumber RepresentationSolution of nonlinear EquationsInterpolationNumerical DifferentiationNumerical Integration

Solution of linear EquationsLeast Squares curve fittingSolution of ordinary differential equationsSolution of Partial differential equations

SE301_Topic1 (c) AL-AMER2006 ٦

Solution of Nonlinear Equations

Some simple equations can be solved analytically

Many other equations have no analytical solution31

)1(2)3)(1(444

solution Analytic

0342

2

−=−=

−±−=

=++

xandx

roots

xx

solution analytic No052 29

⎪⎭

⎪⎬⎫

==+−

− xexxx

SE301_Topic1 (c) AL-AMER2006 ٧

Methods for solving Nonlinear Equations

o Bisection Methodo Newton-Raphson Methodo Secant Method

SE301_Topic1 (c) AL-AMER2006 ٨

Solution of Systems ofLinear Equations

unknowns 1000 in equations 1000 have weif do to What

,,

as it solve can We

12325233

523

12

2221

21

21

=−==⇒=+−−=

=+=+

xxxxxx

xxxx

SE301_Topic1 (c) AL-AMER2006 ٩

Cramer’s Rule is not practical

compute. toyears 10 than more needscomputer super A

needed. are tionsmultiplica102.3 system, 30by 30 a solve To

tions.multiplica 1)N!1)(N(N need weunknowns Nin equations N solve To

problems. largefor practicalnot is Rule sCramer'But

2

21115131

,1

21112513

system thesolve toused becan Rule sCramer'

20

35

21

×

−+

==== xx

SE301_Topic1 (c) AL-AMER2006 ١٠

Methods for solving Systems of Linear Equations

o Naive Gaussian Eliminationo Gaussian Elimination with Scaled

Partial pivotingo Algorithm for Tri-diagonal

Equations

SE301_Topic1 (c) AL-AMER2006 ١١

Curve FittingGiven a set of data

Select a curve that best fit the data. One choice is find the curve so that the sum of the square of the error is minimized.

x 0 1 2 y 0.5 10.3 21.3

SE301_Topic1 (c) AL-AMER2006 ١٢

InterpolationGiven a set of data

find a polynomial P(x) whose graph passes through all tabulated points.

xi 0 1 2 yi 0.5 10.3 15.3

tablein the is)( iii xifxPy =

SE301_Topic1 (c) AL-AMER2006 ١٣

Methods for Curve Fitting o Least Squares

o Linear Regressiono Nonlinear least Squares Problems

o Interpolationo Newton polynomial interpolationo Lagrange interpolation

SE301_Topic1 (c) AL-AMER2006 ١٤

IntegrationSome functions can be integrated analytically

?

solutions analytical no have functionsmany But

=

=−==

∫

∫

− dxe

xxdx

ax

0

3

1

23

1

2

421

29

21

SE301_Topic1 (c) AL-AMER2006 ١٥

Methods for Numerical Integration

o Upper and Lower Sumso Trapezoid Methodo Romberg Methodo Gauss Quadrature

SE301_Topic1 (c) AL-AMER2006 ١٦

Solution of Ordinary Differential Equations

only cases special for available are solutions Analytical *

equations thesatisfies that function a is

0)0(;1)0(0)(3)(3)(

equation aldifferenti theosolution tA

x(t)xx

txtxtx==

=++&

&&&

SE301_Topic1 (c) AL-AMER2006 ١٧

Solution of Partial Differential EquationsPartial Differential Equations are more

difficult to solve than ordinary differential equations

)sin()0,(,0),1(),0(

022

2

2

2

xxutututu

xu

π===

=+∂∂

+∂∂

SE301_Topic1 (c) AL-AMER2006 ١٨

SummaryNumerical Methods:Algorithms that are used to obtain numerical solution of a mathematical problem.We need them whenNo analytical solution exist or it is difficult to obtain.

Solution of nonlinear EquationsSolution of linear EquationsCurve fitting

Least SquaresInterpolation

Numerical IntegrationNumerical Differentiation Solution of ordinary differential equationsSolution of Partial differential equations

Topics Covered in the Course

Lecture 2

Number Representation and accurcy

SE301_Topic1 (c) AL-AMER2006 ١٩

Number RepresentationNormalized Floating Point RepresentationSignificant DigitsAccuracy and Precision Rounding and Chopping

Reading assignment: Chapter 3

SE301_Topic1 (c) AL-AMER2006 ٢٠

Representing Real NumbersYou are familiar with the decimal system

Decimal System Base =10 , Digits(0,1,…9)Standard Representations

21012 10510410210110345.312 −− ×+×+×+×+×=

part part fraction integralsign

54.213±

SE301_Topic1 (c) AL-AMER2006 ٢١

Normalized Floating Point Representation

Normalized Floating Point Representation

No integral part,

Advantage Efficient in representing very small or very large numbers

integer:,0

exponent mantissasign

10.0

1

4321

nd

dddd n

≠

×±

SE301_Topic1 (c) AL-AMER2006 ٢٢

Calculator Examplesuppose you want to compute

3.578 * 2.139using a calculator with two-digit fractions

3.57 2.13 7.60* =

7.653342True answer

SE301_Topic1 (c) AL-AMER2006 ٢٣

Binary System

Binary System Base=2, Digits{0,1}

exponent mantissasign

21.0 432nbbb ×±

10 11 =⇒≠ bb

1010321

2 )625.0()212021()101.0( =×+×+×= −−−

SE301_Topic1 (c) AL-AMER2006 ٢٤

7-Bit Representation(sign: 1 bit, Mantissa 3bits,exponent 3 bits)

SE301_Topic1 (c) AL-AMER2006 ٢٥

FactNumber that have finite expansion in one numbering system may have an infinite expansion in another numbering system

You can never represent 0.1 exactly in any computer

210 ...)011000001100110.0()1.0( =

SE301_Topic1 (c) AL-AMER2006 ٢٦

Representation

Hypothetical Machine (real computers use ≥ 23 bit mantissa)

Mantissa 2 bits exponent 2 bit sign 1 bit

Possible machine numbers

.25 .3125 .375 .4375 .5 .625 .75 .875

1 1.25 1.5 1.75

SE301_Topic1 (c) AL-AMER2006 ٢٧

Representation

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Gap near zero

SE301_Topic1 (c) AL-AMER2006 ٢٨

Remarks

Numbers that can be exactly represented are called machine numbersDifference between machine numbers is not uniformsum of machine numbers is not necessarily a machine

number0.25 + .3125 =0.5625 (not a machine number)

SE301_Topic1 (c) AL-AMER2006 ٢٩

Significant Digits

Significant digits are those digits that can be used with confidence.

SE301_Topic1 (c) AL-AMER2006 ٣٠

Accuracy and Precision

Accuracy is related to closeness to the true value

Precision is related to the closeness to other estimated values

SE301_Topic1 (c) AL-AMER2006 ٣١

Rounding and Chopping

Rounding: Replace the number by the nearest machine number

Chopping: Throw all extra digits

SE301_Topic1 (c) AL-AMER2006 ٣٢

Error DefinitionsTrue Error

can be computed if the true value is known

100* valuetrue

ionapproximat valuetrueError RelativePercent Absolute

ionapproximat valuetrueError True Absolute

t−

=

−=

ε

tE

SE301_Topic1 (c) AL-AMER2006 ٣٣

Error DefinitionsEstimated error

When the true value is not known

100*estimatecurrent

estimate prevoius estimatecurrent Error RelativePercent Absolute Estimated

estimate prevoius estimatecurrent Error Absolute Estimated

−=

−=

a

aE

ε

SE301_Topic1 (c) AL-AMER2006 ٣٤

Notation

We say the estimate is correct to n decimal digits if

We say the estimate is correct to n decimal digits rounded if

n−≤10Error

n−×≤ 1021Error

SE301_Topic1 (c) AL-AMER2006 ٣٥

SummaryNumber Representation

Number that have finite expansion in one numbering system may

have an infinite expansion in another numbering system.

Normalized Floating Point RepresentationEfficient in representing very small or very large numbersDifference between machine numbers is not uniformRepresentation error depends on the number of bits used in the mantissa.

Lectures 3-4

Taylor Theorem

SE301_Topic1 (c) AL-AMER2006 ٣٦

MotivationTaylor Theorem Examples

Reading assignment: Chapter 4

SE301_Topic1 (c) AL-AMER2006 ٣٧

Motivation

We can easily compute expressions like

?)6.0sin(,4.1 computeyou do HowBut,

)4(2103 2

+×x

way?practical a thisis?)6.0sin(

compute todefinition theusecan We

0.6

ab

SE301_Topic1 (c) AL-AMER2006 ٣٨

Taylor Series

∑∞

000

)(

000

)(

30

0)3(

20

0)2(

00'

0

0

)()(!

1)(

can write weconverge series theif

)()(!

1

...)(!3

)()(!2

)()()()(

about )( ofexpansion seriesTaylor The

=

∞

=

−=

−=

+−+−+−+

∑

k

kk

k

kk

xxxfk

xf

xxxfk

SeriesTaylor

or

xxxfxxxfxxxfxf

xxf

SE301_Topic1 (c) AL-AMER2006 ٣٩

Taylor SeriesExample 1

∞xfor converges series The!

)()(!

111)0()(

1)0()(1)0(')('1)0()(

∑∑∞

0

∞

000

)(

)()(

)2()2(

<

=−=

≥==

==

==

==

== k

k

k

kkx

kxk

x

x

x

kxxxxf

ke

kforfexffexffexffexf

0=about =)( ofexpansion seriesTaylor Obtain xexf x

SE301_Topic1 (c) AL-AMER2006 ٤٠

Taylor SeriesExample 1

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 10

0.5

1

1.5

2

2.5

3

1

1+x

1+x+0.5x2

exp(x)

SE301_Topic1 (c) AL-AMER2006 ٤١

Taylor SeriesExample 2

∞xfor converges series The

....!7!5!3

)(!

)()sin(

1)0()cos()(

0)0()sin()(

1)0(')cos()('0)0()sin()(

753∞

00

0)(

)3()3(

)2()2(

∑<

+−+−=−=

−=−=

=−=

====

=

xxxxxxkxfx

fxxf

fxxf

fxxffxxf

k

kk

0about )sin()( ofexpansion seriesTaylor Obtain == xxxf

SE301_Topic1 (c) AL-AMER2006 ٤٢

-4 -3 -2 -1 0 1 2 3 4-4

-3

-2

-1

0

1

2

3

4

x

x-x3/3!

x-x3/3!+x5/5!

sin(x)

SE301_Topic1 (c) AL-AMER2006 ٤٣

Convergence of Taylor Series(Observations, Example 1)

The Taylor series converges fast (few terms are needed) when x is near the point of expansion. If |x-c| is large then more terms are needed to get good approximation.

SE301_Topic1 (c) AL-AMER2006 ٤٤

Taylor SeriesExample 3

( )

( )

( )

....11

1 ofExpansionSeriesTaylor

6)0(1

6)(

2)0(1

2)(

1)0(1

1)('

1)0(1

1)(

01

1f(x) ofexpansion seriesTaylor Obtain

32

4)3(

3)2(

2

++++=−

=−

=

=−

=

=−

=

=−

=

=−

=

xxxx

fx

xf

fx

xf

fx

xf

fx

xf

xaboutx

SE301_Topic1 (c) AL-AMER2006 ٤٥

Example 3remarks

Can we apply Taylor series for x>1??

How many terms are needed to get good approximation???

These questions will be answered using Taylor Theorem

SE301_Topic1 (c) AL-AMER2006 ٤٦

Taylor Theorem

c. andbetween x is)()!1(

)(

where

)(!

)()(

b][a,∈cany for then b][a, interval closed ain 1)(n1,2,..., orders ofsderivative continuous possesf(x)function a If

1)1(

1

1

n

0

)(∑

ξξ andcxnfE

Ecxkcfxf

nn

n

nk

kk

++

+

+=

−+

=

+−=

+

(n+1) terms Truncated Taylor Series

Reminder

SE301_Topic1 (c) AL-AMER2006 ٤٧

Taylor Theorem

.applicablenot is TheoremTaylor defined.not are sderivative

its andfunction then the1],[

1||if0expansion ofpoint with 1

1for remTaylor thoapply can We

⇒

=

<=−

=

xincludesbaif

xcx

f(x)

SE301_Topic1 (c) AL-AMER2006 ٤٨

Error Term

. and between allfor

)()!1(

)(

on boundupper an derivecan We

errorion approximat about the ideaan get To

1)1(

1

cxofvalues

cxnfE nn

n

ξ

ξ ++

+ −+

=

SE301_Topic1 (c) AL-AMER2006 ٤٩

Error Term forExample 4

?2.0=0=about expansion seriesTaylor its of3)=(n terms4first the

by =)( replaced weiferror theis large How

xwhenx

exf x

( ) 0514268.82.0)!1(

)()!1(

)(

1≥≤)()(

41

2.0

1

1)1(

1

2.0)()(

−≤⇒+

≤

−+

=

=

++

++

+

EEneE

cxn

fE

kforefexf

nn

nn

n

kxk

ξ

ξ

SE301_Topic1 (c) AL-AMER2006 ٥٠

Alternative form of Taylor Theorem

hxxwherehnfE

Ehkxfhxf

[a,b]hx[a,b]x[a,b]

xfLet

nn

n

n

n

k

kk

++

=

+=+

∈+∈+

++

+

+=∑

and between is)!1(

)(

!)()(

then and and , intervalan on 1)1,2,...(n orders of sderivative continuous have)(

1)1(

1

10

)(

ξξ

SE301_Topic1 (c) AL-AMER2006 ٥١

Taylor TheoremAlternative forms

hxxwhere

hnfh

kxfhxf

xchxx

cxwhere

cxnfcx

kcfxf

nnn

k

kk

nnn

k

kk

+

++=+

→+→

−+

+−=

++

=

++

=

∑

∑

and between is)!1(

)(!

)()(

,

and between is

)()!1(

)()(!

)()(

1)1(

0

)(

1)1(

0

)(

ξ

ξ

ξ

ξ

SE301_Topic1 (c) AL-AMER2006 ٥٢

Mean Value Theorem

(b-a)dxdf(ξf(a)f(b)

bhxax(b-a)

f(a)f(b)dxdf(ξ

baξ

) ,0,n TheoremTaylor Use:Proof

)],[exist then there

b)(a, intervalopen on the defined is derivative its andb][a, interval closed aon function continuous a isf(x) If

+=

=+==

−=

∈

SE301_Topic1 (c) AL-AMER2006 ٥٣

Alternating Series Theorem

termomittedFirst :n terms)first theof (sum sum partial:

converges series The

0lim

Sseries galternatin heConsider t

1

1

4321

4321

+

+∞→

⎪⎩

⎪⎨

⎧

≤−⎪⎪⎩

⎪⎪⎨

⎧

=

≥≥≥≥

+−+−=

n

n

nnnn

aS

aSSandthen

aand

aaaaIf

aaaa

L

L

SE301_Topic1 (c) AL-AMER2006 ٥٤

Alternating SeriesExample 5

!71

!51

!311)1(s

!51

!311)1(s

0limsince series galternatin convergent a is This

!71

!51

!311)1(susingcomputed becansin(1)

4321

≤⎟⎠⎞

⎜⎝⎛ +−−

≤⎟⎠⎞

⎜⎝⎛ −−

=≥≥≥≥

+−+−=

∞→

in

in

Then

aandaaaa

in

nnL

L

SE301_Topic1 (c) AL-AMER2006 ٥٥

Example 6

? 1with xe eapproximat to

used are terms1)(n when beerror can the largeHowexpansion) ofcenter (the5.0ef(x) of

expansion seriesTaylor theObtain

12x

12x

=

+==

+

+ cabout

SE301_Topic1 (c) AL-AMER2006 ٥٦

Example 6

...!

)5.0(2...!2

)5.0(4)5.0(2

)5.0(!

)5.0(

2)5.0(2)(

4)5.0(4)(

2)5.0('2)('

)5.0()(

22

222

∞

0

)(12

2)(12)(

2)2(12)2(

212

212

∑

+−

++−

+−+=

−=

==

==

==

==

=

+

+

+

+

+

kxexexee

xk

fe

efexf

efexf

efexf

efexf

kk

k

kk

x

kkxkk

x

x

x

5.0,)( ofexpansion seriesTaylor Obtain 12 == + cexf x

SE301_Topic1 (c) AL-AMER2006 ٥٧

Example 6Error term

31

1

12]1,5.0[

11

1121

1)1(

12)(

)!1()5.0(2

max)!1()5.0(2

)!1()5.0(2

)5.0()!1(

)(

2)(

en

xError

en

xError

nxeError

xnfError

exf

nn

nn

nn

nn

xkk

+−

≤

+−

≤

+−

=

−+

=

=

++

+

∈

++

+++

++

+

ξξ

ξ

ξ

SE301_Topic1 (c) AL-AMER2006 ٥٨

RemarkIn this course all angles are assumed to be in radian unless you are told otherwise

SE301_Topic1 (c) AL-AMER2006 ٥٩

Maclurine seriesFind Maclurine series expansion of cos (x)

Maclurine series is a special case of Taylor series with the center of expansion c = 0

SE301_Topic1 (c) AL-AMER2006 ٦٠

Taylor SeriesExample 7

∞xfor converges series The

....!6!4!2

1)(!

)0()cos(

0)0()sin()(1)0()cos()(

0)0(')sin()('1)0()cos()(

642∞

0

)(

)3()3(

)2()2(

∑<

+−+−==

==

−=−=

=−===

=

xxxxk

fx

fxxffxxffxxffxxf

k

kk

)cos()( ofexpansion series MaclurineObtain xxf =

SE301_Topic1 (c) AL-AMER2006 ٦١

Homework problemsCheck the course webCT for the Homework Assignment