A GLIMPSE INTO APPLIED MATHEMATICS UNIVERSITY OF ALBERTA ...irl/IUSEP/lecture_notes/chicone.pdf ·...

A GLIMPSE INTO APPLIED MATHEMATICS

UNIVERSITY OF ALBERTASUMMER 2016

CARMEN CHICONE

1. Mathematics and Applied Mathematics

What is applied mathematics? Every answer to this question is likely to initiatea debate. My definition is the use of mathematics to solve problems or gain insightinto phenomena that arise outside of mathematics. The prototypical example is theuse of mathematics to solve problems in physics. Of course, the world of appliedmathematics is much broader: important applications of mathematics occur in allareas of science, engineering, and technology.

A new book with title “Invitation to Applied Mathematics: Differential Equations,Modeling and Computation” by this author will be available from Elsevier in fall2016. It is intended to help students learn the basic mathematics, modeling, andcomputational skills required to pursue applied mathematics in context, prepare formore advanced training, and approach unsolved applied problems. Perhaps studyingthe material in this book will be helpful in your education.

These notes draw from some of the material presented in the new book and arewritten in the same style and spirit.

Undergraduates interested in applied mathematics should do at least three things:learn how to do and use basic mathematics, learn how write computer programs thatapproximate solutions of mathematical problems, and learn how to do and use basicscience.

The following is an outline of some topics that should be part of your undergrad-uate education.

• Essential (Undergraduate) Pure Math Concepts(1) Calculus and Elementary Algebra(2) Differential Equations(3) Linear Algebra(4) Probability(5) Advanced Calculus

Date: June 14, 2016.1

2 CARMEN CHICONE

(6) Optimization• Essential Numerical Computation Concepts

(1) Approximating Solutions of Linear Equations, Eigenvalues, Null Spaces,Ranges, Orthogonal Bases, and Least Squares.

(2) Approximating Solutions of Nonlinear Equations(3) Approximating Integrals(4) Approximating Derivatives(5) Approximating Solutions of Ordinary Differential Equations(6) Approximating Solutions of Partial Differential Equations

• Essential Science(1) Newton’s Laws of Motion(2) Growth and Decay(3) Vibration(4) Gravity(5) Atomic Theory(6) Electromagnetism(7) Mechanics(8) Conservation of Mass, Momentum, and Energy(9) Fundamental and Constitutive Laws

(10) Something from Astronomy, Chemistry, Biology, Geology, Engineering,Finance, and so on

For our very short course, we will discuss—as time permits—three topics men-tioned in the outline: approximating solutions of nonlinear equations, conservationof mass, and the meaning of fundamental and constitutive laws.

There is nothing more basic in mathematics and applied mathematics than solvingequations. Why nonlinear equations? As long as we don’t go to deeply, there areinteresting, useful , and important aspects of this subject that can be discussed usingconcepts from elementary algebra and calculus. There is also ample opportunityto make interesting and useful computations that require programming computers.Another good reason to start here is that the professor likes this topic!

Some of the basic mathematical models of applied physics arise from conservationof mass and momentum, for example, Fourier’s model of heat flow, mathematicalmodels of fluid mechanics and elasticity, and mathematical models of chemical andbiological processes involving diffusion. Heat flow modeling—the main example wewill discuss—arises from conservation of mass plus Fourier’s constitutive law. Itprovides a beautiful example where fundamental and constitutive laws appear inan important physical context. Also, this example serves to illustrate the basicmodeling process used throughout applied mathematics. Another reason to discuss

A GLIMPSE INTO APPLIED MATHEMATICS UNIVERSITY OF ALBERTA SUMMER 2016 3

a conservation law is simply that doing so is an excellent application of one of themain results from multivariate calculus: the divergence theorem.

There is certainly much more material in these notes than can be covered in 6lectures. The topics not covered might be used for self study or as a source ofproblems for summer projects.

2. Approximating Solutions of Nonlinear Equations

Most of this section is repeated and augmented with interactive computer softwarein the Mathematica notebook NewtonMethodUAlberta.nb, which should now beavailable on the local computer network.

2.1. One equation for one unknown. Certainly one of the most important prob-lems in mathematics and applied mathematics is solving (systems of) equations,which might be linear equations, nonlinear equations, differential equations, and soon. Methods for solving equations and understanding properties of the solutions ofequations lie at the heart of mathematics and applied mathematics.

The simplest case should be familiar: a single equation such as ax = b or x2 +bx+ c = 0, where a, b, and c are supposed to represent given real numbers is given;the problem is to solve for the unknown quantity x. In elementary mathematics, welearn how to solve these equations. Indeed many problems reduce to solving the twotypes just mentioned—a single linear equation or a single quadratic equation. Atleast, these are the most common algebraic equations that are solvable by hand. Thekey fact here is the quadratic formula, which is often used as part of the solution ofmathematical and applied problems.

As you might know, cubic and quartic polynomials can be solved by formulasdiscovered during the Italian renaissance (circa 1530) by Tartaglia. About 300 yearslater, Abel (1824) proved that no such formula exists for solving general quinticpolynomials or general polynomials of higher degree. Moreover, there is a theory(due to Galois circa 1830) that can be used to determine exactly which polynomialequations have roots that can be expressed in formulas using complex numbers andradicals.

By now you would have been introduced to transcendental functions (trig func-tions, exponential functions, logarithms, and perhaps some special functions). Manyequations that involve such functions are not solvable with pencil and paper. For ex-ample, there are no formulas for solving equations such as tanx = 2x, exp(x) = x+2,or log x = 3x + 1. Thus, even for one equation in one real unknown, most often nosolution method is known and the best we can expect is to approximate solutions asclosely as might be desired.

Let us start with polynomial equations. As mentioned, cubic equations can besolved by a formula, which is called Cardano’s formula. At least once in your life, you

4 CARMEN CHICONE

should learn how this formula—a milestone in the history of mathematics—is derived.Here, we simply note that the formula can be programmed on a computer. Indeed, itis implemented in many computer algebra systems, such as Mathematica and Maple.As an example, read the following excerpt from the Mathematica Notebook that youwill be using.

In

ans = Solve[ x^3 - x + 1 == 0, x]

ans // N

Out

x -> -(2/(3 (9 - Sqrt[69])))^(1/3) -

(1/2 (9 - Sqrt[69]))^(1/3)/3^(2/3),

x -> ((1 + I Sqrt[3]) (1/2 (9 - Sqrt[69]))^(1/3))/(2 3^(2/3))

+ (1 - I Sqrt[3])/(2^(2/3) (3 (9 - Sqrt[69]))^(1/3)),

x -> ((1 - I Sqrt[3]) (1/2 (9 - Sqrt[69]))^(1/3))/(2 3^(2/3))

+ (1 + I Sqrt[3])/(2^(2/3) (3 (9 - Sqrt[69]))^(1/3))

x -> -1.32472,

x -> 0.662359 - 0.56228 I, x -> 0.662359 + 0.56228 I

The software is asked to find the roots of the cubic

x3 − x+ 1;

that is, to solve the equation

x3 − x+ 1 = 0.

Note that Mathematica returns a rather complicated formula that involves squareroots, cube roots, and the complex number i :=

√−1 (which is written as capital

I in the Mathematica language). The fundamental theorem of algebra states thata polynomial of degree n has n roots (perhaps some are complex) counting theirmultiplicities. Thus, as expected, Mathematica returns three roots via Cardano’sformula. The next command (given by the double backslash) asks the software forthe numerical values of the roots. One root is real and the remaining two are complex.Can you show by sketching the cubic that there is exactly one real root?

How does the computer extract numerical approximations of the square and cuberoots that appear in the algebraic expressions for the roots of our cubic? This isa good question to start our quest for practical ways of approximating solutions ofmathematical problems.

Consider, for example, cube roots. To extract the cube root of a number a, theusual problem is to find the real root of the cubic equation x3 = a. Two facts are


worth noting: there is a real cube root and there is only one real cube root. Beforetrying to find something, you should know that it exists and is unique.

For definiteness, suppose a = 88. What is its cube root? The answer is somenumber between 4 and 5. Why? We could say 4.5 is an approximation of the desiredcube root. The number 1950.477 is also an approximation, albeit not a very goodone. When we ask for an approximation, we really mean to say something likethe following: Give a procedure that produces a sequence of approximations, eachobtained after a finite number of computable steps, such that the sequence producedconverges to the exact desired value. Even better, provide a test that produces anapproximation that lies within a preassigned distance from the desired value.

We have not yet produced a sequence of approximations that converges to thedesired cube root, but we have found an approximation x = 4.5 that is certain to beno more than 0.5 units from the desired value. If this approximation is good enoughfor our purposes, we can stop here.

Better estimates can be obtained by simply cubing numbers. One way to proceedsystematically is to check the number halfway between 4 and 5 (the first number inour sequence) to see if the value of the function f(x) = x3−88 is positive or negative.In fact, f(4.5) > 0. Using this fact, the root must be between 4 and 4.5; or in otherwords, the root must lie in the interval (4,4.5). Why? Thus we may take x2 = 4.25as the next element of the desired sequence. The process can be repeated to betterapproximate the root. The next approximation is 4.375. So far, the sequence ofcomputed approximations are x1 = 4.5, x2 = 4.25, and x3 = 4.375. What is x4?

It is not too difficult to write a computer program to implement what is calledthe bisection method, just described, for approximating the root of an equation, forexample our cubic. This algorithm is guaranteed to converge to a root—maybe notthe desired root in all cases—and at each step the error of the approximation is nolarger than half the length of the interval where the root resides. The method isviable, but as we will see, it is slow to converge; that is, there are methods thatachieve a specified accuracy with fewer iterations. It seems clear that the distancebetween the computed and exact values is decreased by about 1/2 after each iterationof the bisection method.

Is there a faster algorithm to compute the cube root? As you might expect thereare many possible ways to proceed. The most famous (and usually the best) methodis one that you might have learned during your first semester of calculus; it is calledNewton’s method.

Newton’s idea for his algorithm was probably gained from a geometric interpre-tation. But, a simple way to derive the basic formula is to consider a Taylor ap-proximation (which Newton surely understood before Taylor’s or Maclaurin’s namewas attached). In any case, suppose we desire to find a zero of a function f ; that is,we wish to solve the equation f(x)=0. Let us suppose a zero, x = a, exists so that

6 CARMEN CHICONE

f(a) = 0. For a choice of x near a, there is a number ξ between x and a such that

f(a) = f(x) + f ′(x)(a− x) + f ′′(ξ)(a− ξ)2.

For x close to a, we would expect the remainder term f ′′(ξ)(a − ξ)2 to be small;f ′′(ξ) is some number and (a − ξ)2 is much smaller than (a − ξ). This gives theapproximation

f(a) ≈ f(x) + f ′(x)(a− x).

Because f(a) = 0 by assumption, solving for a gives Newton’s approximation of awith respect to the point x:

a ≈ x− f(x)/f ′(x).

Clearly the process can be repeated. Starting from some approximation x0, the(n+ 1)th iterate of Newton’s method is

xn+1 = xn − f(xn)/f ′(xn).

The sequence of iterates obtained in this way should converge to the desired root.At least one fact can be proved: If the sequence converges to some number b, thenin fact f(b) = 0. Take limits on both sides of the iteration formula as n increaseswithout bound. Under the assumptions that f has a continuous first derivative andthe value of this derivative at the value b is not zero, the result is

b = b− f(b)/f ′(b).

Under the assumptions, we must have f(b) = 0 as stated. It remains to give condi-tions that imply the sequence of iterates generated by Newton’s method converges.

How well does Newton’s method work in practice?To test a new method, a good idea is to choose a problem where the answer is

known. Let us approximate the cube root of 125.For the function f(x) = x3 − 125 with x0 = 4, the iterates are obtained easily in

Mathematica using the function

F (x) := x+ f(x)/f ′(x) :

In

f[x_] := x^3 - 125;

F[x_] := x - f[x]/f’[x];

F[4], F[F[4]], F[F[F[4]]] // N

Out

5.27083, 5.01368, 5.00004


We are not taking full advantage of the computer yet as there is too much handworkin these commands. But, the output of the method seems to be converging rapidlyto the correct answer x = 5.

Computers are very good at iteration! One way to take advantage of this fact isembodied in the next code block, which uses a do-loop to write iterates into a listcalled “iterates.”

In

f[x_] := x^3 - 125;

F[x_] := x - f[x]/f’[x];

x = 4;

iterates = ;

Do[y = F[x], AppendTo[iterates, y // N], x = y, n, 1, 4]

Out

iterates

5.27083, 5.01368, 5.00004, 5.

Again Newton’s method converges very rapidly to the exact solution.The same code can be used to approximate the cube root of 88:

In

f[x_] := x^3 - 88;

F[x_] := x - f[x]/f’[x];

x = 4;

iterates = ;

Do[y = F[x], AppendTo[iterates, y // N], x = y, n, 1, 4]

Out

iterates

4.5, 4.44856, 4.44796, 4.44796

The last two iterates agree to 5 decimal places. More iterates could be computed formore accuracy if desired.

Newton’s method is applicable whenever f is a differentiable function such that thederivative of f does not vanish at the desired root. Of course, it works for transcen-dental equations as well. An example is provided in the companion Mathematicanotebook. You should by now be convinced that Newton’s method seems to work, atleast for simple examples. But you should wonder why and under what circumstancessequences of iterates produced by Newton iterations converge.

2.2. Equation solving for more than one equation. Many real world applica-tions involve solving a system of equations for more than one unknown. This subject

8 CARMEN CHICONE

Figure 1. Mountain Terrain

is often called solving simultaneous equations. The most important special case issolving simultaneous linear equations, for example, the system of three equations inthree unknowns

x+ y + z = 1

2x+ y + z = 2

x+ 2y + z = 3

An algebraic processor (in this case Mathematica) can solve this system exactly.

In

Solve[x + y + z == 1, 2 x + y + z == 2, x + 2 y + z == 3, x, y, z]

Out

x -> 1, y -> 2, z -> -2

Can you solve the system by hand?As you might know, there is a beautiful and useful algorithm for solving a sys-

tem of linear equations called Gaussian elimination. Its analysis, usefulness, andlimitations is one of the main problems that leads to a vast and important subjectcalled numerical linear algebra. This subject will be considered in detail later in youreducation. But as we will see, solving systems of linear equations is an importantingredient in algorithms for approximating solutions of nonlinear systems of equa-tions. We will avoid too much complication by solving only a few equations in a fewunknowns where we could, in principle, do the work by hand.

Out subject is applied mathematics. So, let us consider an applied problem whosesolution requires solving a system of nonlinear equations.

Via satellite imaging (for example) elevations are mapped over a region of theEarth’s surface and the function

F (x, y) := a3x3 − 3ab2xy2 − 0.1(a4x4 + b4y4) + 1500, a = 0.01, b = 0.01

is fit to this data to produce an approximation to the three-dimensional terrain. (Howdo you suppose the fitting is done?) A 3D plot of the function is provided in Fig. 1.An observer resides at the point with coordinates (-550,10, 1242.284) on the mappedterrain, where all distances are measured in meters. The first coordinate measures theeast-west direction with the positive direction pointing east. Likewise, the secondcoordinate measures north-south with the positive direction pointing north. Thefollowing problems might arise in this context.(1) Determine the curve consisting of points that are 700 meters away (via laser shots


in three dimensions) from this observation point and lie farther north. Note: A moredifficult problem is to determine the curve consisting of points that are 700 metersaway when distance is measured along the terrain.(2) Is there a point on the terrain equidistant via laser shots from the first observationpoint, the second observation point (-900, 990, 1800.574), and the third observationpoint (-300, 250, 1517.244)? If so, give the coordinates of such a point.(3) How many solutions of (2) exist within the mapped terrain?(4) Reconsider part (2) in case the third observation point is (1000, 0, 1501)?(5) Determine the shortest distance from observation point 1 to observation point 2along the terrain? Note: The solution of this problem seems to require some newconcepts. It would make a good project.

Questions (1), (3), (4) and (5) are left as exercises.Question (2) boils down to solving a system of two equations. They state the

equality of distances from the observation points to the desired location with un-known coordinates (x, y, z). Well almost. In such problems the distance involves asquare root. Usually the best idea is to measure the square of the distance to avoidcomputing the square root. In this problem there are three unknown quantities: x,y and z. But, the desired point is on the terrain. So, only two equations are needed:the difference of the distances from observation point 1 to observation point 2 setto zero and the distance from observation point 1 to observation point 3 set to zero.Geometric intuition suggests that a solution exists. It is not obvious if this solutionis unique. Also, inspection of the resulting equations makes it seem unlikely thata solution can be expressed explicitly. For this reason, which is typical in appliedproblems, we should seek an approximate solution.

To use Newton’s method, the left-hand sides of the two equations for the posedproblem should be defined as functions g and h so that the problem is converted tofinding a zero of the function defined by (x, y) 7→ (g(x, y), h(x, y)), which takes asinput a point with two coordinates and produces an output of the same type. Weusually say that we have defined a function from two dimensional space to itself.

In fact, the component functions g and h are defined by

g(x, y) := (x+ 550)2 + (y − 10)2 + (F (x, y)− 1242.284)2

− ((x+ 900)2 + (y − 990)2 + (F (x, y)− 1800.574)2),

h(x, y) := (x+ 550)2 + (y − 10)2 + (F (x, y)− 1242.284)2

− ((x+ 300)2 + (y − 250)2 + (F (x, y)− 1517.244)2)

We have not yet discussed how to apply Newton’s method when there is morethan one equation. To do this requires the notion of a partial derivative. You shouldhave already learned to use partial derivatives in your calculus class, but if not, don’tworry. The idea is very simple: Suppose that f is a function from two dimensional

10 CARMEN CHICONE

space to the real numbers so that f(x,y) denotes the real number assigned to thepoint with coordinates (x, y). The partial derivative of f with respect to its firstvariable x is simply the derivative of the function x 7→ f(x, y) with y held fixed.Likewise the partial derivative of f with respect to y is the derivative of the functiony 7→ f(x, y) with x held fixed. One notation for the partial derivative uses subscriptscorresponding to the name of the variable with respect to which the derivative istaken. So, fx denotes the partial derivative of the function f with respect to x andfy its partial derivative with respect to y. The values of these functions at some point(x, y) are of course denoted by fx(x, y) and fy(x, y), respectively. In an alternativenotation, due to Leibniz, these partial derivatives are

∂f

∂x(x, y),

∂f

∂y(x, y).

For example, let f(x, y) = x2+2y3. For this function fx(x, y) = 2x and fy(x, y) = 6y.Can you imagine how to define partial derivatives for a function of three or morevariables? What is fw(2, 1,−1, 1) in case f(x, y, z, w) = xy + ez − zw + w2?

Newton’s method in several variables applies when we seek a simultaneous zero forn functions of n variables. In the case at hand, we have two functions of two variables.Just like for one variable functions, where f(a) ≈ f(x) +f ′(x)(a−x) gives the linearapproximation for f(a), for functions of two variables the linear approximation (ofg(a, b) with respect to a nearby point (x, y) is

g(a, b) = g(x, y) + gx(x, y)(a− x) + gy(x, y)(b− y)

and likewise for h we have

h(a, b) = h(x, y) + hx(x, y)(a− x) + hy(x, y)(b− y).

As before, suppose that the desired simultaneous zero is (a, b) so that g(a, b) = 0and h(a, b) = 0. Then Newton’s approximation for this unknown point with respectto some nearby point (x, y) is obtained by solving the two equations

g(x, y) + gx(x, y)(a− x) + gy(x, y)(b− y) = 0,

h(x, y) + hx(x, y)(a− x) + hy(x, y)(b− y) = 0

for a and b.For the one variable case, the corresponding equation f(x) + f ′(x)(a − x) = 0 is

solved for a to obtain the formula for Newton iterations. Likewise, an explicit solutioncan be obtained in the multivariate case because the simultaneous equations arelinear equations for the unknowns a and b. As remarked earlier, methods for solvinglinear equations are important. As an exercise you should solve these equations. Theauthor has solved the equations by hand at least once in his life! So, to show youthe answer, he used the computer to find the answer:

A GLIMPSE INTO APPLIED MATHEMATICS UNIVERSITY OF ALBERTA SUMMER 201611

a = −((h(x, y)gy(x, y)− xgy(x, y)hx(x, y)− g(x, y)hy(x, y) + xgx(x, y)hy(x, y))

(gy(x, y)hx(x, y)− gx(x, y)hy(x, y))),

b = −((−h(x, y)gx(x, y) + g(x, y)hx(x, y)− ygy(x, y)hx(x, y) + ygx(x, y)hy(x, y))

(gy(x, y)hx(x, y)− gx(x, y)hy(x, y)))

As you can see the explicit solution is complicated. It gets worse as the number ofvariables is increased. These formulas can be simplified to a more compact form usingvector notation. For our numerical work, the exact solution is not important. Appliedmathematicians have developed fast, efficient and stable methods for approximatingsolutions of systems of linear equations. These methods should be employing whenusing Newton’s method. The usual algorithm is set up as follows.

(1) Define new variables X = x− a and Y = y − b.(2) Rearrange the linear equations to the form

gx(x, y)X + gy(x, y)Y = g(x, y),

hx(x, y)X + hy(x, y)Y = h(x, y).

(3) Pick a starting value (x0, y0) and suppose n− 1 iterations have already beencomputed. The nth iteration is performed by solving the linear system

gx(xn−1, yn−1)X + gy(x

n−1, yn−1)Y = g(xn−1, yn−1),

hx(xn−1, yn−1)X + hy(x

n−1, yn−1)Y = h(xn−1, yn−1),(1)

for X and Y . Note that the linear system (once the functions are evaluated)is in the standard form

a11X + a12Y = b1, a21X + a22Y = b2,

which is amenable to numerical algorithms for solving systems of linear equa-tions.

(4) Using the computed values X and Y , the updated iterate is

(xn, yn) = (xn−1 −X, yn−1 − Y ).

Here a and b are replaced by the last approximations of these values.

The algorithm is not complicated in principle. In practice, the main difficulty iscomputing the partial derivatives. This can sometimes be done by hand. One alter-native is to use the computer to perform this task. In some situations the equationsto be solved are so complicated that it is not feasible to compute derivatives. Whenfaced with this problem, cleverly designed alternatives to Newton’s method havebeen developed and are widely used.

Part (2) of the mountain terrain problem is discussed in detain in the companionMathematica Notebook. Newton’s method is used to find an approximate solution.

12 CARMEN CHICONE

2.3. Convergence and Convergence Rates. Consider solving the equation F (x) =0 when F : Rn → Rn is a function that can be differentiated at least twice thesederivatives are continuous. For the mountain terrain problem F : R2 → R2 and thisfunction can be differentiated as many times as desired.

Newton’s method has a special feature that makes it very effective for startingpoints that are sufficiently close to a root of F . The corresponding sequences gener-ated by the iteration process converge quadratically to the desired root. To see whatthis statement means, note first that an alternative way to view Newton’s method isto define a new function G : Rn → Rn by

(2) G(x) = x− [DF (x)]−1F (x),

choose a starting point x0 (the initial guess), and produce a sequence by iteration ofG; that is, after the nth element of the sequence is defined, the (n+ 1)th element isdefined by

(3) xn+1 = G(xn).

Here, DF is the Jacobian matrix of partial derivatives of F , or in one variable, simplythe usual first derivative F ′. For example, in case F = (g, h) as in linear system (1)where x and y are the variables,

DF (x, y) =

(gx(x, y) gy(x, y)hx(x, y) hy(x, y)

).

With this matrix in hand, linear system (1) in matrix form is

DF (xn−1, yn−1)

(XY

)=

(g(xn−1, yn−1)h(xn−1, yn−1)

).

Suppose that x = r is a solution of the equation F (x) = 0. If Newton’s methodconverges to r, then r = G(r); that is, r is fixed point of G. For this reason and manyothers, an important problem is to study fixed points of functions and the sequencesdefined by iteration of these same functions. This subject is one aspect of a widertheory called dynamical systems. In any case, the analysis of Newton’s method isaccomplished by studying the properties of G.

We may as well suppose that a is an approximation of some fixed point r of somefunction G that does not necessarily come from Newton’s method as in Eq. (2).

Recall Taylor’s theorem with integral form of the remainder for the order zeroapproximation of G at r:

(4) G(a)− r =

∫ 1

0

DG(r + t(a− r))(a− r) dt.


This fact is easy to prove. Start with the chain rule to derive

d

dtG(r + t(a− r)) = DG((r + t(a− r))(a− r).

Simply integrate both sides over the range 0 ≤ t ≤ 1 and use the fundamentaltheorem of calculus to obtain Taylor’s formula

(5) G(a)−G(r) =

∫ 1

0

DG(r + t(a− r))(a− r) dt,

and in the special case where G(r) = r, the desired formula (4).Another essential fact that is often used is a basic inequality for definite integrals:

|∫ b

a

f(x) dx| ≤∫ b

a

|f(x)| dx.

The proof is easy, at least for the scalar case. If f has all nonnegative values or allnonpositive values, the inequality is an equality. If f has both positive and negativevalues, break up the integral into a sum over intervals where it is positive and whereit is negative, use the triangle inequality, and then the first statements of this proof.For the vector case, which is technically more challenging, consider the signs of eachcomponent function.

Using Taylor’s formula, we have the identity

(6) |G(a)− r| = |G(a)−G(r)| = |∫ 1

0

DG(r + t(a− r))(a− r) dt|,

and in case the norm of the derivative of G is bounded by M , the estimate

(7) |G(a)− r| ≤M |a− r|.

If we are lucky and M < 1, then G(a) is closer to r than a. Continuing in the samemanner

|G(G(a))− r| ≤M2|a− r|,and so on. Hence, under this hypothesis on the absolute value of DG(r), the iteratesof a obtained by applying G do indeed converge to r as the number of iterates goesto infinity.

In case G is a scalar function of a scalar variable (n = 1), G′ is a continuousfunction, and

0 < |G′(r)| < 1,

Eq. (6) implies

(8) |G(aj)− r| = |∫ 1

0

G′(r + t(aj − r)) dt||aj − r|.

14 CARMEN CHICONE

Using that G′(r) is not zero and passing to the limit as j →∞,

limj→∞

|G(aj)− r||aj − r|

= |G′(r)|.

The convergence rate is called linear (or first order) because each iterate decreasesthe error (which is the absolute value of the difference between the iterate and theroot) by a factor of

|∫ 1

0

G′(r + t(aj − r)) dt| ≈ |G′(r)|

as stated in Eq. (8). Analogous results for G : Rn → Rn are more complicatedbecause iterates are not confined to one direction of approach to the root. But whenthe derivative of G does not vanish, some choices for the starting value will lead toiterations that converge linearly to the root.

For Newton’s method, the important observation about G is that DG(r) = 0, afact that is easy to check and a minor miracle. Not only is the absolute value of thederivative less than one, the absolute value is as small as possible: it is zero. This isthe key fact that produces the rapid convergence rate for Newton’s method.

Using Taylor’s formula (5) again,

(9) |G(a)− r| = |G(a)−G(r)| = |∫ 1

0

DG(r + t(r − a))(r − a)−DG(r)(r − a) dt|.

Under our smoothness assumption, we can apply Taylor’s formula to DG. If thesecond derivative of G is bounded by M , then we have

|DG(r + t(a− r))−DG(r)| ≤M |r − a|.

Using this estimate in Eq. (9), it follows that

|G(a)− r| ≤M |a− r|2.

In this case, the convergence is faster than linear: the ratio

|aj+1 − r||aj − r|

=|G(aj)− r||aj − r|

≤ M |aj − r|2

|aj − r|goes to zero as j increases to infinity (which means the numerator is much smallerthan the denominator). As least, the error |aj+1− r| compared to the previous error|aj−r| is cut by a factor of M |aj−r|. As the iterates approach the root, |aj−r| < 1;thus, in this case the convergence is faster than linear. In the linear case the errormight be cut by only a factor of M ; in the quadratic case the error is cut by M |aj−r|.For this reason, if DG(r) = 0, then the convergence rate is at least what is calledsecond order (or quadratic). It might be faster.


In general, we say the convergence of a sequence aj∞j=1 to a limit r has orderα > 0 in case

limj→∞

|aj+1 − r||aj − r|α

is some positive number. Newton’s method has order of convergence at least 2. Theorder of convergence is greater in case the second derivative of G vanishes.

For some examples, you should check that 12j∞j=1 is linearly convergent and

1

22j ∞j=1 is quadratically convergent. Of course, both sequences converge to zero.

2.4. Newton–Kantorovich Theorem and General Remarks on Newton’sMethod. An important result that gives explicit sufficient conditions for the con-vergence of Newton’s method is the Newton–Kantorovich theorem. This result hasseveral variants; two of them are stated next.

Theorem 2.1. Suppose that F : Ω → Rn is a differentiable function from the opensubset Ω in Rn to Rn, ω ∈ Ω, and the derivative DF (ω) : Rn → Rn is invertible.Define v = −[Df(ω)]−1(ω)F (ω) and U the ball of radius |v| centered at ω + v ∈ Rn.If U ⊂ Ω, there is a positive number M such that

‖Df(x)−Df(y)‖ ≤M |x− y| for all x and y in U,

and

M |F (ω)| ‖[Df(ω)]−1(ω)‖2 ≤ 1

2,

then F has a unique zero in U and for every starting point in U Newton’s methodconverges to this zero1.

Theorem 2.2. Suppose that

(1): F : Ωr → Y is a twice continuously differentiable function from the openball Ωr of radius r > 0 of the Banach space X to the Banach space Y ;

(2): A is a bounded linear operator A : X → Y ;(3): ω ∈ Ωr and there are positive constants η, δ, and C such that

(a): ‖AF (ω)‖ ≤ η,(b): ‖ADF (ω)− I‖ ≤ δ < 1, and(c): ‖AD2F (x)‖ ≤ C for all x ∈ Ωr.

Define

h =Cη

(1− δ)2, α =

(1−√

1− 2h)η

h(1− δ), β =

(1 +√

1− 2h)η

h(1− δ).

1Hubbard, J. and B. Hubbard (2002). Vector calculus, linear algebra, and differential forms: aunified approach. Upper Saddle River, NJ : Prentice Hall.

16 CARMEN CHICONE

If α ≤ r < β and h < 1/2, or α ≤ r ≤ β and h = 1/2, then F has a unique zeroω∞ ∈ Ωr, the sequence ωj∞j=1 of Newton approximates converges to ω∞, and

‖ω∞ − ωj‖ ≤ (2h)2jη

2j(1− δ).

In the second version of the theorem, A is usually taken to be [DF (ω)]−1 in casethis operator is bounded2.

To implement Newton’s method as a numerical algorithm, computation of inversesof matrices is avoided by recasting Eq. (10) to the form

(10) DF (xj)(xj+1 − xj) = −F (xj),

approximating the solution z of the matrix system DF (xj)z = −F (xj), and thendefining xj+1 = z + xj.

For large-scale problems, Newton’s method has some weaknesses: coding the de-rivative DF can be a major problem, solving a large system of linear equations ateach step is computationally expensive, and choosing a starting value close enoughto the desired root may be nontrivial. As a general rule, computing and coding thederivative is good practice when approximating the solution of a particular problem.Writing a general purpose code is another matter. Quasi-Newton methods are of-ten used in general purpose software to avoid computation of the derivative. Giventhe central importance of solving nonlinear systems in applied mathematics, all ofthese problems have been addressed with some success. These issues are discussedin textbooks on scientific computing.3

3. The divergence theorem and conservation of mass

3.1. The divergence theorem. Let us start with a review of few concepts frommultivariable calculus. For our purposes a vector field is a function that assignsa vector to each point in a region of some Euclidean space usually in the planeor on three-dimensional space. In the plane, a vector field X is a function of twovariables (say x and y) and it is given by two component functions (say f and g).In this notation, the Cartesian components of the vector assigned to the point withcoordinates (x, y) is three dimensions

X(x, y) = (f(x, y), g(x, y)).

Likewise, in three dimensional space

X(x, y, z) = (f(x, y), g(x, y), h(x, y))

2Kantorovich, L.V., and G.P. Akilov (1982) Functional Analysis. New York: Pergamon Press.3For example, see the excellent book: O’Leary, D. P. (2009). Scientific Computing with Case

Studies.Philadelphia: SIAM.


and in n-dimensional space

X(x1, x2, x3, . . . , xn) = (f1(x1, x2, x3, . . . , xn), f2(x1, x2, x3, . . . , xn),

f3(x1, x2, x3, . . . , xn), . . . , fn(x1, x2, x3, . . . , xn)).

Because a vector field is a geometric object that assigns a vector to each point, itsrepresentation in components depends on the choice of the coordinate system. Thisis a subtle point that requires more discussion than space allows here. Part of thediscussion is learning how to represent the same vector field in different coordinatesystems.

The same point about geometric objects (which should have definitions that donot depend on the choice of coordinates) applies to the definitions of the gradient of afunction and the divergence of a vector field. These definitions, when stated correctly,don’t depend on the choice of coordinates. But for simplicity, we will simply workwith these concepts in the special case of the usual Cartesian coordinates of two-or three-dimensional space. You should worry about what was just said and take acourse where the correct definitions are given.

The gradient of a function f is a vector field, usually denoted ∇f or grad f , withCartesian coordinates given by partial derivatives in three dimensions as follows:

∇f(x, y, z) = (fx(x, y, z), fy(x, y, z), fz(x, y, z)).

You can guess the formula for two or n dimensions. Remember that subscripts denotepartial derivatives. In Leibniz notation, for example,

fx =∂f

∂x,

and so on.The divergence of a vector field X with components (f, g, h) in three-dimensional

space, usually denoted ∇ ·X or divX, is a scalar function whose representation inCartesian coordinates is

∇ ·X(x, y, z) = fx(x, y, z) + gy(x, y, z) + hz(x, y, z).

Again the formula for two or n dimensions is what you would expect.Let Ω denote a region of space (maybe two, three, or n dimensions) that is bounded

by a hypersurface. In two dimensions Ω might be a disk, which is bounded by a circle;in three dimensions, it might be a ball, which is bounded by a sphere. The boundaryof Ω is denoted ∂Ω. In one dimension, Ω might be an interval and ∂Ω consists of thetwo end points of the interval.

The fundamental theorem of calculus, in one of its many forms, is the formula∫ b

a

fx(t) dt = f(b)− f(a).

18 CARMEN CHICONE

In one dimensional space a vector field has just one component. Thus, we may viewthe function f as the component which represents a vector field X. The notationbecomes a bit silly, but we could write X(x) = (f(x)) for the vector field X writtenin Cartesian coordinates for one-dimensional space. In mathematics, after a while weblur the notation and maybe just think of f as the vector field. Although for mostapplications this does not lead to trouble, the distinction between a vector field andits components can sometimes be essential. Here, we could let Ω be the interval withend points a and b and rewrite the integral in the fundamental theorem of calculusin a different way: ∫

Ω

∇ ·X dL,

where now dL is the element of length. Once we choose to parameterize the inter-val by the usual position coordinate t, the more abstract statement of the integralbecomes the original more familiar form.

What about the right-hand side of the fundamental theorem of calculus? Againwe can write it in a more abstract setting. On the boundary, ∂Ω = a, b, there isa vector field (which in this case is defined at just two points) called the outer unitnormal and denoted here by η. As its name suggests, the vector field is given by

η(a) = −e1, η(b) = e1,

where e1 is the (constant) vector of unit length pointing in the positive directionalong line. The dot product of X and η at a is simply the number −f(a) and thisdot product at b is f(b). Note that the negative sign is due to the direction of thenormal vector at a. It is a small leap to then interpret the integral of X · η over theboundary of Ω as the sum of these two values. In other words,

f(b)− f(a) =

∫∂Ω

X · η dP ,

where P (called the point mass measure) is simply a place holder here to make theintegral have the correct form. Do you see, up to some small leap of faith, that therestatement of the fundamental theorem of calculus in our fancy language is∫

Ω

∇ ·X dL =

∫∂Ω

X · η dP .

This last statement is the divergence theorem in one dimension; it is nothing moreor less than the fundamental theorem of calculus.

All the notation and the theorem generalizes. For example, in three dimensionsthe divergence theorem states∫

Ω

∇ ·X dV =

∫∂Ω

X · η dS,


where now dV is the element of volume and dS is the element of (surface) area. Intwo dimensions, ∫

Ω

∇ ·X dA =

∫∂Ω

X · η dL.

At a basic level, the proof of the divergence theorem for each dimension is a(complicated) corollary of the fundamental theorem of calculus. We will sketch theidea of a proof here. But, before doing so, you should know that it is not necessaryfor you to worry too much about a rigorous proof at this stage of your education.This is good advice for at least two reasons: (1) it is better to learn now what thetheorem says and how to use it and (2) the correct proof, which reveals a muchmore general setting for this theorem with far-reaching consequences, requires newconcepts that are best left to a course in advanced calculus. Just for fun, the perfectedgeneralization of the fundamental theorem of calculus is called Stokes’s theorem; ithas the beautiful looking statement∫

Ω

dα =

∫∂Ω

α.

Some of the ideas underlying the proof of the divergence theorem are revealed bya sketch of a proof in two dimensions.

Dealing with the geometry of Ω is part of the difficulty of the proof. As youknow from calculus, rectangular regions have the simplest geometry to deal with inCartesian coordinates and the usual methodology of calculus. So, let us consider arectangular region bounded above by the line y = d, below by y = c, on the leftby x = a and on the right by x = b. For the outer unit normal η = (η1, η2), theobjective is to show that

(11)

∫Ω

fx + gy dA =

∫∂Ω

fη1 + gη2 dL.

The first integral is the sum of two (double) integrals (one with integrand fx theother with integrand gy) over the rectangle. Parameterizing the rectangle by slicingfirst over the vertical direction and then over the horizontal direction and keepingthe usual orientation from left to right and from bottom to top, the first of theseintegrals can be written as a double integral whose inner integral can be evaluatedby the fundamental theorem of calculus as follows:∫

Ω

fx dA =

∫ d

c

( ∫ b

a

fx(x, y) dx)dy

=

∫ d

c

f(b, y)− f(a, y) dy.(12)

20 CARMEN CHICONE

Likewise, but slicing in the opposite order,

(13)

∫Ω

gy dA =

∫ b

a

g(x, d)− g(x, c) dx.

The sum of the right-hand sides of Eqs. (12) and (13) is the integral of the diver-gence over the rectangle.

Orientation is important in keeping track of signs when computing integrals. Tomaintain the usual orientation, the boundary of the rectangle must be parameterizedcounterclockwise to properly compute the line integral on the right-hand side ofEq. (11). To see what happens, let us compute the integral over the bottom faceB of the rectangle. It is parameterized properly from left to right by the usualcoordinate x. If you like, this boundary is parameterized by the curve γ(x) = (x, c)defined on the interval [a, b]. The outward unit normal η is equal to the constantunit vector field −e2, which has Cartesian components (0,-1). Using this notation∫

B

fη1 + gη2 dL =

∫ b

a

−g(x, c) dx.

Notice that the right-hand side of the last equation is one of the summands thatappears in the integral on the right-hand side of Eq. (13) (with the correct sign).By computing along the remaining three faces of the rectangle respecting the coun-terclockwise orientation, the remaining three summands on the right-hand sides ofEqs. (12) and (13) are matched. This proves the divergence theorem for a rectanglein the plane. A similar argument will work for a rectangular box in three-dimensionalspace. Although visualization is no longer possible and orientation becomes moredifficult to manage, a similar argument will work to prove the divergence theoremfor a rectangular box in an n dimensional Euclidean space.

The next step is to prove the theorem for a region Ω that can be decomposedinto a finite number of rectangular regions. By decomposed, we mean the collectionof rectangular regions must have at least three properties: (1) the union of therectangular regions must be equal to Ω, (2) no two rectangular regions have interiorpoints in common, and (3) when two rectangles intersect they do so along exactlyone common face. Lot’s of pictures are helpful. Draw some!

To prove the theorem for a decomposable region, the idea is simple: compute theintegral of the divergence as a sum of integrals, one over each rectangular region .Also, orient each rectangular region and each boundary of a rectangular region withthe usual positive orientation.

Now comes the key observation that is made, for simplicity, in the plane. Supposetwo rectangles have a face in common and each boundary has counterclockwise ori-entation. For example, imagine the two rectangles meet along a vertical face. Therectangle that lies on the left of this face imparts an up-orientation on this face; the


rectangle on the right imparts a down (opposite) orientation. When the line integralsare computed along this face for the two rectangles they are equal and of oppositesign. Thus they cancel in the grand scheme, which leaves only line integrals alongthe boundary of the original region Ω. When all bookkeeping is done, the sum ofthe integrals of the divergence will be exactly the line integral of the dot product ofthe vector field with the unit outer normal along the boundary of the decomposableregion. This will prove the divergence theorem for decomposable regions.

For regions with more complicated boundaries, which are not decomposable intorectangular regions, the idea is to approximate the entire region with a decomposableregion (perhaps with a large number of small rectangles all the same size) that alllie inside the region. Better and better approximations are made with smaller andsmaller rectangles. Passing to the limit as the number of rectangles goes to infinityproves the divergence theorem for regions with boundaries that are not to jagged.To make this limit process rigorous requires some hypotheses on the smoothness ofthe boundary. Fortunately all this can be done and the divergence theorem is truein great generality. It is one of the essential mathematical theorems in all of appliedphysics. You can’t live without it!

3.2. Fundamental and Constitutive Model Equations. Suppose that somesubstance (considered to have mass) is distributed in Rn and let Ω denote a boundedregion in Rn with boundary ∂Ω and outer normal η. The density of the substanceis represented by a function u : Rn ×R→ R, where u(x, t) is the numerical value ofthe density in some units of measurement at the site with coordinate x ∈ Rn at timet. Usually n ∈ 1, 2, 3, but the following model is valid for an arbitrary dimension.To avoid using the names for geometric objects in each dimension, the derivation isgiven with respect to the geometry of n = 3 where area and volume have their usualmeanings. Also, recall from vector calculus the following concepts: gradient of afunction (grad f or ∇f), divergence of a vector field (divX or ∇·X), and Laplacianof a function (∆f or ∇ · ∇f or ∇2f).

The time rate of change of the amount of the substance in Ω is given by thenegative flux of the substance through the boundary of Ω plus the amount of thesubstance generated in Ω; that is,

d

dt

∫Ω

u dV = −∫∂Ω

X · η dS +

∫Ω

f dV ,

whereX is the vector field on Rn (sometimes called the diffusion flux) of the substancepointing in the direction in which the substance is moving and with the magnitude ofthe amount of substance per area per time passing through the plane perpendicularto this direction; dV is the volume element; dS is the surface element; the vector fieldη is the outer unit normal field on the boundary of Ω; and f (a function of density,

22 CARMEN CHICONE

position, and time) is the amount of the substance generated in Ω per volume pertime. The minus sign on the flux term is required because we are measuring the rateof change of the amount of substance in Ω. If, for example, when the flow is all outof Ω, the inner product X · η is not negative and the minus sign is required becausethe rate of change of the amount of substance in Ω is negative.

Using the divergence theorem (also called Gauss’s theorem) to rewrite the fluxterm and by interchanging the time derivative with the integral of the density, wehave the relation ∫

Ω

ut dV = −∫

Ω

divX dV +

∫Ω

f dV .

Moreover, because the region Ω is arbitrary in this integral identity, it follows that

(14) ut = − divX + f.

To obtain a useful dynamical equation for u from Eq. (14), we need a constitutiverelation between the density u of the substance and the flow field X. In most ap-plications, it is not at all clear how to derive this relationship from the fundamentallaws of physics. Thus, we have an excellent example of a class of important modelingproblems where physical intuition must be used to propose a constitutive law whosevalidity can only be tested by comparing the results of experiments with the predic-tions of the corresponding model. Problems of this type lie at the heart of appliedmathematics and physics.

For Eq. (14), the classic constitutive relation—called Darcy’s, Fick’s, or Fourier’slaw depending on the physical context—is

(15) X = −K gradu+ µV

where K ≥ 0 and µ are functions of density, position, and time; and V denotes theflow field for the medium in which our substance is moving. The minus sign on thegradient term represents the assumption that the substance diffuses from higher tolower concentrations.

By inserting the relation (15) into the balance law [Eq. (14)], we obtain the dy-namical equation

(16) ut = div(K gradu)− div(µV ) + f.

Also, by assuming that the diffusion coefficient K is equal to k2 for some constantk, the function µ is given by µ(u, x, t) = γu where 0 ≤ γ ≤ 1 is a constant thatdetermines the amount of u that moves with the velocity field V , and V is an incom-pressible vector field (div V = 0) such as the velocity field for the motion of water,we obtain the most often used reaction-diffusion-convection model equation

(17) ut + γ gradu · V = k2∆u+ f.


The quantity γ gradu · V is called the convection term, k2∆u is called the diffusionterm, and f is the source term.

In case no diffusion in involved in the motion of the substance (K = 0 in Eq. (16)),all of the substance moves with the velocity field V (that is, γ = 1), the sourcefunction f vanishes, and the velocity V is not necessarily incompressible, the balancelaw reduces to the differential form of the law of conservation of mass, also calledthe continuity equation, given by

(18) ut + div(uV ) = 0.

Because Eq. (17) is derived from physical laws and generally accepted constitutivelaws, this partial differential equation (PDE) is used to model all physical processeswhere reaction, diffusion, or convection is involved.

In case the substrate medium is stationary (that is, V = 0), the model [Eq. (17)]is the diffusion (or heat) equation with a source

(19) ut = k2∆u+ f.

This equation, where u is interpreted as temperature, is the standard model forheat flow. In this application, the conserved substance may be considered to be heat.The total amount of heat in a region is the volume integral of the product of thespecific heat of the underlying medium, its density, and its temperature. Accordingto Fourier’s law, the time rate of change of the total amount of heat in a domain isproportional to the temperature flux through its boundary. Thus, heat conservationagain produces the PDE (19) for the temperature. A deeper understanding of heat,temperature, and energy conservation requires a background in thermodynamics.Here, an intuitive understanding of heat and temperature are sufficient to apply themodel equation for temperature to basic physical scenarios.

An understanding of the derivation of the model equation [Eq. (17)] should alsoexplain the widespread appearance of the Laplacian in applied mathematics: It is thedivergence of the gradient vector field whose flow is supposed to carry some substancefrom regions of its higher concentration toward regions of lower concentration.

Exercise 3.1. Show that the gradient of a function evaluated at a point p points in thedirection of maximum increase of the function at p. Also, the gradient is orthogonal toeach level set of the function at each point on a level set.

Exercise 3.2. Discuss the meaning of the divergence of a vector field. In particular,discuss positive divergence, negative divergence, and zero divergence. Give examples. Hint:You may wish to consider the equation

divX(p) = limΩ→p

1

vol(Ω)

∫∂ΩX · η dS,

24 CARMEN CHICONE

where the limit is taken over all bounded open sets with smooth boundaries that contain thepoint p whose diameters shrink to zero.

3.3. Reaction-Diffusion in One Spatial Dimension: Heat, Genetic Muta-tions, and Traveling Waves.

3.3.1. One-Dimensional Diffusion. Imagine the diffusion of heat in an insulted barwith insulated ends (that is, zero heat flux through the surface of the bar) underthe further assumptions that the temperature is the same over each cross sectionperpendicular to the bar’s axis and there are no heat sources or sinks along the bar.Given the initial temperature distribution along the bar, the basic problem is todetermine the temperature at each point of the bar as time increases.

Under the assumptions, we need only consider the spatial distribution of heatalong the axis of the bar that we idealize as an interval of real numbers. For a barof length L, we thus let x denote the spatial coordinate in the open interval (0, L).The model equation [Eq. (19)] is

(20) ut = κuxx,

where u(x, t) is the temperature at position x along the bar at time t ≥ 0 and κ > 0is a constant (called the diffusivity) that depends on the material used to constructthe bar. The value of κ must be determined by experiment. (How would you set upand conduct such an experiment?) Eq. (20) is often called the heat equation.

A function f : (0, L)→ R representing the initial temperature along the bar givesthe initial condition

(21) u(x, 0) = f(x).

Zero flux conditions at each end of the bar provide the boundary conditions

(22) ux(0, t) = 0, ux(L, t) = 0.

These boundary conditions are also called the zero Neumann boundary conditions.The problem is to find a function u that satisfies the heat equation [Eq. (20)],

the initial condition [Eq. (21)], and the boundary conditions [Eqs. (22)]. This is aclassic problem first solved by Joseph Fourier in 1822. His basic ideas, which havefar-reaching consequences, are milestones in the history of science. Fourier’s law ofheat conduction—heat flows from regions of high temperature to regions of lowertemperature—is used in the derivation of the PDE (20). His mathematical solutionof the heat equation with initial and boundary conditions introduced Fourier series,one of the main tools of mathematical analysis.

A fundamental technique that often works to solve linear PDEs with rectangularspatial domains is separation of variables: Look for solutions of the form u(x, t) =


X(x)T (t), where X and T are unknown functions. Inserting this guess into the heatequation [Eq. (20)] yields the formula

X(x)T ′(t) = κX ′′(x)T (t),

which must hold if (x, t) 7→ X(x)T (t) is a solution. Assume for the moment thatX(x) and T (t) do not vanish and rearrange the last formula to the form

T ′(t)

κT (t)=X ′′(x)

X(x).

If X(x)T (t) is a solution that does not vanish, then the left-hand side of the equationis a function of t alone and the right-hand side a function of x alone. It follows thatthe left-hand side and the right-hand side of the equation are equal to the sameconstant c. In other words, there must be a constant c such that

T ′(t) = cκT (t), X ′′(x) = cX(x);

that is, X and T must be solutions of the given ordinary differential equations(ODEs).

The ODE X ′′(x) = cX(x) is easily solved for each of the usual cases: c > 0, c = 0,and c < 0. Indeed, an essential requirement of the method is to find all solutions ofthis ODE that satisfy the boundary conditions. For c = λ2 > 0, the general solution(expressed with arbitrary constants a and b) is

X(x) = aeλx + be−λx;

for c = 0,

X(x) = ax+ b;

and, for c = −λ2 < 0,

X(x) = a cosλx+ b sinλx.

Solutions of the PDE are required to satisfy the boundary conditions. This ispossible only for c = 0 and X(x) = b or c < 0 and

X(x) = a cosnπ

Lx,

where n is an integer. As cosine is an even function, all of the solutions for c < 0 areobtained with n ranging over the nonnegative integers.

For c = −λ2 = −(nπL

)2, the corresponding solution of T ′(t) = cκT (t) is

T (t) = e−(κn2π2/L2)t.

An infinite number of solutions have been constructed:

un(x, t) = Xn(x)Tn(t) := e−(κn2π2/L2)t cosnπ

Lx, n = 0, 1, 2, . . . ,∞.

26 CARMEN CHICONE

The principle of superposition for the PDE with the zero Neumann boundary con-ditions is valid and easy to prove. It states that if u and v are solutions of the PDEthat also satisfy the zero Neumann boundary conditions, then so is every linear com-bination au + bv of these solutions, where a and b are scalars. As a corollary, everyfinite sum

(23) u(x, t) = b0 +N∑n=1

bne−(κn2π2/L2)t cos

nπ

Lx

is a solution of the PDE and the boundary conditions. Warning: The superposi-tion principle is not valid for nonzero Neumann boundary conditions (for instance,ux(0, t) = a and ux(L, t) = b where a and b are constants and at least one of them isnot zero).

What about the initial condition? One fact is clear: If

f(x) = b0 +N∑n=1

bn cosnπ

Lx

for some choice of b0, b1, b2, . . . , bN , then the function u in Eq. (23) is a solutionof the PDE, boundary conditions, and initial conditions. This result suggests thequestion: Which functions f can be written as a sum of cosines? The surprisinganswer, first given by Fourier, is that most functions defined on the interval [0, L]can be written as an infinite sum of cosines (or sines). We will discuss this result inmore detail in subsequent sections. A more precise (but not the most general) factis that every piecewise continuously differentiable function f defined on [0, L], withat most a finite number of jump discontinuities, can be represented by a (pointwise)convergent Fourier cosine series ; that is,

(24) f(x) = b0 +∞∑n=1

bn cosnπ

Lx.

If in addition f is continuous (no jump discontinuities), then the partial sums of theFourier series converge uniformly to f . This is a powerful result. By applying it, weknow that the PDE model with zero Neumann boundary conditions has a solutionfor all initial conditions that are likely to be encountered. In fact, once the initialcondition f is expressed as a Fourier series, the function u defined by

(25) u(x, t) = b0 +∞∑n=1


nπ

Lx

is a solution of our PDE that satisfies the boundary and initial conditions.It turns out that there is a simple method to determine the Fourier coefficients

bn of a function f defined on [0, L]. Using the convergence theorem and assuming


for simplicity that f is continuously differentiable, this function may be expressed asin Eq. (24) and the sum on the right-hand side of the equation may be integratedterm-by-term. Thus,∫ L

0

f(x) dx =

∫ L

0

b0 dx+∞∑n=1

∫ L

0

bn cosnπ

Lx dx.

Every integral in the infinite summation vanishes. Hence,

b0 =1

L

∫ L

0

f(x) dx.

For each positive integer m, we have that∫ L

0

f(x) cosmπ

Lxdx =

∫ L

0

b0 cosmπ

Lxdx+

∞∑n=1

∫ L

0

bn cosnπ

Lx cos

mπ

Lxdx.

As before, the first integral on the right-hand side of the equation vanishes. Theintegrals of products of cosines behave in the best possible way: they all vanishexcept for the product where n = m. (Check this statement carefully; it is a basicresult that makes Fourier series useful.) By an application of this fact,∫ L

0

f(x) cosmπ

Lxdx =

∫ L

0

bm cos2 mπ

Lxdx = bm

L

2,

and we have the general formula

bm =2

L

∫ L

0

f(x) cosmπ

Lxdx.

All the Fourier coefficients may be computed simply by integrating the product ofthe given function and an appropriate cosine. A similar result holds for Fourier sineseries. Indeed, a function f in the same class of functions may be represented as theFourier sine series

f(x) =∞∑n=1

an sinnπ

Lx.

A solution for the heat equation with zero Neumann boundary conditions andarbitrary piecewise continuously differentiable initial valued has been constructed.This is a wonderful result, but using the PDE with the given boundary and initialdata as a model would be useless if there were other solutions not detected by theconstruction of the solution using separation of variables and Fourier series. Whichsolution would we choose? To be predictive, a proposed model should have a unique

28 CARMEN CHICONE

solution. Suppose there were two solutions u and v in the class of piecewise continu-ously differentiable functions. By superposition, w := u− v is also a solution of thesame boundary value problem (BVP) but with zero initial condition; that is,

wt = κwxx, w(x, 0) = 0, wx(0, t) = 0, wx(L, t) = 0.

Note that ∫ L

0

wtw dx =

∫ L

0

κwxxw dx.

By an application of integration by parts on the right-hand integral and taking thetime derivative outside the left-hand integral, we have the equality

d

dt

∫ L

0

1

2w2 dx = κwwx

∣∣L0− κ

∫ L

0

w2x dx.

Using the boundary conditions,

d

dt

∫ L

0

1

2w2 dx = −κ

∫ L

0

w2x dx ≤ 0.

It follows that the function

t 7→∫ L

0

1

2w2 dx

is nonnegative and it does not increase as t increases. But, at t = 0, the initialcondition is w = 0. Hence, w must be the zero function for all t > 0 for which thesolution exists; therefore, u = v as desired.

The reader may ask: Could a solution exist that is not piecewise continuouslydifferentiable? If so, then perhaps solutions of the model problem are not unique. Afull answer to the question goes beyond the mathematics developed here. At leastthe assumptions used in the proof of uniqueness are reasonable for the basic heatflow model. But, the desire to answer such questions is a very good reason to pursuemore advanced mathematics.

The solution of our diffusion model equation can be used to answer questions andmake predictions . For example, the solution [Eq. (25)] predicts that the temperaturedistribution on the rod will go to a constant steady state as time increases and thissteady state is the average value of the initial heat distribution.

In case heat sources and sinks are included in the model or other physical phenom-ena are taken into account, especially those that produce nonlinear terms, Fouriersolution methods will likely fail. In fact, there are no methods that will produce exactsolutions for all such models. Thus, to make predictions from PDE model equations,approximation methods are often used. The most important approximation meth-ods are called numerical methods; they use arithmetic to approximate solutions ofpartial differential equations.


Ui

j+1

x

t

j

j+1

Ui

j

i-1 i i+1

Ui+1

jUi-1

j

Figure 2. A schematic discretization of space and time is depictedfor numerical solutions of the diffusion equation. The discrete solutionvalues are UJ

i := u(i∆x, j∆t). The unknown interior value U j+1i is

computed using the previously computed values U ji−1, U j

i , and U ji+1.

New methods are best applied to problems where the exact answer is known. Here,the diffusion model serves as a simple example to illustrate a numerical method, calleda finite difference method, that is often used for approximating solutions of PDEs.

To make a numerical approximation of a solution of the heat equation, the diffu-sivity k, the length of the spatial domain L , and the initial data f must be known.The first step of a finite difference method is to discretize space and time. Thismay be done by choosing a positive integer M with corresponding spatial increment∆x := L/M , a temporal increment ∆t, and by agreeing to consider the value of uonly at the interior spatial domain points with coordinates xi := i∆x = iL/M fori = 1, 2 . . .M−1 and temporal coordinates j∆t for j = 0, 1, 2, . . ., where we leave un-specified the (finite) number of time steps that might be computed. In other words,the basic idea is to determine approximate values of u at the interior gridpointsU ji = u(i∆x, j∆t) as depicted in Fig. 2.Having discretized space and time, the second step is to discretize the time and

space derivatives that appear in the PDE. For this, we use Taylor’s theorem. Recallthat if a function f is (n+ 1) times continuously differentiable at a point a, then the

30 CARMEN CHICONE

value of f at a nearby point x is given by

f(x) = f(a) + f ′(a)(x− a) +1

2!f ′′(a)(x− a)2 +

1

3!f ′′(a)(x− a)3

+ · · ·+ 1

n!fn(a)(x− a)n +

1

(n+ 1)!fn+1(c)(x− a)n+1

for some number c that lies between a and x. We often say that the right-hand sideof the last formula is the Taylor expansion of the function f at a.

Consider a point (x, t) and the nearby point (x, t + ∆t). By an application ofTaylor’s formula to the function ∆t 7→ u(x, t+∆t) at ∆t = 0, we have the expansion

u(x, t+ ∆t) = u(x, t) + ut(x, t)∆t+1

2!utt(x, c)∆t

2.

A rearrangement yields the equation

ut(x, t) =u(x, t+ ∆t)− u(x, t)

∆t− 1

2!utt(x, c)∆t.

and the approximation

ut(x, t) ≈u(x, t+ ∆t)− u(x, t)

∆twith an error of order ∆t. The left-hand side of our PDE may be approximated by

(26) ut(i∆x, j∆t) ≈U j+1i − U j

i

∆t.

Using a similar procedure,

u(x−∆x, t) = u(x, t)− ux(x, t)∆x+1

2uxx(x, t)∆x

2 − 1

3!uxxx(x, t)∆x

3,

u(x+ ∆x, t) = u(x, t) + ux(x, t)∆x+1

2uxx(x, t)∆x

2 +1

3!uxxx(x, t)∆x

3,

up to an error of fourth-order in ∆x. These approximations may be added andrearranged to obtain the formula

uxx(x, t) ≈u(x−∆x, t)− 2u(x, t) + u(x+ ∆x, t)

∆x2

with an error of order ∆x2, which is one order more accurate than the approximationchosen for the time derivative. This discrepancy will be addressed further on; finitedifference methods that are second order in both space and time are certainly desir-able and will be constructed. Here, the right-hand side of our PDE is approximatedby

(27) uxx(i∆x, j∆t) ≈U ji−1 − 2U j

i + U ji+1

∆x2.


Equating the last two approximations [Eqs. (26) and (27)] and rearranging theresult, a discrete approximation of the PDE is

(28) U j+1i = U j

i +κ∆t

∆x2(U j

i−1 − 2U ji + U j

i+1).

The left-hand side of Eq. (28) is the value of u at the (j + 1)st time step; allvalues of u on the right-hand side are evaluated at the jth time step. As the valuesof u at the zeroth time step (j = 0) are given by the U0

i via the initial condition,the approximate values of u at the first time step (j = 1) are given by U1

i at thenodes on the grid by Eq. (28). The values U j+1

i , corresponding to the time step(j + 1), are determined using the previously computed values at the jth time step.Well, almost. . . . There is a problem: The values of U0

i for i = 0, 1, 2, 3 . . .M aregiven by the initial data; thus, all is well when computing U1

i at the interior nodes(i = 1, 2, 3, . . . ,M − 1). But, the boundary values U1

0 at the left end and U1M at

the right end, which will be needed in the next time step, must be determined fromthe boundary conditions. (The subject of PDEs would be simple absent boundaryconditions, but there is no reprieve; boundary conditions are essential in modelingand in numerical approximations. Be careful.)

For an approximation of the zero Neumann boundary conditions (which requirethe partial derivative ux to vanish at each end of the computational domain), onepossibility is to insist that for all time steps

(29) U j0 = U j

1 , U jM = U j

M−1.

These conditions impose an approximation of the zero first derivatives at the ends ofthe bar, whose spatial coordinates are x0 and xM , by viewing the approximation of uto be constant over the intervals [x0, x1] and [xM−1, xM ]. This is not the only possibleapproximation of the zero Neumann boundary conditions, but this approximation isconsistent with the desired condition and has the virtue of being simple to implementin computer code. For the case j = 1, simply set U1

0 equal to the already computed(interior) value U1

1 and set U1M equal to the already computed interior value U1

M−1.The same procedure is used for all subsequent time steps.

All the ingredients are now in place to approximate solutions of the heat equation[Eq. (20)] with Neumann boundary conditions [Eqs. (22)] and the initial condition[Eq. (21)]. Determine the initial data U0

i , for i = 1, 2, 3, . . . ,M − 1, from the initialcondition. Set j = 1 and compute U1

i for i = 1, 2, 3, . . . ,M − 1 using Eq. (28) andimpose the end conditions [Eqs. (29)]. Repeat the process to compute U2

i using thepreviously determined values U1

i , and continue to compute in turn U j+1i , for j ≥ 2,

over i = 1, 2, 3, . . . ,M − 1 using the previously computed values U ji . The process is

stopped when j reaches some preassigned integer value N . The size of the increment

32 CARMEN CHICONE

∆t can be adjusted so that after a finite number of steps j = 1, 2, 3, . . . , N , the timeT = N∆t is equal to a preassigned final value.

The numerical scheme may also be viewed in vector form. Define the (M − 1)vector W j to be the transpose of the row vector (U j

1 , Uj2 , U

j3 , . . . , U

jM−1), let α :=

κ∆t/∆x2, and define the (M − 1) × (M − 1) matrix A whose main diagonal is(1−α, 1−2α, 1−2α, 1−2α, . . . , 1−2α, 1−α) (that is, the first and last componentsare 1 − α and the other components are all 1 − 2α), whose first superdiagonal andfirst subdiagonal elements are all α, and all remaining elements are set to zero. Incase M = 5, the matrix is

A =

1− α α 0 0α 1− 2α α 00 α 1− 2α α0 0 α 1− α

.

The iteration scheme [Eq. (28)] (including the boundary conditions) takes the vectorform

W j+1 = AW j.

In other words, the iteration scheme is simply matrix multiplication by A. Thespecial first and last rows of A are due to the Neumann boundary conditions.

The initial vector W 0 is determined by the initial condition for the PDE. Sub-sequent iterates are W 1 = AW 0, W 2 = AW 1, and so on. Or, in a more compactform,

W j+1 = AjW 0,

where Aj denotes the jth power of the matrix A and W j is the jth element in thesequence of iterates whose first three elements are W 1, W 2, and W 3. This typeof iteration scheme, iterating a function (which is a linear transformation here), isubiquitous in applied mathematics. Learning the corresponding theory is certainlyworthwhile.

There are at least two reasons to expect difficulties: (1) The discrete first-orderapproximation of the time derivative ut is less accurate than the second-order ap-proximation of the spatial derivative uxx; and (2) perhaps the approximate valuesU ji begin to grow or oscillate due to discretization errors and thus the approximation

does not remain close to the solution of the continuous model.Alternative numerical algorithms that overcome difficulty (1) are available. In

fact, a viable method that makes the time discretization second-order is discussed inSection ??.

Numerical instability will occur (see Exercise 3.9) unless the space and time in-crements chosen to discretize the model satisfy the Courant–Friedrichs–Lewy (CFL)


condition

(30) κ∆t

∆x2≤ 1

2,

a requirement that is revisited and explained more fully in Section ??. In practice, theCFL condition determines the maximum allowable time-step size for our numericalmethod after a spatial discretization is set.

To appreciate the CFL condition for numerical approximations of the diffusionmodel, suppose that a roundoff error is introduced in the computation at the firsttime step. Instead of computing the exact value W 1 from W 0, the machine computesW 1 + ε (where ε is an (M − 1) vector representing the error). Of course, furthererrors might be introduced at subsequent steps. But, for simplicity, consider onlythe propagation of the first error, and assume that the computed results are exactafter the first error occurs. Under these assumptions, the computed values are

W 2 = AW 1 + Aε, W 3 = A2W 1 + A2ε, W 4 = A3W 1 + A3ε, . . . .

Note that the vector Ajε represents the error at each step. The algorithm will produceuseless results if the norm (which may be taken to be the Euclidean length) of thevector Ajε grows as j increases.

What happens in case the matrix A were the diagonal 2× 2 matrix

A =

(a 00 b

)?

Clearly

Aj =

(aj 00 bj

).

If either |a| > 1 or |b| > 1 and there is a corresponding nonzero element of the vectorε, then the size of the propagated error will grow. For example, if |b| > 1 and εis the transpose of the vector (0.01, 0.035) the number bj0.035 will grow to infinityas j goes to infinity. In this case, roundoff errors are amplified under iteration andthe numerical approximation of the PDE becomes increasingly less accurate as thenumber of time steps increases. On the other hand, if both |a| or |b| are less than orequal to 1, then the propagated error will remain bounded as j goes to infinity.

The matrix A that appears in our numerical scheme is not diagonal. But, it has aspecial form: A is symmetric; that is, A is equal to its transpose. Every symmetricmatrix is diagonalizable; in other words, if A is symmetric, then there is an invertiblematrix B such that B−1AB is diagonal. Also, every eigenvalue of a symmetric matrixis real. Using these facts, let us suppose that every eigenvalue of our matrix A liesin the closed interval [−1, 1]. The matrix C := B−1AB has the same eigenvalues as

34 CARMEN CHICONE

A. (Why?) Iterations of a vector v by C remain bounded because C is diagonal andall its eigenvalues have absolute value less than or equal to 1. In fact,

|Cjv| ≤ |v|

for all vectors v and all positive nonzero integers j. Iteration of v by the matrix Aalso remains bounded because

|Ajv| = |(BCB−1)jv| ≤ |BCjB−1v| ≤ ‖B‖|B−1v| ≤ ‖B‖‖B−1‖|v|.

The CFL condition [Eq. (30)] implies that all eigenvalues of A are in the closedinterval [−1, 1] (see Section ??).

The heat equation is well established as a model and is widely used. But, as for allmathematical models that rely on constitutive laws, it is not in complete agreementwith nature. To reveal a basic flaw, reconsider the Fourier series solution [Eq. (25)]

u(x, t) := b0 +∞∑n=1


nπ

Lx

of this model [Eq. (20)]. The Fourier coefficients are determined by the initial data.Imagine the initial temperature is zero everywhere along the bar except near someplace x = ξ along the rod. In fact, the temperature far away from this point willremain zero for some finite time interval: the effects of the motions of the moleculesnear ξ cannot be transferred instantly to some distant location. In more prosaic lan-guage, there is no action at a distance. The model does not agree with this physicalreality. Indeed, every Fourier coefficients bne

−(κn2π2/L2)t in the series representationof u is changed from its initial value by a different exponential factor for arbitrarilysmall t > 0. At some prespecified distance along the rod from ξ and for arbitrarilysmall t > 0, the model predicts nonzero temperatures at points at least this distancefrom ξ. The influence of the nonzero temperature at ξ is predicted to be felt instantlyat every point along the rod. Of course, the predicted influence is so small in prac-tical situations that it is not measurable. In most practical applications, predictedtemperatures agree with measured temperatures. Thus, although the heat equationmodel predicts a violation of a physical law at a fundamental level, it is an excellentmodel for applications to heat transfer phenomena. The violation is due to Fourier’sconstitutive law X = −K gradu, not the conservation of mass.

Exercise 3.3. (1) Write the Fourier sine series for the function f(x) ≡ 1 on the interval[0, 2] and make graphs of a few of the partial sums to indicate its convergence. (2) Repeatthe exercise for the Fourier cosine series. (3) Write the Fourier sine series for the unit stepfunction that is zero on the interval [0, 1) and one on the interval [1, 2]. (3) Show that theseries evaluated at x = 1 converges to 1/2. Also, draw graphs to indicate the convergence


of the partial sums. Describe the behavior of the partial sums on a small interval [1, 1 + ε).The (perhaps strange) behavior is called the Gibbs phenomenon.

Exercise 3.4. Let h be a smooth function defined on the interval [0, L] and consider thefunction f given by

f(x) =N∑n=1

an sin(2πn

Lx).

Define the error in approximating h by f to be

Λ =

∫ L

0|f(x)− h(x)|2 dx.

(1) Find the numbers a1, a2, a3, . . . , aN that minimizes the error. Compare with theFourier coefficients. (2) Approximate the least error in case N = 4, L = 2, and h(x) = 1.(3) What is the minimum N so that the error is less that 0.01?

Exercise 3.5. (a) Solve the diffusion equation on the spatial domain [0, L] with initialcondition and zero Dirichlet boundary conditions:

ut = κuxx, u(x, 0) = f(x), u(0, t) = 0, u(L, t) = 0.

(b) Show that your solution is unique.

Exercise 3.6. Solve the diffusion equation on the spatial domain [0, L] with initial con-dition and nonzero Neumann boundary conditions:

ux(0, t) = a, ux(L, t) = b,

where a and b are real numbers. Hint: Look for a solution u = v + w, where v is afunction that satisfies the boundary conditions and w satisfies the PDE with zero boundaryconditions.

Exercise 3.7. Solve the diffusion equation on the spatial domain [0, L] with initial con-dition and nonzero Dirichlet boundary conditions:

u(0, t) = a, u(L, t) = b,

where a and b are real numbers.

Exercise 3.8. The zero flux boundary condition for the diffusion equation has the physicalinterpretation that no substance is lost as time increases. Prove this fact by showing thatthe time derivative of the total amount of substance (its integral over the spatial domain)vanishes.

Exercise 3.9. (a) Write computer code to implement the numerical method described inthis section to approximate the solution of the diffusion equation in one space dimensionon a finite interval with zero Neumann boundary conditions and given initial condition.As a test case, consider the spatial domain to be one unit in length, the diffusivity κ = 1,and the initial data given by f(x) = 1 + cosπx. Compare your numerical results with theanalytic solution. (b) Test the CFL number with at least two discretizations, one such that∆t/∆x2 = 0.4 and the other with ∆t/∆x2 = 0.6. Discuss your results.

36 CARMEN CHICONE

Exercise 3.10. (a) Modify your code written for Exercise 3.9 to approximate the solutionof the diffusion equation in one space dimension on a finite interval with mixed boundaryconditions and given initial condition. As a test case, consider the spatial domain to beone unit in length, the diffusivity κ = 10−3, the initial data given by f(x) = 1, and theboundary conditions

ux(0, t) = 0.01, u(1, t) = 1.

Draw a graph of the initial heat profile, its steady state profile, and five profiles at equallyspaced times between these extremes. (b) Determine the value of u at the midpoint of thespace interval (that is, at x = 0.5 at t = 10). The engineers who need this result want itcorrect to 4 decimal places. Can you assure them that your value meets this requirement?(c) Draw a graph of the function t 7→ u(0.5, t) on the time interval 0 ≤ t ≤ 20.

Exercise 3.11. [Ink Diffusion] The following is a constitutive model for the ink diffusionexperiment: a drop of ink is injected at one end of a trough and the front of ink is measuredas time passes. Let u(x, t) denote the concentration of ink at position x at time t and Lthe trough length. Also, let a and b be positive constants. The model equation of motionfor u is the initial BVP

ut = kuxx, ux(0, t) = ux(L, t) = 0, u(x, 0) =

a, 0 ≤ x ≤ b,0, x > b.

(a) What constitutive law is used to construct this model? (b) Show that the model predictsthe presence of ink at every position along the trough for every positive time. Does thisfact invalidate the model? (c) Show that the model predicts that the total amount of inkremains constant in time. (d) Define the diffusion front to be the largest distance from theorigin where the ink concentration is 1% of a. Use the model to determine the diffusionfront. If necessary, choose values for the parameters in the problem. (e) Is the distanceof the diffusion front from the origin a linear function of time? If not, what type is thisfunction? (f) Calibrate the model to the data given in Section ?? and discuss the modelprediction in view of the experimental data. (g) Construct a model that takes into accounttwo (or three) space dimensions. Compare the new front speed with the front speed obtainedfor the one-dimensional model. (h) Can you refine the model to give a more accuraterepresentation of the experiment? You may wish to perform your own experiment. Note:Perhaps diffusion is not the dominant mechanism that causes the ink to disperse. Thewater was left undisturbed for a long period of time to minimize residual fluid motions, butperhaps the effects of temperature cause convection currents that drive the ink movement.Could the proposed model be based on incorrect physics for the process under consideration?Discuss.

Exercise 3.12. (1) Consider a manufactured rod with uniform cross sections. The rodis 1 meter long and the thermal diffusivity of the construction material has been previouslymeasured to be 10−4 square meters per second. Also, the rod is in an insulating jacket sothat no heat can escape to the ambient environment. The initial temperatures along therod, measured in degrees celsius at 21 positions along the rod, are given in the following


table where the first coordinate is the position along the rod measured in meters and thesecond is the corresponding measured celsius temperature.

position temperature position temperature0.00 200.00 0.55 254.370.05 200.25 0.60 264.330.10 201.00 0.65 274.290.15 202.25 0.70 284.310.20 204.00 0.75 294.000.25 206.27 0.80 279.500.30 209.00 0.85 264.180.35 214.40 0.90 249.320.40 224.21 0.95 234.490.45 234.38 1.00 219.050.50 244.88

Heat sources are removed and the heat in the rod is allowed to diffuse. Determine a goodapproximation for the temperature of the rod at its midpoint 10 minutes later. Considera numerical approximation and an approximation using Fourier series. Compare yourresults. (2) Assume the same conditions and data as in part (1) except that the rod is nolonger completely insulated; only its ends are insulated. Heat is transferred from positionsalong the rod to the environment such that the rate of change of temperature is proportionalto their temperature differences with the constant ambient temperature of 20 degrees celsius.The constant of proportionality is 0.05 per minute. Determine a good approximation forthe temperature of the rod at its midpoint 10 minutes later. (3) An early version of part (2)stated a constant of proportionality of 0.5 per minute. A fall of 1/2 a degree per minute doesnot seem unreasonable. But, this reasoning is not a correct interpretation of the influenceoften constant of proportionality. What is a correct interpretation?

3.3.2. Propagation of a Mutant Gene. Consider a population with a mutant genewhose concentration is u. Its allele (the parent gene) has concentration 1 − u. Theindividuals in the population diffuse along a one-dimensional spatial domain (forexample a shoreline) and they interact with each other to produce offspring. A simplemodel (introduced in 1937 independently by R. A. Fisher [?] and A. N. Kolmogorov,I. Petrovskii, and N. Piscounov [?]) is

(31) ut = κuxx + au(1− u).

The choice of the model interaction term, given by f(u) = au(1− u), is akin to theprinciple of mass action and may be more fully justified using probability theory(see [?]). Of course, this term may also be viewed as perhaps the simplest model ofinteraction that agrees (qualitatively) with experiments. The constant a is meant tomodel the utility of the gene for the organism to survive: a > 0 for an advantageous

38 CARMEN CHICONE

mutation; a < 0 for a disadvantageous mutation. When the concentration of themutant gene is zero—no mutant genes in the population—the reaction term au(1−u)has value zero and u = 0 is a solution of the model equation; and, when the entirepopulation has the mutant gene—the concentration is one—the function u = 1 is asolution of the model equation.

There is a natural scientific question: How will an advantageous mutation spreadif it occurs in some individual or group of individuals at a specified spatial location?

The quantity u(x, t) is the number of individuals at time t at shoreline position xwith the mutant gene divided by the total number of individuals in the populationat x. Of course this is an idealization. In reality, we would measure this ratio oversome area (a fixed width times a length of shoreline). The concentration is moreprecisely defined to be the limit of this ratio as the area of the region shrinks tozero at x for the fixed time t. Thus, u—the ratio of two tallies—is a dimensionlessquantity. The time derivative ut has the dimensions of inverse time. In symbols,we write [ut] = 1/T , where in this formula the square brackets denote units of theenclosed expression and T denotes the unit of time (perhaps T is years). With thechoice of a unit L of length, the diffusion term has units [κuxx] = [κ]/L2. These mustagree with the units of the time derivative. Thus, [κ]/L2 = 1/T and [κ] = L2/T .Likewise, the units of the interaction term are carried by a and [a] = 1/T ; thus, ahas the units of a rate.

We are unlikely to know good values of the diffusivity κ (a measure of how fast theorganisms carrying the mutant gene spread along the shoreline) or a (which is thegrowth rate of the population with the mutant gene). Thus, we should not expectto make reliable quantitative predictions; rather, we should use the model to predictthe qualitative behavior of the spread of a mutant gene.

The qualitative behavior of the solutions of our model are independent of the(positive) parameters κ and a. In fact, we can simply eliminate the parameters bya change of variables. Let t = τs, where [τ ] = T (that is, τ has the dimensionsof time), and x = `ξ, where [`] = L. The change of variables is accomplished byapplying the chain rule. Ignoring the temporal variable for this computation, notethat

du

dt=du

ds

ds

dt=

1

τus,

du

dx=du

dξ

dξ

dx=

1

`uξ,

d2u

dx2=

1

`2uξξ.


-100 -50 0 50 100

0.0

0.2

0.4

0.6

0.8

1.0

Figure 3. Graphs of the spatial distribution of the mutant gene mod-eled by PDE (33) at times t = 5, 15, 30, and 45 are depicted for theinitial condition (34) with a = −2, b = 2, and µ = 0.01. The plottedcurves are thicker as time increases. The numerical method is forwardEuler (for the temporal variable) with time step 0.25, with Neumannboundary conditions, and a spatial grid of 200 interior points on thespatial domain [−100, 100].

By substitution into the model PDE [Eq. (31)]

1

τus = κ

1

`2uξξ + au(1− u)

and, by rearranging, we have the equation

us = κτ

`2uξξ + τau(1− u).

We may choose

(32) τ =1

a, ` =

√κ

a,

to obtain the desired dimensionless model

us = uξξ + u(1− u).

Reverting to the usual notation, the PDE

(33) ut = uxx + u(1− u)

is discussed here.For the biological application, the model PDE [Eq. (33)] may be considered for

mathematical convenience with the spatial variable x defined on the whole real line,or we may impose boundary conditions at the ends of some interval. For instance,the zero Dirichlet boundary condition (u vanishes at an end of the portion of theshoreline under consideration) or the zero Neumann boundary condition (ux vanishes)

40 CARMEN CHICONE

may be used. In the present context, the Dirichlet condition means that individualsat the end of the shoreline where it is imposed never carry the mutant gene; theNeumann condition means that no individuals with the mutant gene leave or enterthe population through the end of the shoreline where it is imposed. More generally,nonzero constant or time-dependent boundary conditions might also have plausiblephysical meanings.

The model [Eq. (33)] is a nonlinear PDE; the superposition of solutions is generallynot a solution. There is no known general explicit solution (cf. Exercise 3.13). But,it is possible to prove that unique solutions exist for appropriate initial conditionsu(x, 0) = f(x) (see, for example, [?]). Let us assume these results.

The simplest model has no diffusion. In this case, the PDE reduces to the ODEu = u(1 − u) (see Exercise ??). There are two steady states u = 0 and u = 1. If0 < u(x, 0) < 1 (that is, some individuals at the position x have the mutation), thenthe corresponding solution u(x, t) grows monotonically to u = 1 as t goes withoutbound. Thus, with the passage of time, the mutant gene eventually is establishedin the entire population at each location along the shoreline where there were someindividuals with the mutant gene.

Does the model predict that an advantageous mutant gene will spread to the entirepopulation when diffusion (and thus spatial dependence) is taken into account?

To begin the analysis of the PDE with diffusion, suppose that the initial populationwith the mutant gene is found only in one location along the shoreline. This situationmay be modeled by the initial function

(34) u(x, 0) = f(x) =

0; x < α,µ; a ≤ x ≤ β,0; x > β,

where α < β and 0 < µ ≤ 1. Numerical experiments are used to obtain the popula-tion profiles depicted in Fig. 3. The spread of the mutant gene seems to be a wavespreading in both directions from the spatial location of the initial population thatcarried the mutation. The wave speed for this simulation is approximately 10.

To help determine the wave speed in general, let us note that in our scaling[Eq. (32)], the characteristic velocity is length divided by time. Our length scale

is ` =√κ/a and our time scale is τ = 1/a. There is a unique characteristic velocity

given by

(35) characteristic velocity =√κa.

The wave speed should be a function of this characteristic velocity. Thus the depen-dence of the wave speed of a wave solution of the original model [Eq. (33)] shouldbe h(

√κa) for some scalar function h (see Exercise 3.17). This argument is a simple


-100 -50 0 50 100

0.0

0.2

0.4

0.6

0.8

1.0

Figure 4. Graphs of the spatial distribution of the mutant gene forFisher’s model [Eq. (33)] at times t = 20, 40, 60, and 80 are depictedfor the initial condition (34) with α = −100, β = −90, and µ = 1. Theplotted curves are thicker as time increases. The numerical method isforward Euler on the spatial domain [−100, 100] with Neumann bound-ary conditions, time step 0.1, and a spatial grid of 200 interior points.

example of an application of dimensional analysis, which is often useful to help deter-mine the functional dependence of some phenomenon on the parameters in a model.Dimensioned characteristic quantities (such as wave speed) should be functions ofterms that are ratios of monomials in the scales (such as τ and `), which are used tomake the model dimensionless.

Numerical experiments, using the PDE model for the spread of a mutant genefrom a location where the mutation arises, suggest a basic prediction: The spreadof the mutant gene has a wave front moving in both directions away from the initiallocation with the concentration of the mutant gene and the mutant gene saturatesthe population as the wave passes each remote location.

A related physical phenomenon is the spread of a mutant gene that is alreadydominant on the left side of a location along our beach but not present on the rightside. Some of the results of numerical experiments are reported in Fig. 4; they suggestthe local concentration of the mutant gene rises to dominate the population where themutation is already present and it spreads to the right at a constant speed with eachlocation being fully saturated as the wave front passes. A mathematical idealizationof this situation leads to the question: Is there a solution u of the dimensionlessmodel [Eq. (33)] such that, for each fixed position x,

(36) 0 ≤ u(x, t) ≤ 1, limt→−∞

u(x, t) = 0, limt→∞

u(x, t) = 1?

42 CARMEN CHICONE

-0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2

-0.30

-0.25

-0.20

-0.15

-0.10

-0.05

0.00

Figure 5. A portion of the phase portrait for the ODE system (41)with c = 3 is depicted. The flow crosses each thick line segment in thesame direction. The thick curve is an approximate trajectory connect-ing the rest points at (0, 0) and (1, 0).

The existence of a solution of this type would show that some solutions of the PDEhave the same qualitative behavior as a solution of u = u(1− u) (that is, the modeldifferential equation without diffusion) with initial value u(0) restricted to 0 < u(0) <1.

Inspection of Fig. 4 (or better yet an animation of the wave) suggests that the spa-tial concentration quickly approaches a wave that maintains its profile while movingto the right with constant velocity. Ideally, there is some function φ : R → R suchthat

(37) u(x, t) = φ(x− ct),

where c > 0 is the wave speed. A solution of this form is called a traveling wave withwave form φ.

By substitution of Eq. (37) into PDE (33), we obtain the differential equation

(38) −cφ′(x− ct) = φ′′(x− ct) + φ(x− ct)(1− φ(x− ct)),

which we may view as an ODE for the unknown wave profile φ with auxiliary con-ditions

(39) 0 ≤ φ ≤ 1, lims→−∞

φ(s) = 1, lims→∞

φ(s) = 0.


0 10 20 30 40 50 60

0.0

0.2

0.4

0.6

0.8

1.0

Figure 6. The figure depicts an approximation of the graph of thefunction φ that is the first coordinate function of the solution of theODE system (41) with c = 3 along the unstable manifold of the saddlepoint at (1, 0) that connects this point to the sink at the origin.

A basic fact is that if c ≥ 2, then there is a solution of the ODE

(40) φ+ cφ+ φ(1− φ) = 0

that satisfies the auxiliary conditions (39). This result implies that Fisher’s model[Eq. (33)] has a traveling wave solution that satisfies the original auxiliary condi-tions (36).

The second-order ODE [Eq. (40)] is equivalent to the first-order system of ODEs

φ = v,

v = −cv − φ(1− φ)(41)

in the phase plane.The first-order system (41) has two rest points at coordinates (0, 0) and (1, 0) in

the phase plane. The rest point at the origin is asymptotically stable and the restpoint at (1, 0) is a saddle point. If c ≥ 2, then the triangle depicted by thick linesegments is positively invariant. The horizontal segment connects the rest points,the ray with negative slope is in the direction of the eigenspace corresponding tothe negative eigenvalue of the linearized system matrix at (0, 0) with the largestabsolute value, and the ray with positive slope is in the direction of the eigenspacecorresponding to the unstable manifold of the linearized system matrix at (1, 0). Forc < 2, the eigenvalues of the linearized system at the origin are complex. Thus, theincoming unstable trajectory from the saddle point winds around the origin and φhas negative values, violating the condition that 0 ≤ φ ≤ 1 (see [?] for a detailedproof and Exercise 3.20).

44 CARMEN CHICONE

The connecting orbit in Fig. 5 corresponds to the desired function φ that de-fines a traveling wave solution. Of course, φ is the first coordinate function ofthe corresponding solution of the ODE system (41). Its graph, which has the pro-file of the expected traveling wave, is depicted in Fig. 6. Kolmogorov, Petrovskii,and Piscounov [?] proved that solutions of PDE (33) with initial data such that0 ≤ u(x, 0) ≤ 1, u(x, 0) = 1 for x ≤ α, and u(x, 0) = 0 for x ≥ β > α approach atraveling wave solution with wave speed c = 2 (see Exercise 3.19).

Exercise 3.13. Show that u(x, t) = φ(x−ct) is a solution of PDE (33) in case c = 5/√

6,K is a constant, and

φ(z) = (1 +Kez/√

6)−2.

Exercise 3.14. (a) Repeat the experiment reported in Fig. 3. (b) Repeat the experimentwith Dirichlet boundary conditions and compare the results.

Exercise 3.15. How does the speed of the wave front(s) of solutions of PDE (33) dependon the amplitude of the initial population with the mutant gene?

Exercise 3.16. How does the speed of the wave front(s) of solutions of PDE (33) dependon the length of the spatial interval occupied by the initial population with the mutant gene?

Exercise 3.17. Use numerical experiments to test the characteristic velocity approxima-tion in Eq. (35). Set an initial mutant gene concentration and vary the parameters κ anda in PDE (31).

Exercise 3.18. Reproduce Figs. 4–6.

Exercise 3.19. Use numerical experiments to verify the theorem of Kolmogorov, Petro-vskii, and Piscounov [?] that solutions of Fisher’s model [Eq. (33)] with initial data suchthat 0 ≤ u(x, 0) ≤ 1, u(x, 0) = 1 for x ≤ α, and u(x, 0) = 0 for x ≥ β > α approach atraveling wave solution with wave speed c = 2.

Exercise 3.20. (a) Find the system matrix at each rest point of system 41, find theeigenvalues and eigenvectors, and determine the stability types of the rest points. Showthe triangle as in Fig. 5 is positively invariant by proving the vector field corresponding tothe system of differential equations points into the region bounded by the triangle along theboundary of the region.

Exercise 3.21. Consider Fisher’s equation with a disadvantageous gene (a < 0) inEq. (31). (a) What happens when the disadvantageous gene is carried by all individu-als counted in a finite interval of the spatial domain? (b) Does the disadvantageous genealways disappear as time increases? Discuss.

Exercise 3.22. Consider the PDE

ut = uxx + u(1− u2)


and restrict attention to solutions u such that |u| ≤ 1. Note that u = 0 and u = ±1 aresteady states. (1) Which pairs of these steady states have solutions connecting them in thespace of solutions of the ODE ut = u(1− u2) obtained by ignoring the diffusion term uxx?(2) Which pairs of these steady states have traveling wave solutions connecting them in thespace of solutions of the PDE? Hint: Gather evidence using numerical experiments beforeattempting a pencil and paper solution.

Exercise 3.23. Consider Jin-ichi Nagumo’s equation

ut = uxx + u(1− u)(u− a),

where 0 < a < 1 is a parameter. A traveling wave f(x − ct) is called a front if its profilef has finite distinct limits at ±∞ and a pulse if these limits are equal. Does the Nagumoequation have front or pulse type traveling waves for some parameter values? Discuss usinganalysis and numerical experiments. If such solutions exist, graph typical front and pulseprofiles and determine the corresponding wave speeds.

Exercise 3.24. Consider a population where the concentration of individuals with anadvantageous gene u (which is the number with the gene divided by total number at a givenspatial position) is modeled by Fisher’s equation ut = κuxx + au(1 − u), where time ismeasured in months and distance in kilometers. The population resides on a shoreline 0.8kilometers long where at the left end of the shore no individuals can enter or leave thepopulation, but at the right end of the shoreline, individuals without the advantageous geneenter cyclically so that during the summer months of June, July, August, and Septemberthe concentration u at the right end of the beach is always measured to be approximately20% less than the concentration at the left end. During the winter months (December,January, February, and March) the concentration at the right end of the beach is unaffectedby migration. Spring and fall are transitional periods where the concentration builds up infall and tapers off during spring to the mentioned levels. Suppose that the system constantsare κ = 0.002 square kilometers per month and a = 0.05 per month. The initial populationwith the advantageous gene is measured in June to be 10% over three-quarters of the beachmeasured from left to right and 8% over the remainder of the beach. What is the percentof the population with the gene halfway along the beach after 5 years? Hint: There couldbe more than one viable model for the boundary conditions. Discuss your choice(s).

Date post:	06-May-2018
Category:	Documents
Upload:	buibao
View:	214 times
Download:	0 times

A GLIMPSE INTO APPLIED MATHEMATICS UNIVERSITY OF ALBERTA ...irl/IUSEP/lecture_notes/chicone.pdf ·...

Documents