PRINCIPLES OF APPLIED Transformation and Approximation ...fnarc/m641/keener_chapter_1.pdf · vi...

To Kristine, Sammy, and Justin, Still my best friends after all these years.

PRINCIPLES OF

APPLIED

MATHEMATICS

Transformation and Approximation Revised Edition

JAMES P. KEENER University of Utah Salt Lake City, Utah

ADVANCED BOOK PROGRAM

~ ew I I I I

Member of the Perseus Books Group

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book and Westview Press was aware of a trademark claim, the designations have been printed in initial capital letters.

Library of Congress Catalog Card Number: 99-067545

ISBN: 0-7382-0129-4

Copyright© 2000 by James P. Keener

Westview Press books are available at special discounts for bulk purchases in the U.S. by corporations, institutions, and other organizations. For more information, please contact the Special Markets Department at the Perseus Books Group, 11 Cambridge Center, Cambridge, MA 02142, or call (617) 252-5298.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electric, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher. Printed in the United States of America.

Westview Press is a member of the Perseus Books Group

First printing, December 1999

Find us on the World Wide Web at http:/ /www.westviewpress.com

Contents

Preface to First Edition

Preface to Second Edition

1 Finite Dimensional Vector Spaces 1.1 Linear Vector Spaces ....... . 1.2 Spectral Theory for Matrices . . . 1.3 Geometrical Significance of Eigenvalues 1.4 Fredholm Alternative Theorem . . . . . 1.5 Least Squares Solutions-Pseudo Inverses .

1.5.1 The Problem of Procrustes .... 1.6 Applications of Eigenvalues and Eigenfunctions

1.6.1 Exponentiation of Matrices . . . . . . . 1.6.2 The Power Method and Positive Matrices

1.6.3 Iteration Methods

Further Reading . . . . Problems for Chapter 1

2 Function Spaces 2.1 Complete Vector Spaces .... .

2.1.1 Sobolev Spaces ..... . 2.2 Approximation in Hilbert Spaces

2.2.1 Fourier Series and Completeness 2.2.2 Orthogonal Polynomials . . . . 2.2.3 Trigonometric Series . . . . . 2.2.4 Discrete Fourier Transforms .

2.2.5 2.2.6

Sine Functions Wavelets ....

2.2. 7 Finite Elements . Further Reading . . . . Problems for Chapter 2 . . .

v

xi

xvii

1 1 9

17 24 25 41 42 42 43 45 47 49

59 59 65 67 67 69 73 76 78 79 88 92 93

vi CONTENTS

3 Integral Equations 101 3.1 Introduction .............................. 101 3.2 Bounded Linear Operators in Hilbert Space . . . . . . . . . . . . 105 3.3 Compact Operators ......................... 111 3.4 Spectral Theory for Compact Operators .............. 114 3.5 Resolvent and Pseudo-Resolvent Kernels . . . . . . . . . . . . . . 118 3.6 Approximate Solutions . . . . . . . . . . . . . . . . . . . . . . . . 121 3. 7 Singular Integral Equations . . . . . . . . . . . . . . . . . . . . . 125 Further R.eading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 Problems for Chapter 3 . . . . . . . . . . . . . . . . · . . . . . . . . . . 128

4 Differential Operators 4.1 Distributions and the Delta Function . . . . . . . . . . . . . . . . 4.2 Green's Functions ......................... . 4.3 Differential Operators . . . . . . . . . . . . . . . . . . . . . . . .

4.3.1 Domain of an Operator . . . . . . . . . . . . . . . . . . 4.3.2 Adjoint of an Operator .................. . 4.3.3 Inhomogeneous Boundary Data . . . . . . . . . . . . . . . 4.3.4 The Fredholm Alternative . . . . . . . . . . . . . . . . . .

4.4 Least Squares Solutions . . . . . . . . . . . . . . . . . . . . . . . 4.5 Eigenfunction Expansions . . . . . . . . . . . . . . . . . . . .

4.5.1 'Irigonometric Functions . . . . . . . . . . . . . . . . . . . 4.5.2 Orthogonal Polynomials . . . . . . . . . . . . . . . . . . . 4.5.3 Special Functions . . . . . . . . . . . . . . . . . . . . . . . 4.5.4 Discretized Operators . . . . . . . . . . . . . . . . . . . .

Further R.eading . . . . . . . . . . . . . . . . . · · · · · · · · · · · · · Problems for Chapter 4 . . . . . . . . . . . . . . . . . . . . . . . . . .

133 133 144 151 151 152 154 155 157 161 164 167 169 169 171 171

5 Calculus of V~iations 177 5.1 The Euler-Lagrange Equations ................... 177

5.1.1 Constrained Problems .................... 180 5.1.2 Several Unknown Functions . . . . . . . . . . . . . . . . . 181 5.1.3 Higher Order Derivatives . . . . . . . . . . . . . . . . . . 184 5.1.4 Variable Endpoints ...................... 184 5.1.5 Several Independent Variables . . . . . . . . . . . . . . . . 185

5.2 Hamilton's Principle ......................... 186 5.2.1 The Swinging Pendulum . . . . . . . . . . . . . . . . . . . 188 5.2.2 The Vibrating String . . . . . . . . . . . . . . . . . . . . . 189 5.2.3 The Vibrating Rod . . . . . . . . . . . . . . . . . . . . . . 189 5.2.4 Nonlinear Deformations of a Thin Beam . . . . . . . . . . 193 5.2.5 A Vibrating Membrane . . . . . . . . . . . . . . . . . . . 194

5.3 Approximate Methods ........................ 195 5.4 Eigenvalue Problems . . . . . . . . . . . . . . . . . . . . . . . . . 198

5.4.1 Optimal Design of Structures . . . . . . . . . . . . . . . . 201 Further R.eading . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . 202 Problems for Chapter 5 . . . . . . . . . . . . . . . . . . . . . . . . . . 203

?

I ~

I

I I ii!

I "

I J ~ ~

i -~

I ~ ~ §

I " s I i ~ • I I li

~

I I ! i J iii ~

CONTENTS vii

6 Complex Variable Theory 209 6.1 Complex Valued Functions ..................... 209 6.2 The Calculus of Complex Functions . . . . . . . . . . . . . . . . 214

6.2.1 Differentiation-Analytic Functions ............. 214 6.2.2 Integration . . . . . . . . . . . . . . . . . . . . . . . . . . 217 6.2.3 Cauchy Integral Formula . . . . . . . . . . . . . . . . . . 220 6.2.4 Taylor and Laurent Series . . . . . . . . . . . . . . . . . . 224

6.3 Fluid Flow and Conformal Mappings . . . . . . . . . . . . . . . . 228 6.3.1 Laplace's Equation . . . . . . . . . . . . . . . . . . . . . . 228 6.3.2 Conformal Mappings . . . . . . . . . . . . . . . . . . . . . 236 6.3.3 Free Boundary Problems . . . . . . . . . . . . . . . . . . 243

6.4 Contour Integration . . . . . . . . . . . . . . . . . . . . . . . . . 248 6.5 Special Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

6.5.1 The Gamma Function . . . . . . . . . . . . . . . . . . . . 259 6.5.2 Bessel Functions . . . . . . . . . . . . . . . . . . . . . . . 262 6.5.3 Legendre Functions . . . . . . . . . . . . . . . . . . . . . . 268 6.5.4 Sine Functions . . . . . . . . . . . . . . . . . . . . . . . . 270

Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 Problems for Chapter 6 . . . . . . . . . . . . . . . . . . . . . . . . ~ . 27 4

7 Transform and Spectral Theory 283 7.1 Spectrum of an Operator ...................... 283 7.2 Fourier Transforms ........ , ................. 284

7.2.1 Transform Pairs .................... , .. 284 7.2.2 Completeness of Hermite and Laguerre Polynomials ... 297 7.2.3 Sine Functions ........................ 299 7.2.4 Windowed Fourier Transforms ............... 300 7.2.5 Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

7.3 Related Integral Transforms ..................... 307 7.3.1 Laplace Transform ...................... 307 7.3.2 Mellin Transform ....................... 308 7.3.3 Hankel Transform ...................... 309

7.4 Z Transforms ............................. 310 7.5 Scattering Theory .......................... 312

7.5.1 Scattering Examples ..................... 318 7.5.2 Spectral Representations ................... 325

Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327 Problems for Chapter 7 . . . . . . . . . . . . . . . . . . . . . . . , . . 328 Appendix: Fourier Transform Pairs . . . . . . . . . . . . . . . . . . . . 335

8 Partial Differential Equations 337 8.1 Poisson's Equation .......................... 339

8.1.1 Fundamental Solutions .................... 339 8.1.2 The Method of Images .................... 343 8.1.3 Transform Methods . . . . . . . . . . . . . . . . . . . . . 344 8.1.4 Hilbert Transforms ...................... 355

viii CONTENTS

8.1.5 Boundary Integral Equations ................ 357 8.1.6 Eigenfunctions . . . . . . . . . . . . . . . . . . . . . . . . 359

8.2 The Wave Equation . . . . . . . . . . . . . . . . . . . . . . . . . 365 8.2.1 Derivations .......................... 365 8.2.2 Fundamental Solutions . . . . . . . . . . . . . . . . . . . . 368 8.2.3 Vibrations . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 8.2.4 Diffraction Patterns . . . . . . . . . . . . . . . . . . . . . 376

8.3 The a:eat Equation . . . . . . . . . . . . . . . . . . . . . . . . . . 380 8.3.1 Derivations .......................... 380 8.3.2 Fundamental Solutions . . . . . . . . . . . . . . . . . . . . 383 8.3.3 Transform Methods . . . . . . . . . . . . . . . . . . . . . 385

8.4 Differential-Difference Equations .................. 390 8.4.1 Transform Methods ..................... 392 8.4.2 Numerical Methods . . . . . . . . . . . . . . . . . . . . . 395

Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400 Problems for Chapter 8 . . . . . . . . . . . . . . . . . . . . . . . . . . 402

9 Inverse Scattering Transform 411 9.1 Inverse Scattering .......................... 411 9.2 Isospectral Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . 417 9.3 Korteweg-deVries Equation ..................... 421 9.4 The Toda Lattice . . . . . . . . . . . . . . . . . . . . . . . . . . . 426 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432 Problems for Chapter 9 . . . . . . . . . . . . . . . . . . . . . . . . . . 433

10 Asymptotic Expansions 437 10.1 Definitions and Properties ..................... 437 10.2 Integration by Parts ......................... 440 10.3 Laplace's Method . . . . . . . . . . . . . . . . . . . . . . . . . . . 442 10.4 Method of Steepest Descents . . . . . . . . . . . . . . . . . . . ; 449 10.5 Method of Stationary Phase . . . . . . . . . . . . . . . . . . . . . 456 Further Reading . . . . . . . . ~ . . . . . . . . . . . . . . . . . . . . . 463 Problems for Chapter 10 . . . . . . . . . . . . . . . . . . . . . . . . . . 463

11 Regular Perturbation Theory 469 11.1 The Implicit Function Theorem .................. 469 11.2 Perturbation of Eigenvalues ..................... 475 11.3 Nonlinear Eigenvalue Problems ................... 478

11.3.1 Lyapunov-Schmidt Method ................. 482 11.4 Oscillations and Periodic Solutions ................. 482

11.4.1 Advance of the Perihelion of Mercury . . . . . . . . . . . 483 11.4.2 Vander Pol Oscillator .................... 485 11.4.3 Knotted Vortex Filaments .................. 488 11.4.4 The Melnikov Function . . . . . . . . . . . . . . . . . . . 493

11.5 Hopf Bifurcations ...... _. .................... 494 Further Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496

.~ I ~

I i i ~ ~

I I

~ I I I 'I ! I * I I ~

CONTENTS ix

Problems for Chapter 11 . . . . . . . . . . . . . . . . . . . . . . . . . . 498

12 Singular Perturbation Theory 505 12.1 Initial Value Problems I . . . . . . . . . . . . . . . . . . . . . . . 505

12.1.1 Van der Pol Equation .................... 508 12.1.2 Adiabatic lnvariance . . . . . . . . . . . . . . . . . . . . . 510 12.1.3 Averaging . . . . . . . . . . . . . . . , . . . . . . . . . . . 511 12.1.4 Homogenization Theory ....... , ........... 514

12.2 Initial Value Problems II . . . . . . . . . . . , . . . . . . . . . . . 520 12.2.1 Operational Amplifiers . . . . . . . . . . . . . . . . . . . . 521 12.2.2 Enzyme Kinetics . . . . . . . . . . . . . . . . . . . . . . . 523 12.2.3 Slow Selection in Population Genetics . . . . . . . . . . . 526

12.3 Boundary Value Problems . . . . . . . . . . . . . . . . . . . . . . 528 12.3.1 Matched Asymptotic Expansions . . . . . . . . . . . . . . 528 12.3.2 Flame Fronts . . . . . . . . . . . . . . . . . . . . . . . . . 539 12.3.3 Relaxation Dynamics . . . . . . . . . . . . . . . . . . . . 542 12.3.4 Exponentially Slow Motion . . . . . . . . . . . . . . . . . 548

Further Reading .............................. 551 Problems for Chapter 12 . . . . . . . . . . . . . . . . . . . . . . . . . . 552

Bibliography

Selected Hints and Solutions

Index

559

567

596

1 1

I ~

-?£

I I t I t

I • (1

Preface to First Edition

Applied mathematics should read like good mystery, with an intriguing beginning, a clever but systematic middle, and a satisfying resolution at the end. Often, however, the resolution of one mystery opens up a whole new problem, and the process starts all over. For the applied mathematical scientist, there is the goal to explain or predict the behavior of some physical situation. One begins by constructing a mathematical model which captures the essential features of the problem without masking its content with overwhelming detail. Then comes the analysis of the model where every possible tool is tried, and some new tools developed, in order to understand the behavior of the model as thoroughly as possible. Finally, one must interpret and compare these results with real world facts. Sometimes this comparison is quite satisfactory, but most often one discovers that important features of the problem are not adequately accounted for, and the process begins again.

Although every problem has its own distinctive features, through the years it has become apparent that there are a group of tools that are essential to the analysis of problems in many disciplines. This book is about those tools. But more than being just a grab bag of tools and techniques, the purpose of this book is to show that there is a systematic, even esthetic, explanation of how these classical tools work and fit together as a unit.

Much of applied mathematical analysis can be summarized by the observation that we continually attempt to reduce our problems to ones that we already know how to solve.

This philosophy is illustrated by an anecdote (apocryphal, I hope) of a mathematician and an engineer who, as chance would have it, took a gourmet cooking class together. As this course started with the basics, the first lesson was on how to boil a pot of water. The instructor presented the students with two pots of water, one on the counter, one on a cold burner, while anot_her burner was already quite hot, but empty .

In order to test their cooking aptitudes, the instructor first asked the engineer to demonstrate to the rest of the class how to boil the pot of water that was already sitting on the stove. He naturally moved the pot carefully from the cold burner to the hot burner, to the appreciation of his classmates. To be sure the class understood the process, the pot was moved back to its original spot on the cold burner at which time the mathematician was asked to demonstrate how to heat the water in the pot sitting on the counter. He promptly and confidently

xi

xii PREFACE TO FIRST EDITION

exchanged the position of the two pots, placing the pot from the counter onto the cold burner and the pot from the burner onto the counter. He then stepped back from the stove, expecting an appreciative response from his mates. Baffled by his actions, the instructor asked him to explain what he had done, and he replied naturally, that he had simply reduced his problem to one that everyone already knew how to solve.

We shall also make it our goal to reduce problems to those which we already know how to solve.

We can illustrate this underlying idea with simple examples, we know that to solve the algebraic equation 3x = 2, we multiply both sides of the equation with the inverse of the "operator" 3, namely 1/3, to obtain x = 2/3. The same is true if we wish to solve the matrix equation Ax = b where

A=(~!). b=(~)· Namely, if we know A-t, the inverse of the matrix operator A, we multiply both sides of the equation by A-t to obtain x : A - 1b. For this problem,

A-t=! ( 3 -1 ) 8 -1 3 ' x=~(!)·

H we make it our goal to invert many kinds of linear operators, including matrix, integral, and differential operators, we will certainly be able to solve many types of problems. However, there is an approach to calculating the inverse operator that also gives us geometrical insight. We try to transform the original problem into a simpler problem which we already know how to solve. Fbr example, if we rewrite the equation Ax= bas T-1 AT(T-1x) = T-1b and choose

- ( 1 -1 ) -1- 1 ( 1 1 ) T- 1 1 I T - 2 -1 1 I

we find that

T-1 AT = ( ~ -~ ) .

With the change of variables y = T-1x, g = T-1b, the new problem looks like two of the easy algebraic equations we already know how to solve, namely 4yt = 3/2, -2y2 = 1/2. This process of separating the coupled equations into uncoupled equations works only if T is carefully chosen, and exactly how this is done is still a mystery. Suffice it to say, the original problem has been transformed, by a carefully chosen change of coordinate system, into a problem we already know how to solve.

This process of changing coordinate systems is very useful in many other problems. For example, suppose we wish to solve the boundary value problem u"- 2u = l(x) with u(O) = u(1r) = 0. As we do not yet know how to write down an inverse operator, we look for an alternative approach. The single most important step is deciding how to represent the solution u(x). For example, we

PREFACE TO FIRST EDITION xiii

might try to represent u and I as polynomials or infinite power series in x, but we quickly learn that this guess does not simplify the solution process much. Instead, the ''natural" choice is to represent u and I as trigonometric series,

00

u(x) = L u~,: sinkx, k=l

00

l(x) = L 1~.: sin kx. k=l

Using this representation (i.e., coordinate system) we find that the original differential equation reduces to the infinite number of separated algebraic equations (k2 + 2)u~,: = -1~.:· Since these equations are separated (the kth equation depends only on the unknown u~.:), we can solve them just as before, even though there are an infinite number of equations. We have managed to simplify this problem by transforming into the correctly chosen coordinate system.

For many of the problems we encounter in the sciences, there is a natural way to represent the solution that transforms the problem ip.to a substantially easier one. All of the well-known special functions, including Legendre polynomials, Bessel functions, Fourier series, Fourier integrals, etc., have as their common motivation that they are natural for certain problems, and perfectly ridiculous in others. It is important to know when to choose one transform over another.

Not all problems can be solved exactly, and it is a mistake to always look for exact solutions. The second basic technique of applied mathematics is to reduce hard problems to easier problems by ignoring small terms. For example, to find the roots of the polynomial x2 + x + .0001 = 0, we notice that the equation is very close to the equation x2 + x = 0 which has roots x = -1, and x = 0, and we suspect that the roots of the original polynomial are not too much different from these. Finding how changes in parameters affect the solution is the goal of perturbation theory and asymptotic analysis, and in this example we have a regular perturbation problem, since the solution is a regular (i.e., analytic) function of the "parameter" 0.0001.

Not all reductions lead to such obvious conclusions. For example, the polynomial 0.0001x2 + x + 1 = 0 is close to the polynomial x + 1 = 0, but the first has two roots while the second has only one. Where did the second root go? We know, of course, that there is a very large root that "goes to infinity" as "0.0001 goes to zero," and this example shows that our naive idea of setting all small parameters to zero must be done with care. As we see, not all problems with small parameters are regular, but some have a singular behavior. We need to know how to distinguish between regular and singular approximations, and what to do in each case.

This book is written for beginning graduate students in applied mathematics, science, and engineering, and is appropriate as a one-year course in applied mathematical techniques (although I have never been able to cover all of this material in one year). We assume that the students have studied at an introductory undergraduate level material on linear algebra, ordinary and partial differential equations, and complex variables. The emphasis of the book is a working, systematic understanding of classical techniques in a modern context. Along the way, students are exposed to models from a variety of disciplines.

xiv PREFACE TO FIRST EDITION

It is hoped that this course will prepare students for further study of modem techniques and in-depth modeling in their own specific discipline.

One book cannot do justice to all of applied mathematics, and in an effort to keep the amount of material at a manageable size, many important topics were not included. In fact, each of the twelve chapters could easily be expanded into an entire book. The topics included here have been selected, not only for their scientific importance, but also because they allow a logical flow to the development of the ideas and techniques of transform theory and asymptotic analysis. The theme of transform theory is introduced for matrices in Chapter one, and revisited for integral equations in Chapter three, for ordinary differential equations in Chapters four and seven, for partial differential equations in Chapter eight, and for certain nonlinear evolution equations in Chapter nine.

Once we know how to solve a wide variety of problema via transform theory, it becomes appropriate to see what harder problems we can reduce to those weknow how to solve. Thus, in Chapters ten, eleven, and twelve we give a survey of the three basic areas of asymptotic analysis, namely asymptotic analysis of integrals, regular perturbation theory and singular perturbation theory.

Here is a summary of the text, chapter by chapter. In Chapter one, we review the basics of spectral theory for matrices with the goal of understanding not just the mechanics of how to solve matrix equations, but more importantly, the geometry of the solution process, and the crucial role played by eigenvalues and eigenvectors in finding useful changes of coordinate systems. This usefulness extends to pseudo-inverse operators as well as operators in HUbert space, and so is a particularly important piece of background information.

In Chapter two, we extend many of the notions of finite dimensional vector spaces to function spaces. The main goal is to show how to represent objects in a function space. Thus, Hilbert spaces and representation of functions in a Hilbert space are studied. In this context we meet classical sets of functions such as Fourier series and Legendre polynomials, as well as less well-known sets such as the Walsh functions, Sine functions, and finite element bases.

In Chapter three, we explore the strong analogy between integral equations and matrix equations, and examine again the consequences of spectral theory. This chapter is more abstract than others as it is an introduction to functional analysis and compact operator theory, given under the guise of Fredholm integral equations. The added generality is important as a framework for things to come.

In Chapter four, we develop the tools necessary to use spectral decompositions to solve differential equations. In particular, distributions and Green's functions are used as the means by which the theory of compact operators can be applied to differential operators. With these tools in place, the completeness of eigenfunctions of Sturm-Liouville operators follows directly.

Chapter five is devoted to showing how many classical differential equations can be derived from a variational principle, and how the eigenvalues of a differential operator vary as the operator is changed.

Chapter six is a pivotal chapter, since all chapters following it require a substantial understanding of analytic function theory, and the chapters preceding it require no such knowledge. In particUlar, knowing how to integrate and differ-

PREFACE TO FIRST EDITION XV

entiate analytic functions at a reasonably sophisticated level is indispensable to the remaining text. Section 6.3 (Applications to Fluid Flow) is included because it is a classically important and very lovely subject, but it plays no role in the remaining development of the book, and could be skipped if time constraints demand.

Chapter seven continues the development of transform theory and we show that eigenvalues and eigenfunctions are not always sufficient to build a transform, and that operators having continuous spectrum require a generaliZed construction. It is in this context that Fourier, Mellin, Hankel, and Z transforms, as well as scattering theory for the SchrOdinger operator are studied.

In Chapter eight we show how to solve linear partial differential and difference equations, with special emphasis on (as you guessed) transform theory. In this chapter we are able to make specific use of all the techniques introduced so far and solve some problems with interesting applications.

Although much of transform theory is rather old, it is by no means dead. In Chapter nine we show how transform theory has recently been used to solve certain nonlinear evolution equations. We illustrate the inverse scattering transform on the Korteweg-deVries equation and the Toda lattice.

In Chapter ten we show how asymptotic methods can be used to approximate the horrendous integral expressions that so often result from transform techniques.

In Chapter eleven we show how perturbation theory and especially the study of nonlinear eigenvalue problems uses knowledge of the spectrum of a linear operator in a fundamental way. The nonlinear problems in this chapter all have the feature that their solutions are close to the solutions of a nearby linear problem.

Singular perturbation problema fail to have this property, but have solutions that differ markedly from the naive simplified problem. In Chapter twelve, we give a survey of the three basic singular perturbation problema (slowly varying oscillations, initial value problems with vastly different time scales, and boundary value problems with boundary layers).

This book has the lofty goal of reaching students of mathematics, science, and engineering. To keep the attention of mathematicians, one must be systematic, and include theorems and proofs, but not too many explicit calculations. To interest an engineer or scientist, one must give specific examples of how to solve meaningful problema but not too many proofs or too much abstraction. In other words, there is always someone who is unhappy.

In an effort to minimize the total displeasure and not scare off too many members of either camp, there are some proofs and some computations. Early in the text, there are disproportionately more proofs than computations because as the foundations are being laid, the proofs often give important insights. Later, after the bases are established, they are invoked in the problem solving process, but proofs are deemphasized. (For example, there are no proofs in Chapters eleven and twelve.) Experience has shown that this approach is, if not optimal, at least palatable to both sides.

This is intended to be an applied mathematics book and yet there is very

xvi PREFACE TO FIRST EDITION

little mention of numerical methods. How can this be? The answer is simple and, I hope, satisfactory. This book is primarily about the principles that one uses to solve problems and since these principles often have consequence in numerical algorithms, mention of numerical routines is made when appropriate. However, FORTRAN codes or other specific implementations can be found in many other good resources and so (except for a code for fast Walsh transforms) are omitted. On the other hand, many of the calculations in this text should be done routinely using symbolic manipulation languages such as REDUCE or MACSYMA. Since these languages are not generally familiar to many, included here are a number of short programs written in REDUCE in order to encourage readers to learn how to use these remarkable tools. (Some of the problems at the end of several chapters are intended to be so tedious that they force the reader to learn one of these languages.)

This text would not have been possible were it not for the hard work and encouragement of many other people. There were numerous students who struggled through this material without the benefit of written notes while the course was evolving. More recently, Prof. Calvin Wilcox, Prof. Frank Hoppensteadt, Gary deYoung, Fred Phelps, and Paul Arner have been very influential in the shaping of this presentation. Finally, the patience of Annetta Cochran and Shannon Ferguson while typing from my illegible scrawling was exemplary.

In spite of all the best efforts of everyone involved, I am sure there are still typographical errors in the text. It is disturbing that I can read and reread a section of text and still not catch all the erors. My hope is that those that remain are both few and obvius and will not lead to undue confusion.

Preface to the Second Edition

When the first edition of this book was published, l thought that it was an up-to-date look at classical methods of applied mathematics. I did not expect the basic tools of applied mathematicians to change dramatically in the near future. Indeed the basic ideas remain the same, but there are many ways in which those basic ideas have found new applications and extensions.

Some of the most significant new techniques of the last decade are wavelet analysis, multigrid methods, and homogenization theory, all of which are extensions of methods already discussed in the first edition. Additionally, software tools have become much more sophisticated and reliable, and it is not possible to ignore these developments in the training of an applied mathematician.

So the first reason to revise this book is to bring it up to date, by adding material describing these new and important methods of applied mathematics.

The second reason for this revision is to make it easier to read and use. To that end, the text has been thoroughly edited, with emphasis on clarity and filling in the gaps. I have tried to eliminate those annoying places where a step was described as obvious, but is far from obvious to the reader. I have also tried to eliminate many of the (mostly typographical, but nonetheless noisome) errors that were in the first edition. I have added equation numbers to those equations that are referenced in the text, and many figures have been added to aid the geometrical interpretation and understanding. Finally, I have added a section of hints and solutions for the exercises. Many of the exercises are difficult and both students and instructors find such a section to be most helpful.

Many new exercises have been added, mostly those which are intended to use modern software for symbolic computation and graphical interpretation. The first edition of this book referred to REDUCE, which, sadly, disappeared from widespread use almost immediately after the publication of that edition. The good news is that the new software tools are much better. My personal preferences are Maple and Matlab, but other choices include Mathematica, MathCad, Derive, etc. Exercises that are marked with .II!, are designed to encourage computer usage with one of these software packages. I have mentioned Maple throughout the text, but the intent is "generic software package with symbolic computation and graphics capability".

xvii

xviii PREFACE TO SECOND EDITION

While the basic philosophy of the text remains unchanged, there are important ways in which most chapters have been modified. In particular, here are some of the major modifications and additions by chapter:

Chapter 1. A section describing the problem of Procrustes (who devised an unusual method for coordinate transformations) has been added. Also, a section on applications of eigenvalues and eigenfunctions of matrices has been added, wherein matrix iterative methods and the ranking of sports teams are discussed.

Chapter 2. A section introducing wavelets has been added, replacing the discussion of Walsh functions.

Chapter 4. The discussion of extended operators, which I always found to be awkWard, is replaced by a much more transparent and straightforward discussion of inhomogeneous boundary conditions using integration by parts.

Chapter 5. An example of how variational principles are used to design optimal structures has been added.

Chapter 7. Further development and construction of wavelets, as well as other transforms (such as windowed Fourier transforms) is included.

Chapter 8. A discussion of the Hilbert transform and boundary integral methods has been added, along with some interesting new applications. There is also added emphasis on diffusion-reaction equations, with a discussion of the Turing instability, and derivations of the CalmAllen and Calm-Hilliard equations. Finally, a discussion of multigrid methods for the numerical solution of Poisson's equation has been added.

Chapter 11. A discussion of the Lyapunov-Schmidt technique for bifurcation problems has been added, as has a description of the Melnikov function.

Chapter 12. The discussion of the averaging theorem has been expanded, while discussions of homogenization theory and exponentially slow motion of transition layers have been added.

The content of Chapters 3, 6, 9 and 10 has changed little from the first edition.

As with any project of this size, I owe a debt of gratitude to many people who helped. Eric Cytrynbaum, Young Seon Lee, Todd Shaw and Peter Bates found many of the errors that I missed and made numerous helpful suggestions about the presentation, while David Eyre and Andrej Cherkaev made many helpful suggestions for new material to include. Steve Worcester did the initial 'J.EX-ing of the manuscript in record time. Nelson Beebe is a 'J.EX-pert, whose wizardry never ceases to astound me and who8e help was indispensable. Thanks to the

PREFACE TO SECOND EDITION xix

folks at Calvin College for providing me with a quiet place to work for several months, and to the University of Utah for a sabbatical leave during which this revision was finished.

Further corrections or modifications can be found at the Internet site http://www.math.utah.edu/-keener/AppliedMathBook.html.

James P. Keener University of Utah

June 1999

Chapter 1

Finite Dimensional Vector Spaces

1.1 Linear Vector Spaces

In any problem that we wish to solve, the goal is to find a particular object, chosen from among a large collection of contenders, which satisfies the governing constraints of the problem. The collection of contending objects is often a vector space, and although individual elements of the set can be ca.lled by many different names, it is common to ca.ll them vectors. Not every set of objects constitutes a vector space. To be specific, if we have a collection of objects S, we must first define addition and sca.la.r multiplication for these objects. The operations of addition and sca.lar multiplication are defined to satisfy the usual properties: H x,y,z E S, then

• x + y = y + x (commutative law)

• x + (y + z) = (x + y) + z (associative law)

• 0 E S, 0 + x = x (additive identity)

• -xES, -x + x = 0 (additive inverse)

and if xES, a, {3 E IR (or <U) then

• a({jx) = (a{j)x

• (a + {j)x = ax + {jx

• a(x + y) = ax+ ay

• lx = x, Ox =0.

Once the operations of addition and scalar multiplication are defined, we define a vector space as follows:

1

2 CHAPTER 1. FINITE DIMENSIONAL VECTOR SPACES

Definition 1.1 A set of objects Sis called a linear vector space if two properties hold:

1. If x, y E S, then x + y E S (called closure under vector addition),

2. If a E IR (or a E ([;') and x E S, then ax E S (called closure under scalar multiplication).

If the scalars are real, S is a real vector space, while if the scalars are complex, S is a complex vector space.

Examples:

1. The set of real ordered pairs (:z:,y) (denoted m?) is a linear vector space if addition is defined by (:z:1, yt) + (:z:2, y2) = (:z:1 + x2, Yt + y2), and scalar multiplication is defined by a(:z:,y) = (a:z:,ay).

2. The set of real n-tuples (:z:t, x2, ... , :Z:n) (denoted lRn) forms a linear vector space if addition and scalar multiplication are defined component-wise as in the above example. Similarly the set of complex n-tuples (denoted <IJR) forms a linear vector space with the complex scalars.

3. The set of all polynomials of degree n forms a linear vector space, with "vectors" Pn(:z:) = :E.i=o ai:z:i, with Clj E 1R (or Clj E IV), if addition and scalar multiplication are defined in the usual way.

4. The set of all continuous functions defined on some interval [a, b] forms a linear vector space.

There are many sets that are not linear vector spaces. For example, the set of all ordered pairs of the form (x, 1) is not closed under addition or scalar multiplication and so does not form a vector space. On the other hand, ordered pairs of the form (x, 3x) do comprise a linear vector space. Similarly, the set of continuous functions /(x) defined on the interval [a,b) with f(a) = 1 does not form a vector space, while if instead the constraint /(a) = 0 is imposed, the properties of closure hold.

In a vector space, we can express one element of S in terms of additions and scalar multiplications of other elements. For example, if Xt. x2, ... , Xn, are elements of S, then x = a1x1 + a2x2 + · · · + anXn is also inS, and xis said to be a linear combination of Xt, x2, ... , Xn· If there is some linear combination with a1x1 + a 2x2 + · · · + anXn = 0, and not all of the scalars ai are zero, then the set { x1, x2, ••• , Xn} is said to be linearly dependent. On the other hand, if the only linear combination of Xt, x2, ... , Xn which is zero has all ai equal to zero, the set { x1, x2, •.• , Xn} is said to be linearly independent.

Examples:

1. The single nontrivial vector x E lRn, :z: ::j: 0 forms a linearly independent set because the only way to have a:z: = 0 is if a = 0.

1.1. LINEAR VECTOR SPACES 3

2. A linear combination of two vectors in m?, Clt(Xt,Yl) + Cl2(:Z:2,Y2), is the zero vector whenever :z:1 = -/3:z:2 and Yt = -f3y2 where f3 = a2/a1. Geometrically, this means that the vector (:z:t, yl) is collinear with the vector (:z::~, y2), i.e., they are parallel vectors in 1R2.

Important examples of linearly independent vectors are the monomials. That is, the powers of x, {1, x, x2 , ••• , xn} form a linearly independent set. An easy proof of linear independence is to define f(x) = ao + alx + a2x2 + ... + anxn and to determine when f(x) is identically zero for all x. Clearly, /(0) = ao and if f(x) = 0 for all x, then /(0) = 0 implies a0 = 0. The kth derivative of f(x) at x = 0,

dk f(x) I = k!ak d,xk z=O

is zero if and only if ak = 0. Therefore, f(x) = 0 if and only if ai = 0 for j = 0, 1, 2, ... , n.

It is often convenient to represent an element of the vector space as a linear combination of a predetermined set of elements of S. For example, we might want to represent the continuous functions as a linear combination of certain polynomials or of certain trigonometric functions. We ask the question, given a subset of S, when can any element of S be written as a linear combination of elements of the specified subset?

Definition 1.2 A set of vectors T C S is a spanning set if every x E S can be written as a linear combination of the elements ofT.

Definition 1.3 The span of a set of vectors T is the collection of all vectors that are linear combinations of the vectors in T.

Definition 1.4 If a spanning set T c S is linearly independent then T is a basis for S. If the number of elements ofT is finite, then S is a finite dimensional vector space whose dimension is the number of elements ofT.

One can easily show (see Problem 1.1.1) that every basis of S has the same number of elements. For a given basis T, a vector inS has a unique representation in terms of the elements of T.

Examples:

1. In 1R2, any three vectors are linearly dependent, but any two noncollinear

vectors form a basis.

2. In 1R3 the vectors { (1, 0, 0), (0, 1, 0), (0, 0, 1), (1, 1, 0), (0, 1, 1)} form a spanning set, but the last two vectors are not necessary to form a spanning set. From this set, there are eight different ways to choose a subset of 3 linearly independent elements which is a basis. In lRn, the so-called natural basis is the set of vectors {et, e2, ... , en} where the jth entry of e~c is Okj where o,i = 0 if j ::j: k and O~ok = 1. The object o"i is called the Kronecker delta.


3. The set {1,z,z2,z2 -1} is linearly dependent. However, any of the subsets {1,z,z2}, {1,z,z2 - 1}, or {z,z2,z2 - 1} forms a basis for the set of quadratic polynomials.

4.· The set of all polynomials does not have finite dimension, since any linear combination of polynomials of degree n or less cannot be a polynomial of degree n + 1. The set of all continuous functions is also infinite since the polynomials are a subset of this set. We will show later that the polynomials form a "basis" for the continuous functions, but this requires more technical information than we have available now.

When you first learned about vectors in an undergraduate Physics or Mathematics course, you were probably told that vectors have direction and magnitude. This is certainly true in JR3 , but in more complicated vector spaces the concept of direction is hard to visualize. The direction of a vector is always given relative to some other reference vector by the angle between the two vectors. To get a concept of direction and angles in general vector spaces we introduce the notion of an inner product.

Definition 1.5 For x, y E S, the inner product (also called scalar product or dot product) of x andy, denoted (x,lf), is a function{-,·): S x S ~ IR, (or (U if Sis a complex vector space) with the properties:

1. (x,y) = (y,z), (overline denotes complex conjugate)

2. (ax,y) = a(x,y},

3. (x+y,z)=(x,z)+(y,z),

4. (x,x) > 0 if x :/: 0, (x,x) = 0 if and only if x = 0.

A linear vector space with an inner product is called an inner product space.

Examples:

1. In JR.", suppose z = (zt, :1:2, ... , Zn) and Sl =(Sit, f12, ... , Sin) then

n

<x, 11> = E x,y, lo•l

is the usual Euclidean inner product. H x, Sl e (U" I then

n

(:~:, y) = E :l:loYio

lo=l

is an inner product.

(1.1)

2. For two real continuous functions l(x) and g(z) defined for :1: e [0,1], we discretize l(x) to define a vector in mn IF= (f(xt), l(z2), ... ,/(xn)) and

1.1. LINEAR VECTOR SPACES

similarly we discretize g(x) to define G = (g(zl),g(z2), ... ,g(xn)), where :z:,. = kfn, and it makes sense to define

1 n

(F, G)n = - 2: f(x~c)g(xlc)· n k=l

(While this is an inner product on JR", this is not ap inner product for the continuous functions. Why?) Taking the limit n -+ oo, we obtain

(/,g} = 11

f(x)g(x)dz. (1.2)

It is an easy matter to verify that (1.2) defines an inner product on the vector space of continuous functions.

3. There are other interesting ways to define an inner product for functions. For example, if the complex valued functions I and g are continuously differentiable on the interval [0, 1], we might define

(/,g)= 11

(!(z)g(x) + /'(z)gl(z)) dz.

More generally, if I and g are n times continuously differentiable on [0, 1] we might define

(f,g} = r ( t di !(~)dig(~)) dx lo \j,.,

0 dx1 dxJ

(1.3)

as an inner product. Indeed, one can show that these satisfy the required properties of inner products.

The magnitude of a vector can also be defined for a general vector space.

5

Definition 1.6 For x E S, the norm (also called amplitude, magnitude, or length) of x, denoted llxll, is a function 11·11: S-+ [O,oo) with the properties:

1. llxll > 0 if x :/: 0 and llxll = 0 implies x = 0,

2. llaxll = lal·llxll for a E IR (or a E <U if Sis a complex vector space),

3. llx + Yll $ llxll + IIYII (triangle inequality).

In JRn, we can take llxll = (L:~=1 Ixkl2 ) 1 12 or more generally, llxll = (L:~=1 1xkiP)11P, 1 < p < oo. With p = 2, this is the Euclidean norm or "2"-norm, however, for differing values of p the meaning of size changes. For example, in JR2 the "unit sphere" llxll = 1 with p = 2 is the circle x~ + x~ = 1 while with p = 1 the "unit sphere" lx1l + lx2l = 1 is a diamond (a square with vertices on the axes). H we let p ~ oo, then llxll = maxk lxkl = llxlloo is called the "infinity" or "sup" (for supremum) norm, and the "unit sphere" llxll = 1 is a square (Fig. 1.1).

A linear vector space with a norm is called a normed vector space. Although there are many ways to define norms, one direct way to find a norm is


(a) (b) (c)

Figure 1.1: The unit sphere llxll = 1 for (a) the "1"-norm, {b) the "2"-norm, (c) the sup norm.

if we have an inner product space. If x E Sand (x,y} is an inner product for S then llxll = J(x,x} is called the induced norm (or natural norm) for S. For example, in (Jln, the Euclidean norm

(

n )1/2 llxll = t; lx~~:l2 , {1.4)

is induced by the Euclidean inner product {1.1), and for real continuous func

tions with inner product (/,g)= J: f(x)g(x)dx, the induced norm is

(

b )1/2 11/11 = L1f(x)l2dx

For n times continuously differentiable complex valued functions, the inner product (1.3) induces the norm

1\/1\ = (!.' ~ ldl £,<;l r dz r For any induced norm, llxll > 0 if x :/: 0, and llaxll = lal· llxll for scalars a.

To see that the triangle inequality also holds, we must first prove an important result:

Theorem 1.1 (Schwarz1 inequality) For x,y in an inner product space,

l(x, v>l ~ llxii·IIYII· {1.5)

Proof: For x, yES, a a scalar,

0 ~ llx- ayll2 = llxll2 - 2Re((x, ay}) + lai2 IIYII2

•

1This theorem is also associated with the names of Cauchy and Bunyakowsky.

1.1. LINEAR VECTOR SPACES 7

If y =f 0, we pick a= fMtt, (thereby minimizing llx- ay!l2) from which the Schwarz inequality is immediate. 1

Using the Schwarz inequality, we see that the triangle inequality holds for any induced norm, since

or

llx + Yll2 = llxW + 2Re{ (x, y)) + IIYII2

~ llxll2 + 2llxll·llvll + llvll2

= (llxll + llviD2,

llx +vii ~ llxll + IIYII· One can show that in JR2, cos(}= ~ where (x, y) is the usual Euclidean

inner product and(} is the angle between the vectors x andy. In other real inner product spaces, such as the space of continuous functions, we take this to be the definition of cosO. The law of cosines and the Pythagorean theorem are immediate. For example,

llx + vW = llxll2 + 2llxll . IIYII cos(} + IIYW

is the law of cosines, and if cos() = 0 then

llx + Yll2 = llxll2 + IIYW (Pythagorean Theorem).

Following this definition of cos (), we say that the vectors x and y are orthogonal if (x, y) = 0.

Examples:

1. The vector (1, 0) ia orthogonal to (0, 1) in JR? using the Euclidean inner product since ((0, 1), (1, 0)} = 0.

2. With inner product (!,g)= J;.,. f(x)g(x)dx, sinx ap.d sin2x are orthogonal · r"J1r · · 2 dx 0 smce Jo smxsm x = .

3. With inner product (!,g)= J01 f(x)g(x)dx, the angle 0 between the func

tions f(x) = 1 and g(x) = x on [0, 1) is 30° since cosO= ~ = .;'3/2.

Orthogonal vectors are nice for several reasons. If the vectors { ¢1, ¢2, ... , ¢n} E S, cfoi =f 0, are mutually orthogonal, that is, (¢i,¢;) = 0 fori =f j, then they form a linearly independent set. To see this suppose there are scalars a1, a2, ... , an so that

a1cfo1 + a2¢2 + · · · + ancfon = 0.

The inner product of this expression with ¢; is

(¢;, 01¢1 + 02¢2 + · · · + ancfon) = (¢;,a;¢;) = 0:; llc/Jj 112 = 0,


so that ai = 0 for j = 1, 2, ... , n, provided t/J; is nontrivial. H the set {1/Jt. t/J2, ... , t/Jn} forms a basis for S, we can represent any element

f E S as a linear combination of 1/J;,

n

I= Lf3itP3· j=l

To determine the coefficients /3;, we take the inner product of f with tPi and find a matrix equation B/3 = '7 where B = (bi;), btj = (1/Jj,tPi}, {3 = ({3i), '1i = (/, t/Jt}. This matrix problem is always uniquely solvable since the tPt 's are linearly independent. However, the solution is simplified enormously when the tPi 's are mutually orthogonal, since then B is a diagonal matrix and easily inverted to yield

{3 (!, tPi}

i = llt/Jill2 •

The coefficient f3t carries with it an appealing geometrical interpretation in m,n. H we want to qrthogonally "project" a vector f onto ¢ we might imagine shining a light onto tjJ and measuring the shadow cast by f. From simple trigonometry, the length of this shadow is II/II cosO where 8 is the angle in the plane defined by the vectors f and 1/J. This length is exactly ~ and the vector

of the shadow, ~1/J, is called the projection off onto 1/J. Notice that the quantity llx- 'i'YII is minimized by picking 'i'Y to be the projection of x onto y, . ~ 1.e., 'i' = TTUfP"Y·

The Gram-Schmidt orthogonalization procedure is an inductive technique used to generate a mutually orthogonal set from any linearly independent set of vectors. Given the vectors Xt, x2, ... , Xn, we set t/J1 = Xt. To find a vector t/J2 that is orthogonal to t/J1, we set t/J2 = x2 - a¢1, and use the requirement ( t/J2, t/J1) = 0 to determine that

so that {x2,tPt)

tP2 = X2- llt/Jtll2 tPt•

In other words, we get tjJ2 by subtracting from x2 the projection of x2 onto tPt (See Fig. 1.2}. Proceeding inductively, we find t/Jn by subtracting the successive projections,

n-1 ( ) .J. ~ Xn, tPi t/J; 'l'n = Xn - LJ 111/J 112 '

j=1 j

from which it is clear that (1/J~;, t/Jn) = 0 for k < n. We are left with mutually orthogonal vectors c/>1, t/J2, .•. , c/>n. which have the same span as the original set.

1.2. SPECTRAL THEORY FOR MATRICES 9

.. ~\),

\

\, q, 2 \

Xz

Figure 1.2: Graphical representation of the Gram-Schmidt orthogonalization procedure to produce tP2 from Xt = c/>1 and x2.

Example:

Consider the powers of x, {1, x, x2, ••• , xn}, which on any interval [a, b] (a<

b) form a linearly independent set. With [a,b] = [-1, 1] and the inner product

(!,g) = [1

1 f(x)g(x)dx,

the Gram-Schmidt procedure produces

if>o = 1, l/>1 =x, l/>2 = x 2 -1/3, l/>3 = x3

- 3x/5,

and so on. The functions thus generated are called the Legendre polynomials. Other sets of orthogonal polynomials are found by changing the underlying interval or the definition of the inner product. We will see more of these in Chapter 2.

1.2 Spectral Theory for Matrices

In a vector space, there are many different possible choices for a basis. The main message of this section and, in fact, of this text, is that by a careful choice of basis, many problems can be "diagonalized." This is also the main idea behind transform theory.

Suppose we want to solve the matrix problem

Ax=b,

where A is an n x n matrix. We view the entries of the vectors x and b as the coordinates of vectors relative to some basis, usually the natural basis. That is,

n

x = (x1,x2, ... ,xn)T = L.x;ef. j=l


The same vector x would have different coordinates if it were expressed relative to a different basis.

Suppose the matrix equation Ax = b is expressed in coordinates relative to the basis {t/J1 , '1/J?., ••• , t/Jn} and we wish tore-express the problem relative to some other basis { tPb tP2, ... , t/>n}. What does this change of basis do to the original representation of the matrix problem?

Since { t/Ji} forms a basis, for each vector t/>;, there are numbers c~i), i =

1, 2, ... , n for which t/>; = E;=t c~i) tPi. The numbers c~i) are the coordinates of the vector t/>; relative to the basis {t/JJ}· In the case that {tPJ} is the natural basis, the number c~i) is the jth element of the vector tj>;.

H $; and :z:~ are coordinates of a vector x relative to the bases { t/J;} and { tPi}, respectively, then

n n n (n ) n (n ) x = I;x~:t/J~c = 'Ex~tj>; = I;x~ I;c~')t/Ji = 2:; 'Ec~'>x~ t/JJ, A:=l i=t i=t j=l i=l i=l

so that, since the representation of a vector in terms of a given basis is unique, ZJ = E~1 c~')x~. Written in vector notation,$= C:z:' where Cis the matrix of coefficients that expresses the basis { t/>;} in terms of the basis { t/J;}, with the number c~') in the jth row and ith column of C.

Now the original problem A$= b becomes A'x' = c-tACx' = b', where A' = c-tAC, z, bare the coordinates of x,b relative to the original basis { t/J;}, and x', b' are the coordinates of :z:, b relative to the new basis { 1{1;}. The transformation A' = c-1 AC is called a similarity transformation, and we see that all similarity transformations. are equivalent to a change of basis and vice versa. Two matrices are said to be equivalent or similar if there is a similarity transformation between them.

This transformation can be made a bit clearer by examining what happens when the original basis is the natural basis. H AE is the representation of A with respect to the natural basis and { t/>;} is the new basis, then 0 1 is the matrix whose columns are the vectors tj>;. The representation of A with respect to {t/>;} is At = 01t AECt. Similarly, if C~ has as its columns the basis vectors {t/Ji}, then the representation of A with respect to {t/J;} is A~ = C:J1 AEC~. Thus, if we are given A~, the representation of A with respect to {t/J;}, and wish to find At, the representation of A with respect to {t/>1}, we find that

At= 01t AECt = C11 (C~A~c;t)Ct = (C;tCt)-1 A~(c;tCt)·

In other words, the matrix C that transforms A~ to At is C = c;tct where Ct has t/>; in its ith column and C~ has t/J; in its ith column.

Under what conditions is there a change of basis (equivalently, a similarity transformation) that renders the matrix A diagonal? Apparently we must find a matrix C so that AC = C A where A is diagonal, that is, the column vectors of C, say :z:;, must satisfy A:z:, = ~;:z:;, where A; is the ith diagonal element of A.


Definition 1. 7 An eigenpair of A is a pair (A, :z:), A E <V, :z: E <lin satisfying

Ax = A:z:, :z: i- 0.

The vector :z: is called the eigenvector and A is called the eigenvalue of A.

A number.>. is an eigenvalue of A if and only if the equation (A- >.I)x = 0 has a nontrivial solution, that is, if and only if the matrix A- >.I is singular. It follows that A is an eigenvalue of A if and only if it is a root of the nth order polynomial

PA(A) = det(A- >.I), (1.6) called the characteristic polynomial for A.

There are always n roots, counting multiplicity, of an nth order polynomial. The order of any particular root is called its algebraic multiplicity while the number of linearly independent eigenvectors for a given eigenvalue is called its geometric multiplicity. The geometric multiplicity can be no greater than the algebraic multiplicity for an eigenvalue A.

Our motivation for finding eigenvectors is that they provide a way to represent a matrix operator as a diagonal operator. Another important interpretation of eigenvectors is geometrical. If we view the matrix A as a transformation that transforms one vector into another, then the equation Ax = .>.:z: expresses the fact that some vector, when it is transformed by A, has its direction unchanged, even though the length of the vector may have changed. For example, a rigid body rotation (in three dimensions) always has an axis about which the rotation takes place, which therefore is the invariant direction.

Theorem 1.2 Suppose A is an n x n matri:z:.

1. If the matri:z: A has n linearly independent real (or complex} eigenvectors, there is a real (or complex) change of basis in Jl(l (or <lin) so that relative to the new basis A is diagonal.

2. lfT is the matri:z: whose columns are the eigenvectors of A, then T-1 AT= A is the diagonal matri:z: of eigenvalues.

The factorization A= TAT-1 is called the spectral representation of A. Proof: Suppose :z:1, X2, ... , :Z:n are linearly independent eigenvectors of A, with Axi = Ai:z:;. LetT be the matrix with columns of vectors Xi. Then we have

AT = A(:z:1 :z:2 ... Xn)[ ~1{A:z:1 A:z:2 ... Ax]n) = (A1:z:1 A2:z:2 ... Ana:n)

.>.2 0 = (:z:1 :z:2 ... :z:n) = TA,

0 An

where A is diagonal, or T-1AT=A.

I


Examples:

1. The matrix

A=(~!) has eigenpairs ~~ = 4, XI = (1,1)T, and ~2 = 2, x2 = (1,-1)T2

• The vectors X1, x2 are real and linearly independent. The matrix

T= ( 1 1) 1 -1

gives the required change of basis and

T- 1AT= ( 4 0) 0 2 .

2. The matrix A= ( c?s(} -sin(} )

smO cos(}

transforms the vector (cos¢, sin c/J)T into the vector (cos(¢+0),sin(¢+0))T and so is called a rotational matrix. This matrix has eigenvalues ~ = sin(}± icos(} and eigenvectors (=Fi, l)T. There is no real change of basis which diagonalizes A, although it can be diagonalized using complex basis vectors.

3. The matrix

A=(~~) has characteristic polynomial ~2 = 0, so there is one eigenvalue .>. = 0 with algebraic multiplicity two. However, there is only one eigenvector x1 = (1, O)T, so the geometric multiplicity is one. There is no change of basis which diagonalizes A.

We need to determine when there are n linearly independent eigenvectors. As we see from the last example, the dilemma is that even though the characteristic polynomial (1.6) always has n roots, counting multiplicity, there need not be an equal number of linearly independent eigenvectors. It is this possible deficiency that is of concern. For any eigenvalue >., the geometric multiplicity is at least one, since det(A- >.I) = 0 implies there is at least one nontrivial vector x for which (A- >.I)x = 0. In fact, every square matrix A has at least one eigenpair (although it may be complex).

Theorem 1.3 If A has n distinct eigenvalues, then it has n linearly independent eigenvectors.

2Remark about notation: When dealing with vectors and matrices, it is important to distinguish between row vectors and column vectors. For example, (1, 2) is a row vector and

( ~ ) is a column vector. However, it is often convenient to write column vectors as the

transpose of a row vector, for example, ( ~ ) = (1, 2)T.


Proof: The proof of this statement is by induction. For each eigenvalue >..~:, we let x.~: be the corresponding eigenvector. For k = 1, the single vector x1 =F 0 forms a linearly independent set. For the induction step, we suppose x1,x:l,· .. ,x.~:-1 are linearly independent. We try to find scalars a1, a2, ... ,a.~: so that

a1x1 + a2x2 + · · · + a~cx1c = 0.

If we multiply this expression by A and use that Axi = AiXi, we learn that

a1>.1x1 + a2>.2x2 + · · · + a~c>.~cx~c = 0.

If we multiply instead by >..~:, we obtain

>.~ca1x1 + >.1ca2x2 + · · · + >.~ca~cx~c = 0.

Subtracting the second of these expressions from the first, we find that

a1(>.1- >..~:)xl + · · · + a.~:-1(>..~:-1- >..~:)x.~:-1 = 0.

It follows that a1 = a2 = ··· = a1c-1 = 0, since xl,X2,···•XA:-l are assumed to be linearly independent and Ale is distinct from >.1, >.2, ... , Aie-l, and finally a.~:= 0 since x~c is assumed to be nonzero.

I There is another important class of matrices for which diagonalization is

always possible, namely the self-adjoint matrices, but before we can discuss these, we must define the adjoint of a matrix.

Definition 1.8 For any matrix A, the adjoint of A is defined as the matrix A* where (Ax,y) = (x,A*y), for all x, yin (i;ffl.

Definition 1.9 If A= (a;J), the transpose of A is AT= (a;,).

To find the adjoint matrix A • explicitly, note that (using the Euclidean inner product) n,n n (n )

(Ax, y) = L L aiJXJfii = L Xj L aiJili i=l j=l j=l i=l

so that A • = AT. For example, if

(

au A= a21

aa1

a1:l ) a22 , aa2

then A • = ( au ~~ aa1 ) .

Cii2 a:l2 aa:l

Definition 1.10 A matrix A is self-adjoint if A • = A.


H A is self-adjoint and complex, then A is called a Hermitian matrix whereas, if A is self-adjoint and real, it is symmetric. Notice that by definition, a self-adjoint matrix is square.

Theorem 1.4 If A is self-adjoint the following statements are true:

1. (Ax, x) is real for all x,

f. All eigenvalues are real,

9. Eigenvectors of distinct eigenvalues are orthogonal,

4. There is an orthonormal basis formed by the eigenvectors,

5. The matrix A can be diagonalized.

The proof of this theorem is as follows:

1. H A= A*, then (Ax,x) = (x,A*x) = (x,Ax) = (Ax,x), so (Ax,x) is real.

2. H Ax= Ax, then (Ax,x) = (Ax,x} = ,\(x,x). Since (Ax,x} and (x,x} are real, ,\ must be real.

3. Consider the eigenpairs ( ..\, x) and (#-', y). Then

..\(x, y) = (.\x, y) = (Ax, y) = (x, Ay) = (x, 1-'Y} = J.l(x, y)

so that (,\- J.l){X, y) = 0.

H ,\and 1-' are distinct(,\ f: J.l), then (x, y) = 0, i.e., x andy are orthogonal.

The proof of item 4 requires more background on linear manifolds.

Definition 1.11

1. A linear manifold M C S is a subset of S which is closed under vector addition and scalar multiplication.

2. An invariant manifold M for the matrix A is a linear manifold M C S for which x E M implies that Ax E M.

Examples:

1. N(A), the null space of A, is the set of all x for which Ax= 0. If x andy satisfy Ax= 0 and Ay = 0, then A( ax+ {3y) = 0 so that N(A) is a linear manifold. If x e N(A), then Ax= 0 so since A(Ax) = AO = 0, Ax is in N(A) as well, hence N(A) is an invariant manifold.

2. R(A), the range of A, is the set of all x for which Ay = x for some y. Clearly R(A) is a linear manifold and it is invariant since if x E R(A) then surely Axe R(A).


We observe that if M is an invariant manifold over the complex scalar field for some matrix A, there is at least one vector x E M with Ax = ..\x for some ..\ E a::'. To verify this, notice that since M lies in a k-dimensional space, it has a basis, say {xi. x2, ... , xk}· For any x E M, Ax E M so x and Ax have representations relative to the basis {xi, x2, ... , Xk}· In particular, take

x == I:~=l aixi and Axi = I:7=l f3jiXj· To solve Ax- ..\x = 0, we must have

k k

0 = L:ai(Axi)- ..\ L:aixi i=l i=l

k (k ) k k = ~ai j;f3jiXj- ..\xi = t;xi ~(f3ji- AOij)ai.

Observing that {XI, x2, ... , Xk} is a linearly independent set, we have

k

L(f3ii- Mii)ai = 0, j = 1,2, ... ,k, i=I

which in matrix notation is (B- ,\l)a = 0 where B = ((3ji) is a k x k matrix, and I is the k x k identity matrix. Of course, as we noted earlier, every square matrix has at least one eigenpair, which concludes the verification .

We are now ready to prove item 4, namely that the eigenvectors of an n x n self-adjoint matrix form an n-dimensional orthogonal basis. Suppose A = A*. Then there is at least one eigenpair (..\IJ x 1) with Axi = ..\1xi and ..\1 real and the eigenvector x1 forms a linearly independent set (of one element). For the induction step, suppose we have found k - 1 mutually orthogonal eigenvectors Xi, Axi = AiXi with Ai real fori= 1, 2, ... , k- 1. We form the linear manifold

Mk = {xi (x, Xj) = 0, j = 1, 2, ... , k - 1 },

called the orthogonal complement of the k- 1 orthogonal eigenvectors x1, x2, ... , Xk-l· This manifold is invariant for A since, ifx E Mk, then (x,xj) = 0 and

(Ax,xj) = (x,Axj) = ,\j(x,x3) = 0

for j = 1, 2, ... , k- 1. Therefore, Mk contains (at least) one eigenvector Xk corresponding to a real eigenvalue ..\k and clearly (xk, Xj) = 0 for j < k, since Xk E Mk. The eigenvalue Ak is real since all eigenvalues of A are real. In summary, we can state the main result of this section:

Theorem 1.5 (Spectral Decomposition Theorem) If A is ann x n selfadjoint matrix, there is an orthogonal basis {x1, x2, ... , Xn} for which

1. Axi = AiXi with Ai real.

2. (xi,Xj) = Dij (orthogonality).


9. The matrix Q with x; as its jth column vector is unitary, that is,

q-1 = Q*.

4. Q* AQ =A where A is a diagonal matrix with real entries Ai·

Suppose we wish to solve Ax = b where A • = A. We now know that relative to the basis of eigenvectors, this is a diagonal problem. That is, setting x = Qx', b = Qb' we find that Q* AQx' = Ax' = b'. H all the eigenvalues are nonzero, the diagonal matrix A is easily inverted for x' = A - 1b'. Rewriting this in terms of the original basis we find that x = QA-1Q*b. These operations can be summarized in the following commuting diagram.

Ax=b A-l ~ X= A-1b

x=Qx' .t. t x' =Q*x

b=Qb' b' = Q*b

Ax'= b' A-1 ~ x' = A-1b'

The solution of Ax = b can be found directly by applying the inverse of A (the top of the diagram) or indirectly in three steps by making a change of coordinate system to a diagonal problem (the leftmost vertical descent), solving the diagonal problem (the bottom of the diagram), and then changing coordinates back to the original coordinate system (the rightmost vertical ascent).

The geometry of diagonalization can be illustrated on a piece of graph paper. Suppose we wish to solve Ax = b where

A= ( -~ 1) -1 1 . (1.7)

We first calculate the eigenvalues and eigenvectors of A and find At = !, with eigenvector

Xt = ( ~) and A2 = -1, with eigenvector

X2 = ( i)' We plot these eigenvectors on a piece of rectangular graph paper. Any vector b can be represented· uniquely as a linear combination of the two basis vectorE XlJ x2, say b = a1x1 + a2x2. To find the solution x of Ax = b (since x = 2a1x1 - a2x2) we take twice the first coordinate of b added to the negative of the second coordinate of b with respect to the eigenvector basis. These arE depicted in Fig. 1.3.

As was suggested in the preface, this diagonalization procedure is the basif on which most transform methods work, and is reiterated throughout this text. Hopefully, this commuting diagram will become emblazoned in your mind beforE the end of this text.

1.3. GEOMETRICAL SIGNIFICANCE OF EIGENVALUES

I I

I

I I

I I

I I

I

I

I I

I

I ; I ;

I ; I ;;

~-

I I

I I

X

I I

I I

I

,.""'""'' ' I

,' I ;; I

I I

I

I

I I

I

I ;' r-

I I

-

I ,I

17

-~ b

Figure 1.3: Graphical solution of the 2 x 2 matrix equation Ax = b with A given by (1.7), using the eigenvectors x 1 and x2 of the matrix A as coordinates.

1.3 Geometrical Significance of Eigenvalues

Aside from providing a natural basis in JR" in which to represent linear problems, eigenvectors and eigenvalues have significance relating to the geometry of the linear transformation A.

For a real symmetric matrix A, the quadratic form q(x) = (Ax, x) produces a real number for every vector x in JR". Since q( x) is a continuous function in IR", it attains a maximum on the closed, bounded set of vectors x with llxll = 1. Suppose the maximum is attained at x = x1• Then for every unit vector x orthogonal to x1, q(x) :::; q(xt). But, on the subset (x,x1) = 0, llxll = 1, q(x) again attains a maximum at, say, x = x2. Continuing inductively in this way we produce a set of mutually orthogonal unit vectors, x1, X2, ••• , Xn at which the local extremal values of q(x) are attained on the set llxll = 1. We will show that the x, are the eigenvectors of A and that the values q(xi) are the corresponding eigenvalues.

Theorem 1.6 (Maximum Principle) If A is a real symmetric matrix and q(x) = (Ax,x}, the following statements hold:

1. At = maxllzll=l q(x) = q(x1) is the largest eigenvalue of the matrix A and x1 is the eigenvector corresponding to eigenvalue Al·

2. (inductive statement) Let Ak = maxq(x) subject to the constraints

a. (x,x;} = 0, j = 1, 2, ... 'k - 1,


b. llxll = 1.

Then .X~c = q(x~c) is the kth eigenvalue of A, .Xt 2: .X2 2: · · · 2: .X~~: and x~c is the corresponding eigenvector of A.

Examples:

1. Consider the matrix

A=(~ !)• then q(x) = (Az,z) = 3vl + 2111112 + 3vt where

z = ( t/1 ) t/2 •

We change to polar coordinates by setting 111 = cos 6, t/2 = sin 6 so that q(6) = 3+sin26. Clearly, maxq(6) = 4 occurs at 6 = 1r/4 and minq(6) = 2 occurs at 6 = -'fr/4. It follows that ~1 = 4, z1 = (~, ~)T and ~2 = 2, .., _ ( 1 1 )T .u2 - V'2• -V'2 •

2. In differential geometry, the curvature tensor of a surface is a syiilmeiric 2 x 2 matrix. As a result, the directions of maximal and minimal curvature on any two-dimensional surface are always orthogonal. You can check this out by a careful look at the shape of your forearm, for example.

To understand the geometrical significance of the maximum principle, we view the quadtatic form q(x) = (.Az,x) as a radial map of the sphere. That is, to each point on the sphere llxll = 1, associate q(x) with the projection of Ax onto x, q(x)x = (Ax, x)x. H A is positive definite (a positive definite matrix A is one for which (Ax, x) > 0 for all x :/= 0), the surface q(x)x looks like an ellipsoid (an American or rugby football) in JRn. (It is not exactly an ellipsoid; the level surface q(x) = 1 is an ellipsoid.) The maximum of q(x) occurs along the major axis of the football-shaped object. H we intersect it with a plane orthogonal to the major axis, then restricted to this plane, the maximum of q(x) occurs along the semi-major axis, and so on.

To verify statement 1, we apply the method of Lagrange multipliers to q(x). Specifically, we seek to maximize the function Pt(x) = q(x)- p((x,x) -1). H the maximum occurs at x11 then P1(x1 +h) -Pt(xt) ~ 0 for any h. Expanding Pt(Xt +h) we find that

Pt(Xt +h)- Pt(Xt) = 2{(Azt, h)- p(x1, h))+ (Ah, h)- p(h, h).

Notice that (Az1 , h) - p(x1 , h) is linear in h and (Ah, h) - p(h, h) is quadratic in h. The sign of this quadratic expression cannot be changed by changing the sign of h. However, unless the linear expression is identically zero, its sign can be changed by reversing the sign of h. Thus, if h is sufficiently small, and the linear expression is not identically zero, the sign of P1(x1 +h)- Pt(Xt) can be

GEOMETRICAL SIGNIFICANCE OF EIGENVALUES 19

changed by reversing the sign of h, which is not permitted. Thus, we must have (Az1 - J.'Zl, h) = 0 for all h. Taking h = Azt - J.'Xt, we conclude that

Azt - J.'Zl = 0,

so that Xt is the eigenvector of A. Furthermore, q(xt) = (Axt, Xt) = p(xlt Xt) = · p, since llxtll = 1 so that max q( x) = p is the eigenvalue and the vector that maximizes q(x) is an eigenvector of A.

To verify the inductive step, we again use Lagrange multipliers and maximize

k-1 P~c(x) = (.Az,x)- p((x,x) -1)- L qJ(X,xj),

J=l

where X1,x2, ... , Xk-1 are eigenvectors of A corresponcling to known eigenvalues .X1 2: A2 ... 2: Aie-l, and p, Ctj, j = 1, ... , k- 1 are Lagrange multipliers. Notice that 8?Je> = 0 implies that (x,x) = 1 and 8-?a~z) = 0 implies that (x,XJ) = 0. Suppose x~,: maximizes P~.:(x) so that P~.:(x~.: + h) - P~.:(x~.:) ~ 0. Using that (x~c,Xj) = 0 for j = 1, 2, ... , k- 1, we find that

P~,:(x~c +h)- P~,:(x~,:) = 2((.Az~,:,h}- p(x~,:,h}) + (Ah,h)- p(h,h} .

Following the same argument as before, we require (Ax~~:- pz~,:, h) = 0 for all h. In other words, Ax~~: - J.'Xk = 0 and x~~: is an eigenvector of A. Finally, q(x~.:) = (Az~~:,x~~:) = p(x~c,x~.:) = p since llx~~:ll = 1 so that maxq(x) is the eigenvalue Ale and the vector that maximizes q(x) is an eigenvector of A. Clearly Ale ~ Alc-t ~ .. · ~At· I

From the maximum principle we can find the kth eigenvalue and eigenvector only after the previous k - 1 eigenvectors are known. It is sometimes useful to have a characterization of Ale that makes no reference to other eigenpairs. This is precisely the motivation of

Theorem 1.7 (Courant Minimax Principle) For any real symmetric matrix A,

.X~c =min max (.Az,x), C llzll=l

Cez:O

where C is any (k- 1) x n matrix.

The geometrical idea of this characterization is as follows. For any nontrivial matrix C, the constraint Cx = 0 corresponds to the requirement that x lie on some n - k + 1 dimensional hyperplane in JRn. We find A~,: by first maximizing q(x) with llxll = 1 on the hyperplane and then we vary this hyperplane until this maximum is as small as possible.

For example, imagine slicing a football with a plane through the origin to get an ellipse, and finding the principal axis of the ellipse, then varying the orientation of the slicing plane until the length of the principal axis on the slice


plane is minimized. The slice that minimizes the principal axis contains the two smaller eigenvectors of A, but not the largest, so we find (x2, ..X2).

Proof: To prove the minimax principle we note that since A is symmetric, there is a unitary matrix Q that diagonalizes A, so that A = Q AQ* where A is the diagonal matrix of eigenvalues ..X1 ? ..X2 • • • ? ..Xn. It follows that

n

(Ax,x} = (QAQ*x,x} = (Ay,y} = 'E..Xiy~, i=l

where y = Q*x and Cx = 0 if and only if CQy =By= 0, where B = CQ. We define p. by

p. =min { max (Ax,x}} =min {max i: ..XiYl}. 0 11:~:11=1 B 111111=1 i=1 Oz=O .811=0

If we pick the matrix B so that By = 0 implies that Y1 = Y2 = · · · = Yk-1 = 0, then p. :::; max11 1111=1l:~=k ..Xiy[ = ..Xk. Now pick a different matrix B. Since B has rank $ k - 1, the solution of By = 0 is not unique, and we can satisfy the equation By = 0 with then- k additional restrictions YH1 = Yk+2 = · · · = Yn = 0. With these n- k restrictions, the augmented system, call it BaY= 0, has rank$ k- 1 + (n- k) = n- 1 so always has nontrivial solutions. For this restricted set of vectors (i.e., with YH1 = Yk+2 = · · · = Yn = 0) the maximum for each B might be reduced. Therefore, (restricting y so that IIYII = 1)

p. = min {max t ..Xiy[ ~~ min { m~ t ..Xiy[} B B11=0 i=l J . B B.,l/=0 i=1

? min {~ax ..xk tyr} = ..xk. B B.,l/=0 .

=1

Since p.? ..Xk and p.:::; ..Xk, it must be that p. = ..Xk, as proposed. I Using the minimax principle we can estimate the eigenvalues of the matrix

A.

Theorem 1.8 Suppo6e the quadratic fonn q(x) = (Ax,x} is constrained by k linear constraints Bx = 0 to the quadratic fonn q in n - k variables. The relative extremal values of q, f!.enot~ ..Xt ? ..X2 ? .... · · · ? ..Xn and the relative extremal values of q, denoted ..X1 ? ..X2 ~ · · · ? ..Xn-k satisfy the interlacing inequality

..Xj ~ ~j ~ ..xi+k• j = 1, 2, ... , n - k.

1.3. GEOMETRICAL SIGNIFICANCE OF EIGENVALUES 21

Proof: According to the minimax principle, for any quadratic form q, the relative maxima are given by

..Xj =min {max q(x)}, 0 llzll=l

O:z:=O

where Cx = 0 represents j- 1linear constraints on x. If Xi, i = 1, 2, ... ,j- 1, are the vectors for which the first j - 1 relative maxima of (j(x) are attained, then the minimum for q(x) is attained by taking the rows of C to be xi, and (restricting x so that llxll = 1)

~i =min {max q(x)} = Il_!ax q(x) = max q(x) ~ ..Xi+k• 0 Oz=O (z,z;)=O (z,x;)=O

Bz=P

since the combination of Cx = 0 and Bx = 0 places j + k- 1 constraints on q(x). Similarly, (restricting x so that llxll = 1)

~i =min { max q(x)} $ max q(x) $ max q(x) = ..Xj, 1/i (z,l/;)=0 (z,x;}=O (x,z;)=O

where XiJ i = 1, 2, ... , j - 1 are those vectors at which the first j - 1 relative maxima of q(x) are attained. 1

For a geometrical illustration of these results, consider a 3 x 3 positive definite matrix A and its associated quadratic form q(x) = (Ax,x}. If we view q(x) as the "radial" map of the unit sphere llxll = 1, x -t q(x)x, the surface thus constructed is football shaped with three principal axes, the major axis, the semi-major axis, and the minor axis with lengths ..X1 ~ ..X2 ? ..Xa, respectively. If we slice this football-shaped surface with any plane through the origin, the intersection of q(x) with the plane is "elliptical" with a major and minor axis with lengths :X1 ?. :X2, respectively. It is an immediate consequence of the preceding theorem that these are interlaced,

..Xt ~ ~1 ?. ..X2 ?. ~2 ~ ..Xa.

Example:

To illustrate the usefulness of these results, suppose we wish to estimate the eigenvalues of the symmetric matrix ·

(

1 2 3 ) A= 2 3 6

3 6 8

using the minimax principle. First notice that if we increase the diagonal elements of A a bit, we get the modified Illatrix

(

1 2 3 ) A= 2 4 6

3 6 9


which is of rank 1 and therefore has two zero eigenvalues. The range of A is spanned by the vector (1, 2, 3)T and A has the positive eigenvalue ..\1 = 14. It follows (see Problem 1.3.2) that the eigenvalues of A satisfy ..\1 ~ 14, ..\2 ~ 0, ..\a~ 0. To use the minimax principle further we impose the single constraint :Ct = 0. With this constraint the extremal values of q(:c) are the eigenvalues of the diagonal su bmatrix

~ ( 3 6) A= 6 8 .

For this matrix we calculate that ~1 = 12 (with eigenvector (2, 3)T), ~2 = -1 (with eigenvector (3,-2)T), so that ..\a~ -1 ~ ..\2 ~ 0, and ..\1 ~ 12. To further improve our estimate of ..\1, observe that q(:c) = 12 for

·=_,b(!)· We try the nearby unit vector

•=;.. 0) and find that q(:c) = ¥/ = 13.07 so that 13.07 ~ ..\t ~ 14. An improved estimate of ..\2 is found by imposing the single constraint :c2 = 0 to get the restricted quadratic form q(:c) = :c~ + 6:ct:ca + 9:c~. For this quadratic form the extremal values are ~· It follows that -0.1098 ~ .tfB ~ ..\2 ~ 0. These estimates are actually quite good since the exact eigenvalues are ..\ = -1, -0.076, and 13.076.

To understand further the physical significance of eigenvalues, it is useful to think about vibrations in a lattice. By a lattice we mean a collection of objects (balls or molecules, for example) which are connected by some restoring force (such as springs or molecular forces). As a simple example consider a onedimensional lattice of balls connected by linear springs (Fig. 1.4). Let m; and u; denote the mass and displacement from equilibrium, respectively, of the jth ball. The equation of motion of the system is given by

cflui ffij dt2 =k;(Uj+l-Uj)+kj-1(Uj-1-Uj)•

Here we have assumed that the restoring force of the jth spring is linearly proportional to its displacement from equilibrium with constant of proportionality ki > 0 (i.e., Hooke's law).

Suppose that the masses at the ends of the lattice are constrained to remain fixed so that u0 = Un+l = 0. This system of equations can be written as

rPu =Au, dt2

(1.8)

1.3. GEOMETRICAL SIGNIFICANCE OF EIGENVALUES

u j-1

m j-1

-mi

ui uf+l __..,

mf+J

Figure 1.4: One dimensional lattice of balls connected by springs.

23

where u is the n-vector with components u1, u2, ... , Un and the n x n matrix A is tridiagonal with entries

. . - kj+ki-1 . - 1 a3,3 - - mi , J - , .•. , n, kj-1 • 2 aJJ-1 = m , J = , ... , n, k j

aJ,J+l = .;;;, j = 1. ... , n- 1,

and all other entries are zero. Of course, one could envision a more complicated lattice for which connections are not restricted to nearest neighbor interactions in which case the matrix A would not be tridiagonal. Something to try with the system (1.8) is to assume that the solution has the form u = 4Jeiwt in which case we must have

A4J = -w24J. Thus, the eigenvalues >. = -w2 of the matrix A correspond to the natural frequencies of vibration of this lattice, and the eigenvectors 4J determine the shape of the vibrating modes, and are called the natural modes or normal modes of vibration for the system.

The matrix A for the simple lattice here is symmetric if and only if the masses m; are identical. H the masses are identical we can use the minimax principle to draw conclusions about the vibrations of the lattice.

H mi = m for j = 1, 2, ... n, we calculate that

n n

m(Au, u} = m L L aiJUiUJ i=l j=1

n-1

= -kou~- knu;- L kj(Uj- UJ+1)2. j=l

Since the constants ki are positive, A is a negative definite matrix (a negative definite matrix is one for which (Ax,x} < 0 for x ::f: 0). Notice further that increasing any kj decreases the quantity {Au, u} and hence, the eigenvalues of A are decreased. From the minimax principle we have some immediate consequences:

1. H the mass m is increased, all of the eigenvalues of A are increased, that is, made less negative, so that the natural frequencies of vibration are decreased.


2. If any of the spring constants k; is increased then the eigenvalues of A are decreased (made more negative) so that the natural frequencies of vibration are increased.

The natural frequencies of vibration of a lattice are also called the resonant frequencies because, if the system is forced with a forcing function vibrating at that frequency, the system resonates, that is, experiences large amplitude oscillations which grow (theoretically without bound) as time proceeds. This observation is the main idea behind a variety of techniques used in chemistry (such as infrared spectroscopy, electron spin resonance spectroscopy and nuclear magnetic resonance spectroscopy) to identify the structure of molecules in liquids, solids, and gases.

It is also the idea behind microwave ovens. If an object containing a water molecule is forced with microwaves at one of its resonant frequencies, the water molecule vibrates at larger and larger amplitudes, increasing the temperature of the object. As you may know, microwave ovens do not have much effect on objects whose resonant frequencies are much different than that of water, but on objects containing water, a microwave oven is extremely efficient. Microwave ovens are set at frequencies in the 1011hz range to resonate with the rotational frequencies of a water molecule.

1.4 Fredholm Alternative Theorem

The Fredholm Alternative Theorem is arguably the most important theorem used in applied mathematics (and we will see it many times in this text) as it gives specific criteria for when solutions of linear equations exist.

Suppose we wish to solve the matrix equation Ax = b where A is an n x m matrix (not necessarily square). We want to know if there is a solution, and if so, how many solutions are possible. ·

Theorem 1.9 (Fredholm Alternative Theorem) The equation Ax = b has a solution if and only if ( b, v) = 0 for every vector v satisfying A • v = 0.

It is interesting to note that this theorem is true for any inner product on mn I although if one changes the inner product, the adjoint matrix A. changes as do the vectors v for which A*v = 0.

Theorem 1.10 A solution of Ax = b (if it emts) ~ unique if and only if x = 0 ~the only solution of Ax= 0.

Proof: We prove Theorem 1.10 first. Suppose Ax = 0 for some x I- 0. If Axo = b then Xt = xo +ax also satisfies Ax1 = b for any choice of a so that the solution is not unique. Conversely, if solutions of Ax = b are not unique then there are vectors Xt and x2 with x = x1 - x2 I- 0 satisfying Axt = Ax2 = b. Clearly Ax = A(xt - x2) = Axt _: Ax2 = 0.

1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES 25

The first half of the Fredholm alternative (Theorem 1.9) is easily proven. If A*v = 0 and xo satisfies Axo = b then

(b, v) = (Axo, v) = (xo, A"v) = (xo, 0) = 0.

To prove the last half, suppose (v, b) = 0 for all v with A*v = 0. We write b = br + bo where br is the component of b in the range of A and bp is the component of b orthogonal to the range of A. Then 0 = (bo, Ax) = (A*b0 , x) for all x, so that A"bo = 0. Since (b, v) = 0 for all v in the null space of A*, and since bo is in the null space of A*, we conclude that (b0 , b) = 0. Expanding b = br+bo we find 0 = (bo,br+bo) = (bo,br)+(bo,bo). Since (b0 ,br) = 0 it must be that bo = 0 so that b = br is in the range of A, i.e., Ax= b has a solution. 1

Example:

To illustrate these two theorems in a trivial way, consider the matrix

A=(!~)· The null space of A is spanned by the vector (2, -l)T so that solutions of Ax = b, if they exist, are not unique. Since the null space of A • is spanned by the vector (3, -l)T, solutions of Ax= b exist only if b is orthogonal to v so of the form

b=a(!), which is no surprise since (1, 3)T spans the column space of A.

One useful restatement of the Fredholm alternative is that the null space of A* is the orthogonal complement of the range of A and that together they span IR.m, so that IRm = R(A) EB N(A*). Stated another way, any vector b E JRm can be written uniquely as b = br + bo where br is in the range of A and b0 is in the null space of A•, and br is orthogonal to b0 (see Fig. 1.5).

1.5 Least Squares Solutions-Pseudo Inverses

Armed with only the Fredholm alternative, a mathematician is actually rather dangerous. In many situations the Fredholm alternative may tell us that a system of .equations Ax = b has no solution. But to the engineer or scientist who spent lots of time and money collecting data, this is a most unsatisfactory answer. Surely there must be a way to "almost" solve a system.

Typical examples are overdetermined systems from curve fitting. Suppose the data points (ti, Yi), i = 1, 2, ... , n, are thought to be linearly related so that Yi = a(ti -l) + /3, where t = ~ L:~=l ti. This is a system of n linear equations


Figure 1.5: Graphical interpretation of the Fredholm alternative theorem.

in the two unknowns a and /3 of the form

(

t1 - t 1 ) ( Yl )

A(~)= t.~: ~ (~)= ~ . tn-t 1 Yn

(1.9)

If there are more than two data points this system of equations does not in general have an exact solution, if only because collected data always contain error and the fit is not exact.

Since we cannot solve Ax = b exactly, our goal is to find an x that minimizes I lAx- bll.

The minimal solution is norm dependent. One natural, and common, choice of norm is the Euclidean norm (1.4) while other choices such as llxlloo or llxll = E~=1 lxsl = llxlh are useful as well, but lead to different approximation schemes.

Definition 1.12 For an m x n matrix A and vector b E <lim, the least squares problem is to find x E ~ for which the Euclidean norm of Ax- b is minimized. A vector x (not necessarily unique) that minimizes the Euclidean norm of Ax-b is the least squares solution.

To find the least squares solution, recall from the Fredholm alternative that b can always be written as b = br + bo, where br is in the range of A and bo is orthogonal to the range of A. Since Ax - br is in the range of A then (by the Pythagorean theorem)

IIAx- bll2;, IIAx- brll2 + llboll2·


We have control only over the selection of x, so the minimum is attained when we minimize II Ax - brW. Since br is in the range of A, Ax = br always has a solution and minx IIAx- bll = llboW· We know that br is the projection of b onto the range of A and bo is in the orthogonal complement of the range of A. Therefore, 0 = (bo,Ax) = (A•bo,x) for all x so that A•bo = 0. Now Ax= br is equivalent to Ax = b - bo so that Ax - b = -bo must be in the null space of A•, which is true if and only if A• Ax= A•b. Thus the least squares solution of Ax = b is any vector x satisfying A • Ax = A • b. One such x always exists. The equation

A•Ax = A•b (1.10)

is called the normal equation. Another derivation of the equation (1.10) uses ideas of calculus. Since we

wish to minimize IIAx- bll2 =(Ax- b,Ax- b}, we let x = xo + ay, where xo is the minimizing vector and y is an arbitrary perturbation. Since xo renders IIAx- bll2 a minimum, (A(xo + ay)- b,A(xo + ay)- b)~ (Axo- b,Axo- b) for ay nonzero. Expanding this expression, we find that we must require

a(y, A•(Axo- b)}+ a(A.(Axo- b), y} + a 2 (Ay, Ay} ~ 0.

This last expression is quadratic in a, and unless the linear terms vanish, for sufficiently small a, a change of sign of a produces a change of sign of the entire expression, which is not permitted. Thus we require (y, A*(Axo- b)} = 0 for ally. That is, we require A• Axo = A*b, which is the normal equation.

We know from our first derivation that the normal equation always has a solution. Another check of this is to apply the Fredholm alternative theorem directly to (1.10). We require (v,A"'b} = 0 for all v with A• Av = 0. (Notice that A• A is a square, self-adjoint matrix). If A* Av = 0 then Avis simultaneously in the null space of A • and in the range of A. Since these two subspaces are orthogonal, Av must be 0, the only element common to both subspaces. Thus (v, A•b) = (Av, b) = 0, as required.

As an example of the least squares solution, consider the curve fitting problem described earlier. We seek a least squares solution of (1.9). Premultiplying by A*, we find (use that E~=1 (t,- f)= 0)

n n

a '.L:(t,- f) 2 = '.L:Yi(ti- f), i=l i=l

n

n/3 = LYi! i=l

which is always solvable for a and /3. Other types of curve fitting are possible by changing the underlying as

sumptions. For example, one could try a quadratic fit Yi = at~+ j3t, + 'Y· Other choices of basis functions (such as are discussed in Chapter 2) can also be used, but in all cases we obtain a linear least squares problem. For the exponential fit y1 = aef3t,, the parameters a, /3 occur nonlinearly. However, the equation


ln Yi = ln a + /3ti is linear in ln a and /3 and leads to a linear least squares problem.

The least squares solution always exists, however, it is unique if and only if A* Ax= 0 has only the trivial solution x = 0. H A*(Ax) = 0 then AxE N(A*) and Ax E R(A) so that Ax = 0. Thus, A• A is invertible if and only if Ax = 0 has no nontrivial solutions, that is, if A has linearly independent columns. H A has linearly independent columns, then the least squares solution of Ax = b is given by x = (A* A)-1 A*b, called the Moore-Penrose least squares solution. The matrix A'= (A* A)-1 A* is called the Moore-Penrose pseudo-inverse of A. Notice that when A is square and invertible, A* is also invertible and (A* A)-1 A* = A-1 A*-1 A* = A-1 so that the pseudo-inverse of A is precisely the inverse of A. Notice also that the linear fit discussed earlier has a unique solution if there are at least two distinct data points ti (so that the columns of A are linearly independent).

H A does not have linearly independent columns, the least squares problem does not have a unique solution. Suppose Xp is one solution of the least squares problem A • Ax = A • b. Since A has linearly dependent columns, there are vectors w that satisfy Aw = 0, and x = Xp + w is also a least squares solution. One reasonable way to specify the solution uniquely is to seek the smallest possible solution of A* Ax= A*b. To minimize llxll, we want x to satisfy (x,w} = 0 for all w with Aw = 0. That is, we want x to be orthogonal to the null space of A, and therefore, in the range of A •.

Definition 1.13 The least squares pseudo-inverse of A is the matrix A' for which x = A'b satisfies:

1. A* Ax = A*b.

2. (x, w) = 0 for every w satisfying Aw = 0.

Example:

The geometrical meaning of the pseudo-inverse can be illustrated with the simple 2 x 2 example

A=(!!)· (1.11)

For this matrix,

R(A) = span { ( l ) } • R(A*) =span { ( ~ ) } '

N(A*) =span{ ( _; ) } ·

The pseudo-inverse of A must project the vector

·b=(:~)


R(A*)

N(A)

X.' Ax=br

Figure 1.6: Graphical solution of the least squares problem for the matrix (1.11). Every vector in the manifold x : Ax = b,. minimizes IIAx- bll· The smallest such vector, denoted x', must be orthogonal to N(A).

onto a vector in R(A) (orthogonal to N(A*)), and then find the inverse image of this vector somewhere in R(A•) (orthogonal to N(A)). (See Figure 1.6.) That is, x must have the form

x=a(!) and must satisfy

Ax = ( :~ ) - ~ ( _; ) ( ( _; ) ' ( :~ ) ) = ~ ( ~ ~ ) ( :~ ) .

Since

A(!)=(~). it follows that

(~b1 + ~b2) A ( ! ) = ~ ( ~ ) b1 + ~ ( ~ ) b2 = ~ ( ~ ~ ) ( ~ ) so that

( 2 1 ) ( 1 ) ( 2/5 1/5 ) ( bl ) X = Sbl + Sb2 1 = 2/5 1/5 b2

and that the pseudo-inverse of A is

I ( 2/5 1/5 ) A = 2/5 1/5 .

There are a number of different numerical algorithms one can use to calculate .the pseudo-inverse A'. In most situations it is not a good idea to directly calculate (A* A)-1 since, even if A has linearly independent columns, the calculation of (A* A)-1 is numerically poorly posed. The algorithms that we describe


below are included because they illustrate geometrical concepts, and also, in the case of the last two, because of their usefulness for numerical calculations.

Method 1: Gaussian Elimination

The first idea is to mimic Gaussian elimination and find A' via elementary row operations. Recall that Gaussian elimination is the process of reducing the augmented matrix [A, I] to [J,A-1] by means of elementary row operations. The goal of reduction is to subtract appropriate multiples of one row from lower rows of the matrix so that only zeros occur in the column directly below the diagonal element. For example, one step of the row reduction of the matrix

au a12 a22

A=

0

BA:A:

BA:+1,A:

BnA:

B1n

B2n

BA:n

BA:+1,n

ann is accomplished by premultiplying A by the matrix (only nonzero elements are displayed)

1

1 M.~:= I

-O'A:+l 1 -0'1:+2 1

-O'n 1

where O'J = BJA:/a,~:,~:. Specifically, the nonzero entries of M.~: are ones on the diagonal and -u i in the kth column, below the diagonal. The numbers u i are called multipliers for the row reduction. The result of a sequence of such row operations is to triangularize A, so that MnMn-1 · · · M1A = U is an upper triangular matrix. The product of elementary row operations MnMn-1 · · · M1 = L-1 is a lower triangular matrix which when inverted gives A = LU. This representation of A is called an LU decomposition.

By analogy, our goal is to find some way to augment the matrix A with another matrix, say P, so that the row reduction of [A, P] yields the identity and the pseudo-inverse [I, A']. The least squares solution we seek must satisfy Ax= br, subject to the conditions {w,,x) = 0 where {wi} spans the null space of A, and the vector br is the projection of the vector b onto the range of A. We

. A: ( A: T) can represent br as br = Pb = b.- Li=t {u,, b)u, = I- Li=t u,u, b, where

1.5. LEAST SQUARES SOLUTIONs-PSEUDO INVERSES 31

the k vectors {uj} form an orthonormal basis for the null space of A•. Thus, the projection matrix P is given by P = I- 2::=1 u,uf. Notice that if the null space of A• is empty, then k = 0 and P =I. If the null space of A is empty we row reduce the augmented matrix [A, P], but if not, we append the additional constraint (w,, x} = 0 for each linearly independent w, E N(A), and row reduce the augmented matrix

[fr ~]· wT 0 I

(equivalent to the system of equations Ax = Pb, {w1, x) = 0, i = 1, ... , l), discarding all zero rows, which leads to [I, A'].

To see that this can always be done, notice that the projection P is guaranteed by the Fredholm alternative to project onto the span of the columns of A. Furthermore, the rows of A span the range of A • and the vectors { Wi} span the null space of A, so that, from the Fredholm alternative JRn = R(A•) E9 N(A), the rows of A augmented with { wf} span mn and hence can be row reduced to an identity matrix (with possibly some leftover zero rows).

Example:

Consider the matrix

( 2 4 6) A= 1 2 3 .

Since A has linearly dependent columns, A has no Moore-Penrose pseudo inverse. The null space of A is spanned by the two vectors

w,=(_D· --=( -0· and the null space of A • is spanned by the single unit vector

u= ~ ( -~ )· AB a result,

1 ( 1 -2 ) 1 ( 4 2 ) p = I- 5 -2 4 = 5 2 1 .

We row reduce the augmented matrix

(

A p) (2 4 6 wr 0 - 1 2 3 wf 0 - 1 1 -1

2 -1 0

4/5 2/5 0 0

Elementary row operations reduce this matrix to

(

1 0 0 2/70 1/70 ) 0 1 0 4/70 2/70 1

0 0 1 6/70 3/70

2/5) 1/5 0 .

0


(we have discarded one identically zero row) so that

A' = _!_ ( ~ ~ ) . 70 6 3

It is easy to check that A • AA' = A •.

Method 2: L U Decomposition

Any matrix A can be written as A = LU where L is lower triangular and U is upper triangular (with the proviso that some row interchanges may be necessary). In practice, A is usually not reduced to a diagonal matrix, but simply to triangular form. The decomposition is unique if we require L to have ones on the diagonal and U to have linearly independent rows, except for possible zero rows.

To see why the LU decomposition is desirable, suppose we want to solve Ax= band we know that A= LU. We let Ux = y so that Ly =b. Solving Ly = b involves only forward substitution, and solving U x = y involves only backward substitution. Thus, if both L and U are full rank, the solution vector x is easily determined.

The LU decomposition is found by direct elimination techniques. The idea is to reduce the augmented matrix [A, I] to the form (U, L -l] by lower triangular elementary row operations. When implementing this on a computer, to save memory, one can simply store the multipliers from the row reduction process in the sub diagonal element that is zeroed by the row operation (see Problem 1.5.12). The sub-diagonal half of the resulting matrix is the sub-diagonal part of L and the diagonal elements of L are all ones. If A has linearly independent rows, then zero rows of U and corresponding columns of L can be harmlessly eliminated.

Example:

Consider the matrix

A=u n By row reduction, storing multipliers below the diagonal, we find that A row reduces to

(: -~) so that

U=O -D· ( 1 0 0) L= 3 1 0 .

5 2 1

1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES

Eliminating the last row of U and last column of L (which are superfluous) we determine that

A = LU = ( ~ ~ ) ( 1 2 ) . 5 2 0 -2

33

Notice that since L has linearly independent columns, the null space of A is the same as the null space of U and similarly, since the rows of U are linearly independent, the null space of A* is the null space of L*. The range of A is spanned by the columns of L and the range of A • is spanned by the columns of U*. If A = A •, then U = o:L * for some scalar o:.

Once the LU decomposition is known, the pseudo-inverse of A is easily found.

Theorem 1.11 If A = LU, then

A'= U*(UU*)- 1 (L*L)- 1U.

This statement can be directly verified by substituting x:::: A'b into A* Ax= A*b and using that A= LU. Since U has linearly independent rows and L has linearly independent columns, UU* and L* L are both invertible. Notice also that if Aw = 0, then Uw = 0 and

{w, A' b) = {w, u•wu•)- 1(L* L)-1 L*b) = (Uw, (UU*)-1(L* L)-1 L*b) = 0,

so that x is the smallest possible least squares solution. The usefulness of this representation of A' is primarily its· geometrical interpretation.

Method 3: Orthogonal Transformations

Another important way to solve least squares problems is to transform the matrix A to upper triangular form using carefully chosen transformations. The advantage of transforming to a triangular system is that if the diagonal elements are nonzero, it can be solved by backward substitution, which is not much more difficult than solving a diagonal system. However, there is an additional bonus if the matrix transformations we use are orthogonal matrices.

Definition 1.14 A square complex matrix Q is unitary (also called orthogonal) if Q* = Q-1, i.e., Q*Q = I.

(Recall that unitary matrices first appeared in Thm. 1.5.) Unitary matrices have two important geometrical properties, namely they preserve angles and lengths, that is, for any vectors x and y, (Qx, Qy) = (x, Q*Qy) = (x, y), and IIQxll = llxll. In other words, unitary transformations are rotations and reflections.

The transformation we seek represents A as A= QR where Q is unitary and R is upper triangular, and this is called the QR decomposition of A. This decomposition can always be found because this is exactly what the Gram-Schmidt


____ ... -

----------- Uv

-Pv

Figure 1. 7: Sketch of the geometry of an elementary reflector, where Pv = f1:ii~ u is the projection of v onto u.

procedure does, namely, it represents a vector as a linear combination of previously determined orthogonal vectors plus a new mutually orthogonal vector. Here, the columns of Q are the orthogonal basis that results from applying the Gram-Schmidt procedure to the columns of A. The matrix R indicates how to express the columns of A as linear combinations of the orthogonal basis Q.

To carry this out we could use the Gram-Schmidt procedure, however, it turns out that this is a numerically unstable process. Instead, we use a method that looks similar to Gaussian elimination, which, instead of using elementary row operations as in Gaussian elimination, uses an object called an elementary reflector, or more commonly, Householder transformation.

Definition 1.15 An elementary reflector (Householder transformation) is a matrix of the form

2uu• U =I -llull2' provided u ':/: 0.

It is easy to verify that U = u• and u•u = I. The identification of the Householder transformation as a reflector comes from the fact that if v is parallel to u, then Uv = -v, while if vis orthogonal to u, then Uv = v. Thus, if Pv is the projection of v onto u, then v = Pv +(I- P)v, and Uv = -Pv +(I- P)v. In other words, U "reflects" the vector v through the plane that is orthogonal to u. A sketch of this is shown in Fig. 1. 7.

A useful property of elementary reflectors is that, as is also true of elementary row operations, they can be chosen to introduce zeros into a column vector.

Theorem 1.12 For z E IEr', let u = ±llzll, and suppo1e z ':/: -ue1. If u = z + ue1, then U = I - ~ u an elementary reflector whose action on z u Ux = -ue1 (e1 u the first natural basis vector).

1.5. LEAST SQUARES SOLUTIONS-PSEUDO INVERSES

I I I I

I

u = x -llxlle I

I

llxll e 1

I I I

I I

X

llxlle I

u = x+ llxlle

e I

35

1

Figure 1.8: Sketch of the geometry of a Householder transformation applied to the vector u = x±llxllel. The two choices of a= ±llxll give different reflections.

Proof: Since x f. -cre1, u f. 0 and U is an elementary reflector. Since cr2 = llxll2,

llull2 = 2a2 + 2crxl.

where X1 = (x, e1) is the first component of x, and

uu*x (x + cre1,x)u U X = X - = X - = X - U = -CTel.

U 2 + O"Xl CT2 + CTX1

It is wise (for numerical reasons, to avoid cancellation errors) to choose a to have the same sign as Xt, so that x f. -cre1 unless x = 0 already, in which case, the transformation is not needed. I

The geometry of Theorem 1.12 is depicted in Fig. 1.8. We now see that we can place zeros in the first column of A, below the diag

onal, by premultiplying A by the appropriate Householder transformation. To carry out the transformation on the entire matrix, we do not actually calculate the matrix U but rather note that for any column vector y

Uy = Y _ 2uu•y = y _ 2(u,y) u, u*u (u,u)

so that U y can be viewed as a linear combination of two column vectors with a multiplier related to the inner product of the two vectors. Premultiplying by a sequence of Householder transformations, we sequentially triangularize the matrix A. We first apply the Householder transfor:rp.ation that places zeros in the first column below the diagonal. Then we apply a second Householder transformation that leaves the first row and first column undisturbed and places zeros in the second column below the diagonal, and so on sequentially. Note that if

2UkUk uk = h- llukll2


is a k x k Householder transformation that places zeros in the first column below the diagonal of a k x k matrix, then the n x n matrix

(In-k 0 )

V~c = 0 U~c

is the Householder transformation for an n x n matrix that transforms only the lower diagonal k x k submatrix and leaves the remaining matrix unscathed.

H Vi, V:l,· · ·, Vn are the chosen sequence of transformation matrices, then with Q* = Vn Vn-1 · · · V1 we have

IIAx- bll = IIQ* Ax- Q*bll = II&- Q*bll,

where

R= ( ~)' with R upper triangular. H the columns of A are linearly independent (the same condition as required for the Moore-Penrose pseudo-inverse) the matrix R is square, has nonzero diagonal elements, and is therefore invertible.

H we denote the vector

Q*b= (~)I where the vector b1 has the same number of elements as R has rows, then the least squares solution is found by solving Rx = b1 and for such an x, IIAx- bll = 11~11, and this error cannot be made smaller. It follows that the pseudo-inverse of A is given by A'= R'Q*, where R' is the pseudo-inverse of R.

This method is most useful in the case that the matrix A has linearly independent columns so that R is invertible and the solution of Rx = b1 is found by back substitution. H, however, R is not invertible, one must find the smallest solution of 1ix = i1]., which, unfortunately, cannot be found by back substitution.

There is another important use for Householder transformations that should be mentioned here. As is well known, there is no way to find the eigenvalues of a matrix larger than 4 x 4 with a finite step algorithm. The only hope for larger matrices is an iterative procedure, and one of the best is based on the Q R decomposition.

Two matrices have the same eigenvalues if they are related through a similarity transformation. The goal of this algorithm is to find an infinite sequence of similarity transformations that converts a matrix Mo into its diagonal matrix of eigenvalues.

The method is to use Householder transformations to decompose the matrix Mn into the product of an orthogonal matrix and an upper triangular matrix Mn = QnRn· To make this into a similarity transformation we reverse the order of matrix multiplication and form Mn+l = RnQn so that Mn+l = Q;;1 MnQn. Now the amazing fact is that if M0 is a real symmetric matrix, then Mn converges to the diagonal matrix of eigenvalues. Of course, if Mo is not symmetric,


this cannot happen, since the transformation that diagonalizes Mo is not orthogonal. However, if Mo is not symmetric, then Mn converges to a "nice" matrix whose eigenvalues are easy to find.

Householder transformations provide a very efficient way to find the QR decomposition of a matrix. Good software packages to do these calculations on a computer are readily available, and a good description of the corresponding theory is found in [103).

Method 4: Singular Value Decomposition {SVD)

The final method by which A' can be calculated is the singular value decomposition. This method extends the idea of the spectral decomposition of a matrix to nonsquare matrices and is often used in applications.

As we will see, the singular value decomposition of a matrix A is similar to the spectral representation of a square matrix A = TAT-1• Once the spectral decomposition is known (if it exists), the inverse is easily found to be A-1 = TA-1T-1 , provided the diagonal matrix A has only nonzero diagonal entries. This does not, however, work to find a pseudo-inverse.

If A is self-adjoint, then the spectral decomposition of A is A= QAQ* where Q is orthogonal. To find the pseudo-inverse of A, we want to minimize

IIAx- bll:l = IIQAQ*x- bll = IIAQ*x- Q*bll = IIAy- Q*bll, (1.12)

where y = Q*x. It is an easy matter to verify (see Problem 1.5.2) that the pseudo-inverse of an m x n diagonal matrix D of the form

D=(D 0) 0 0 '

where D is a (square) diagonal matrix with nonzero diagonal elements is the n x m diagonal matrix

D' = ( Dt 0) 0 0 .

It follows that the pseudo-inverse of the n x n diagonal matrix A is

A'= ( x-l o) 0 0 '

where A is the nonzero part of A. Therefore we take

y = Q*x = A'Q*b

andy= Q*x is the smallest minimizer of (1.12). However,

X= QA'Q*b

has the same norm as Q*x and so is the smallest least squares solution of (1.12). It follows that the pseudo-inverse of A is A'= QA'Q*.


The geometrical interpretation of this pseudo-inverse is as follows. H A = QAQ*, then Q represents the basis relative to which A has a diagonal representation. H one of the diagonal elements of A is zero, then of course, the inverse of A does not exist. To find A'b, we form Q*b which expresses b relative to the new coordinate system. The expression A'Q*b projects Q*b onto the range of A and determines the smallest element whose image under A is Q* b. Finally Q(A'Q*b) expresses this vector in terms of the original coordinate system (see Problem 1.5.6).

The change of coordinates Q identifies both the range and null space of A. What we have done here is to project the vector b onto the range of A and then find another vector in the range of A whose image is this projection. Since A* =A, the matrix Q is appropriate for both of these transformations.

This construction of A' can work only for symmetric matrices where R( A •) = R(A). Unfortunately, this construction fails for nonsymmetric matrices for two important reasons (see Problem 1.5.9}. First, since A* :f. A, R(A*) :f. R(A) so projecting onto R(A) does not also put us on R(A*). Furthermore, if A* :f. A, the basis provided by the diagonalization of A is not an orthogonal basis and hence projections are not orthogonal.

What is needed is an analogous construction for nonsymmetric matrices. This construction is provided by the singular value decomposition, which we now state.

Theorem 1.13 Suppose A is any m x n matrix.

1. A can be factored as A= UEV*, (1.13}

where U ( m x m) and V ( n x n) are unitary matrices that diagonalize AA • and A • A respectively, and E is an m x n diagonal matrix {called the matrix of singular values) with diagonal entries O'ii = ...;>:i1 where Ai 1 i = 1,2, ... ,min(m,n), are the eigenvalues of AA* {see Problem 1.B.3}.

B. The pseudo-inverse of A is

A'= V!:'U., (1.14)

where !:' 1 the pseudo-inverse of !: 1 is an n x m diagonal matrix with diagonal entries o11 = -(1'

1 =-it- provided O'ii :f. 0 and 0 otherwise. II V AI

The factorization A = U!:V* is called the singular value decomposition of A. Proof: To show that this decomposition always exists, notice that the matrix A• A is a symmetric n x n matrix with non-negative eigenvalues A1, A2 1 ••• , An· Let V be the n x n matrix that diagonalizes A • A. The columns of V are the orthogonal eigenvectors x1,x2, .•. ,xn of A* A, normalized to have length one. Then

A* A= VAV*.


Suppose A* A bas k positive eigenvalues A1, A2 1 ••• , Ak > 0, and Ak+l = ).k+2 = · · · =An = 0. Let Yi = $t. i = 1, 2, ... , k, and notice that

(y;,yi} = .JX~\f>;;"(Ax;,Axi} = .JXIJ>:;(xi,A'"Axi}

= .:§f(x;,xi} = 5ii·

Since y; E mm I this guarantees that k :::; m. We can use the Gram-Schmidt procedure to extend the y;'s to form an orthonormal basis for !Rm, {y;}~1 . Let U be the matrix with columns Yl, Y2, ... , Ym. The vectors y; are the orthonormal eigenvectors of AA •. We now examine the entries of U* AV. For i ::; k, the ijth entry of u• AV is

(y;,AxJ} = -it-(Ax;,Axj} = +(x;,A*Axi} 'S.~' v Ai = ~(Ax;,Axi) = .J>:iaiJ·

Fori> k, note that AA*y; = 0 so that A*y; is simultaneously in the null space of A and the range of A*. Therefore (the Fredholm alternative again}, A*y; = 0 and (y;,Axi} = (A*y;,xi} = 0 fori> k. It follows that U*AV = E where !: = (O';j), O'ij = .J>:i5;i, i = 1,2, ... , m, j = 1, 2, ... ,n, and this is equivalent to (1.13).

It is noteworthy that the orthogonal vectors {x1 , x2 , ••• , xk} span the range of A • and the vectors { x k+l, ... , Xn} span the null space of A. Similarly, the vectors {y1, Y2, ... Yk} span the range of A and {Yk+l• ... , Ym} span the null space of A*. Thus the orthogonal matrices U and V provide an orthogonal decomposition of !Rm and !Rn into !Rm = R(A) EB N(A*) and !Rn = R(A"') EB N(A), respectively. Furthermore, the singular value decomposition A= UEV* shows the geometry of the transformation of X by A. In words, v· X decomposes x into its orthogonal projections onto R(A•) and N(A), then EV*x rescales the projection of x onto R(A*), discarding the projection of x onto N(A). Finally, multiplication by U places !:V*x back onto the range of A.

It is now clear that to minimize

I lAx- bll2 = IIU!:V*x- bll2 ,

we take X= V!:'U*b

so that the pseudo-inverse of A is as stated in (1.14). I The singular value decomposition gives a nice geometrical interpretation for

the action of A. That is, first there is a rotation by V*, followed by rescaling by E and finally a second rotation by U. Similarly, to find the least squares solution of Ax = b, we first use U to decompose b into its projection onto R(A) and N(A*). The component on R(A) is rescaled by E' and then transformed onto R(A*) by V.

This again illustrates nicely the importance of transform methods. As we know, the least squares solution of Ax = b can be found by direct techniques.


However, the change of coordinates provided by the orthogonal matrices U, V transforms the problem Ax= b into ~y = b where y = V'"x, b = U'"b, which is a diagonal (separated) problem. This least squares problem is easily solved and transforming back we find, of course, x = V~'U'"b. This transform process is illustrated in the commuting diagram

Ax=b A' --t x=A'b

y=V'"x .t. t

x=Vy b=U'"b b=Ub

~y=b E' ~ y= ~'b

where the top line represents the direct solution process, while the two vertical lines represent transformation into and out of the appropriate coordinate representation, and the bottom line represents solution of a diagonal system.

The singular value decomposition also has the important feature that it allows one to stabilize the inversion of unstable (or ill-conditioned) matrices. To illustrate this fact, consider the matrix

A=(1+-Jfo 1-1fo), 2-710 2+-fto

where € is a small number. We can visualize what is happening by viewing the equation Ax = b as describing the intersection of two straight lines in JR2 by noting that for € small the lines are nearly parallel. Certainly small changes in b give large changes to the location of the solution. Thus, it is apparent that the inversion of A is sensitive to numerical error.

We calculate the singular value decomposition of A to be

A--1 (1 2)(v'10 0)(1 1) - v'10 2 -1 0 € 1 -1 '

and the inverse of A to be

--L(1 1)(fto 0)(1 2) - v'IO 1 -1 0 € 2 -1

= ( 0.1 + eJro 0.2 - e{to ) . 0.1 - eJrn 0.2 + e:;/i6

A-1

It is clear that when the singular value € is small, the matrix A-1 is very large and the solution x = A-1b is unstable.

If instead of using the full inverse, we replace € by zero and use the singular value decomposition to find the pseudo-inverse,

A' __ 1 ( 1 1 ) ( 'fto 0 ) ( 1 2 ) _ 1:_ ( 1 2 ) - v'iO 1 -1 0 0 2 -1 - 10 1 2 '

the solution is now stable. That is, small changes in the vector b translate into only small changes in the solution vector x = A'b and, in fact, x varies only


in the direction orthogonal to the nearly parallel straight lines of the original problem. In other words, the singular value decomposition identified the stable and unstable directions for this problem, and by purposely replacing € by zero, we were able to keep the unstable direction from contaminating the solution.

In a more general setting, one can use the singular value decomposition to filter noise (roundoff error) and stabilize inherently unstable calculations. This is done by examining the singular values of A and determining which of the singular values are undesirable, and then setting these to zero. The resulting pseudo-inverse does not carry with it the destabilizing effects of the noise and roundoff error. It is this feature of the singular value decomposition that makes it the method of choice whenever there is a chance that a system of equations has some potential instability.

The singular value decomposition is always recommended for curve fitting. The SVD compensates for noise and roundoff error prppagation as well as the fact that solutions may be either overdetermined or underdetermined. In a least squares fitting problem it is often the case that there is simply not enough experimental evidence to distinguish between certain basis functions and the resulting least squares matrix is (nearly) underdetermined. The SVD signals this deficiency by having correspondingly small singular values. The user then has the option of setting to zero small singular values. The decision of how small is small is always up to the user. However, with a little experimentation one can usually learn what works well.

1.5.1 The Problem of Procrustes

The Householder transformation solves the problem of finding an orthogonal transformation that transforms one vector into another, provided the lengths of the two vectors are the same. A generalization of this problem, called the problem of Procrustes3 is to seek an orthogonal transformation that transforms one set of vectors into another, as best possible.

Suppose we have two sets of vectors in JRn, represented as the columns of two n x k matrices A and B, and we wish to find the best orthogonal transformation Q that aligns A with B. That is, we wish to minimize

IIQA-BII2•

Before we can solve this problem we must define an appropriate norm for matrices. A useful matrix norm for this problem is the Frobenius norm

IIAII~ = Tr(A* A) = Tr(AA*).

(see Problem 1.5.14). In terms of this norm, the problem is to minimize

IIQA- Bll~ = Tr(AA'") + Tr(BB'")- 2Tr(QAB*). 3Procrustes was a legendary Greek outlaw, living in Athens, who would offer his hospitality

to travelers, and then force them to lie on an iron bed, stretching them on a rack if they were too short or lopping off limbs if they were too long for the bed - an unusual method of coordinate transformation,--to be sure.


Thus, it is equivalent to maximizing Tr(QAB*). Now suppose that the matrix AB* has the singular value decomposition

AB* =U~V*.

Then,

Tr(QAB*) = Tr(QU~V*) = Tr(V*QU~) = 'Eziiqi $ L:ai, i i

where Z = V*QU. This inequality is valid since the diagonal elements of a real unitary matrix cannot be larger than one, and the singular values ai are non-negative. However, it is also clear that equality is obtained if and only if Z = I, in which case

Q=VU*.

1.6 Applications of Eigenvalues and Eigenfunctions

It is not possible to overstate the importance of eigenvalues and eigenfunctions in applications. In this section, we give a short introduction to a few of the ways in which an understanding of eigenvalues and eigenfunctions gives important insight or capability in applied problems.

1.6.1 Exponentiation of Matrices

The solution of the scalar ordinary differential equation ~ = au with u(O) = Uo is u(t) = uae'". It would be nice if we could make a similar statement for the vector differential equation

dx=Ax dt

(1.15)

where A is ann x n matrix and x(t) is a vector in m.n. The problem, however, is that we do not know how to interpret the matrix eAt.

The answer is readily found if the matrix A can be diagonalized. If there is a similarity transformation so that A= T-tAT, then (1.15) becomes

so that

dx T- = T AT-1Tx = ATx

dt

dy -=Ay dt

where Tx = y. Since A is a diagonal matrix, the solution of the system !!If = Ay is readily written as

y(t) = eAty(O),

).6. APPLICATIONS OF EIGENVALUES AND EIGENFUNCTIONS 43

where eAt is the matrix whose only nonzero components are the diagonal elements e).it, j = 1, 2, ... , n. It follows that

x(t) = T-1eAtTx(O}

so that eAt ~ T-leAtT.

1.6.2 The Power Method and Positive Matrices

There are many situations where the only eigenvalue of interest is the one with largest magnitude. It is easy to calculate the largest eigenvalue and corresponding eigenvector using a method called the power method. The power method involves "powering" a matrix. We start with some initial vector x0 , and then form the sequence of vectors

XI;

X1;+1 = Allx~;ll" (1.16}

Theorem 1.14 If the largest (in magnitude) eigenualue At of A is unique with algebraic multiplicity one, and has a one-dimensional eigenspace spanned by ifJ,., and if (:eo, l/11) :f: 0, lll/Jtll = 1, then

lim XI; = AtlPl· k-+oo

Proof: We suppose that At is the largest eigenvalue of A and that t/11 is the corresponding eigenvector of A •, (l/Jt. tPt) = 1. We represent the initial vector as xo = a1l/Jt + r1 with a1 = (xo, t/11) and note that rt is orthogonal to tPI. (rt, tPt) = 0. The orthogonal complement of tPt is an invariant subspace of A since (Ar, t/Jt) = (r, A*t/Jt) = (r, At tPt) = 0 if (r, tPt) = 0. Furthermore,

IIArll $ Kllrll

for some constant K < IAtl, whenever (r,t/Jt) = 0. Now we calculate that

where

so that

Akxo = a1Atl/J1 + r~;

llr~;ll = IIAAiroll $ Kkllroll,

lim ,1~;A"xo-+ atlPl· k-+oo "l

Thus, if a1 :f: 0, our result is established. Furthermore, the rate of convergence is approximately f , which is a measure of the separation between the largest eigenvalue At and till other eigenvalues of A. 1


Example: The Ranking of Teams

Every Fall a controversy rages throughout the United States trying to decide which is the best collegiate football team {division I-A). The dilemma is that there are over a hundred teams vying for the title, but no team plays more than 12 or 13 opponents, so there are often good teams that have not been matched up head to head. It is certain that this same controversy occurs in many other sports and in many other countries. A possible resolution to this controversy is to devise a ranking scheme that assigns a rank to each team based on its performance. One idea (there are several) for how to calculate a team's rank is as follows: We assume that team i has rank r;, a number between 0 and 1. The team earns a score based on its games with other teams

8; = _!_ L a;Jrh n; J'f.i

where a;J is a number that reflects the result of the head to head contest between teams i and j. The score depends on the product of a;; and r; in order to give some weight to the strength of opposition. It is reasonable to take a;i + ai; = 1 with a;J > aii if team i beat team j in their head to head contest. The division by n;, the total number of games played by team i, is necessary to normalize the score 8;. The entry a;; is zero if team i has not played team j, and of course, a;t = 0. It is now reasonable to suppose that the score and the rank of a team should be related by ..\r, = s;, so that the determination of rank reduces to an eigenvalue problem Ar = ..\r, where the entries of A are the weighted elements a;; /n;. The question is to determine if this equation has a useful solution.

This leads us to consider an important class of matrices called positive matrices. A positive matrix is a matrix A all of whose entries are positive. Similarly, a nonnegative matrix is one all of whose elements are nonnegative. The matrix that we constructed to rank football teams is nonnegative. The classically important statement about these matrices is called the Perron Frobenius Theorem.

Theorem 1.15 (Perron Frobenius) A nonnegative matrix A has a nonnegative eigenfunction with corresponding positive eigenvalue. Furthermore, if the matrix is irreducible, the eigenfunction is unique and strictly positive, and the corresponding eigenvalue is the largest of all eigenvalues of the matrix.

We leave the proof of this theorem to the interested reader, which can be , found in several sources, including [56]. There are several equivalent ways to de

scribe irreducibility. Perhaps the most geometrical is that a matrix is irreducible if for every pair of integers i, j, there is a sequence k1, k2 , ••• , km for which the product Oik1 OA:1 A:2 ••• akmi is nonzero. In practical terms for football teams, this means that between any two teams there is a path of common competitors.

The implementation of the Perron Frobenius theorem uses the power method. There is a positive number, say p.,_ so that the eigenvalues of the matrix A+ p.I

1.6. APPLICATIONS OF EIGENVALUES AND EIGENFUNCTIONS 45

are all strictly positive. FUrthermore, the largest eigenvalue of the matrix A+ p.I is ,\ + p., where tjJ is the positive eigenvector and ,\ the corresponding positive eigenvalue of A. Hence, if we power the matrix A + JLl, convergence to tjJ is assured (provided A is irreducible), and tjJ is the positive ranking vector we seek.

1.6.3 Iteration Methods

The power method is one of the many iteration methods that allow rapid extraction of important information about a matrix. In fact, matrix iteration is the only practical method by which to solve many important problems associated with partial differential equations. Direct methods, such as Gaussian eliminartion, are certain to work if the problem is small enough, but direct methods have serious difficulties when matrices are large and sparse (a sparse matrix is one which has mostly zero entries), as they often are with the simulation of partial differential equations.

The main idea of many iteration methods is to split the matrix A into two parts, A = A1 + A2, where A1 is invertible and easily solved, and then to write the equation Ax = b as

A1x = b- A2x.

We then define an iterative procedure by

AlXHl = b- A2Xk· (1.17)

The convergence of this iteration is easily seen to depend on the eigenvalues of the matrix A11 A2 . That is, if xis a solution of the problem Ax= b, then

Xk+l - x = A11 A2(xk - x).

If A11 A2 has eigenvectors t/J; with corresponding eigenvalues J.l.j, and if Xo -x = Ej=l a;t/J;, then

Xk- X=~ OjJ.l.Nj· j

The iteration converges provided the largest eigenvalue of A11 A2 is smaller than 1 in absolute value.

Now the trick is to split A in such a way as to make the largest eigenvalue of A11 A2 as small as possible, thereby making convergence as fast as possible. The easiest split of A is perhaps

A=D+L+U

where D is the diagonal part of A, and L and U are lower and upper triangular (off-diagonal) parts of A, respectively. Then, if we choose A1 = D, and A2 = L + U, iterates are called Jacobi iterates. These iterates are certainly easy to calculate, since they only involve solving the system

Dxn+l = b- (L + U)xn,


which, if the diagonal elements are all nonzero, is easy and fast. However, it is not much harder to take At= D + U and A2 = L, in which case the iterates

(D + L)xn+l = b- Uxn

require solution of a lower triangular system of equations at each step. This method goes by the name of Gauss-Seidel iterates.

The key determinant to the success of these methods is the size of the eigenvalues of A}1 A2 • While these cannot be calculated easily for many matrices, it is illustrative to see what these are for some simple examples.

Suppose D is a scalar multiple of the identity matrix, D = dl. H t/J is an eigenvector of A with eigenvalue ..\, At!J = ..\t/J, then t/J is also an eigenvector of D-1(L + U). That is,

n-1(£ + U)t/J = (~ -1)€/J.

It is an immediate consequence that the Jacobi method converges whenever all eigenvalues ..\ of A satisfy 0 < ~ < 2. Thus, for example, if A is diagonally dominant (a matrix A= (atJ) for which IBid~ L#t latJD , Jacobi iterates are convergent.

It is always possible to try to accelerate the convergence of an iterative method. Acceleration is the goal of

Atz~+l = b- A2Zn 1 Zn+l = wx~+l + (1 - w)xn

for some parameter w. Notice that if w = 1, the iterations are unchanged. With w i- 1, this method is called successive over-relaxation or SOR, and the parameter w, usually taken to be bigger than 1, is called the relaxation parameter. Notice that if the relaxation parameter is less than one then the new guess Zn+t is an interpolation between z~+l and Zn, whereas if the relaxation parameter is greater than one, the new guess is an extrapolation.

It is known that for many matrices the Jacobi method converges more slowly than the Gauss-Seidel method, and with careful choice of the relaxation parameter, the SOR modification of Gauss-Seidel iterates converges fastest of them all. It is not hard to see how to choose the relaxation parameter to get improved convergence from the SOR method.

Suppose the eigenvalues of A}1 A2 are known, with

Ai1 A2tPJ = l'itPi·

The convergence of the SOR method is assured if the eigenvalues of

A}1( -wA2 + {1 ·~ w)A1) = -wA}1 A2 + (1- w)I

are less than one in magnitude. However, if tPi is an eigenvector of Ai1 A2, then

( -wAi1 A2 + (1- w)I)t!JJ = (1- w(JJJ + 1))tPi·

1.6. APPLICATIONS OF EIGENVALUES AND EIGENFUNCTIONS 47

Now suppose that !Jmsn and /Jmaz are the smallest and largest eigenvalues of A11 A2 , and that -1 < l'min < l'maz < 1. Then the optimal relaxation parameter is found by balancing the numbers 1- w(JJj + 1) between -1 and 1, taking

2 Wopt = 1

2 + Jlomln + Jloma3l

in which case the eigenvalue that determines the rate of convergence for the SOR method is

/Jmaz - Jlomln

..\soR = 2 + l'maz + Jlomln

This is an improvement unless JLmsn = -l'maz·

Multigrid Methods

The solution of large systems of equations which come from the discretization of partial differential equations has been revolutionized with the discovery of a new approximate solution technique called the multigrid method. As we will see, the success of this method is completely dependent upon its relationship to the eigenvalues and eigenfunctions of the underlying problem.

However, at this point in our development of transform theory, it is not yet appropriate to discuss this important solution technique, so we defer the discussion of multigrid methods to Chapter 8.

Further Reading

There are numerous linear algebra books that discuss the introductory material of this chapter. Three that are recommended are:

• P. R. Halmos, Finite Dimensional Vector Spaces, Van Nostrand Reinhold, New York, 1958.

• G. Strang, Linear Algebra and its Applications, 3rd ed., Academic Preas, New York, 1988.

• R. A. Horn and C. R. Johnson, MatriX Analysis, Cambridge University Press, Cambridge, 1985.

The geometrical and physical meaning of eigenvalues is discussed at length in the classic books

• R. Courant and D. Hilbert, Methods of Mathematical Physics, Volume I, Wiley-Interscience, New York, 1953.

• J. H. Wilkinson, The Algebraic Eigenvalue Problem, Oxford University Preas, London, 1965.

Numerical aspects of linear algebra are discussed, for example, in


• G. W. Stewart, Introduction to Matrix Computations, Academic Press, New York, 1980,

• G. E. Forsythe, M.A. Malcolm and C. B. Moler, Computer Methods for Mathematical Computations, Prentice-Hall, 1977,

• G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed., Johns Hopkins University Press, Baltimore, 1996,

• L. N. 'frefethen and D. Bau, Numerical Linear Algebra, SIAM, Philadelphia, 1997,

and for least squares problems in

• C. L. Lawson and R. J. Hanson, Solving Least Squares Problems, SIAM, Philadelphia, 1995,

while many usable programs are described in

• W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling, Numerical Recipes 1n .Fbrtran 77: The Art of Computing, Cambridge University Press, Cambridge, 1992.

This book (in its many editions and versions) is the best selling mathematics book of all time.

The best and easiest way to do numerical computations for matrices is with Matlab. Learn how to use Matlabl For example,

• D. Hanselman and B. Littlefield, Mastering Matlab, Prentice-Hall, Upper Saddle River, NJ, 1996.

While you are at it, you should also learn Maple or Mathematica.

• B. W. Char, K. 0. Geddes, G. H. Gonnet, B. L. Leong, M. B. Monagan, and S. M. Watt, Maple V Language Reference Manual, Springer-Verlag, New York, 1991,

• A. Heck, Introduction to Maple, 2nd ed., Springer-Verlag, New York, 1996.

• S. Wolfram, Matbematica, 3rd ed., Addison-Wesley, Reading, MA, 1996.

Finally, the ranking of football teams using a variety of matrix algorithms is summarized in

• J.P. Keener, The Perron Frobenius Theorem and the ranking of football teams, SIAM Rev., 35, 80-93, 1993.

PROBLEMS FOR CHAPTER 1 49

Problems for Chapter 1

Problem Section 1.1

1. Prove that every basis in a finite dimensional space has the same number of elements.

2. Show that in any inner product space

llx+vll2

+ llx -vll2 = 2llxW + 2llvW.

Interpret this geometrically in JR?.

3. (a) Verify that in an inner product space,

1 Re (x, y} = 4(11x + vW -llx- vll2

).

(b) Show that in any real inner product space there is at most one inner product which generates the same induced norm.

(c) In IRn with n > 1, show that llxiiP = CE~=1 IxkiP) 11P can be induced · by an inner product if and only if p = 2.

4. Suppose f(x) and g(x) are continuous real valued function defined for x E [0, 1). Define vectors in IRn, F = (f(xt), j(x2), ... , f(xn)) and G = (g(xt), g(x2), ... , g(xn)), where Xk = kjn. Why is

1 n (F, G}n = n L f(xk)g(xk)

k=l

with Xk = ~'not an inner product for the space of continuous functions?

5. Show that

(!,g)= 11

(!(x)g(x) + f'(x)g'(x)) dx.

is an inner product for continuously differentiable functions on the interval [0, 1).

6. Show that any set of mutually orthogonal vectors is linearly independent.

7. (a) Show that IRn with the supremum norm llxlloo = maxk{lxkl} is a normed linear vector space.

(b) Show that IRn with norm llxl It = :E~=l lxk I is a normed linear vector.

8. Verify that the choice-y =~minimizes llx--yyll2 • Show that l(x, y}l2 =

llxll2 ·llvll2 if and only if x andy are linearly dependent.


9. A Starting with the set {1, x, x2 , ••• , xk, ... }, use the Gram-Schmidt procedure and the inner product

(f,g} = ib f(x)g(x)w(x)dx, w(x) > 0

to find the first five orthogonal polynomials when

(a) a= -1, b = 1, w(x) = 1 (Legendre polynomials)

(b) a= -1, b = 1, w(x) = (1- x2)-112 (Chebyshev polynomials)

(c) a= 0, b = oo, w(x) = e-~ (Laguerre polynomials)

(d) a= -oo, b = oo, w(x) = e-:~~2 (Hermite polynomials)

Remark: All of these polynomials are known by Maple.

10. ,!;!. Starting with the set {1, x, x2 , ••• , xn, ... } use the Gram-Schmidt procedure and the inner product

(f,g} = /_1

1 (f(x)g(x) + f'(x)g'(x))dx

to find the first five orthogonal polynomials.

Problem Section 1.2 1. (a) Represent the transformation whose matrix representation with re

spect to the natural basis is

( 1 1 2) A= 2 1 3

1 0 1

relative to the basis {(1, 1, O)T, (0, 1, l)T, (1, 0, l)T}.

(b) The representation of a transformation with respect to the basis {(1,1,2}T,(1,2,3)T,(3,4,1)T} is

(

1 1 1 ) A= 2 1 3 .

1 0 1

Find the representation of this transformation with respect to the basis {(1, 0, O)T, (0, 1, -l)T, (0, 1, 1)T}.

2. (a) Prove that two symmetric matrices are equivalent if and only if they have the same eigenvalues {with the same multiplicities).

(b) Show that if A and B are equivalent, then

detA = detB.

pROBLEMS FOR CHAPTER 1 51

(c) Is the converse true?

3. (a) Show that if A is ann x m matrix and B is an m x n matrix, then AB and BA have the same nonzero eigenvalues.

(b) Show that the eigenvalues of AA • are real and non-negative.

4. Show that the eigenvalues of a real skew-symmetric (A = -AT) matrix are imaginary.

5. Find a basis for the range and null space of the following matrices:

(a)

A~o D· (b)

( 1 2 3) A= 3 1 2 .

1 1 1

6. Find an invertible matrix T and a diagonal matrix A so that A = TAT-1

for each of the following matrices A:

(a)

( 1 0 0 ) 1/4 1/4 1/2 0 0 1

(b)

( -~ ~) (c) c 0 0) 1 2 0

2 1 3

(d) c 1 0) 0 1 0 0 1 1


(e)

(

1/2 1/2 ~/6 ) 1/2 1/2 ~/6 . ~/6 ~/6 5/6

7. Find the spectral representation of the matrix

A= ( _; ~). illustrate how Ax = b can be solved geometrically using the appropriately cliosen coordinate system on a piece of graph paper.

8. Suppose P is the matrix that projects (orthogonally) any vector onto a manifold M. Find all eigenvalues and eigenvectors of P.

9. The sets of vectors {1/>i}f.:tJ {,P,}f::t are said to be biorthogonal if (1/>,,P;} = 6,;. Suppose {1/>slr:t and {,P,}f:t are biorthogonal.

(a) Show that {1/>i}f::t and {1/li}f::t each form a linearly independent set.

(b) Show that any vector in JRn can be written as a linear combination · of {t/>i} as

where a, = (x, 1/Js)·

n

x= :Eait/>i i=t

(c) Express (b) in matrix form; that is, show that

n

x= LP,x i=t

where P1 are projection matrices with the properties that P( = P, and P,P; = 0 fori¥:. j. Express the matrix Pi in terms of the vectors t/>1 and ,p,.

10. (a) Suppose the eigenvalues of A all have algebraic multiplicity one. Show that the eigenvectors of A and the eigenvectors of A • form a biorthogonal set.

(b) Suppose At/>i = Ast/>i and A•,p, = >..,,p,, i = 1, 2, ... , n and that Ai ¥:. A; for i ¥:. j. Prove that A = E~1 A,P, where P, = tf>,,p; is a projection matrix. Remark: This is an alternate way to express the spectral decomposition theorem for a matrix A.

(c) Express the matrices C and c-t, where A= CAc-t, in terms of t/>1 and ¢ 1•

(d) Suppose At/> = At/> and A,P = X,p and the geometric multiplicity of A is one. Show that it is. not necessary that (I/>, ,P} ¥:. 0.

pROBLEMS FOR CHAPTER 1

Problem Section 1.3

1. Use the minimax principle to show that the matrix

(

2 4 5 1 ) 4 2 1 3 5 1 60 12 1 3 12 48

has an eigenvalue~ < -2.1 and an eigenvalue At > 67.4.

53

2. (a) Prove an inequality relating the eigenvalues of a symmetric matrix before and after one of its diagonal elements is increased.

(b) Use this inequality and the minimax principle to show that the smallest eigenvalue of

( 8 4 4)

A= 4 8 -4 4 -4 3

is smaller than -1/3.

( 1 2 3) 2 2 4 3 4 3

is not positive.

4. The moment of inertia of any solid object about an axis along the unit vector x is defined by

J(x) = L ~(y)pdV, where d11 (y) is the perpendicular distance from the point y to the axis along x, p is the density of the material, and R is the region occupied by the object. Show that J(x) is a quadratic function of x, J(x) = :z;T Ax where A is a symmetric 3 x 3 matrix.

5. Suppose A is a symmetric matrix with eigenvalues At ~ ..\2 > As ~ •.•• Show that

max(u,v)=O(Au, u} + (Av, v} = At + ..\2

where !lull = llvll = 1.

Problem Section 1.4

1. Under what conditions do the matrices of Problem 1.2.5 have solutions Ax = b? Are they unique?


2. Suppose P projects vectors in JRn (orthogonally) onto a linear manifold M. What is the solvability condition for the equation Px = b?

3. Show that the matrix A= (lliJ) where a,i = (</Jit</JJ) is invertible if and only if the vectors <Pi are linearly independent.

4. A square matrix A (with real entries) is positive-definite if (Ax, x) > 0 for all x :/: 0. Use the Fredholm alternative to prove that a positive definite matrix is invertible.

Problem Section 1.5 1. Use any of the algorithms in the text to find the least squares pseudo

inverse for the following matrices:

(a) c 0 0) 0 1 0 1 1 0

(b)

( ~ -~) -1 3

(c)

( 3 1 2) -1 1 -2

(d)

( -1 0 1) -1 2/3 2/3 . -1 -2/3 7/3

(e) Jl. The linear algebra package of Maple has procedures to calculate the range, null space, etc. of a matrix and to augment a matrix with another. Use these features of Maple to develop a program that uses exact computations to find A' using the Gaussian elimination method (Method 1), and use this program to find A' for each of the above matrices.

2. Verify that the least squares pseudo-inverse of an m x n diagonal matrix D with ~i = t1i6i; is the n x m diagonal matrix D' with ~i = ;

1 6,;

whenever q' :/: 0 and d~i = 0 otherwise.

pROBLEMS FOR CHAPTER 1 55

3. (a) For any two vectors x,y E mn with llxll = IIYII find the Householder (orthogonal) transformation U that satisfies Ux = y.

(b) Verify that a Householder transformation U satisfies U*U =I.

4. Use the Gram-Schmidt procedure (even though Householder transformations are generally preferable) to find the QR representation of the matrix

( 2 l 3) A= 4 2 1 . 9 1 2

5. For the matrix

( 5 -3) A= 0 4 I

illustrate on a piece of graph paper how the singular value decomposition A = UEV* transforms a vector x onto Ax. Compare this with how A= TAT-1 transforms a vector x onto Ax.

6. For the matrix

A=(~!), illustrate on a piece of graph paper how the least squares pseudo-inverse A' = Q-1 A'Q transforms a vector b into the least squares solution of Ax=b.

7. For each of the matrices in Problem 1.2.6, \lSe the QR algorithm to form the iterates An+l = Q;;1 AnQn, where An = QnRn· Examine a few of the iterates to determine why the iteration works and what it converges to.

8. For the matrices

A=(~~) and

A ( 1.002 0.998 ) = 1.999 2.001 I

illustrate on a piece of graph paper how the least squares pseudo-inverse A' = VE'U* transforms a vector b onto the least squares solution of Ax = b. For the second of these matrices, show how setting the smallest singular value to zero stabilizes the inversion process.

9. For a nonsymmetric matrix A= T-1 AT, with A a diagonal matrix, it is not true in general that A' = T-1 A'T is the pseudo-inverse. Find a 2 x 2 example which illustrates geometrically what goes wrong.


10. Find the singular value decomposition and pseudo-inverse of

( 2v'S -2v'5)

A= 3 3 . 6 6

11. Find the least squares solution of then linear equations

a;x + b;y = Ci i = 1,2, ... ,n,

where a;b1 - a1b, :f: 0 for i :f: j. H r1, j = 1, 2, ... , k = n(n- 1)/2 are the solutions of all possible pairs of such equations, show that the least squares solution

r = (:)

is a convex linear combination of r i, specifically

where

k

r = LPJTj, J=l

D2 P - i i- """ D2' L..ti=l i

and Di is the determinant of the jth 2-equation subsystem. Interpret this result geometrically.

12. The matrices M~c, k = 1, 2, ... , n, represent elementary row operations if they have the form

fflij = 1 if i = j m,k = -aile if i > k m;1 = 0 otherwise.

Suppose L is a triangular matrix with entries l;j = 1 if i = j, l,i = a,i if i > j, l;j = 0 otherwise. Show that (MnMn-1 · · · Mt)-1 = L.

13 . .1!. Find the SVD of then x n Hilbert segment A= (at1),au = ,~.,for several values of n. Why is this matrix ill-conditioned?

3

14. (a) Show that IIAIIF = Tr(A* A) is a norm on the space of n x k matrices.

(b) Show that if A and B are related through an orthogonal similarity transformation, then IIAIIF = IIBIIF·

(c) Show that IIAIIF = IIA*IIF, even though A and A* may have different dimensions.

PROBLEMS FOR CHAPTER 1 57

Problem Section 1.6

1. A Write a computer program (in MATLAB) to find the largest eigenvalue of a matrix using the power method. Use this program to solve the following problems:

(a) Find the largest eigenvalue of the matrix in Problem 1.3.1.

(b) Find the smallest eigenvalue of the matrix in Problem 1.3.1. (Shift the eigenvalues by subtracting an appropriate constant from the diagonal.)

2. ~ Develop a ranking scheme for your favorite sport. First collect the data for all the games played between teams (or individuals).

(a) Take a;i = 1 if team i beat team j and zero otherwise. Does this give a reasonable ranking?

(b) Take aii to be the percentage of points earned by team i in the contest between team i and j. Does this give a reasonable ranking?

(c) Propose your own method to assign values to the matrix A.

3. Prove that a diagonally dominant matrix has a nonnegative eigenvector.

I ·~

i :iJ ~ •'ffi

t • -~

I :}§ -~

J Bibliography j ·~ 1 t ~

I ·~ ~ ::1 t .~ };~

.~ ~ ~ •';! 'i , -_'4

.~

I I I " 1 '$

! '$1

[1] M. J. Ablowitz, P. A. Clarkson, and M. A. Ablowitz. Solitons, Nonlinear Evolution Equations and Inverse Scattering, volume 149 of London Mathematical Society Lecture Notes. Cambridge University Press, 1991. {432}

[2] M. J. Ablowitz and A. S. Fokas. Complex Variables: Introduction and Applications. Number 16 in Cambridge Texts in Applied Mathematics. Cambridge University Press, Cambridge, 1997. {273}

[3] M. J. Ablowitz and H. Segur. Solitons and the Inverse Scattering Thinsform. SIAM, Philadelphia, 1981. { 432}

[4] M. Abramowitz and I. A. Stegun. Handbook of Mathematical Functions. Dover, New York, 1974. {402}

[5] K. Aki and P. G. Richards. Quantitative Seismology, Theory and Methods: Vol. JJ. Freeman, 1980. {132}

[6) R. Askey. Orthogonal Polynomials and Special Functions. SIAM, Philadelphia, 1975. {93}

[7] K. E. Atkinson. A Survey of Numerical Methods for the Solution of Fredholm Integral Equations of the Second Kind. SIAM, Philadelphia, 1976. {127}

[8] G. K. Batchelor. An Introduction to Fluid Dynamics. Cambridge University Press, Cambridge, 1967. { 488, 497}

[9] M.S. Bauer and D. S. Chapman. Thermal regime at the Upper Stillwater dam site, Uinta Mountains Utah. Tectonophysics, 128:1-20, 1986. {408}

[10] C. M. Bender and S. A. Orszag. Advanced Mathematical Methods for Scientists and Engineers. International Series in Pure and Applied Mathematics. McGraw-Hill, 1978. { 463}

[11] P. G. Bergmann. Introduction to the Theory of Relativity. Dover, New York, 1976. {203}

559

560 BIBLIOGRAPHY

[12] N. Bleistein and R. A. Handelsman. Asymptotic Expansions of Integrah. Dover, New York, 1986. {463}

[13] A. Brandt. Multi-level adaptive solutions to boundary-value problems. Math. Comp., 31:333-390, 1977. {402}

[14] W. L. Briggs. Multigrid Thtorial. SIAM, Philadelphia, 1987. {402}

[15] J. Buckmaster and G. S. S. Ludford. Theory of Laminar Flames. Cambridge University Press, New York, 1982. {551}

[16] G. F. Carrier, M. Krook, and C. E. Pearson. Functions of a Complex Variable. McGraw-Hill, New York, 1966. {273}

[17] K. M. Case. On discrete inverse scattering problems. J. Math. Phys., 14:916-920, 1973. { 431, 432}

[18] B. W. Char, K. 0. Geddes, G. H. Gonnet, B. L. Leong, M. B. Monagan, and S.M. Watt. Maple V Language Reference Manual. Springer-Verlag, 1991. {48}

[19] A. Cherkaev. Variational Methods for Structuml Optimization. Springer, New York, 1999. {203}

[20) Y. Choquet-Bruhat and C. DeWitt-Moretti. Analysis, Manifolds and Phusics. North-Holland, Amsterdam, 1982. { 497}

[21] S. N. Chow and J. K. Hale. Methods of Bifurcation Theory. SpringerVerlag, New York, 1982. { 496}

[22] S. N. Chow, J. K. Hale, and J. Mallet-Paret. An example of bifurcation to homoclinic orbits. J. Diff. Eqns., 37:351-373, 1980. {497}

[23] C. K. Chui. An Introduction to Wavelets. Academic Press, 1992. {93, 307, 328}

[24) E. T. Copson. Theory of Functions of a Complex Variable. Oxford at the Clarendon Press, 1935. {273}

[25) E. T. Copson. Asymptotic Expansions. Cambridge University Press, Cambridge, 1967. {463}

[26) A. M. Cormack. Representation of a function by its line integrals, with some radiological applications. J. Appl. Phys., 34:2722-2727, 1963. {128}

[27] R. Courant and D. Hilbert. Methods of Mathematical Physics, volume 1. Wiley-Interscience, New York, 1953. {47, 171}

[28] S. J. Cox and M. L. Overton. On the optimal design of columns against buckling. SIAM J. Appl. Math., .23:287-325, 1992. {203}

' ~ BIBLIOGRAPHY

·~ 561

"

1

I 1 il ?l i ·' ·~

' ~~ j ll ·J

1 ~ 3 ~1

<<

~j ~~ ;j

~ :l

j .; ~ ,')

I ~ ;j

~ ~

l 'P

1 'j J

-~ ·•

~ 1 ! oW

~ '1

[29] I. Daubechies. Ten Lectures on Wavelets. SIAM, Philadelphia, 1992. {93, 328}

[30] C. deBoor. A Pmctical Guide to Splines, volume 27 of Applied Mathematical Sciences. Springer-Verlag, New York, 1978. {93}

[31] W. Eckhaus and A. Van Harten. The Inverse Scattering Transformation and the Theory of Solitons. North Holland, Amsterdam, 1981. {432}

[32] A. Erdelyi. Asymptotic Expansions. Dover, New York, 1987. {463}

[33) P. C. Fife. Models for phase separation and their mathematics. In M. Mimura and T. Nishiuda, editors, Nonlinear Partial Differential Equations and Applications. Kinokuniya Publishers, 1991. { 402}

[34] H. Flaschka. On the Toda lattice, II-inverse scattering solution. Prog. . Theor. Phys., 51:703-716, 1974. {431, 432}

[35] G. E. Forsythe, M.A. Malcolm, and C. B. Moler. Computer Methods for Mathematical Computations. Prentice-Hall, 1977. {48}

[36] A. C. Fowler: Mathematical Models in the Applied Sciences. Cambridge Texts in Applied Mathematics. Cambridge University Press, Cambridge, 1997. {551}

[37) B. Friedman. Principles and Techniques of Applied Mathematics. Dover, New York, 1990. {171, 328, 401}

[38] P. R. Garabedian. Partial Differential-Equations. Chelsea Publishing Co., New York, 1998. {401}

(39) I. M. Gel'fand and S. V. Fomin. Calculus of Variations. Prentice-Hall, Englewood Cliffs, 1963. {202}

[40) I. M. Gel'fand and B. M. Levitan. On the determination of a differential equation from its spectral function. Am. Math. Soc. Transl., pages 259-309, 1955. Series 2. { 432}

[41] I. M. Gel'fand and G. E. Shilov. Genemlized Functions, volume 1. Academic Press, New York, 1972. {171}

[42) H. Goldstein. Cl£Usical Mechanics. Addison-Wesley, Reading, MA, 2nd edition, 1980. {203}

[43] G. H. Golub and C. F. Van Loan. Matrix Computations. Johns Hopkins University Press, 3rd edition, 1996. {48}

[44) D. Gottlieb and S. A. Orszag. Numerical Analysis of Spectral Methods; Theory and Applications. SIAM, Philadelphia, 1977. { 401}

562 BIBLIOGRAPHY

(45] J. Guckenheimer and P. Holmes. Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields, volume 42 of Applied Mathematical Sciences. Springer-Verlag, New York, revised edition, 1990. {496}

(46] R. Haberman. Elementary Partial Differential Equations. Prentice-Hall, 3rd edition, 1997. { 401}

(47] J. K. Hale. Ordinary Differential Equations. Wiley-Interscience, New York, 1969. {513, 551}

(48] P.R. Halmos. Finite Dimensional Vector Spaces. Van Nostrand Reinhold, New York, 1958. {47}

(49] D. Hanselman and B. Littlefield. Mastering Matlab. Prentice-Hall, Upper Saddle River, NJ, 1996. {48}

[50] A. Heck. Introduction to Maple. Springer-Verlag, New York, 2nd edition, 1996. {48}

[51] P. Henrici. Applied and Computational Complex Analysis. Wiley and Sons, 1986. {273}

[52] H. Hochstadt. Integral Equations. Wiley-Interscience, New York, 1973. {127}

[53] H. Hochstadt. The FUnctions of Mathematical Physics. Dover, New York, 1986. {93}

[54] M. H. Holmes. Introduction to Perturbation Methods. Springer-Verlag, New York, 1995. {551}

(55] F. Hoppensteadt. Mathematical Theories of Populations: Demographics, Genetics and Epidemics. SIAM, Philadelphia, 1975. {127}

[56] R. A. Hom and C. R. Johnson. Matrix Analysis. Cambridge University Press, Cambridge, 1985. {44, 47}

[57] P. Horowitz and W. Hill. The Art of Electronics. Cambridge Press, Cambridge, MA, 2nd edition, 1989. { 497}

[58] C. M. Hutchins. The acoustics of violin plates. Scientific American, 245:17Q-187, 1981. {401}

[59] F. M. Phelps Ill, F. M. Phelps IV, B. Zorn, and J. Gormley. An experimental study of the brachistochrome. Eur. J. Phys., 3:1-4, 1984. {204}

(60] G. Iooss and D. Joseph. Elementary Stability and Bifurcation Theory. Springer-Verlag, New York, 1989. {496}

(61] E. Jahnke and F. Emde. Tables of FUnctions. Dover, New York, 1945. {402}

BIBLIOGRAPHY 563

[62] 0. Kappelmeyer and R. Haenel. Geothermics with Special Reference to Application. Number No 4 in Geoexploration Monographer, Ser. 1. Gebriider Borntraeger, Berlin, 1974. {401}

[63] J. Keener and J. Sneyd. Mathematical Physiology. Springer-Verlag, New York, 1998. {551}

(64] J.P. Keener. Analog circuitry for the van der Pol and Fitzhugh-Nagumo equations. IEEE 7rans on Sys., Man, Cyber., pages 101Q-1014, 1983. SMG-13. {497}

(65] J. P. Keener. The Perron Frobenius theorem and the ranking of football teams. SIAM Rev., 35:8Q-93, 1993. {48}

[66] J. B. Keller. Inverse problems. Am. Math. Monthly, 83:107-118, 1976. {432}

[67] J. B. Keller and S. Antman. Bifurcation Theory and Nonlinear Eigenvalue Problems. Benjamin, New York, 1969. { 496}

[68] J. Kevorkian and J. D. Cole. Multiple Scale and Singular Perturbation Methods, volume 114 of Applied Mathematical Sciences. Springer-Verlag, Berlin, 1996. {546, 551}

(69] S. Kida. A vortex filament moving without change of form. J. Fluid Mech., 112:397-409, 1981. { 497} .

(70] D. J. Korteweg and G. deVries. On the change of form of long waves advancing in a rectangular canal, and on a new type of long stationary waves. Phil. Mag., 39:422-443, 1895. { 432}

[71] Y. Kuznetsov. Elements of Applied Bifurcation Theory. Springer-Verlag, New York, 2nd edition, 1998. {496}

(72] G. L. Lamb, Jr. Elements of Soliton Theory. John Wiley and Sons, New York, 1980. {328, 432, 497}

[73] L. D. Landau and E. M. Lifshitz. Mechanics. Butterworth-Heineman, 3rd edition, 1976. {203}

[74] C. L. Lawson and R. J. Hanson. Solving Least Squares Problems. SIAM, Philadelphia, 1995. {48}

[75] P. Lax. Integrals of nonlinear equations of evolution and solitary waves. Comm. Pure Appl. Math., 21:407-490, 1968. {432}

(76] G. A. Lion. A simple proof of the Dirichlet-Jordan convergence theorem. Am. Math. Monthly, 93:281-282, 1986. {165}

(77] W. V. Lovitt. Linear Integral Equations. Dover, New York, 1950. {127}

564 BIBLIOGRAPHY

[78] J. Lund and K. L. Bowers. Sine Methods for Quadrature and Differential Equatiom. SIAM, Philadelphia, PA, 1992. {93, 274}

[79] W. S. Massey. Algebraic Topology: An Introduction. Springer-Verlag, New York, 1990. { 497}

[80] S. F. McCormick, editor. Multigrid Methods. Frontiers in Applied Mathematics. SIAM, Philadelphia, 1987. { 402}

[81] L. M. Milne-Thomson. Theoretical AerodJmamics. Dover, New York, 1973. {274}

[82) J. D. Murray. Asymptotic Analysis. Springer-Verlag, New York, 1984. {463}

[83) J. D. Murray. Mathematical Biology. Springer-Verlag, New York, 1989. {402}

[84) F. Natterer. The Mathematics of Computerized Tomography. Wiley, New York, 1986. {128}

[85] A. W. Naylor and G. R. Sell. Linear Operator Theory in Engineering and Science. Holt, Rinehart and Winston, New York, 1971. {127} ·

[86) F. W. J. Olver. Asymptotics and Special F'u.nctions. A. K. Peters, Ltd., London, 1997. {463}

[87] R. E. O'Malley, Jr. Introduction to Singular Perturbations. Academic Press, New York, 1974. {551}

[88) M. Pickering. An Introduction to Fast Fourier Transform Methods for Partial Differential Equations, with Applicatiom. Wiley and Sons, New York, 1986. { 401}

[89] D; L. Pqwers. Boundary Value Probletm. Academic Press, 4th edition, 1999. {401}

(90] W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling. Numerical Recipes in Fortran 77: The Art of Computing. Cambridge University Press, Cambridge, 1992. {48, 77, 402}

[91] R. H. Rand. Topics in Nonlinear Dynamics with Computer Algebra. Gordon and Breach, Langhorne, PA, 1994. {497}

[92] R. H. Rand and D. Armbruster. Perturbation Methods, Bifurcation Theory and Computer Algebra, Applied Mathematical Sciencu, volume 65. Springer-Verlag, New York, 1987. {497}

(93] G. F. Roach. Green's FUnctions: Introductory Theory with Applications. Van Nostrand Reinhold, London, 1970. {171}

l l ., 'i .:t

l ~

~

~~ 1

~ ~~

:1 :1 J'l

J ~

BIBLIOGRAPHY 565

[94] T. D. Rossing. The physics of kettledrums. Scientific American, 247:172-179, Nov. 1982. {401}

[95] W. Rudin. Principles of Mathematical Analysis. McGraw-Hill, New York, 3rd edition, 1976. {92, 165}

[96] W. Rudin. Real and Complex Analysis. McGraw-Hill, New York, 3rd edition, 1987. {164}

[97] D. R. Smith. Singular Perturbation Theory. Cambridge University Press, Cambridge, 1985. {551}

[98] I. N. Sneddon. Fourier Transforms. Dover, New York, 1995. {328}

[99] I. Stakgold. Branching of solutions of nonlinear equations. SIAM Rev., 13:289-289, 1971. { 496}

[100] I. Stakgold. Boundary Value Problems of Mathematical Physics. Macmillan, New York, 2nd edition, 1998. {171, 328}

[101] F. Stenger. Numerical methods based on Whittaker cardinal, or sine functions. SIAM Rev., 23:165-224, 1981. {274}

[102] F. Stenger. Numerical Methods Based on Sine and Analytic F'u.nctions, volume 20 of Springer Series in Computational Mathematics. SpringerVerlag, New York, 1993. {93, 274}

[103) G. W. Stewart. Introduction to Matrix Computations. Academic Press, New York, 1980. {48}

[104] J. J. Stoker. Nonlinear Elasticity. Gordon and Breach, New York, 1968. {203}

[105) J. J. Stoker. Differential Geometry. Wiley-Interscience, New York, 1969. {202, 497}

[106] G. Strang. Linear Algebra and its Applications. Academic Press, New York, 3rd edition, 1988. {47}

[107) G. Strang and G. Fix. An Analysis of the Finite Element Method. PrenticeHall, 1973. {93}

[108] G. Strang and T. Nguyen. Wavelets and Filter Banks. WellesleyCambridge Press, Wellesley, MA, 1997. {93, 328}

(109] A. F. Timin. Theory of Approximation of Functions of a Real Variable. Macmillan, New York, 1963. {328}

[110) E. C. Titchmarsh. Theory of F'u.nctions. Oxford University Press, 1939. {273}

566 BIBLIOGRAPHY

[111] E. C. Titchmarsh. Eigenfunction Expansions Associated with SecondOrder Differential Equations. Oxford University Press, Oxford, 1946. {328}

[112] L. N. Trefethen and D. Bau. Numerical Linear Algebra. SIAM, Philadelphia, 1997. {48}

[113] F. G. 'Iiicomi. Integral Equations. John Wiley and Sons, New York, 1957. {127}

(114) A. M. Turing. The chemical basis of morphogenesis. Phil. 7rans. Roy. Soc. Lond., B237:37-72, 1952. {390}

[115] M. Van Dyke. Perturbation Methods in Fluid DynamiC8. Academic Press, New York, 1964. {551}

[116) M. Van Dyke. An Album of Fluid Motion. The Parabolic Press, Stanford, 1982. {274}

(117) R. Wait and A. R. Mitchell. Finite Element Analysis and Applications. Wiley and Sons, New York, 1968. {93}

[118] J. W. Ward and R. V. Churchill. Fourier Series and Boundary Value Problems. McGraw Hill, New York, 5th edition, 1993. {92}

(119] H. F. Weinberger. Variational methods for eigenvalue approximation. SIAM, 1974. {203}

[120] H. F. Weinberger. A First Course in Partial Differential Equations. Dover, New York, 1995. { 401}

(121} G. B. Whitham. Linear and Nonlinear Waves. Wiley and Sons, New York, 1974. {401}

[122) J. H. Wilkinson. The Algebraic Eigenvalue Problem. Oxford University Press, London, 1965. {47}

[123] F. A. Williams. Combustion Theory. Addison-Wesley, Reading, Mass., 1965. {551}

[124) T. J. Willmore. An Introduction to Differential Geometry. Oxford University Press, London, 1959. {203}

[125) S. Wolfram. Mathematica. Addison-Wesley, Reading, MA, 3rd edition, 1996. {48}

(126] E. Zauderer. Partial Differential Equations of Applied Mathematics. Wiley and Sons, New York, 2nd edition, 1998. { 401}

[127) A. H. Zemanian. Distribution Theory and 7ransform Analysis: An Introduction to Generalized Functions, with Applications. Dover, New York, 1987. {171}

j

Appendix A


[jt.l.l; Hint: What if one basis was smaller than another?

i 1.1.3; (a) Follows from direct verification.

, (b) Follows from (a). H the norm is known to be induced by an inner '! product, then (a) shows how to uniquely calculate the inner product.

(c) Suppose llxll = c~=~llxkiP)l/p. (-¢:::) If p = 2, then (x, y) = l':~=l XkYk induces the norm. (:::}) If the norm is induced by an inner product, then from (a)

1 ( n . n ) (x, y) = 4 (L ixk + YkiP) 21P- (L ixk- YkiP) 21P .

k-1 k=l

Take x = (1, 0, 0, ... , 0), and y = 0, 1, 0, ... , 0). Then (x, x) = 1, (x, y) = 0, and (x, x + y) = ! ( (2P + 1?1P - 1). Since for an inner product, (x, x + y) = (x, x) + (x, y), it must be that (2P + 1)2/P = 5. Since (2P + 1)2/P is a monotone decreasing function of p which approaches 1 for large p and is unbounded at the origin, the solution of (2P + 1)2/P = 5 at p = 2 is unique. We conclude that p = 2.

1.1.4; (F,F) = 0 does not imply that f(x) = 0.

1.1.8; Observe that with {3 = (x, y) /llvll 2 , x- {3y is orthogonal toy, so that

ilx- avW = llx- f3vW + li(a- f3)vW,

which is minimized when a= {3. Clearly, if x = ay, then

i(x,v)l2 = llxWIIvW·

If so, then we calculate directly that llx- {3yjj 2 = 0, so that x = {3y.

567

566 BIDLIOGRAPHY

[111] E. C. Titchmarsh. Eigenfunction Expansions Associated with SecondOrder Differential Equations. Oxford University Press, Oxford, 1946. {328}

[112] L. N. Trefethen and D. Bau. Numerical Linear Algebra. SIAM, Philadelphia, 1997. {48}

[113) F. G. 'llicomi. Integral Equations. John Wiley and Sons, New York, 1957. {127}

[114] A. M. Turing. The chemical basis of morphogenesis. Phil. Thlns. Ro1J. Soc. Lond., B237:37-72, 1952. {390}

[115] M. Van Dyke. Perturbation Methods in Fluid Dynamics. Academic Press, New York, 1964. {551}

[116] M. Van Dyke. An Album of Fluid Motion. The Parabolic Press, Stanford, 1982. {274}

[117) R. Wait and A. R. Mitchell. Finite Element Analysis and Applications. Wiley and Sons, New York, 1968. {93}

[118] J. W. Ward and R. V. Churchill. Fourier Series and Boundary Value Problema. McGraw Hill, New York, 5th edition, 1993. {92}

[119] H. F. Weinberger. Variational methods for eigenvalue approximation. SIAM, 1974. {203}

(120] H. F. Weinberger. A Firat Course in Partial Differential Equations. Dover, New York, 1995. {401}

(121] G. B. Whitham. Linear and Nonlinear Waves. Wiley and Sons, New York, 1974. {401}

[122] J. H. Wilkinson. The Algebraic Eigenvalue Problem. Oxford University Press, London, 1965. {47}

[123] F. A. Williams. Combustion Theory. Addison-Wesley, Reading, Mass., 1965. {551}

[124) T. J. Willmore. An Introduction to Differential Geometry. Oxford University Press, London, 1959. {203}

[125) S. Wolfram. Mathematica. Addison-Wesley, Reading, MA, 3rd edition, 1996. {48}

[126) E. Zauderer. Partial Differential Equations of Applied Mathematics. Wiley and Sons, New York, 2nd edition, 1998. { 401}

[127] A. H. Zemanian. Distribution Theory and Thlnsform Anal1Jais: An Introduction to Generalized FUnctions, with Applications. Dover, New York, 1987. {171}

Appendix A


1.1.1; Hint: What if one basis was smaller than another?

1.1.3; (a) Follows from direct verification.

(b) Follows from (a). If the norm is known to be induced by an inner product, then (a) shows how to uniquely calculate the inner product.

(c) Suppose llxll = o=~=llxkiP) 11P. (-<=) If p = 2, then (x, y} = L:~=l XkYk induces the norm. (:::::?)If the norm is induced by an inner product, then from (a)

1 ( n n ) (x, Y} = 4 (L lxk + YkiP) 21P- (L lxk- YkiP) 21P . k-1 k=l

Take x = (1, 0, 0, ... , 0), and y = 0, 1, 0, ... , 0). Then (x, x} = 1, (x, y} = 0, and (x, x + y} = ~ ( (2P + 1 )2/P - 1). Since for an inner product, (x, x + y} = (x, x} + (x, y), it must be that (2P + 1 )2/P = 5. Since (2P + 1)2/P is a monotone decreasing function of p which approaches 1 for large p and is unbounded at the origin, the solution of (2P + 1)2/P = 5 at p = 2 is unique. We conclude that p = 2.

1.1.4; (F, F} = 0 does not imply that f(x) = 0.

1.1.8; Observe that with {3 = (x, y) /llvll2 , x- {3y is orthogonal toy, so that

llx- ayll 2 = llx- f3Yll 2 + ll(o:- f3)yjj 2,

which is minimized when a= {3. Clearly, if x = ay, then

l(x,y)j2 = llxii2 IIYW· If so, then we calculate directly that llx- {3yjj 2 = 0, so that x = {3y.

567

568 APPENDIX A. SELECTED HINTS AND SOLUTIONS

1.1.9; (a) ¢o(x) = 1, c/J1(x) = x, c/J2(x) = x2 - ~, ¢a(x) = x3 - ~x, ¢4(x) = /x4- li.x2 + .1...

7 35.

(b) ¢o(x) = 1,¢1(x) = x,¢2(x) = x2 - ~ = ~cos(2cos- 1 x),¢3 (x) = x3

- ~x =-! cos(3cos-1 x), ¢4(x) = x4 - x2 + k = k cos(4cos-1 x).

(c) ¢o(x) = 1,¢1(x) = x -1,¢2(x) = x2 - 4x + 2,¢3 (x) = x3 - 9x2 + 18x- 6, ¢4(x) = x4 - 48x3 + 72x2 - 96x + 24.

(d) ¢o(x) = 1,¢1(x) = x,¢2(x) = x2 - !,¢a(x) = x3 - ~x,¢4(x) = x4- 3x2 + i·

1.1.10; ¢o(x) = 1,¢1(x) = x,¢2(x) = x2 - ~,¢a(x) = x3 - t0x,¢4(x) = x4 -

33 2 27 ... ( ) 5 1930 3 4411 28 X + 140' '1'5 X = X - 'i359X + 1057X.

(

2 3 (a) Relative to the new basis, A= 1 1

0 0 n 1.2.1;

( 1 1 3 ) ( 1 (b) Set C = 1 2 4 , and D = 0 2 3 1 0

sentation of A in the new basis is

0 0) 1 1 . Then the repre-

-1 1

(

53 19 4 )

A'= v-'cAc-'v = ~ ~'~ =! . (1o) (lo) 1.2.2; (c) A = 0 1 , and B = 0 2

have the same determinant but

are not equivalent.

1.2.3; (a) Notice that if ABx =Ax, then BA(Bx) = A(Bx).

(b) If AA•x =AX, then A(x, x) = (AA*x, x) = (A*x, A*x) :2: 0.

1.2.4; If Ax= AX then A(x, x) = (Ax, x) = (x, AT x) = -(x, Ax) = -X(x, x}.

1.2.5; (a) R(A) = {(1,1,2)T,(2,3,5)T},N(A) = O,R(A*) = m?,N(A*) = {(1, 1, -1)T}.

(b) R(A) = R(A*) = IR3, N(A) = N(A*) = 0.

1.2.6; (a) T = ( i 1 0 ) c 0 n ~ 1 , T-1 AT = 0 1 1 0 0 0

(b) T = ( ~ ~i) ,T-1AT= ( ~i ~).

(c) T= ( ~1 0 0) coo) 1 0 , T-1 AT= 0 2 0 . -1 1 . 0 0 3

1 " ¥,

i

l ·~

569

(d) T does not exist.

(e) T = -1 1 (

1 1

0 -v'3 1) (000) } , T-1 AT = 0 ~ ~ . ~ 0 0 3

( 1 2 ) -1 ( 3 0) 1.2.7; T= _2

_1

,T AT= O 6

.

1.2.8; If x is in M, then Px = x, and if x is in the orthogonal complement of M, the Px = 0. Therefore, P has two eigenvalues, A= 0, 1.

1.2.10; (d) Examine the eigenvectors of A and A* where A= ( ~ ~ ).

1.3.1; Hint: Minimize (Ax,x) with a vector of the form xT = (1, -1,z,O).

1.3.2; (a) Prove that if the diagonal elements of a symmetric matrix are increased, then the eigenvalues are increased (or, not decreased) as well.

(b) Find the eigenvalues and eigenvecto"' of B = ( :

use them to estimate the eigenvalues of A.

4 4 ) 8 -4 , and -4 8

( 1 2 3) 1.3.3; The matrix 2 2 4 has a positive, zero, and negative eigenvalue.

3 4 7 Apply 1.3.2a.

1.4.1; (a) b must be orthogonal to (1, 1, -1)T, and the solution, if it exists, is unique.

(b) The matrix A is invertible, so the solution exists and is unique.

1.4.2; b must be in the range of P, namely M.

1.4.3; ( =>) Suppose A is invertible, and try to solve the equation ~i ai</Ji = 0. Taking the inner product with <Pi, we find 0 = Aa, so that a = 0, since the null space of A is zero.

( ¢:) Suppose {<Pi} form a linearly independent set and that Ax = 0. Then (x, Ax} = (~i Xi</Ji, ~i xi <Pi) = 0, so that ~i Xi</Ji = 0, implying that x = 0, so that A is invertible (by the Fredholm alternative).

1.4.4; Since (Ax, x) = (x, A*x) > 0 for all x =J:. 0, the null spaces of A and A • must be empty. Hence, (b, x) = 0 for all x in N(A*) so that Ax = b has a solution. Similarly, the solution is unique since the null space of A is empty.


( 2 -1 1) 1.5.1; (a) A'= i -1 2 1

0 0 0

(b) A'= 2~ ( i ! ~ ) (c) A' = 1\ ( : ~ )

0 -4

( -6 2 2) (d) A' = i -5 4 1

-4 2 2

1.5.3; (a) Take u = :c -v.

1.6.4; Q = (~,.P,,.P,), where~ = ;ky (:) ,¢>. = }.;. ( ~0 ) ,.P, =

1( 2

) -(v'ffi * ¥) 75 ~1 , and R- ~ 1Ji5 1f . 1.5.5; Use that

1 ( 1 -2 ) ( ViO 0 ) ( 1 -1 ) A = ViQ 2 1 0 2v'iQ .. 1 1

or

A = ( i ~ ) ( ~ ~ ) ( ~ !a ) · 1.5.6; Use that A = i ( ~ ~2 ) ( ~

AI 1 ( } -2) = 5 -2 4 .

~ ) ( !2 ~ ) , so that

1.5.8; For A = ( ~:: ~::~ ) , singular values are ViO and eVlO with E = . f:t'n I ( 0.} + Jw 0.2 - ~ ) ( 200.1 -99.8 )

0.001 v LV, A = 0.1 - :Jrn 0.2 + :Jrn = -199.8 100.2 .

Using instead singular values ViO and 0, A' = ( ~:~ ~:; ) .

( v'5 o o ) ( 2V10 o ) 1.5.10; A= UJJV where U = ~ 0 -1 -2 , I:= 0 3VW , 0 -2 1 0 0

( 1 -1 ) ( ..Yi 1 1 ) 1 I 20 SO U V = ~ _1 _ 1 , so that A = _rl

1 .!. .

. 20 30 15

l ~

' ' j

~ ' :1 -~

l ~

j

571

2 1 3. F1 - ~n 1 th .l!Ll' 1 1 _ ~m 1 1 ~m-1 1 •. , or Xn- ~1.:=1 kf• e wuerence Xn-Xm - ~k=n+t kf ~ (n+1)1 ~k=O ~

~ (n~t)l 1 ~'! < (n!t)l is arbitrarily small for m and n large.

2.1.4; The functions {sinmrx}~=1 are mutually orthogonal and hence linearly independent.

2.1.5; maxt lfn(t) - fm(t)l = !(I- '!!;) if m > n, which is not uniformly small

form and n large. However, J; lfn(t)- fm(t)!Zdt = (~2:':~2

< 1~n. 2.1.6; (a) Hint: Show that l(lul-lvl)l ~ ju- vi.

2.1.8; J; X(t) = 0.

2.1.11; J; (I; f(:c,y)dx) dy = - J; (I; f(:c,y)dy) dx = -~. Fubini's theorem

fails to apply because J; (I; lf(x, y)ldx) dy = f; (I; a:'!v' dx) dy =

J;; tan-1 ;dy does not exist.

2.2.1; With w(x) = l,p(z) = \:x2 + 136 , with w(:c) = v'f='?,p(x) = 1:1T (6x2 + 1), and with w(x) = ~,p(:c) = 3~(4x2 + 1).

2.2.2; Define the relative error e~,: by e~ = 1 - ~· where /A:(x) is the kth

D.J

0.0

.... ·1.0

partial sum, f~.:(x) =~!::a ant/>n(x). Then

(a) H(:c) - 2H(x -?r) = ~ ~::0 2n~l sin(2n + l)x, e5 = 0.201,

(b) :c -?r = -2~~1 ~ sinnx, e5 = 0.332,

( ) { X 0 < X < 11" 1r 4 ~oo 1 ( 1) -

c 211" - x 11" < x < 2?r = 2 - 1T ~n=o (2n+t)!l cos 2n + x, e5 -

0.006,

(d) x2 (2?r- x)2 = 81~· -48 ~::1 ~ cosnx, es = 0.002.

0.0 D.J 1.0 :til

1.1 2.0

·1

0.0 D.J !.0 no:

,.. 2.0

Figure A.1: Left: Partial sum f~.:(x) for Problem 2.2.2a with k = 5; llight: Partial sum f~.:(x) for Problem 2.2.2b with k = 5.

2.2.3; f(x) =x(x-?r)(x-211") = 12~~1 ;?s-sinnx.

572

:u :u 1.0

u

1.0

o.a

0.0 . 0.0 o.a

APPENDIX A. SELECTED HINTS AND SOLUTIONS

1.0 .til

1.1

100

10

10

40

10

1.0 CLO CL6 1.0 .til

... 1.0

Figure A.2: Left: Partial sum fk(x) for Problem 2.2.2c with k = 5; llight: Partial sum /k(x) for Problem 2.2.2d with k = 3.

2.2.4; With the additional assumption that the function f(x) to be fit intersects the straight line at the two points x = x1 and x = x2, x1 < x2, we must have that x2- Xt =!,and x2 + x1 = 1, so that a=£/(})- !f(~),p = 2/(f)- 2f(i>·

2.2.6; (a) You should have a procedure that generates the Legendre polynomials from Problem 1.1.9, or use Maple, which knows about Legendre polynomials.

I.Q I.Q

o.a ... 0.0 0.0

.... .... •1.0 .... 0.0 CL6 1.0 ·1.0 .... 0.0 CL6 1.0

X X

Figure A.3: Left: Legendre polynomials P1(x),P2(x) and P3 (x); llight: Legendre polynomials P4 (x),P5(x).

(b) g(x) = ax+bx3 +cx5 , where a= ~(1r4 -15311"2 + 1485) = 3.10346, b = -m<11"4 - 12511"2 + 1155) = -4.814388, c = ~(11"4 _ 10511"2 + 945) = 1.7269.

(c) g(x) = 3.074024x-4.676347x3 + 1.602323x5 • A plot of g(x) is barely distinguishable from sin 'II"X on the interval -1 :$ x :$ 1.

2.2. 7; Use integration by parts to show that the Fourier coefficients for the two representations are exactly the same for any function which is sufficiently smooth.

2.2.9; (b) Write tPn+l - AnXtPn = r:,:=o PktPk, and evaluate the coefficients by taking inner products with tPb and using part (a), and the fact that llt/Jkll = 1. Show that Bn = -An(XtPn 1 tPn) 1 On= An/An-1·

2.2.14; Direct substitution and integration yields h(t) = 211" r:,:._00

fkgkeikt.

573

2.2.15; Suppose the transforms of j,g, and hare fJ_, and F• respectively. Use direct substitution and the fact that -k L,i=~1 e21

riJk/N = 1 if k is an integer multiple of N (including 0), and = 0 otherwise to show that hj =

h9i· 2.2.16; Rewrite the definition of the discrete Fourier transform as a matrix mul

tiplication. Show that the matrix is orthogonal.

2.2.19; See Fig. A.4.

0.4

0.2

o.o-1---~

.0.2

.(),4

3 0 ·1 -2 X

Figure A.4: The wavelet generated by the B-spline N3 (x), for Problem 2.2.19.

2.2.24; Solve the equation Aa = P where

( 2 1 0) A= 1 4 1 , (

/o- h) P= -3 /o -h ·

h-h

The solution is

0 1 2

(

_2

a= _1 12 i

! -1! ) ( lo) 0 ~ h . -! i h

2.2.26; (a) Solve Aa = P where

11 N 11 aij = ¢~'1//jdx, Pi=-~h t/J~'t/ljdx.

0 i=O 0

Use Problem 2.2.23 to evaluate the coefficients, and find that the ith equation is at-1 + 4at + ai+l = *(fi+l- /i-t), fori ::fi 0, N.

{b) Solve Aa = P where 1 · N 1

aij = 1 ¢~¢jdx, PJ = - L: /i 1 t/J~t/Jjdx. 0 ~0 0

Use Problem 2.2.23 to evaluate the coefficients, and find that the ith equation is -ai-1 + 8ai- at+l = lUi+l- /i-t), fori ::fi O,N.


(c) Require ai-l+ 4ai + Gi+t = *Ui+t -/i-t), fori =f 0, N.

(d) Solve Aa = f3 where

Uii = 11

'1/J~'(x)'l/J'j(x)dx, 1 N 1

/3i = 1 g(x)'l/Ji(x)dx- 2.:::: /i 1 </Ji(x)'l/lj(x)dx. 0 i=O 0

2.2.27; Observe that the equations for a 1, ... , GN-1 from Problem 2.2.26a and c are identical.

3.1.1; Use Leibniz rule to differentiate the expression u(x) = I01 y(x-l)f(y)dy+ I: x(y- l)f(y)dy twice with respect to x.

3.2.1; Find a sequence of functions whose £ 2 norm is uniformly bounded but whose value at zero is unbounded. There are plenty of examples.

3.2.2; The proof is the same for all bounded linear operators; see page 107.

3.2.3; (a) The null space is spanned by u = 1 when A= 2, therefore solutions exist and are unique if A =f 2, and solutions exist (but are not unique)

if A= 2 and I0112 f(t)dt = 0.

(b) The null space is spanned by u = x when A= 3, therefore solutions exist and are unique if A =f 3, and solutions exist, but are not unique, if A= 3 and J0

1 tf(t)dt = 0.

(c) The null space is spanned by ¢(x) = cosjx if A = *· Therefore, if A =f * for j = 1, · · ·, n, the solution exists and is unique, while if

A = * for some j, then a solution exists only if J02

'/r f ( x) cos j xdx = 0.

3.3.1; u(x) = f(x)+A I02

'/r 2:}=1 i-~'lr cosjtcosjxf(t)dt = sin2 x- ~ 2_1r>.1r cos 2x, provided A =f :. For A=:, the least squares solution is u(x) = ~·

3.3.2; u(x) = 3 _!6>.Po(x) + 3!2>.P1(x) + 15~6>.P2(x), provided A =f ~~~· It is helpful to observe that x2 + x = !Po(x) + P1(x) + ~P2(x).

3.4.1; (a) Eigenfunctions are ¢n(x) = sinn1l'x for An= n2~2 .

(b) f(x) = -;!-r E~o (~~~\)2 sin(2n- 1)11'x.

3.4.2; (b) </11 (x) = sinx, At = ~' ¢2(x) = cosx, A2 = ~'/r.

(c) There are no eigenvalues or eigenfunctions. Remark: The existence of eigenfunctions is only guaranteed for self-adjoint operators.

(d) ¢n(x) = sinanx, An= ai• where an= 2n:f1.

575

3.4.3; An = n281r2 is a double eigenvalue with ¢n(x) =sin n;x, '1/Jn(x) =cos n~x for n odd.

3.4.4; k*(x,y) = k(y,x):~~~·

3.5.1; (a) u(x) = f(x) +fox ex-tf(t)dt =ex when f(x) = 1.

(b) u(x) = f(x) + J; sin(t- x)f(t)dt = cosx when f(x) = 1.

(c) u(x) = f(x) + J; sin(x- t)f(t)dt =ex when f(x) =I-+ x.

3.5.2; (a) u(x) = f(x) + 1 ~>. + f01 f(t)dt = x + 2 ( 1~>.) when f(x) = x, provided

A# 1.

(b) u(x) = f(x) + ~ f 01 xtf(t)dt = x when f(x) = 5

:.

(c) u(x) = p(x) + a(x) I:o exp (-I: b(t)a(t)dt) b(y)p(y)dy.

II>.Kr+1"[" 3.6.1; (a) Show that llu- unll < 1_ j>.KII ·

(b) Use that IIKII :::; ! to conclude that n 2: 2. The exact solution is u(x) = 3~~ and some iterates are u0 (x) = 0, u1(x) = 1, u2(x) =

2{,

ua(x) = 9 .

3.6.2; If f(x"') = 0, for convergence to x*, require II- f'(x"')l < 1 or 0 < f'(x"') < 2.

3 6 3 D • • 1 f(x·v"<x·) 1 1 h' h 'f f( "') o d • . ; ror convergence to x , reqmre 1 fl ,. \'1 2 < , w IC 1 x = , an f'(x*) =f 0 is always satisfied.

3.6.4; (a) Let Xn+l = D-1b- D-1 Rxn-

3.6.5; Show that Un(x) = 2:::~=0 ~ ·

3.7.1; y = y(x) where a2x = sin-1 y'y + yJl- y2, a=~·

3.7.2; (a) A straight line.

(b) y = y(x) where x = hJY2=1- ~ ln(y+ Jy2 -1), provided y 2: 1, which means this problem is not physically meaningful, since y cannot reach zero.

3.7.3; (a) T(p) = I;(p) V 1 ~dz. c(z) 1-c~(z)p

(b) Let y = ljc2(z), then use Abel's technique to show that z(c) = IJl/c ~

-:; 1/co yc2p2-t'

(c) T(x) = ~lnJl + y2 - y, where y = ~~·

( ( tsin(k+~)n d 4.1.1; a) Use that Sk x) = 2 sin if for -1 < x < 1, an then observe that

siV¥ Sk(x) = (k + !) sinc((k + !)x) is a delta sequence.

576 APPENDIX A. SELECTED illNTS AND SOLUTIONS

4.1.4; Observe that X= f~oo 1/l(x)dx is a test function, x(O) = 0, so that X= x</J for some test function </J. Hence, 1/J(x) = Jx (x<P(x)).

4.1.5; u(x) = c1 + c2H(x) + ca5(x). Show that a test function 1/J is of the form 1/J = .Jx(x2</J) if and only if f~oo 1/Jdx = f0

00 1/Jdx = 1/J(O) = 0.

4.1.6; u(x) = 5(x) + c1x + c2.

4.1.7; Set u = xv, so that x2v' = 0, and then u(x) = c1x + c2xH(x) (using that x5(x) = 0).

4.1.8; u(x) = c1 + c2H(x).

4.1.9; 5(x2 - a2) = 2lal(5(x- a)+ 5(x +a)).

4.1.10; In the sense of distribution, x'(x) = 5(x)- 5(x- 1), since (x'(x)<P(x)) =

-(x(x), <P'(x)) =-I: x(x)<P'(x)dx =- f01

<P'(x)dx = <P(o)- </1(1).

4.1.11; (a) For distributions f and g, define (! * g, <P) = (g, 1/J), where 1/J(t) = (f(x), </J(x + t)).

{b) 5*5=5(x).

4.2.1; g(x,y) = -x for 0 ~ x ~ y,g(x, y) = g(y,x).

4.2.2; U(x) =xis a solution of the homogeneous problem. There is no Green's function.

( ) cosa(;-lx-v\) 'ded ..J.. 2 4.2.3; g x, y = 111n f prov1 a .,... mr.

{ -(1- y)2x for 0 $ x < y ~ 1

4.2.4; g(x, Y) = (2 _ y)yx- y for 0 ~ y < x ~ 1

4.2.5; g(x,y) = lny for x < y, g(y,x) = g(x,y).

{

_x(3y5/2+2) 4.2.6; g(x, y) = 113tf( -a/2 )

- 5 3x+2x

for 0 ~ x < y

for x > y

4.2.7; u(x) = -£:. f~oo e-alx-eiJ(e)cJe.

4.2.8; The functions sin 2x and cos 2x are elements of the null space of the operator.

4.2.9; u(x) = J; g(x, y)f(y)dy-).. J01 g(x, y)u(y)dy+a(1-x)+{3x, where g(x, y) =

x(y- 1) for 0 ~ x < y ~ 1, g(x, y) = g(y, x).

4.2.10; u(x) = fg g(x,y)f(y)dy-)..jg g(x,y)u(y)dywhereg(x,y) = i(x+1)(y-2) for 0 ~ x < y, and g(x, y) = g(y, x).

4.2.11; u(x) = f01 g(x, y)f(y)dy- ).. J; g(x, y)u(y)dt where g(x, y) = 2~ xn(yn -

y-n) for 0 ~ x < y ~ 1,g(x,y) =g(y,x).

1 i ' 1 j l ~ !

577

4.2.12; g(x,y) = -!e-lv-xl, u(x) = ~ J.:O e-lv-xlu(y)dy- f~oo !e-lv-xlf(y)dy.

4.2.13; u(x) = cosx +).. J; p(e) sin(x- e)u(e)~.

4.2.14; u(x) = 1 + J; eln(Vu<e)~. 4.3.1; L*v = v"- (a(x)v)' + b(x)v, with boundary conditions v(O) + v'(l)

a(l)v(l) = 0, v(l) + v'(O)- a(O)v(O) = 0.

4.3.2; L*v = -(p(x)v')' + q(x)v with p(O)v(O) = p(1)v(1), and p(O)v'(O) = p(l)v' (1).

4.3.3; L*v = v"- 4v'- 3v with v'(O) = O,v'(1) = 0.

4.3.4; (a) g*(y,x)w(y) = (g*(e,x),<>(e- y))e = (g*(e,x),Len(e,v))e = (Leg*(e, x), g(e, y))e =(<>(e-x), g(e, y))e = g(x, y)w(x).

(b) u(x)w(x) =(<>(e-x, u(e))e = (Leg*(e, x), u(e))e = (g*(e, x).J(e))e

= w(x) J: g(y, x)f(x)dx.

4.3.6; This follows from Problem 4.3.2.

4.3.7; M*y =!fit+ AT(t)y, with the vector ( ~~~~~ ) E N..L([L, R]).

4.3.8; (a) The operator is formally self-adjoint, but not self-adjoint.

(b) Require u'(O) = u'(l) = 0, for example.

4.3.9; Require J:1r f(x) sinxdx =a, and J:1r f(x) cosxdx = {3.

~.3.10; Require f01

f(x)dx = -{3.

4.3.11; Require J0! f(x) sin 1rxdx = {3 + 1ra.

4.3.12; Require Jg f(x)dx =a- {3.

4.4.1; g(x,y) = yx+~(y-x)+l2 -~(x2+y2 ),for0 ~ x < y ~ l,g(x,y) = g(y,x).

4.4.2; g(x, y) = (~ + y2(4y- 3))(x- ~) + t- ~-X: - xH(y- x).

4.4.3; g(x, y) = -~ cos21r(x- y)- ~sin 21r(x- y)- 4~ sin 21r(y- x).

4.4.4; g(x, y) = ~ cosxsiny+~ sinxcosy- ~ sinxsin y-2H(y-x) sinxcos y-2H(x - y) cos x sin y.

4.4.5; g(x,y) = ~xy- x- -T(x2 + y2) for x < y,g(x,y) = g(y,x).

4.4.6; g(x, y) = ~ ln(l-x)+ ~ ln(l +y)+ ~for -1 ::; x < y S 1, g(x, y) = g(y, x).

4.4.7; u(x) = l cos2x + ({3- a)~;+ ax-~~ (a+~).


4.4.8; u(x) = -~xcosx + cosx + f2 sin3x- /; sinx.

4.4.9; u(x) = 0.

4.5.1; u(x) = aiJ3 - f +ax + 0;a x2 + E:.:1 n~:-2 cos mrx, where bn = -2 J: (f(x) +a - /3) cos mrxdx.

4.5.2; Use Fourier series on [0, 211"].

4.5.3; Use Fourier series on [0, 1].

4.5.4; No eigenfunction expansion solution exists.

4.5.5; u(x) =ax+ /3- a1r + E:.,1 Cln cos(2n- 1)~, where

_ 8 1-coe(:ln-1)! Cln - ll'(:ln 1)2 (:ln-1)2/:1-1 •

4.5.6; u(x) = -~(Jx:l- !) - !x. Solution is exact if a+! = 0.

4.5.7; u(x) = -~(x) + (b + 4c)L1(x), where L 1(x) = 1- x, and ~(x) = i(x:l - 4x + 2) are Laguerre polynomials.

4.5.8; Use Hermite polynomials: u(x) = (-&x - ~ -a - f )ez2/:l.

4.5.9; Eigenfunctions are r/Jn(x) =sin~"' (x + 1), ,\ = n2t, so an = 0 for n even,

Cln = - ~ n:~4 for n odd.

4.5.11; rp!:> = cos(<n;!~b), Ak = -4sin2 :~c:k': 1), k = 1, 2, ... , N -1.

4.5.12; For ,\ = 411":1, eigenfunctions are 1, cos 21rx, sin 21rx. For ,\ = 411"2n2 with n > 1, eigenfunctions are cos 21rnx and sin 21rnx.

4.5.14; With u(x;) = u;, require u;-1 - 2u; + u;H = h:l J::~11 r/J;(x)f(x)dx.

5.1.1; (a) (~ )' = 0.

(b) fl-y=- cosx.

(c) y' -y = -ez.

5.1.2; y(x) = l{x:l- ax+ 1).

5.1.4; Find the increase in energy due to rotation and determine how this affects the moving velocity. Then determine how this affects the fastest path. Show that the rotational kinetic energy of a rolling bead is 2/5 that of the 't t lat' al kin t' Minimize' • T rzt J 7(l+V'2> -~-1 s rans 1on e 1c energy. = Jo iog(IIO-II) u;c;.

5.1.6; Maximize f01

y(x)dx subject to f01 vfl + y12dx =l-a-b, y(O) = a,y(1) =

b.

5.1. 7; The arc of a circle.

5.1.9; The Euler-Lagrange equations are y" = z, z" = y.

5.1.10; The Euler-Lagrange equation is ~ - y = 0.

5.1.12; y(x) = 2sink1rx.

5.2.1; Uz = 0 at X= O,l.

5.2.2; Uzz = 0 and f..l.tUzzz- f..1.2Uz = 0 at X= 0, 1.

579

5.2.3; H u is the vertical displacement of the string, then T = ! J~ pu~dx +

muHO, t) + mu~{l, t), U = ~ J~( .;(1 + u!)- 1)dx + fu2(0, t) + ~u2 (l, t). Then, require puu = f..l./z ~ subject to the boundary conditions

yl+u;: mUtt + ku = f..I.--J!.&...; at x = 0, and mUtt + ku = -f..I.--J!.&...; at x = l.

yl+u;: yl+u;:

5.2.4; puu = f..I.Uzz on 0 <X< 4 and!< X< 1, and mUtt= f..I.Uz(~,t)l;~~~.

5.2.5; Use that T = !ml202 + To:~z2 sin2 8, U = mgl(1 -cos 8).

5.2.6; H = !m:i:2+!k1x2+tk2x4 • Haridlton'sequationsarep = -k1q-k:~q3,q = !!l, where p = m:i:, q = x.

1 • :1 1 • 2 • • • 2 5.2.7; T = 2mtlt8t + 2(l~81 + 2hl:~ cos(81- 8:~)8t8'J + l~82 ), U = mtltg(1-

cos8t) + m:~g(h(l- cos81) + l'J(l- cos8:~)).

5.2.9; Set x = p+cosp,y = sinp, and then T = mp2(1-sinp),U = mgsinp, so that 2ji(l- sinp)- p:l cosp- gcosp = 0.

5.3.1; (a) u(x) = a(1 + ~~x + ~x:l). (b) u(x) = a(~:g + ~x + ~~x2).

5.3.2; (a) u(x) = a(1+ ~~~~x+ 29:1~l x2 + ~::!x3 ) = a(l.O+ 1.013x+0.4255x2 + 0.2797:~:3).

(b) u(x) = a(U:i; + :~jWx + ~:~~~x:l + ::6~7 :~:3 ) = a(0.99997 + 1.013:~: + 0.4255x2 + 0.2797x ).

{ W O<x<l

5.3.3; u' = Ui' 1 < < I . 199' 2 X

5.3.4; (a) u(x) = 1- x -112x(1- x) = 1- x- 0.230x(1- x).

(b) u(x) = 1-x-f2x(1-x) = 1-x-0.227x(1-x). The exact solution is u(x) _ elnh(z-1)

- alnhm •

5.3.5; Using (non-orthogonal) basis functions rp1(x) = x, r/J:I(x) = x2, r/Ja(x) = x3, u(x) = a(l + ;r/Jt(x) + /&r/J:I(x) + /forp3(x)). The Sobolev inner product does not give a useful answer.


5.4.1; Use the functional D(¢) = J01

p(x)¢'2 (x)dx + ap(1)¢2 (1), a 2:: 0, subject

to H(¢) = J01

w(x)¢2 (x)dx = 1 and

(a) ¢(0) = ¢(1) = 0,

(b) ¢(0) = 0,

(c) ¢(0) = 0, and a= 0.

Show that >.(a) 2:: >.(b)~ >.(c).

5.4.2; Minimize J01 (x¢'2 (x) + ~ ¢ 2 (x))dx, subject to J; x¢(x)dx = 1, with

¢(0) = ¢(1) = 0, or ¢(0) = 0 with ¢(1) unspecified.

5.4.3; Minimize D(¢) = J; ¢12 (x)dx subject to J; ¢ 2 (x)dx = 1. Use ¢(x) = v'30x(l- x) to find D(¢) = 10 2:: 1r2 •

5.4.4; The first eigenvalue is approximated by 10, from Problem 5.4.3. The second eigenfunction is approximated by ¢ 2(x) = V84ox(x- 1)(x- !), and D(¢2) = 42. The exact values are .>. 1 = 1r2 , .>. 2 = 471'2.

6.1.1; (a) f(-3) = -iVM,f(!) = -ji,J(5) = -.,fiO.

(b) There are square root branch points at z = ±1 and there is a logarithmic branch point on the negative branch of the square root at Z - 13

- 12'

6.1.2; (e21ri)z is not single valued.

6.1.3; (a) z = ! + 2n7r- i In(2 ± v'3). (b) z = (2n + 1)7r- i In( v'2 + 1), z = 2n7r- i In( J2- 1).

/

(c) No such values exist.

6.1.4; ii = e-(n/2+2n?r) for all integer n; ln(l + i}i1r = -7r2(t + 2n) + i; ln2; arctanh 1 has no value.

6.1.6; It is not true. For example, consider z 1 = e1ril2 , z2 = e31ri/2 •

6.1.7; The two regions are lzl < 1 and izi > 1; There are branch points at w = ±1.

6 2 1 . f( ) - 15-8i • • ' Z - 4(z-2)2(z-!)'

6.2.2; fc f(z)dz = -27rv'I9(15)113e-i1r/3.

6.2.3; The integral is independent of path. fc z- 113dz = !z213 i~+~ = -3(2)1f3ei1r/

6.2.4; Find the real part of f(z) = z112 and use that f is an analytic function.

6.2.5; (a) iz- il < v'2, lz- il > v'2. (b) iz- il < 2, iz- il > 2.

I . g

i :2

(c) iz- il < 2, iz- il > 2.

(d) iz- il < 1,1 < iz- il < v'2, iz- il > J2. (e) iz- il < 1, 1 < iz- il < v!z, v'2 < iz- il < 2, iz- il > v!z.

6.2.6; ~zl=l/2 z21z~l dz = 0.

6.2.7; ~zl=l/2 exp[z2 ln(1 + z)]dz = 0 (There is a branch point at z = -1).

6.2.8; ~zl=l/2 arcsin zdz = 0 (There are branch points at z = ±1).

6 2 9 . r sin z d . h 1 ' ' ' Jjzl=1 2z+i Z = 7r Sill 2·

6.2.10; ~zl=l ln~~~2)dz = 0.

6.2.11; flzl=l cot zdz = 27ri.

581

6.2.13; Hint: Use the transformation z = e where p = ~ and apply the Phragmen-Lindelof theorem to g(O = f(z).

6.2.14; The function G(z) = F(z)ei(a+<)z satisfies IG(iy)i :S A and jF(x)l :S SUP-oo<x<oo IG(x)j. Apply problem 6.2.13.

6.3.4; !), = R-q r2-R'

635. A. ·.J,_ 2(b ) 4i(b-a)l cz-1) • • ' 'f' + t'f' - a - 3 -a - 311" n z+l

6.3.6; (b) w( z) = Az2 + C~i - i)2, Fx - iFy = -8p1riA.

(c) Fx- iFv = p7r(4rA- 8A2i).

6.3.7; The upper half.; plane.

6.3.8; Flow around a corner with angle(}= {31r. This makes sense only for (3 < 2.

6 3 9 . F ·p - 2 U2 . 2 ia • • , x - t y - - 1r p a sm ae .

6.3.11; f(z) = UJZ2 + a2•

6.3.12; Show that 1; = iJ (e-wf2k + vfl + e-w/k ).

Joo dx 271'

6.4.1; -oo ax2+bx+c = -/4ac-bl'

6 4 2 . roo xsinx dx = .l!:e-lal. • • ' Jo al+x2 2

roo dx - ~. 6.4.3; Jo l+xk - ksin.,.

roo dx - _71'_. 6.4.4; Jo (x+l)xP - sin 1rp

. Joo x dx = 71' • 6.4.5, 1 (x2+4)-/x'Ll ' 2J5


6.4.6; fooo i!t dx = 72· 6.4.7; J~ £~~dx = 211'H(ka)e-k4

,

6.4.8; Consider f0 4:~1. on the rectangular contour with comers at z = ±11' and z = ±11' - iR, and let R -+ oo. Show that f~w _,. !!~~~ .. dx = ~(H(a -1)1na -In( a+ 1)).

• Joo .... - w 6.4.9, _00 ciJ.Lzdx- coeJi"V•

6.4.11; / 000

a:'.tiH = iin\"2- ~(arctanV1-1r).

6.4.12; J:w ln(a + bcos9)d8 = 211'1n(!l±¥). Hint: Differentiate the integral with respect to b, and evaluate the derivative.)

6.4.13; / 000 .T!&';r;zcix = ! tanh f·

6.4.14; f0w ln{sin x )dz = -11' In 2.

6.4.16· J00 ~~- dz - w(l+w2J ' -oo ~ - x -L v •

. Joo e1'"• tl~h a: .J_ _ iww 6.4.16, -OO ._ _ u;f;- "-1-L1W,

6 4 17 roo ln(1+1112) .J- In 2 • • ; Jo , ...L-2 u;.G = w •

6.4.18; /~00 co::2 a:dz = '82•

6.4.20; Evaluate f0 P~(z) on some contour C that contains all the roots.

6.4.22; The change of variables p2 = r 2 cos2 9+z2 sin2 9 converts this to an integral for which complex variable techniques work nicely.

6.4.23; f000

t-112e'"'dt =.[fir exp(iifi[r).

6.4.24; J:O tl/2ei"tdt = vrre::!' •. 6.4.27; f~oo ( .. ~:~)1! dz = 2e0111e-wi/J/'J sin ~IPI/3-t r(1- /3)H( -p).

6.4.28; Use that V'z+lat!v'z+id = Yif{fa:'fJf"', but be careful to apply Jordan's

1 1 Th Joo f"", - ../i -iw/4e-110 -e-""H( ) emma correct y. en, _00 v'z+at+OI+Vl - ~e 01-jJ P •

6.6.4; Examine the expression fo" (:l:JE. .. , make the change of variables x = ycos

2() + '7sin2

6 and then use the beta function to show that f(y) = ~Ju Jt (,~Jjt .. dz. For another way to solve this problem, see Problem 7.3.4.

6.5.5; (a) W(Jv, Yv) = ;z. (b) W(J. H(t)) = 2i

1/t II 11"% •

(c) W(H~1>,H~2>) = -~!·

6.5.6; (b) Yn(z) = - ~: r(~~l) + higher order terms.

6.5.8; Use that '.E~=-oo Jn(z)tn = e<t-1/t)z/2.

6.5.13; :E:.0 s2n J~1 P~(t)dt = f~1 4>2 (z,s)dz =-!In(~:;::>= 2'E:.o 2~~1 .

6.5.23; Period= jifB(~, t) = fif-r(rt~ ) ·

583

7.1.2; (a) A with IAI < 1 is residual spectrum, with IAI = 1 is continuous spectrum, and with IAI > 1 is resolvent spectrum.

(b) Notice that L2 = Li. Then, A with IAI < 1 is residual spectrum, with IAI = 1 is continuous spectrum, and with IAI > 1 is resolvent spectrum.

(c) An = ~for positive integers n are point spectrum, there is no residual spectrum since La is self adjoint, and A -:F ~ is resolvent spectrum, A = 0 is continuous spectrum.

7.1.3; Use improper eigenfunctions to show (L4 - A)-1 is unbounded. Show that { Xn} with Xn = sin n9 is an improper eigenfunction.

7.1.4; Show that f/>(x) = sinpz is ·an eigenfunction for all p. Notice that the operator is not self-adjoint.

7.2.1; (a) o(x-~)=2:E~=lsin( 2n2-l1Tx)sin( 2n2-l11'~).

{b) o(x- ~) = ~ fo00 coskxcosk~dk.

(c) o(x- ~) = ~ fo00 sink(x + 4>) sink(~+ l/>)dk where tan</>=~·

7.2.6; (a) ~2~42· (b) l!i~.

- :1 (c) .j!e=if-.

7.2.7; H(p+1r) -H(p-1r).

7 2 8· ..!.. f 00 -

1-(iuF(u))e-'":cdu- roo e:c-•J'(s)ds • • ' 211" -oo l+iiS ,.. ,.. ,.. - J:c '

7.2.9; Use the convolution theorem to find u(x) = f(x)- t f~oo f(t)e- 31:z:-tldt.

7.2.10; f~oo J(e)J(x- e)cie.

7.2.11; (a) E:z:E~ = ~ which is optimal.


2 (b) E111Ep = ti = 0.274.

(c) E111 Ep = !· 7.2.12; (a) J~00 eiJ.Iz Jo(ax)clx =f. J:,.. (J:O eilll(p+aaln8)dx) d(J

= fo~,.. 6(1-4 + a sin 8)d8 = J ; if II' I < lal and = 0 otherwise. a -p2

(b) Take the inverse Fourier transform of the answer to part (a).

7.2.15; Show that Ne(1) = 1/120, N6 (2) = 13/60, and N8 (3) = 11/20. Then Es( ) - _L ~ 1l + 11 13 + 1 z - wZ + eoZ 20 + GoZ 120.:2 '

7.3.2; (a) 8-(a+I>r(a + 1).

(b) a.:. provided Re 8 > a.

7.3.3; u(z) = J; f(y)K(z- y)dy where K = L-1( 1+thl,n)·

7.3.4; f(t) = linw7r(! 1t f~ Ta-1T(t- T)dT.

7.3.6; (a) M[H(z)- H(z- 1)] = ~· (b) M[(1 + z)-1] = aln,..n'

(c) M[e-111] = r(3).

(d) M = 1:,.

(e) M[eilll] = i'r(8).

(f) M[cosz] =! COB11'8r(s).

7.3.7; F(J-4) = f000

r/(r)sinJ.4rdr,rf(r) = ~ f000 F(J.4)sinJ.4rdJ.4.

7.3.8; (a) Show that J000

Jo(z)dx = 1, and then G(p) = ~·

(b) G(p) = ~ sinap.

(c) G(p) = sinh-1 ~·

7.4.2; Let Un = EJ 9nJIJ where 9nJ = 0, n $ j,gnJ = 11

f_ 1 (J.'n-J- J.'i-n) where I'~ - >.1-4 + 1 = 0.

{ cosz,z > 0 { sinz1 z > 0 7•5•1 i Ut(z)= coshz,z<O ,u~(z)= sinhz,z<O ·

7.5.3; (a) Eigenvalues are>.= -1-42 where tanhJ-4 = -A~,.,J-4 > 0.

(b) >.~ = A-1-4 where tan a~=~~ which has positive solutions

if and only if Aa~ > "": .

(d) C _ 1 + 2ik(a+P}+agce-2111"-1) c _ -e2ika (2i/!k-,1a)+~2iak+,8a) 11 - 4k I 12 - 41; '

>.2 = -p where tanh ap = - p(a+J-+2P) (a+p)( +JA)+p2.

7.5.5; (a) RL = RRe-2ik<P.

(b) R};> = R~>e-2ik<P.

7 5 6 . R = -e2ikta ~~:a cos kaa+ik, sin kaa where k· = .!!!. ' ' ' ka coa kaa-illt aln ll:~a ' c1 •

585

7.5.7; The gene~al solution is u(z) = aeikz(k2 + 1 + 3iktanhx- 3tanh~ x) + ,Be-ikz(k2 + 1- 3iktanhx- 3tanh2 x). Tr(k) = ~!Z+~H!Z+!~. The bound

states are ¢1(x) = tanhx sechx fork= i 1 and ¢J2(x) = sech2x fork= 2i.

7.5.13; Use (7.37) to show that cu = 2~~:(Li), Ct2 = c~1 = 2~~$~) 1 C22 = - 2A:(}H) ·

{ ez/-./2 z < 0

There is a bound state having f/J(z) = e-zf../2(1 + ~tanhx), x > 0

at k= ~·

~ 1inh~ 2 b. 8.1.2; (a) Vn(x) =an~ +bn sllih , where an= -1i f0 am Tg(y)dy,

bn = ~ J: sin T f(y)dy.

(b) u. ( ) = aalnh(Df{u-11)) ~~~sinh~!: (e)t~e+ aalnh ~ t sinh(!!!!:(e-n Y nwalnh li£1 0 a n mrslnh • 11 a

b))F(e)de1 where Fn({) = !!!!:(!({) - ( -1)ng({)). In the special a a

case f(y) = y(b - y) 1 g(y) = y2 (y - b), an = - n'Y',..s (1 + 2( -l)n)),

bn = n1~1 (1- ( -1)n). The first representation converges much faster than the second and gives a good representation of the solution with only a few terms.

8.1.3; (a) Require f~oo f~oo z(<f>t + cl/Jz)dxdt = 0 for all test functions f/J(x, t).

(b) If U = f(x- ct) 1 make the change of variables e =X+ ct, '7 =X- ct to find J~oo f~oo x(f/Jt + Cl/Jz)dxdt = f~oo f~oo f({)l/J11d{d1J = 0 since

f~oo f/Jfld'1 = 0.

8.1.4; u(r,8) = Jrsin8- i-r3 sin38.

8.1.5; G(z,Zo) = - 2~Inl:.:~:_o1 1·

8 1 6 ( 8) "' rlnl in8 h 1 r2w (8) -in8d(J Req . • • ; u r, = .Lm,PO an lniRini-1 e , w ere an = 211' Jo g e . mre

I:1f g(8)d8 = 0.

00 ( ~ ~h~) 8.1.8; u(x,y)=En=t aniiiilillf£+bn •in cosT, where

an =- ~?!~ (120( -l)n- 60n2~ + n41r4 + 600),

bn = ~?!~ {120 + 600(-l)n- 60n21r2( -l)n + n411'4).


8.1.9; G(x,y;~,r~) = ""'oo 1 ..Lsin~sin~e-~lx-el . . , L.,.,n= nn- a a

8 110. ( ) G( ) _ 1 Joo ( 'k( ))coshk(yo-a)sinhi:Jl.dk £ • . , a x,y,Xo,yo - - 2,. _00

exp -t X- Xo nrnah'-n Or y >Yo· (Use the Fourier transform in x.)

(b) Using Fourier series in y, G(x,y,xo,Yo) = 2::;:'=1 at exp(-AnlxXo I) sin Any sin An Yo where An = 2n

2±1 ~.

8 111· u(r 8) = ""'00 anbn a (!.)n- (!!)neinll- b (r)n- (f!.)neinll where • • ' ' L..,..n=-oo b!ln -a"'2n n a r n b r '

an = 2~ I:,. g(8)d8, bn = 2~ Io2,. f(8)d8.

8.1.12; If f(O) = l::':0 ancosn(8- <Pn), then u(r,O) = l::'=oan(~)ncosn(O<Pn) = 2~ Io

2,. f(O)dO+ # 2:::': 1 (~)n Itr f(<P)cosn(O- </J)d</J. This infinite

sum can be summed by converting the cosine to complex exponentials and using geometric series.

8.1.13; u(r, 8) = -~ Iooo F(J.L) sina~nlb<~:a~ sin(J.t In fi)dJ.L, where F(J.L) = fo00

J(Re-t)sinj.ttdt.

8.1.14; u(r,8) = 2:::'=-ooan (ft)7 sin n:o, where an=~ j0° f(O)sin n:11 dO.

8.1.16; Hint: Evaluate L:f=1 rJ cosj</J using Euler's formula cosj</J = !(eiN + e-iJ<P) and geometric series.

8.1.17; Using Mellin transforms,

u(r,O) = 2~ Io00

e-iklnr kco;hk7r (G(k)sinhk8- kF(k)coshk(B- 1r)) dk where F(k) = Iooo f(r)eiklnrd;, G(k) =It g(-r)eiklnrdr.

8.1.18; u(r,8) = 2::'=-oo J:{<:R)einll, where an= 2~ I:,. e-inllf(8)d8, and In(x) is the modified Bessel function of first kind.

8.1.20; u = a2!jj2 (af- /3H(f)).

8 1 21· u(r 0) = ""'00

n_ ~-rlnleinll where a = .1.. f 2,. g(O)e-inll df) ' • ' ' L..,n=-oo ...... ci+lnli3 ' n 2,. Jo •

8.1.22; -v(~') = Inn . v ~G(x, e)J(x)dVx - Ian n . v eG(x, e)v(x)dSx, where V

_ 8u - lfn·

8.1.24; Eigenfunctions are <Pnm(x, y) =sin n:x sin m;11 , with eigenvalues Anm = 2(n~ m~) -7r ~+v.

8.1.25; Eigenfunctions are Jn(J.'n!cfi)sinnfJ for n > 0. Thus, eigenvalues are the same as for the full circle, with n = 0 excluded.

8.1.26; Eigenfunctions are <Pnmk = Jn(J.'nkfi)einll sin m;z with eigenvalues A = I'~ m~ -(1ft-+ (iT).

8.1.27; Eigenfunctions are <Pnm(</J,fJ) = P~(cosfJ)sinn</J (or cosn</J) with Anm = bm(m+l).

587

8.1.29; Eigenfunctions are u(r,fJ, <P) = frJm+l/2(J.'mkfi)P~(cosfJ) cosn¢> with

eigenvalues Amk = ( ~ )2, where Jm+l/2(J.tmk) = 0 for m ~ n. Note that J.tot = 1r, J.tu = 4.493, J.t21 = 5. 763, J.to2 = 27r, J.tst = 6.988, etc.

8.2.1; Require h'(x)- ah(x) = f'(x) + af(x).

8.2.2; Ptt + (>.+ + >.-)Pt = c(A+- A-)P:r: + c2P:r::r:·

8.2.4; G = !H(t- r-lx- W + !H(t- r-lx+W.

8.2.7; With appropriately scaled space and time variables, ~-~=sin¢>.

8.2.8; u(x, t) = L::'::0 (an sin cyt + bn cos¥) sin T where

(a) bn = n~~2 sin~,., an = 0,

(b) an = ni~a sin n211', bn = 0,

(c) bn = 9~~2 sin~~, an= 0,

(d) an=- c!~~2 (cos n511' -cos~~), bn = 0.

8.2.9; For a rectangle with sides a and b, w = .f11' = ! J ;b + b. Set A = ab,

and find that the minimum is at a = VA.

8.2.10; For a square of side L, the fundamental eigenvalue is>.= v'2~, whereas for a circle of radius R the fundamental eigenvalue is.\= ~· Take 1r R2 = L2 and use that 2~w = .\. The fundamental frequency for the circle is smaller than for the square, >.ftrc!e = 0.959>.~Iuare.

8.2.12; G(r) = -~H~1)(r) is outgoing as r-+ oo.

8.2.13; Construct the Green's function from H~1) (>.jr - el) with A = ~, and then the solution is proportional to (up to a scalar constant) 1/J(r, 0) = 1 ei>.r sin(>.asln 9) for large r. 7r >.ainiJ

8 3 1. C8V _ 1 82 V V ' ' 1 8t - R1 8z'I" - R2 .

8.3.3; G(x, t; ~, r) = 2L:;:'=1 (1- e-n211'~(t-r)) cosn1rxcosn1r~, fort> r.

~

8.3.4; G(x, t) = 2J;te-tf-at.

8.3.5; G(r, t) = (47rt)-nl2e-r2

/ 4t, where r = lxl. -a. 00 -a.J72 :z::2

8.3.7; T(x, t) =To+ ax+ (Tt- To)7 f_ 00 ~2+1 dTJ, where a= m·

8.3.8· X = ln 2 {ijj = 0.S2ro, t = In 2 = 3.47 X 106s = 40 days. , y-w w


8.3.9; t = At~ In ~ = 5.5 hours. At this time the temperature at the bottom of the cup is 116° F. This seems like a long time. What might be wrong with the model?

8.3.10; (a) Require ut = Di"iJ2u on regions i = 1,2 subject to the conditions that u and n · kiVu be continuous at the interface.

(b) In a spherical domain, eigenfunctions are

cp(r) = /. r-h' { lain~

ram ,

0 < r < rp

rp < r < R

The tempera! behavior is cp(r)e-1h and the values I" must satisfy the

transcendental equation ~tan~ = --jt; tan e~>. (c) /-'2

j::::j 6.6 X 10-4 fa.

8.3.11; Solve the transcendental equation Jh:ie-~-ot = (J fort. The velocity is h t'

8.3.12; Propagation occurs provided h3(J < /(l) = 0.076, where

8.3.13;

f(y) = (41ry)-312e-1/ 4v. The propagation velocity is v where /( lg.) = h30, and the minimal velocity is i~·

u{:c, t) = ei<.lt (1 + iw: ia:-y)eio(L-z) - (1 + iw _ ia:-y)eio(z-L) (1 + IW + ia:-y)eioL _ (1 + iw _ ia-y)e-iaL ,

where a2 = 1 + iw.

8.3.14; p(:c, t) = e"-'t P(:c), where P(:c) = exp( -q(w):c), q:l(w) = k+~""'.

8.4.1; Un(t) = exp( -e•lniw/k) )2) sin(.ar). H we set n = llf, and h = l, we - have in the limit k -+ oo, u(:c,t) = exp(-!jf.)sin(ar), which is the

correct solution of the continuous heat equation with periodic initial data.

8.4.2; Un(t) = Jn(-f).

8.4.5; Un(t) = J2n(f).

8.4.6; k(w) = cos-1(1- ~~~~h2 ).

8.4. 7; For stability, require -2 < 6t.\ < 0, for all eigenvalues ,\ of A.

8.4.8; For stability, require that all eigenvalues of A be negative.

9.1.3; q(:c) = -2H(:c)sech2 :c. (See Problem 7.5.13.)

9.2.2; (a.) !l9f = ian(a~+l -a~).

589

(b) 2Jt = e-Un-1 _ e-Un+l,

9.2.3; qxt = sinh q.

9.3.2; The bound state has J.L = -4· There is a single soliton with amplitude !A2 and speed c = 4J.L2 = A2

.

9.3.3; Use that r(x) = 6e-x+Bt + 12e-2x±64t and find a solution of the form

(9.18) with~= x- 4t, 'f1 = 2x- 32t.

9.3.4; One soliton will emerge.

9 3 5. !l£u. i !ku - 0 • • ' dt = - 2k cu' dt - •

9.4.3; Verify (9.20) and (9.21).

9.4.4; (a) a= 1~~2 z-2i, (3 = 1- a, R = ~·

(b) = _.! (4a5-l~f4a5zl-1) (3 _ 1 zl(16a~-4a5+1)-4afi Th t

a 4 a5z z~-1) ' - 4 a5(zL1) . ere are wo 2 4a2

rootsof(3(z)=Oatz = ._, ?~ ...

(c) a= _ 2B (3 = zl±2Bz-1 ~~ z2-1 ·

9.4.5; anWn = a~z) (z- ~).

9.4.6; With q0 > 0, set a0 = !e-qo/2 , a 1 = !eqo/2 , and then use Problem 9.4.4b to show that two solitons moving in opposite directions emerge.

9.4.7; Set b1 = -~ =F 0 and then use Problem 9.4.4c to show that one soliton

emerges.

9.4.8; Choose spring constants kn with 4k; = 1 + sinb2

wsech2nw.

10.2.1; En(x) = r(:) L~o(-1)krtltf>. 10 2 2· r1(cosm + t2)eixtdt = (-i. + 2 + 2i )eix _ 2i + i(1 + eix) '\'oo (2!:)2k. • ' ' JO X ~ ~ ~ X L.Jk=O X

10.2.3; I; eixtt-lf2dt = Iooo eixtrtf2dt- It eixtt-ll2dt

_ (;i + ei"' '\'oo (-1)k-1 r(k±l/2) - V x 7/r L.Jk=O xk+t ·

10.2.4; C(x) = !A- P(x) cosx2 + Q(x)sinx2, S(x) = h/I- P(x) sinx

2-

Q(x)cosx2 , where P(x) = ~ (~- ~ + · · ·), Q(x) = 1 (l - ~ + ~ + .. ·) . 2 x 4x 16x

10.3.1; E1(x) =e-xl:~0(-1)k Jb. roo e-•t '\'00 ( )k ( 4k)l

10.3.2; Jo l+t' dt = L.Jk=O -1 z'41'+T.

10 3 5 rl -xtt-ll2dt /Jf -x r;; '\'00 1 • • i Jo e = V z - e Y 1r L.Jk=O xk+it(l/2-k) ·


10.3.6; f000

e:ctt-tclt = y'21jiell (1 - 2!11 - 6~3112 + · · · ), where 71 = ez-1.

10.3. 7; I: F sin2 1rtdt = ~ - 1~~2 + (50 - 81r2);; + .. ·.

10 3 8. rff ezt2t-1/3 cos tdt - 1 "00 <ct~; r(lt+tfJ> • • 1 Jo - 2 L.tlt=O 2 z'+t a •

10.3.11; (a) I:~=O ( ~ ) kin-" =nIt en(ln(l+z)-z)dx ,.., ~for large n.

(b) 2:~=0 ( ~ ) kl..\" = Iooo e-z(1 + ..\x)ndx,.., (n..\)ne-n+! ex:1 >fii· 10.3.13; Iooo flee-t Intdt = v'21rxz+te-z lnx(

2zb -

2..;m + · · ·).

10.3.16; ( 11~ ) ,..., 1.82 X 1013.

10.4.1; Io e:~~~;> dx = 21rie-3"14 - 2ie-• l:J!o( -4)i ri~ttfl>.

10.4.3; I(x) =I: exp[ix(t + t3 /3)]dt = J!"e-2z/3(1-~ + 48~\~2 + O(x-3)).

10.4.4; Jn(z) = {£ cos(z- Df- f)- ~{J; sin(z- Df- f)+ O(z-612 ).

10.4.5; l'nlc = (2n + 4k- 1)f - 2t,; 2~:;!Z~1 + O(b ).

10.4 6· r1 cos(xt")dt = ,1(.!)1/Prl)- ie1•

• , Jo ,. z ,. PIC •

10.4.7; Ioff/2 {1- ~}112 cos(:ccosO)dD

=& {ew(~-~(~+iv'ID+···)} -yt~+···. 10.4.8; H x > 0, the change of variables s = VZf converts this to !v'zi(x312),

where I(x) is defined in Problem 10.4.3. H :c < 0, Iooo cos(sx + s8 /3)ds"'

2(-~1/4 cos('f +f). 10.4.9; Use the answer to Problem 10.4.8 to show that the kth zero of the Airy

function is at x, -J( -4k + 1)1r.

f oo -•'12 . fl[ 1 3 10.4.10; (a) -co wdq,..., v Q"(l- ~ + ici2' •• ·), for large a.

10.5.1; I: f(x)e'"'(:c)dx = f(a)r(~)("'<~ca) )2/3eiltg(a).

10.5.4; Set u(x, t) = e-d/2w(x, t) so that Wtt = W:cz + ~w.

11.1.1; X1 = -0.010101, X2,3 = -0.49495 ± 0.86315i.

11.1.2; x1 = 0.98979. Since x = 0 is a double root of x3 - x2 = 0, the implicit function theorem does not apply for r.oots near x = 0.

11.1.3; X1 = 1- E + 3E2 - 12c:3 + 0(E4), X2,3 = ±~ -! ± ~iE + !e2 + 0(~).

11.1.4; ¢ + i.,P = U(z- a:)+ iEa(i)20 + 0(E2).

11.1.5; ¢ + i.,P = z + ic:e1z + O(e).

591

11.1.6; u(x,y) = a(1-y)+by+E(a+(b-a)y)~8

-w(x,y)+O(c:2), where V2v = 0 2 bz2

and Vz(O,y) = O,vz(1,y) = a/2 + (b- a)y/2, v(x, 0} = a; , v(x, 1) = T· Use separation of variables to find v.

11.1.8; u(r,O} = rcosO + c:(£{1- r4 ) + -fi(r2 - r 4 ) cos28) + O(e).

11.1.9; u(r,O) = rcosO- !c:r2 sin29 + O(e2).

11.1.11; sfv = c:2k2 /2 + c:4k4 /4 + O(c:6).

11112 u h 1 f •d h 2l 2 wk sinh2 2kl+2k21

2 • . ; ror a c anne 0 Wl t , 8 = E T _, L!l nLI nL'ii'I + .. ·.

11.2.1; (a) Eigenvalues are 1, 2, 3, independent of E. Eigenvectors are ( -2, 0,1-c:}T, (0, 1,0)T, and (0,0, l)T.

(b) The eigenvalue ,\ = 1 has algebraic multiplicity 2, but geometric multiplicity 1, so the "standard" perturbation method fails. The eigenvalues are ..\ = ±~ = 1 ± ( c:112 - !c:3/ 2 -}E6/ 2 + 0( c:7/2)), with

corresponding eigenvectors (1,z2)T where x2 = ±.j6 = ±(c:1/ 2 +

1/2c:3/2 + 3/&6/2 + O(e7/2)).

11.2.2; y(x) = sinnx + ~(-xsinnx -1rnxcosnx + nx2 cosnx), ,\ = n2 - E~ + o(e).

11.2.3; (b) For X1 sufficiently large (since f(x) has compact support), ,\ ""Ao + e-zt J/(~tofgj)dz J 0 :» dz

11.3.1; Steady solutions are at 8 = 0 and at cosO= ;&r provided 0 2 > f. 11.3.2; y(x) = c:sinn1rx

+ €2(1-coemrz)2 +2zcoamrz -E2( 2 _ 1)s;nn'/r:C +2E2e\nmrz-n7rZCOinlTZ 3~211'2 Srtll* ~ 8nlirJ cos mr '

..\ = n21r2 + t,i;r(1- cosn1r) + 0\~).

11.3.3; Bifurcation points occur at solutions of ~(1-/:4) = n21r2 and there are a finite number of such solutions. The nth solution exists if ~ > n21r2.

11.3.4; u(z, y) = c:sinn1rxsin m;v + 0(E2), ,\ = -(n2 + !m2)1r2 - !6 E2a + 0(E4).

11.3.5; u = e¢ + O(c:4), ..\ = .\o - ~a4 f ::: if a4 f:. 0 whereas u = c:¢ + O(c:6),

4 f<P:dz . ,\ = Ao - c: a5 t/J dz if a4 = 0, a5 f:. 0.

11.3.6; The exact solution is ¢ = 1 + ..\a where a = (1 + .\a)2. A plot of the amplitude a as a function of ,\ is shown in Fig. A.5.

592

11.3.7;

"" 15

-.: 10

APPENDIX A. SELECTED HINTS AND SOLUTIONS

"" 15

.. 10

-o.• -o.2 0.0 ),

0.2 o .•

Figure A.5: Plot of ifJ as a function of A for Problem 11.3.6.

{a) u(x) = Aa(x), where A satisfies the quadratic equation A2 fol a3(y)dy- ~ +! fol a(y)dy = 0.

(b) u(x) = Aa(x), where 1 = Asin(A) J; a 2 (y)dy.

a.o

2.11

2.0

~ ... c 1.0

o.o 011 I I I i I :p=:, I 0.0

o.o o.z 0.4 o.e o.a 1.0 1..2 1.4 1.1 ),

... .. -:z 0 ),

Figure A.6: Left: Plot of A as a function of A for Problem 11.3.7a with a(x) = sin(x); Right: Plot of A as a function of A for Problem 11.3.7 with a(x) = sin(x).

11.3.0; u(x) = exp(Acosx), where A= Af~1r eAcoau cosydy.

11.3.10; Solve the system of equations v" +n2 rr2v = e(<P+v) 2 -1'¢1-Jlv, J01 vf/Jdx =

0, I'= 2e J;(q, + v) 2q,dx, where ¢J =sin nrrx.

11.4.3; (a) a= 1- 112 e2 + O(e3), u(t) = sin(t) + tecostsint + O(e2), or a=

1 + 162 e2 + O(e3), u(t) = cos(t) + je(cos t- 2) + O(e2).

(b) The two curves are a- 4- .!..e2 - _lll_e4 +0(e6) and a= 4- .!..e2 + - 30 21600 , 30 _!ll_e4 + O(e6) 21600 .

11.4.5; Set u = Aeiwt and find that A2 ( (b2 A2 + 1 - w2 ) 2 + a2w2 ) = i F 2 •

11.4.6; Set U = a(x)uij(x) and show that for a(x) to be bounded, it must be that f~oo g(x)uti(x)dx = 0.

11.4.7; (a) Use the implicit function theorem to show that u = _l+fw~ coswt + O(e2).

(b) Require a= ~(ifJ) = 3~wsech w21r sinifJ.

593

11.4.8; Require a= <I>(¢) = -23 . ~w~ cos(¢).

stn :l

11.5.1; There are two Hopf bifurcation points, at A= ! ± ~-

11.5.3; Steady state solutions have s = >., x = (>. + a)(l- >.). A Hopf bifurcation occurs at >. = !(a+ 1).

11.5.4; Suppose '1/Jo satisfies AT '1/Jo = 0, and 'l/;1 satisfies AT '1/Jt = -iA'l/;1. Then a small periodic solution takes the form x = e(a¢o + bei>.t¢1 + be-i>.t¢)1) + O(e2) provided 2lbi2(Q(¢1, ¢}1), '1/Jo) + a(B¢o, '1/Jo) + a2(Q(¢o, ¢o), '1/Jo) = 0, where 2a(Q(¢o, ¢1), 'l/;1) = -(B¢1, 'l/;1).

12.1.1; u(t, E)= a(1 + ~) cos((l- ~t- E)+ O(e3).

12.1.2; u(t) = A( Et) sin( (1 + e2w2)t) + eB( d) cos( (1 + e2w2)t) + 0( e2) where At =

!A(1- ~A2), Bt = !B(1- ~') + {!,(A2 - 4)(6A2 -1), and w2 = -l6 •

12.1.3; The solution is u(t) = eA(r) sin(t) + O(e2 ) where AT= !A(1J(r)- ~A2 ), and f.L = E

21J, T = d.

12.1.4; (a) The Landau equation is AT= -iA3 , with solution A(r) = ,;3]t~,,, where A(O) = Ao, so that u(t) = V Af sin(t + ¢o) + O(e).

1+ 4 A oft

(b) The Landau equation is AT = - 3~ A 2 , with solution A( r) = , ,31r_Ap~_ where A(O) = Ao.

12.1.5; Take a= 1 + E2a2, take the slow time to beT= E2t and then the solution is u(t) = A(r) sin(t + ¢(r) + O(E) where f/JT = !(a2 - 1

52 + ! cos2 ¢),

AT = ~sin ¢cos¢.

12.1.6; The leading order solution is u(t) = A(r)sin(t + ¢(r)) where r = d and AT= iA2 cos(¢), ifJT =~A sin¢.

12.1.7; (a) The Landau equation is AT = -~A3 cos(r), with solution A(r) =

V ;A_o , where A(O) = A0. This solution exists for all time 3A0 smT+4

provided A5 < ~. (b) The Landau equation is AT= -%A3q(r).

12.1.8; Let T = e2t and then u(t) = A(r) sin(t + ¢(r)) where AT = -!A, and -~.. _ 1A2 '/'T- B •

12.1.9; With u = RcosB, then (from higher order averaging) Rt = -\'-R3- ~R+

O(e3 ). The solution is R2(t) = 16e.R5e-•2t(16e + 3.R5(1- e-•

2t)- 1. The

behavior predicted by the leading order equation Rt = -\'-R3 is incorrect, because it predicts algebraic rather than exponential, decay.

2.1.11; The exact solution is u(x) = aJ0 ( ~eax) + bYo( ~eax). The approximate solution is (set E = % and employ adiabatic invariance, or approximate the Bessel functions for large arguments) is u(x) "'e-ax/2 sin( ~eax + ¢).


12.1.12; Write the equation (12.16) as the system u 111 = r(~)v, Vz = J(x, ~),and then make the exact change of variables u = Uo + EW(~)z, v = z + EZ, where~= r(u)-r, and Z = J; J(y,u)du-u J: f(y,u)du, with u = ~·

12.1.13; Set v = z + Eh(~)u where h' = g- g, and then !!if = z + eh(~)u, ~ = -gu- Eh(~)z -·f2h(~)2u.

12.1.15; In dimensionless time, rc has units of length-2• A good dimensionless parameter is E = ~, r = r c + r m/ h. The function W is a piecewise linear "sawtooth" function with slope - roh?frm •

12.1.16; Write the equation as the system u' = v,v' = u- (1 + g'(~))u8 + Eav. Then the transformation v = z + Eg( ~) transforms this into a system to which Melnikov's method can be applied. The requirement is that af~oo U/?(t)clt = J~00 9'(!.¥)U8(t)Ub(t)clt, where Uo(t) = v'2sech t.

12.2.2; The device is a hysteretic switch. There are two stable steady outputs, Vo = V~ if V<V~ and Vo = VR_ if V_ > VR-.

{

VR+ 12.2.3; (a) i = "Ri'0 where v0 = i

VR-

Bl~'iz;·

for v > AvR+ for AVR- < v < AvR+ , where A =

for v < AVR-

(b) The current in the left branch of the circuit satisfies C RpR• ~ + R.i2 = v- v1 , thus the effective inductance is L = CRpR •.

12.2.5; u(t) = rh + e-t/e + O(E).

12.2.6; u(t) = 1~1 + O(E}, v(t) = (t+!>:~ + e-t/'- + O(E).

12.3.2; To leading order in E, u(t) = -ln(~) - ln(2)e-t/2'-11'(.~ sin l3ta +

cos~).

12.3.3; u(t) = -tan-1(t) + E113 exp( 2;ih) (l sin( 2~a)- cos(~))

+ 1 El/322/3 exp(21

/1(t-l)) i ,_1/1

12.3.4; For all {3 there is a solution with u(x) ""'-1+ a boundary layer correction at x = 1. For {3 > 0.2753 (the real root of 3x3 + 12x2 + llx- 4 = 0}, there is a solution with u(x) ""'2+ a boundary layer correction at x = 1.

·12.3.5; The solutions are u(x) = ;h - tanh( 111;[

1) + O(E) and u(x) = ~ -

tanh( 1112?) + O(E) with 1/J. and fb appropriately chosen.

12.3.6; The two solutions are u(x) = ::;:~ + 2e-z/e + O(E}, and u(x} = ::~ -fe- 111

/'- + O(E).

12.3.7; (a) u(x) = ;h- tanh(fe- tanh-1{j)).

595

(b) u(x) = 4::~:-:_13 - 2tanh(z;t + tanh-1 (i))

(c) u(x) = H(x- i> 51:+h + t(l- H(x- i)) 1ia:-:!3 - ~ tanh(lf(x -!)) where H(x) is the usual Heaviside function.

12.3.8; u(x) = a+~-l + dn(cosh( 2z-<;!.B-l)) + O(e).

L2.3.11; (a) T"' 2ln 2~a.

(b) T(v) = -r!1 ln !~ !+-r :, where c(v) = v':~v!i·

Index

Abel's equation, 158 Abel's integral equation, 103 activation energy, 539 adiabatic invariant, 511 adjoint matrix, 13 adjoint operator, 106, 152 admissible functions, 177 age structure, 102 airfoil, 241, 243 Airy function, 467 algebraic multiplicity, 11, 43, 52, 476 amplifier, 523 analytic continuation, 227 analytic function, 214 Argand diagram, 209 Arrhenius rate function, 539 associated Legendre equation, 268 associated Legendre functions, 167 asymptotic sequence, 438 asymptotic series, 437 averaging theorem, 512

B-splines, 81, 89, 304 Banach space, 61 band limited, 299 basis, 3 Battle-Lemarie function, 305 Belousov-Zhabotinsky reaction, 557 Bernoulli's law, 234, 243 Bernstein polynomials, 69 Bessel functions, 69, 169

modified, 265 of the first kind, 263 of the second kind, 264 of the third kind, 265 spherical, 364

Bessel's equation, 169, 262

596

Bessel's inequality, 68, 111, 136, 163 beta function, 261, 281 bifurcating solution branch, 479 bifurcation point, 479 biharmonic equation, 474 bilinear transformation, 237 binomial expansion, 70, 279, 443 biorthogonality, 52 Biot-Savart law, 343 bistable equation, 549 Blasius theorem, 233 blood flow, 367 bound states, 317 boundary conditions

Dirichlet, 338 inhomogeneous, 154 mixed, 338 natural, 185, 205 Neumann, 338 periodic, 152 separated, 147, 152

boundary element method, 358 boundarylayer,521 boundary operator, 151 bounded linear functional, 108 bounded linear operator, 105 branch cut, 211 branch point, 211 Bunyakowsky inequality, 6

Calm-Allen equation, 382, 549 Calm-Hilliard equation, 383 canonical variables, 188 CAT scan, 105 Cauchy inequality, 6 Cauchy integral formula, 220 .Cauchy sequence, 61, 68

INDEX

Cauchy's equation, 234 Cauchy-Riemann conditions, 215, 355 causality, 149, 369, 414 characteristic polynomial, 11 Chebyshev polynomials, 50, 73, 168,

281 chemostat, 503 circulation, 235 coaxial cable, 365, 380, 406 commutator, 418 compact

disc, 300 operator, 111 support, 87, 137

complementary error function, 442 complete set, 68, 113 complete space, 61 complex

calculus, 214 conjugate, 209 differentiation, 214 functions, 211 integration, 217 numbers, 209 velocity, 231

computer aided tomography, 105 conformal map, 236 conservation law, 380 constitutive relationship, 380 continuous linear operator, 105 continuous spectrum, 284, 312 contraction mapping principle, 122,

313,498 convergence, 61,437 convolution, 172

integral, 78 kernel, 296 theorem,294,295,304,356,384

cosine integral transform, 290 Courant minimax principle, 19 cubic splines, 89 curve fitting, 25 cycloid, 180, 185, 204

Daubechies wavelets, 82, 307 decompositions

LU, 32 QR, 33,36 singular value, 37 spectral, 15, 37, 52

degenerate kernel, 112 delta distribution, 139, 340 delta sequences, 134 dendrites, 409

597

difference equation, 170, 324 difference operator, 310, 332 differential-difference equations, 390 diffraction pattern, 376, 407 diffusion coefficient, 380 diffusion-reaction equation, 380, 382,

383, 389 digital recording, 300 digital signal, 463 dilation equation, 80 Dilbert space, 65 dimension, 3 dipole, 230 dipole distribution, 139 dipole source, 342, 358 Dirichlet boundary conditions, 338 Dirichlet-Jordan convergence theo-

rem, 164 discrete Fourier transform, 76 discrete sine transform, 77, 170 dispersion, 395 dispersion relation, 463 distributions, 138

delta, 139, 340 dipole, 139 Heaviside, 139 higher dimensional, 339 regular, 139 singular, 139

divergence, 228, 234 divergence theorem, 338, 380 domain of an operator, 106, 151 dot product, 4 drag, 232 dual manifold, 153 Duffing's equation, 194

eigenfunctions, 161, 283, 359, 362

598

improper, 296 nodal curves, 361

eigenpair, 11, 161 eigenvalue, 11, 161, 199, 283 eigenvector, 11 elastic membrane, 194 elastic rod, 189 elastica equation, 193 elementary reflector, 34 elementary row operations, 30 entire function, 226 enzyme kinetics, 523 equivalent matrices, 10 essential singularity, 227 Euclidean inner product, 4 Euclidean norm, 5 Euler column, 193 Euler identity, 209 Euler-Lagrange equation, 178 exponential integral, 440

fast Fourier transform, 77 Fermat's problem, 179 Fick's law, 380 filter, 78, 296 finite element space, 88 finite elements, 88 FitzHugh-Nagumo equations, 503 formally self-adjoint, 153 Fourier coefficients, 68, 136, 162 Fourier convergence theorem, 74 Fourier cosine integral transform, 290 Fourier integral transform, 290 Fourier series, 68, 74, 164, 303 Fourier sine integral transform, 289 Fourier transform theorem, 290 Fourier's law of cooling, 380 Frechet derivative, 178 Fredholm alternative, 24, 110, 156,

476, 493, 507, 514, 516 Fredholm integral equation

of the first kind, 101 of the second kind, 102

free boundary problems, 243 Frenet frame, 489 Frenet-Serret equations, 489

Fresnel integrals, 464, 466 Frobenius norm, 41 Fubini's theorem, 64 function spaces, 60 functional, 108, 177

INDEX

fundamental theorem of algebra, 276 fundamental theorem of calculus, 218

Gabor transform, 301 Galerkin approximation, 92, 124, 131,

176, 197 Galerkin method, 349 Galerkin projections, 399 gamma function, 259, 441 gauge function, 438 Gauss-Seidel iterates, 46 Gaussian elimination, 30 Gegenbauer polynomials, 73, 168 Gel 'fand-Levitan-Marchenko ( GLM)

equation, 413, 415, 433 generalized Green's function, 159 generalized Laguerre functions, 168,

281 generating function

for Bessel functions, 266 for Legendre polynomials, 269,

280 geometric multiplicity, 11, 52, 476 Gram-Schmidt procedure, 8, 34, 39,

65, 68, 114 Green's formula, 338 Green's functions, 146, 284, 339, 383

generalized, 159 modified, 159, 175

Green's theorem, 217

Haar functions, 83 Hamilton's equations, 188 Hamilton's principle, 186 Hamiltonian, 187 Hammerstein integral equations, 500 Hankel functions

of the first kind, 265, 407 of the second kind, 265

Hankel transform, 309, 352 Hardy-Weinberg proportions, 527

INDEX

heat equation, 338 Heaviside distribution, 139 Heisenberg uncertainty principle, 334 Helmholtz equation, 360 Hermite functions, 89 Hermite polynomials, 50, 73, 169,

298 Hermitian matrix, 14 Hilbert space, 65, 283 Hilbert transform, 355 Hilbert-Schmidt kernel, 103, 112 Hilbert-Schmidt operator, 112, 161 hodograph transformation, 245 Hooke's constant, 189, 319 Hooke's law, 368 Hopf bifurcation, 494, 496, 510 Householder transformation, 34

ill-conditioned matrices, 40 implicit function theorem, 470, 475,

513 improper eigenfunctions, 296 improper eigenvalues, 296 indicia! equation, 263 induced norm, 6 inequality

Bessel, 68 Bunyakowsky, 6 Cauchy, 6 Schwarz, 6 Sobolev, 66 triangle, 5, 60

influence function (see also Green's function), 146

inner product, 4, 65, 108 inner product space, 4 integral equations

Abel, 103 Fredholm, 101, 102 Hammerstein, 500 singular, 103 Volterra, 101

integration by parts, 217, 338 complex, 217 contour, 220, 291, 353

Lebesgue, 63 line, 217 Riemann, 63, 217

interpolation, 78, 397 invariant manifold, 14 inverse integral operator, 110 inviscid fluid, 228 irrotational flow, 229 isolated singularities, 226 isoperimetric problem, 181 isospectral flow, 417, 418 iterative solution technique, 121

Jacobi iterates, 45 Jacobi polynomials, 73, 168 jet nozzle, 243, 246

599

Jordan's lemma, 253, 291, 292, 353, 413, 450, 461

Josephson junction, 406 Joukowski transformation, 242, 311 jump condition, 147, 159

Kaplan matching principle, 530, 541 kinetic energy, 186, 188, 204 Kirchhoff's laws, 365, 485, 523 Korteweg-deVries {KdV) equation,

418 Kronecker delta, 3 Kutta condition, 242

.L'Hopital's rule, 251, 264 lady bugs, 366 Lagrange multiplier, 18, 180, 198 Laguerre polynomials, 50, 73, 168,

298 Landau equation, 509, 593 Langrangian, 187 Laplace transform, 307, 331 Laplace's equation, 223, 229, 337 Laplace's method, 442 Laplacian operator, 337 lattice, 22, 310, 420 Laurent series, 225, 267 least squares problem, 26 least squares pseudo-inverse, 28 least squares solution, 26

600

Lebesgue dominated convergence theorem, 64, 94, 134

Lebesgue integration, 63, 94 Legendre function

of the first kind, 269 of the second kind, 269

Legendre polynomials, 9, 50, 72, 167 Legendre's equation, 262, 268 Lewis number, 540 lift, 232 linear combination, 2 linear functional, 108, 137

bounded, 108 linear manifold, 14, 107 linear operator, 105 linear vector space, 2 linearly dependent, 2 linearly independent, 2 Liouville's theorem, 221 Lipschitz continuity, 136, 164 logistic equation, 471 LU decomposition, 30, 32 lung tissue, 409 Lyapunov-Schmidt method, 482

Maple, 48, 50, 453, 455, 492 Mathematica, 48 Mathieu's equation, 502, 552 Matlab, 48 maximum modulus theorem, 222 maximum principle, 17, 115, 117 mean value theorems, 222 Mellin transform, 308, 354 Melnikov function, 494 method of averaging, 511 method of images, 343, 370 method of lines, 391 Meyer functions, 304 Michaelis-Menten law, 525 microwave ovens, 24 midline strain, 191 minimax principle, 19, 199 minimum principle, 199 modified Bessel function, 265, 266 modified Green's function, 159 modulus of continuity, 71

INDEX

moment of inertia, 53 Moore-Penrose least squares solution,

28 Moore-Penrose pseudo-inverse, 28 mother wavelet, 80 multigrid method, 4 7 multiplicity

algebraic, 43, 52, 476 geometric, 52, 476

multiscale method, 509

natural basis, 3 natural modes, 23 natural norm, 6 near identity transformation, 512 Neumann boundary conditions, 338 Neumann iterates, 122 Newton's law, 233 Newton's method, 131 nodal curves, 361 normal equation, 27 normal modes, 23 normed vector space, 5 norms,5,59

Euclidean, 5 Frobenius, 41 induced, 6 £2,60 LP,60 IJ', 59 natural, 6 Sobolev, 66 supremum, 5, 60 uniform, 60

null space, 106

operational amplifier (op-amp}, 521 orthogonal complement, 15, 108 orthogonal functions

associated Legendre, 167 Haar, 83 Laguerre, 168

orthogonal matrices, 33 orthogonal polynomials, 9, 69

Chebyshev, 50, 73, 168, 281 Gegenbauer, 73, 168

INDEX

Hermite, 50, 73, 169, 298 Jacobi, 168 Laguerre, 50, 73, 168, 298 Legendre, 9, 50, 72, 167

orthogonality, 7

Parseval's equality, 69, 113, 271, 293 path independence, 219 pendulum, 148, 188, 510 perihelion of Mercury, 183, 483 periodic boundary conditions, 152 Perron Frobenius theorem, 44 Phragmen-Lindelof theorem, 222 pitchfork bifurcation, 482 Plancherel's equation, 293 Poisson equation, 337, 392 Poisson kernel, 223 pole of order n, 226 population dynamics, 101, 472 populations genetics, 526 positive definite

matrices, 18 operator, 161, 338

positive matrices, 44 potatoes, 386 potential energy, 187-189, 194 potential function, 228 power method, 43 precompact, 111 principal branch, 211 problem of Procrustes, 41 projection, 8, 98, 119, 124, 347, 397 pseudo-inverse, 28 pseudo-resolvent operator, 120 Pythagorean theorem, 7

QR decomposition, 33 quadratic form, 17, 186 quasi-steady state solution, 523, 525

range, 14 range of an operator, 106 ranking of teams, 44 Rayleigh-Ritz technique, 92, 197 recurrence formulas, 266 reflection coefficient, 316

reflectionless potential, 323 region of influence, 369

601

Reisz representation theorem, 138 relaxation oscillations, 545 removable singularity, 226 renewal processes, 102 residue, 225 residue theorem, 225, 285, 414 resolvent kernel, 120, 128 resolvent operator, 110, 118 resolvent set, 283 resonant frequencies, 24 Riemann mapping theorem, 237 Riesz representation theorejll, 108 Rodrigue's formula, 268 root cellar, 388 rotating top, 206 rotational matrix, 12

saddle point, 449 saddle point method, 450 scalar product, 4 scattering transform, 411 SchrOdinger equation, 169, 313, 411

nonlinear, 420 SchrOdinger operator, 312 Schwarz inequality, 6 Schwarzschild metric, 182 secular terms, 506, 532 self-adjoint, 13, 153 separable kernel, 112 separated boundary conditions, 147,

152 separation of variables, 345 sequence spaces, 59 sequentially compact, 111 set of measure zero, 63, 94, 134, 163 shallow water waves, 421 Shannon sampling function, 78 similar matrices, 10 similarity solution, 384 similarity transformation, 10 similarity variable, 384 sine function, 78, 270, 302 sine integral transform, 289 sine-Gordon equation, 419

602

singular integral equations, 103 matrices, 11 value decomposition, 37 values, 38

singularities, 292, 308 essential, 227 isolated, 226 pole of order n, 226 removable, 226

sink, 230 smoke rings, 488 smoothing algorithm, 397 Snell's law, 132, 180 Sobolev inequalities, 66 Sobolev inner product, 66 Sobolev norm, 66 Sobolev space, 65, 66 solitary pulse, 548 solitary wave, 421 SOR, 46 source, 230 span, 3 spanning set, 3 special functions

Airy function, 467 · beta function, 261, 281

gamma function, 259, 441 spectral decomposition, 15, 37, 52 spectral representation, 11, 283, 297 spectroscopy, 24 spectrum

continuous, 284, 312 discrete, 283 point, 283 residual, 284

splines B, 81 cubic, 89

square well potential, 321 stagnation points, 231 stationary phase, 456 Stirling's formula, 444, 453 strain matrix, 190 stream function, 474 streamlines,230

INDEX

Sturm-Liouville operator, 153, 160, 163, 198, 500

successive over-relaxation, 46 superposition principle, 200 support, 137 supremum norm, 60 swimming protozoan, 473 symbolic function, 140

Taylor series, 224 telegrapher's equation, 366, 373, 406,

461 test function, 137, 339 Toda lattice, 420 tomography, 103 transcendentally small, 439, 540 transfer function, 296 transform theory, 9 transformations

bilinear, 237 hodograph, 245 Householder, 34 Joukowski, 242, 311 near identity, 512 orthogonal, 33 similarity, 10 unitary, 33

transforms discrete Fourier, 76 discrete sine, 77, 170 Fourier, 290 Fourier cosine transform, 290 Fourier sine integral, 289 Gabor, 301 Hanke1,309,352,405 Hilbert, 355 Laplace, 307, 331 Mellin, 308, 354 scattering, 411 windowed Fourier, 300

transmission coefficient, 316 transpose, 13 trefoil knots, 488 triangle inequality, 5 trigonometric functions, 69, 73

. tunnel diode, 485

INDEX

two component scattering, 419 two-timing method, 509

uniform norm, 60 unitary matrices, 33

van der Pol equation, 486, 494, 542 Van Dyke matching principle, 530 variational principle, 177 vector space, 1 vectors, 1 Volterra integral equation, 101 vortex, 230 vortex filament, 488 vorticity, 342, 488

Watson's lemma, 446 wave equation, 189, 337 wavelets, 69, 80

Battle-Lemarie, 305 Daubechies, 82, 307 Haar, 83 hat, 83 Meyer, 304 mother, 80

weak convergence, 142 weak formulation, 142, 340 Weierstrass approximation theorem,

69 Whittaker cardinal function, 78, 270,

300 Wiener-Paley theorem, 299 winding number, 490 windowed Fourier transform, 300 Wronskian, 147, 175, 263

X-ray tomography, 103

Young's modulus, 192, 368

z-transform, 311 Zeldovich number, 540 zero sequence, 138

603

Date post:	17-Jul-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

PRINCIPLES OF APPLIED Transformation and Approximation ...fnarc/m641/keener_chapter_1.pdf · vi...

Documents