7075.tp.indd 1 10/20/09 9:15:20 AM The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.This page intentionally left blank This page intentionally left blank The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.7075.tp.indd 2 10/20/09 9:15:20 AM The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.British Library Cataloguing-in-Publication DataA catalogue record for this book is available from the British Library.Forphotocopyingofmaterialinthisvolume,pleasepayacopyingfeethroughtheCopyrightClearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission tophotocopy is not required from the publisher.ISBN-13 978-981-283-625-0ISBN-10 981-283-625-XAll rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means,electronic or mechanical, including photocopying, recording or any information storage and retrievalsystem now known or to be invented, without written permission from the Publisher.Copyright 2010 by World Scientific Publishing Co. Pte. Ltd.Published byWorld Scientific Publishing Co. Pte. Ltd.5 Toh Tuck Link, Singapore 596224USA office:27 Warren Street, Suite 401-402, Hackensack, NJ 07601UK office:57 Shelton Street, Covent Garden, London WC2H 9HEPrinted in Singapore.THEBIRTHOFNUMERICALANALYSISRokTing - The Birth of Numerical.pmd 1/27/2010, 4:47 PM 1 The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.Preface1 ThelimitationsofcomputersIn Wikipedia, numerical analysis is described as that part of mathematics wherealgorithms for problems of continuous mathematics are studied (as opposed todiscretemathematics). Thismeansthatitisespeciallydealingwithreal andcomplexvariables, thesolutionofdierential equationsandothercomparableproblems that feature in physics and engineering. A real number has in principleaninnitenumberof digits, butonadigital computer, onlyanitenumberof bitsisreservedtostorea(real)number. Thismemoryrestrictionimpliesthat only rounded, approximating values of only nitely many real numbers canbe stored. The naive idea of the early days of digital computers was that theywouldnot makethesamestupiderrorsthat humancomputers sometimesmade, like transcriptionerrors, readingerrors, wrongsigns,etc. Thiseuphoriawashoweversoontemperedwhenitwasrealizedthatcomputersinfactmakeerrorsinpracticallyeverycalculation. Small errorsindeed, butneverthelessalotoferrors. Andall thesesmall errorscanaccumulateandgrowlikeavirusthroughthemanyelementarycomputationsmadewhichcouldeventuallygivea result that is quite dierent from the exact one.2 Abirthday?Acareful analysisof thispropagationof errorswhensolvingalinearsystemof equationswasrstpublishedinapaperbyJohnvonNeumannandHer-man Goldstine: Numerical inverting of matrices of high order, published in theNovember issue of the BulletinoftheAmericanMathematical Society in 1947.Becausethiswasthersttimethatsuchananalysiswasmade, thispaperissometimes considered to be the start of modern numerical analysis. Of course nu-merical calculations were done long before that paper and problems from physicsand engineering had been solved earlier, but the scale and the complexity of thecomputationsincreaseddrasticallywiththeintroductionofdigitalcomputers.The large systems to which the title of the paper refers, would not be calledlarge at all by current standards. It is claimed in the paper that serious prob-lems can occur if one wants to solve systems of more than ten equations. In asubsequentfootnote, itissuggestedthatitwouldprobablybepossibleinthefuture to solve systems of a hundred equations. If we know that the PageRank ofGoogle can be computed by manipulating systems of approximately ten billionequations, then it should be clear we have come a long distance.v The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.vi Preface3 SixtyyearsyoungbacktotherootsofthefutureIfthepublicationof thevonNeumann-Goldstinepaperisindeedthestartofnumerical analysis, then November 2007would bethe moment that numericalanalysis can celebrate its sixtieth birthday. This inspired the scientic researchnetwork Advanced Numerical Methods for Mathematical Modeling, a consortiumofnumericalanalysisgroupsinFlanders,toorganizeatwo-daysymposiumatthe Department of Computer Science of the K.U.Leuven (Belgium) entitled Thebirth of numerical analysis. The idea of this symposium was to invite a numberof speakerswhowerealreadyactivenumerical analystsaroundthemiddleofthe twentieth century or shortly after and hence were co-founders of the moderndiscipline.Theycametowitnessabouttheirexperienceduringtheearlydaysand/orhowtheirrespectivesubdomainshaveevolved.Backtotherootsisanimportant general cultural paradigm, and it is none the less true for numericalanalysis. To build a sound future, one must have a thorough knowledge of thefoundations. ElevenspeakerscametoLeuvenonOctober29-30, 2007forthisevent to tell their personal story of the past and/or give their personal vision ofthe future. In the rest of this preface we give a short summary of their lectures.Most of them have also contributed to these proceedings.4 ExtrapolationThe start of the symposium was inspired by extrapolation. In many numericalmethods, a sequence of successive approximations is constructed that hopefullyconverges to the desired solution. If it is expensive to compute a new approxi-mation,itmightbeinterestingtorecombineanumberofthelastelementsinthesequencetogetanewapproximation.Thisiscalledextrapolation(tothelimit). In the talk of Claude Brezinski, a survey was given of the development ofseveral extrapolation methods. For example for the computation of , ChristiaanHuygensusedintheseventeenthcenturyanextrapolationtechniquethatwasonly rediscovered by Richardson in 1910, whose name has been attached to themethod. AnotherextrapolationmethodwasdescribedbyWernerRombergin1955. He tried to improve on the speed of convergence of approximations to anintegralgeneratedbythetrapeziumrule.ThismethodisnowcalledRombergintegration. After the introduction of digital computers, the improvements, gen-eralizations, and variations of extrapolation techniques were numerous: Aitken,Steensen,Takakazu,Shanks,Wynn,QDandepsilonalgorithmsarecertainlyfamiliar to most numerical analysts. Read more about this in Brezinskis contri-bution of this book.Claude Brezinski (1941) is emeritus professor at the University of Lilleand is active on many related subjects among which extrapolation meth-ods but also Pade approximation, continued fractions, orthogonal poly-nomials, numerical linear algebra and nonlinear equations. He has alwaysshown a keen interest in the history of science, about which he has severalbooks published. The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.Preface viiThe talk of James Lyness connected neatly with the previous talk. Rombergintegration for one-dimensional integration also got applications in more-dimen-sional integration, but the rst applications only came in 1975 for integrals over asimplex and integrands with a singularity. Nowadays this has become an eleganttheory forintegration offunctionswithanalgebraicorlogarithmicsingularityinsomeverticesof apolyhedral domainof integration. Theintegrationrelieson three elements. Suppose we have a sequence of quadrature formulas Q[m]{f}that converge to the exact integral:I{f} = Q[]{f}. First one has to write anasymptotic expansion of the quadrature around m = . For example Q[m]{f} =B0 +B1/m2+B4/m4+ +B2p/m2p+R2p(m) withB0 = I{f}. This is justan example and the form of the expansion should be designed in function of thesingularityof theintegrand. Next, thisisevaluatedforsayndierentvaluesof m,whichresultsinasystemof nlinearequationsthatiseventuallysolvedfor theBk, in particularB0 =I{f}. The moral of the story is that for furtherdevelopment of multidimensional extrapolation quadrature one only needs threesimple elements: a routine to evaluate the integrand, a routine implementing thequadraturerule, andasolverforalinearsystem. Ofcoursethemostdicultand most creative part is to nd the appropriate expansion.The contribution of Lyness is included in these proceedings.James Lyness (1932) is employed at the Argonne National Laboratoryand the University of New South Wales. His rst publications appearedmainlyinphysics journals, but sincehepublishedhis rst paper onN-dimensional integration (co-authored by D. Mustard and J.M. Blatt)inTheComputerJournal in1963, hehasbeenaleadingauthorityinthis domain with worldwide recognition.5 FunctionalequationsThe afternoon of the rst day was devoted to functional equations.From an historical point of view, we could say that the method of Euler forthe solution of ordinary dierential equations (1768) is the seed from which allthe other methods were deduced. That is how it was presented in the lecture ofGerhard Wanner. Runge, Heun and Kutta published their methods around 1900.These were based on a Taylor series expansion of the solution and the idea wasto approximate it in the best possible way. This means that for a small steph,the dierence between the true and the approximating solutions was O(hp) withorderp as high as possible. Such an approach quickly leads to very complicatedsystemsofequationsdeningtheparametersofthemethod. Thereforeinthesixties and seventies of the previous century, much eort was put in a systematictreatment by, e.g., Butcher, Hairer and Wanner.On the other hand, multistep methods are among the ospring of techniquesby Adams and Bashforth which date back to 1885. These predict the next valueof thesolutionusingnotonlythelastcomputedpoint, suchasRunge-Kuttamethods do, but they use several of the previously computed points to make the The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.viii Prefaceprediction. Dahlquist published in 1956 the generalized linear multistep methods.Important eorts have been made to improve the step control and the stability.GerhardWanner(1942)isprofessorattheUniversityofGen`eveandex-president of the Suisse Mathematical Society. He wrote together withHairer several books on analysis and dierential equations. The historicalaspects always played an important role. He has had scientic contactsat all levels from 2 meter below sea level (the Runge-Kutta symposiumat the CWI in Amsterdam on the occasion of 100 years of Runge-Kuttamethods in 1995) to the top of the Mont Blanc at 4807 meters above sealevel where he hiked together with Hairer.The further development was picked up in the talk by Rolf Jeltsch. His maintopic was the evolution of the concept of stability when solving sti dierentialequations.Sti dierential equations form a problem for numerical solution methods be-cause the dynamics of the solution have components at quite dierent scales. Re-searchers wanted to design methods that computed numerically stable (bounded)solutionswhenthetheoretical solutionwassupposedtobestable. Dahlquistprovedin1963hiswellknownsecondbarrierformultistepmethods.Itstatedthat there was not an A-stable method of order higher than two. The A standsfor absolute, which means that thenumerical method computed astable solu-tion, whatever the step size is. This started a quest for other types of methodsand gave rise to a whole alphabet of weaker types of stability.Rolf Jeltsch (1945) is professor at the ETH Z urich. He is a former pres-identoftheEMS(1999-2002)andoftheSuisseMathematical Society(2002-2003). Since2005heispresidentof theGesellschaftf urAnge-wandteMathematikundMechanik(GAMM). Inthenineteenseven-tieshismainresearchtopicwasordinarydierential equations. Sincethe nineteen eighties, he focussed more on hyperbolic partial dierentialequations and large scale computations in engineering applications.Unfortunately, thecontributionsbyWannerandJeltschcouldnotbein-cludedintheseproceedingsbuttheeditorswerehappytondavaluablere-placement tocover theareaof dierential equations. JohnButcherprovided atext in which he reports on the contribution of New Zealanders, which includeshis own, to numerical analysis in general and dierential equations in particular.SohelinksuptheEuropeanandNewZealandnumerical scene. Hispersonalreminiscences bring about a broader historical perspective.John Butcher (1933) is professor emeritus at the Department of Math-ematics, of theUniversityof Auckland. Hismaininterestsareinthenumerical solution methods for ordinary dierential equations. He is con-sidered to be the worlds leading specialist on Runge-Kutta methods. His1987bookonthenumerical solutionofordinarydierential equationsand their subsequent editions of 2003 and 2008, are considered the bestreference work on this subject. The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.Preface ixHerbert Keller was the dean of the company. He replaced Philip Davis, whorst agreed to attend but eventually was not able to come to the meeting. ThemessageofKellerwasthatsingularitieshavealwaysplayedanimportantroleinnumericalcomputationsandthattheywerenotalwaysgiventheattentiontheydeserve. Thisstartswithsuchasimplethingasdividingbyzero(orbysomethingalmostzero).Thisisobviouslyafundamentalissue.InGaussianelimination for solving a linear system of equations, dividing by a small diagonalelementmaycompletelydestroytheaccuracyof thealgorithm. But, itisasimportanttotakecareof asingularityof anintegrandwhendesigninggoodnumerical quadrature formulas. This was already shown in the talk by Lyness.AnotherexampleisencounteredwhensolvingnonlinearequationswheretheJacobian evolves during the iteration towards a matrix that is (almost) singular.This kind of diculties certainly appears in more complex large scale problemsconnected with dierential or integral equations, dynamical systems etc.Herb Keller was in excellent shape during the symposium and full of travelplans. However, few months after the symposium, we received the sad news thatKeller had passed away on January 26, 2008. So there is no contribution by himabouthistalk. HinkeOsinga, whohadaninterviewwithKellerpublishedintheDSWebMagazine, waskindenoughtoslightlyadapthertextandthisisincluded instead.Herbert Keller (1925 2008) was emeritus professor at the CaliforniaInstitute of Technology. Together with E. Isaacson, he is the author of thelegendary book Analysis of Numerical Methods that was published in1966 by J. Wiley. His scientic contributions mainly dealt with boundaryvalue problems and methods for the solution of bifurcation problems.TherstdaywasconcludedbythelectureofKendall Atkinsonwhospokeabout his personal vision on the evolution in research related to the solution ofintegral equations. Heemphasizedtheuseoffunctional analysisandoperatortheoryfortheanalysisof numerical methodstosolvethiskindof equations.TheoriginpointstoapaperbyKantorovichFunctionalanalysisandappliedmathematicsthatappearedinRussianin1948. ItdealsamongotherthingswiththesolutionofFredholmintegralequationsofthesecondkind.Atkinsonsummarizesthemostimportantmethodsandtheresultsthatwereobtained:degeneratekernel approximationtechniquesinwhichthekernel iswrittenasK(s, t) =

i i(s)i(t), projection methods (the well known Galerkin and col-location methods where the solution is written as a linear combination of basisfunctionswithcoecientsthatarexedbyinterpolationorbyorthogonalityconditions), andtheNystrommethodthatisbasedonnumerical integration.The details of his lecture can be found in these proceedings.Kendall Atkinson (1940) is emeritus professor at the University of Iowa.He is an authority in the domain of integral equations. His research en-compasses radiosity equations from computer graphics and multivariateapproximation, interpolation and quadrature. He wrote several books onnumerical analysis and integral equations. The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.x Preface6 Theimportanceof softwareandtheinuenceofhardwareTherstlectureoftheseconddaywasgivenbyBrianFord. Hesketchedthestart and the development of the NAG (Numerical Algorithms Group) softwarelibrary. Thatlibrarywastherstcollectionofgeneral routinesthatwerenotjust focussing on one particular kind of numerical problem or on one particularapplicationarea. Alsonewwasthatitcameoutof thejointeortof severalresearcherscomingfromdierentgroups. Fordstarteduphislibraryin1967,stimulated by his contacts with J. Wilkinson and L. Fox. Being generally appli-cable,welltested,andwithgooddocumentationitwasanimmediatesuccess.The ocial start of the Algol and Fortran versions of the NAG library is May 13,1970. The algorithms are chosen on the basis of stability, robustness, accuracy,adaptability and speed (the order is important). Ford then tells about the fur-ther development and the choices that had to be made while further expandingthe library and how this required an interplay between numerical analysts andsoftwaredesigners.Heconcludeshistalkwithanappeal toyoungresearchersto work on the challenge put forward by the new computer architectures wheremulticore hardware requires a completely new implementation of the numericalmethods if one wants to optimally exploit the computing capacity to cut downon computer time and hence to solve larger problems.ThecontributionofB.Fordisincludedintheseproceedings.Moreonthe(r)evolution concerning hardware and its inuence on the design and implemen-tation of numerical software can be found in the next contribution byJ. Dongarra.Brian Ford (1940) is the founder of the NAG company and was directoruntil hisretirementin2004. HereceivedahonorarydoctorateattheUniversityof BathandwasgivenaOBE(Ocerof BritishEmpire)forhisachievements. Underhisleadership, NAGhasdevelopedintoarespectedcompanyfortheproductionofportableandrobustsoftwarefor numerical computations.The conclusion of B. Ford was indeed the main theme in the lecture of JackDongarra. In his lecture he describes how, since about 1940, the development ofnumerical software, thehardware, andtheinformaticstoolsgohandinhand.Earlysoftwarewasdevelopedonscalararchitectures(EISPACK, LINPACK,BLAS 70) then came vector processors (70-80) and parallel algorithms, MIMDmachines(ScaLAPACK90), andlaterSMP, CMP, DMP, etc. Themulticoreprocessors are now a reality and a (r)evolution is emerging because we will haveto deal with multicore architecture that will have hundreds and maybe thousandsof cores. Using all this potential power in an ecient way by keeping all theseprocessors busy, will require a total re-design of numerical software.A short version of the lecture, concentrating on a shorter time-span, has beenincluded in these proceedings. The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.Preface xiJack Dongarra (1950) is professor at the University of Tennessee whereheistheleaderoftheInnovativeComputingLaboratory.Heisspe-cializedinlinearalgebrasoftwareandmoregenerallynumerical soft-wareonparallel computersandotheradvancedarchitectures. In2004he received the IEEE Sid Fernbach Award for his work in HPC (HighPerformance Computing). He collaborated on the development of all theimportantsoftwarepackages:EISPACK,LINPACK,BLAS,LAPACK,ScaLAPACK, etc.7 ApproximationandoptimizationMore software in the lecture of RobertPlemmons. The thread through his lec-ture is formed by nonnegativity conditions when solving all kinds of numericalproblems. First a survey is given of historical methods for the solution of non-negative least squares problem. There one wants to solve a linear least squaresproblem where the unknowns are all nonnegative. Also the factorization of twononnegative matrices (NMF) was discussed. The latter is closely connected withdata-analysis. OthertechniquesusedhereareSVD(singularvaluedecompo-sition)andPCA(principal componentanalysis). Thesehoweverdonottakethe nonnegativity into account. Around the nineteen nineties ICA (independentcomponentanalysis)wasintroducedforNMF.ThiscanalsobeformulatedasBSS (blind source separation). In that kind of application, a mixture of severalsources is observed and the problem is to identify the separate sources. In morerecent research, one tries to generalize NMF by replacing the matrices by ten-sors. There are many applications: lter out background noise from an acousticsignal, ltering of e-mails, data mining, detect sources of environment pollution,space research SOI (space object identication) etc.Read more about this in the contribution by Chen and Plemmons in theseproceedings.Robert J. Plemmons (1938) is Z. SmithReynolds professor at theWake Forest University, NC. His current researchincludes computa-tional mathematics withapplications insignal andimageprocessing.For example, images that are out of focus are corrected, or atmosphericdisturbances are removed. He published more than 150 papers and threebooks about this subject.Michael Powell gaveasurveyof thesuccessivevariants of quasi-Newtonmethods or methods of variable metric for unconstrained nonlinear optimization.He talked about his contribution to their evolution of the prototypes that weredesigned in 1959. These methods were a considerable improvement over previousmethods that were popular before that time which were dominated by conjugategradients, pure Newton, and direct search methods. In these improved methods,an estimate for the matrix of second derivatives is updated so that it does nothavetoberecomputedineveryiterationstep. Thesemethodsconvergefastandtheunderlyingtheoryisquitedierentfromthecorrespondingtheoryfor The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.xii PrefaceapureNewtonmethod. MikePowell thenevokedmethodsderivedfromtheprevious ones that are important in view of methods for optimization problemswithconstraints. Furthermethodsdiscussedarederivativefreemethods, andthe proliferation of other types like simulated annealing, genetic algorithms etc.A summary of this presentation can be found in these proceedings.Michael J.D. Powell (1936)isprofessoratCambridgeUniversity. Hehas been active in many domains. His name is for example attached tothe DFP (Davidon-Fletcher-Powell) method (a quasi-Newton method foroptimization), and a method with his name that is a variant of the Mar-quardt method for nonlinear least squares problems. But he is also wellknownforhisworkinapproximationtheory.ThePowell-Sabinsplinesare still an active research area in connection with subdivision schemesfor the hierarchical representation of geometric objects or in the solutionof partial dierential equations.8 AndsomehistoryThe closing lecture of the symposium was given by Alistair Watson. With somepersonal touch, he sketched the early evolution of numerical analysis in Scotland.In a broader sense, numerical analysis has existed for centuries of course. If yourestrict it to numerical analysis as it was inuenced by digital computers, then1947 is a good choice to call that the start. But thinking of the more abstractconnection between numerics and computers, then one should probably go backto 1913 when papers by Turing were published. That is where Watson starts hisaccount of the history. More concretely, in Scotland, the start is associated withtheworkofWhittakerandlaterAitkenwhowereappointedinEdinburgh.IntheMathematicalLaboratory,foundedin1946,computationsweredonebymeansofpocketcalculators. Itwasonlyin1961thatthiswascoinedtobeaNumerical Analysis course. In that year Aitken claimed to have no need for adigital computer. It arrived anyway in 1963. Then things start to move quickly.More numerical centers came into existence in St Andrews and later in Dundee.The University of Dundee became only independent of St Andrews in 1967 andsoon the gravity center of numerical analysis had moved to Dundee. From 1971until 2007, the biennial conference on numerical analysis was held in Dundee. Inthe most recent years, the University of Strathclyde (Glasgow) seems to be thenew attraction pole for numerical analysis in Scotland.A longer write-up of this historical evolution is included in these proceedings.Thefocusofthework ofAlistairWatson(1942)isnumericalapprox-imationandrelatedmatters.Thiscanbetheoreticalaspects, butalsoelementsfromoptimizationandlinearalgebra. HeisFRSE(Fellowofthe Royal Society of Edinburgh) and he is probably best known by manyfor his involvement in the organization of the Dundee conferences on nu-merical analysis. The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.Preface xiii9 AndthereismoreOftheelevenlecturersatthesymposium,theyoungestwas57andtheoldest82,withanaveragejustover68.Allofthemshowedalotofenthusiasmandmade clear that whatever the exact age of numerical analysis, be it sixty yearsor a hundred years or even centuries, there still remains a lot to be done and thechallenges of today are greater than ever.Of course we would have liked to have invited many more people to lectureandcovermoresubjectsatthesymposium, buttimeandbudgetwasnite.Someimportantpeoplewhohadplannedtocome, nallydecidedfordiversereasons to decline. One of these was Gene Golub (1932 2007), the undisputedgodfather of numerical linear algebra, who, sad to say, passed away just after thesymposium on November 19, 2007. Another name we would have placed on ourlist as a speaker for his contributions to numerical integration would have beenPhilipRabinowitz(19262006)hadhestillbeamongus.WearefortunatetohaveobtainedpermissionoftheAmericanMathematicalSocietytoreprintan obituary with reminiscences of Phil Davis and Aviezri Fraenkel that was rstpublished in the Notices of the AMS in December 2007.Phil Davis also wrote up some personal reminiscences of what it was like inthe very early days, when numerical analysis was just starting in the years beforeand during WW-II.Philip Davis (1923), currently emeritus professor at Brown University.He is well known for his work in numerical analysis and approximationtheory, butwithhismanycolumns, andbooks, healsocontributedalot to the history and philosophy of mathematics. His books on quadra-ture(togetherwithPh.Rabinowitz)andoninterpolationandapprox-imationareclassics. HealsocollaboratedontheAbramowitz-Stegunproject Handbook of Mathematical Functions. He started his career as aresearcher in the Air Force in WW-II, and joined the National Bureauof Standards before going to Brown.Finally, we found Robert Piessens kind enough to write another contributionforthisbook. Hisapproachisagainhistorical andsketchesrsttheworkofChebyshev about linkage instruments, a mechanical tool to transform a rotationintoastraightline. ThisishowChebyshevpolynomialscameabout. Hecon-tinues by illustrating how the use of Chebyshev polynomials has inuenced theresearch of his group at K.U.Leuven, since it turned out to be a powerful tool indevelopingmethodsforthenumericalsolutionofseveral problemsencompass-ing inversion of the Laplace transform, the computation of integral transforms,solution of integral equations and evaluation of integrals with a singularity. Thelatterresultedinthedevelopmentof theQUADPACKpackagefornumericalintegration.Robert Piessens (1942) is emeritus professor at the Department of Com-puter Science at the K.U.Leuven, Belgium. He was among the rst profes-sors who started up the department. His PhD was about the numerical The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.xiv Prefaceinversionof theLaplacetransform. Heisoneof thedevelopersoftheQUADPACK package which has been a standard package for automaticnumerical integration in one variable. Originally written in Fortran 77,it has been re-implemented in dierent environments. It is available vianetlib,severalofitsroutineshavebeenre-codedandareintegratedinOctave and Matlab, the Gnu Scientic Library (GSL) has a C-version,etc.10 AcknowledgementsThe symposium was sponsored by the FWO-Vlaanderen and the FNRS, the re-gional science foundations of Belgium. We greatefully acknowledge theirsupport.Leuven, March 2009 Adhemar Bultheel and Ronald CoolsFromlefttoright:H.Keller,C.Brezinski,G.Wanner,B.Ford,A.Watson,K.Atkinson,R.Jeltsch,R.Plemmons,M.Powell,J.Lyness,J.Dongarra.Infront:theorganizers:R.CoolsandA.Bultheel The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.Preface xvG.Wanner,C.Brezinski,M.Powell J.Dongarra,B.Ford,H.KellerJ.Lyness,R.Jeltsch B.Plemmons,M.PowellR.Jeltsch,A.Watson B.Ford,K.Atkinson The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.This page intentionally left blank This page intentionally left blank The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.TableofContentsPreface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vSome pioneers of extrapolation methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Claude BrezinskiVery basic multidimensional extrapolation quadrature. . . . . . . . . . . . . . . . . 23James N. LynessNumerical methods for ordinary dierential equations: early days . . . . . . . 35John C. ButcherInterview with Herbert Bishop Keller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45Hinke M. OsingaApersonalperspectiveonthehistoryofthenumericalanalysisofFredholm integral equations of the second kind. . . . . . . . . . . . . . . . . . . . . . . 53Kendall AtkinsonMemoires on building a general purpose numerical algorithms library . . . . 73Brian FordRecent trends in high performance computing . . . . . . . . . . . . . . . . . . . . . . . . 93JackJ.Dongarra,HansW.Meuer,HorstD.Simon,andErichStrohmaierNonnegativity constraints in numerical analysis. . . . . . . . . . . . . . . . . . . . . . . 109Donghui Chen and Robert J. PlemmonsOn nonlinear optimization since 1959 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141M. J. D. PowellThehistoryanddevelopmentofnumericalanalysisinScotland:apersonal perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161G. Alistair WatsonRemembering Philip Rabinowitz. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179Philip J. Davis and Aviezri S. FraenkelMy early experiences with scientic computation . . . . . . . . . . . . . . . . . . . . . . 187Philip J. DavisApplications of Chebyshev polynomials: from theoretical kinematics topractical computations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193Robert PiessensName Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207Subject Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215xvii The Birth of Numerical Analysis Downloaded from www.worldscientific.comby 116.203.74.250 on 11/13/14. For personal use only.