This page intentionally left blank - Julian Oliver · · 2009-10-01Introducing students to...

http://www.cambridge.org/9780521828628

This page intentionally left blank

Introductory Computational Physics

Computers are one of the most important tools available to physicists,whether for calculating and displaying results, simulating experiments, orsolving complex systems of equations.

Introducing students to computational physics, this textbook shows how touse computers to solve mathematical problems in physics and teachesstudents about choosing different numerical approaches. It also introducesstudents to many of the programs and packages available. The book reliessolely on free software: the operating system chosen is Linux, which comeswith an excellent C++ compiler, and the graphical interface is the ROOTpackage available for free from CERN.

This up-to-date, broad scope textbook is suitable for undergraduates startingon computational physics courses. It includes exercises and many examplesof programs. Online resources at www.cambridge.org/9780521828627feature additional reference information, solutions, and updates on newtechniques, software and hardware used in physics.

Andi Klein is a Technical Staff member at Los Alamos NationalLaboratory, New Mexico. He gained his Ph.D. from the University ofBasel, Switzerland. He held the position of Professor of Physics at OldDominion University, Virginia, from 1990 to 2002, where he taught coursesin computational physics.

Alexander Godunov is Assistant Professor at the Department of Physics,Old Dominion University, Virginia. He gained his Ph.D. from Moscow StateUniversity, Russia and has held research positions at Tulane University,Louisiana, and visiting positions at research centers in France and Russia.

Introductory ComputationalPhysics

Andi Klein and Alexander GodunovLos Alamos National Laboratoryand Old Dominion University

cambridge university pressCambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo

Cambridge University PressThe Edinburgh Building, Cambridge cb2 2ru, UK

First published in print format

isbn-13 978-0-521-82862-8

isbn-13 978-0-521-53562-5

isbn-13 978-0-511-16650-1

© Cambridge University Press 2006

Information on this title: www.cambridge.org/9780521828628

This publication is in copyright. Subject to statutory exception and to the provision ofrelevant collective licensing agreements, no reproduction of any part may take placewithout the written permission of Cambridge University Press.

isbn-10 0-511-16650-8

isbn-10 0-521-82862-7

isbn-10 0-521-53562-x

Cambridge University Press has no responsibility for the persistence or accuracy of urlsfor external or third-party internet websites referred to in this publication, and does notguarantee that any content on such websites is, or will remain, accurate or appropriate.

Published in the United States of America by Cambridge University Press, New York

www.cambridge.org

hardback

eBook (Adobe Reader)

eBook (Adobe Reader)

hardback

http://www.cambridge.org

http://www.cambridge.org/9780521828628

Contents

Preface page ix

1 Introduction 11.1 The need for computers in science 11.2 What is computational physics? 11.3 Linux and C++ 2

2 Basics 52.1 Basic computer hardware 52.2 Software 72.3 How does it work? 9

3 Short introduction to Linux 113.1 Getting started and logging in 113.2 Getting help 123.3 The filesystem, or where is everything? 123.4 Moving around in your system 133.5 Listing your directory 143.6 Creating your own files 153.7 Doing some work 173.8 Good programming 193.9 Machine representation and precision 203.10 Exercises 23

4 Interpolation 254.1 Lagrange interpolation 274.2 Neville’s algorithm 294.3 Linear interpolation 304.4 Polynomial interpolation 31

v

vi Contents

4.5 Cubic spline 334.6 Rational function interpolation 344.7 Exercises 35

5 Taking derivatives 375.1 General discussion of derivatives with computers 375.2 Forward difference 385.3 Central difference and higher order methods 385.4 Higher order derivatives 405.5 Exercises 40

6 Numerical integration 416.1 Introduction to numerical integration 416.2 The simplest integration methods 426.3 More advanced integration 446.4 Exercises 49

7 Solution of nonlinear equations 517.1 Bisection method 517.2 Newton’s method 527.3 Method of secants 527.4 Brute force method 537.5 Exercises 53

8 Differential equations 558.1 Introduction 558.2 A brush up on differential equations 558.3 Introduction to the simple and modified Euler methods 578.4 The simple Euler method 588.5 The modified Euler method 628.6 Runge–Kutta method 658.7 Adaptive step size Runge–Kutta 708.8 The damped oscillator 728.9 Exercises 81

9 Matrices 839.1 Linear systems of equations 839.2 Gaussian elimination 849.3 Standard libraries 86

Contents vii

9.4 Eigenvalue problem 869.5 Exercises 88

10 Random processes and Monte Carlo simulation 8910.1 Random processes in science 8910.2 Random number generators 9010.3 The random walk 9210.4 Random numbers for nonuniform distributions 9710.5 Monte Carlo integration 10110.6 Exercises 103

References 105

Appendix A The ROOT system 107A.1 What is ROOT 107A.2 The ROOT basics 107A.3 The first steps 108A.4 Lab ROOT 113A.5 Exercises 115

Appendix B Free scientific libraries 117B.1 LAPACK 117B.2 SLATEC 118B.3 Where to obtain ROOT 118

Appendix C FORTRAN and C++ 119C.1 Calling FORTRAN from C++ 120

Appendix D Program listings 121D.1 Simple Euler 121D.2 Runge–Kutta program 123D.3 Random walk in two dimensions 131D.4 Acceptance and rejection method with sin(x) distribution 134

Index 137

Preface

Computers are one of the most important tools in any field of science andespecially in physics. A student in an undergraduate lab will appreciate thehelp of a computer in calculating a result from a series of measurements.The more advanced researcher will use them for tasks like simulating anexperiment, or solving complex systems of equations. Physics is deeplyconnected to mathematics and requires a lot of calculational skills. If one isonly interested in a conceptual understanding of the field, or an estimate ofthe outcome of an experiment, simple calculus will probably suffice. We cansolve the problem of a cannon ball without air resistance or Coriolis force withvery elementary math, but once we include these effects, the solution becomesquite a bit more complicated. Physics, being an experimental science, alsorequires that the measured results are statistically significant, meaning wehave to repeat an experiment several times, necessitating the same calculationover and over again and comparing the results. This then leads to the questionof how to present your results. It is much easier to determine the compatibilityof data points from a graph, rather than to try to compare say 1000 numberswith each other and determine whether there is a significant deviation. Fromthis it is clear that the computer should not only “crunch numbers,” but shouldalso be able to display the results graphically.

Computers have been used in physics research for many years and thereis a plethora of programs and packages on the Web which can be used tosolve different problems. In this book we are trying to use as many of theseavailable solutions as possible and not reinvent the wheel. Some of thesepackages have been written in FORTRAN, and in Appendix C you will find adescription of how to call a FORTRAN subroutine from a C++ program. Aswe stated above, physics relies heavily on graphical representations. Usually,the scientist would save the results from some calculations into a file, whichthen can be read and used for display by a graphics package like gnuplot ora spreadsheet program with graphics capability. We have decided to pursue

ix

x Preface

a different path, namely using the ROOT package [1] developed at the highenergy physics lab CERN in Switzerland. ROOT, being an object orientedC++ package, not only provides a lot of physics and math C++-classes butalso has an excellent graphics environment, which lets you create publicationquality graphs and plots. This package is constantly being developed andnew features and classes are being added. There is an excellent user’s guide,which can be found on the ROOT website in different formats. In order toget started quickly we have given a short introduction in Appendix A.

Chapter 1Introduction

1.1 The need for computers in science

Over the last few decades, computers have become part of everyday life.Once the domain of science and business, today almost every home has a per-sonal computer (PC), and children grow up learning expressions like “hard-ware,” “software,” and “IRQ.” However, teaching computational techniquesto undergraduates is just starting to become part of the science curriculum.Computational skills are essential to prepare students both for graduate schooland for today’s work environment.

Physics is a corner-stone of every technological field. When you have asolid understanding of physics, and the computational know-how to calculatesolutions to complex problems, success is sure to follow you in the high-techenvironment of the twenty-first century.

1.2 What is computational physics?

Computational physics provides a means to solve complex numerical prob-lems. In itself it will not give any insight into a problem (after all, a computeris only as intelligent as its user), but it will enable you to attack problemswhich otherwise might not be solvable. Recall your first physics course. Atypical introductory physics problem is to calculate the motion of a cannonball in two dimensions. This problem is always treated without air resistance.One of the difficulties of physics is that the moment one goes away fromsuch an idealized system, the task rapidly becomes rather complicated. If wewant to calculate the solution with real-world elements (e.g., drag), thingsbecome rather difficult. A way out of this mess is to use the methods ofcomputational physics to solve this linear differential equation.

1

2 Introduction

One important aspect of computational physics is modeling large complexsystems. For example, if you are a stock broker, how will you predict stockmarket performance? Or if you are a meteorologist, how would you try topredict changes in climate? You would solve these problems by employingMonte Carlo techniques. This technique is simply impossible without com-puters and, as just noted, has applications which reach far beyond physics.

Another class of physics problems are phenomena which are represented bynonlinear differential equations, like the chaotic pendulum. Again, computa-tional physics and its numerical methods are a perfect tool to study such sys-tems. If these systems were purely confined to physics, one might argue thatthis does not deserve an extended treatment in an undergraduate course. How-ever, there is an increasing list of fields which use these equations; for exam-ple, meteorology, epidemiology, neurology and astronomy to name just a few.

An advantage of computational physics is that one can start with a simpleproblem which is easily solvable analytically. The analytical solution illus-trates the underlying physics and allows one the possibility to compare thecomputer program with the analytical solution. Once a program has beenwritten which can handle the case with the typical physicist’s approximation,then you add more and more complex real-world factors.

With this short introduction, we hope that we have sparked your interest inlearning computational physics. Before we get to the heart of it, however, wewant to tell you what computer operating system and language we will be using.

1.3 Linux and C++Linux

You may be accustomed to the Microsoft Windows or Apple MAC operatingsystems. In science and in companies with large computing needs, however,UNIX is the most widely used operating system platform. Linux is a UNIX-type operating system originally developed by Linus Torwald which runs onPCs. Today hundreds of people around the world continue to work on thissystem and either provide software updates or write new software. We useLinux as the operating system of choice for this text book because:

• Linux is widely available at no cost;• Linux runs on almost all available computers;• it has long-term stability not achieved by any other PC operating system;• Linux distributions include a lot of free software, i.e., PASCAL, FOR-

TRAN, C, C++.

1.3 Linux and C++ 3

In today’s trend to use networked clusters of workstations for large computa-tional tasks, knowledge of UNIX/Linux will provide you with an additional,highly marketable skill.

C++In science, historically the most widely used programming language wasFORTRAN, a fact reflected in all the mathematical and statistical librariesstill in use the world over (e.g., SLATEC, LAPACK, CERNLIB). One disad-vantage of FORTRAN has always been that it was strongly decoupled fromthe hardware. If you wanted to write a program which would interact directlywith one of the peripherals, you would have to write code in assembly lan-guage. This meant that not only had you to learn a new language, but yourprogram was now really platform dependent.

With the emergence in the late 1970s of C [2] and UNIX, which is writtenin C, all of a sudden a high level language was available which could do both.C allowed you to write scientific programs and hardware drivers at the sametime, without having to use low level processor dependent languages. In themid 1980s Stroustrup [3] invented C++, which extended C’s capabilitiesimmensely. Today C and C++ are the most widely used high level languages.

Having “grown up” in a FORTRAN environment ourselves, we still con-sider this to be the best language for numerical tasks (we can hear a collectivegroan in the C/C++ community). Despite this, we decided to “bite the bullet”and switch to C++ for the course work.

The GNU C/C++ compiler is an excellent tool and quite versatile. Com-pared to the Windows C++ compilers (e.g., Visual C++ [Microsoft] or Bor-land C/C++), the user interface is primitive. While the Windows compiler-packages have an extensive graphical user interface (GUI) for editing andcompiling, the GNU compiler still requires you first to use a text editor andthen to collect all the necessary routines to compile and link. One “disadvan-tage” to the Windows compiler packages is that many of them automaticallyperform a number of tasks necessary to building a program. You might bewondering how that could be a disadvantage. We have noticed that whenstudents have used such packages, they often have a poor understanding ofconcepts like linking, debuggers, and so on. In addition, if a student switchesfrom one Windows compiler package to another, s/he must learn a newenvironment. Therefore, this text will use/refer to the GNU C/C++ com-piler; however the programs can be easily transported to the afore mentionedproprietary systems.

4 Introduction

Having extolled the virtues of C/C++, we must mention here that some ofthe sample programs in this book reflect our roots in FORTRAN. There aremany functions and subroutines available for scientific tasks which have beenwritten in FORTRAN and have been extensively tested and used. It wouldbe foolish to ignore these programs or attempt to rewrite them in C/C++. Itis much less time consuming to call these libraries from C++ programs thanit is to write your own version. In Appendix C we describe how FORTRANlibraries and subroutines can be called from C++.

Chapter 2Basics

Before we start we need to introduce a few concepts of computers and theinteraction between you, the user, and the machine. This will help you decidewhen to write a program for solving a physics or science problem and whenit is much easier or faster to use a piece of paper and a pocket calculator. Inthinking about computers, remember there is a distinction between hardwareand software. Software is divided into the operating system and your partic-ular application, like a spreadsheet, word-processor or a high level language.In this book we will spend most of the time in dealing with issues relevantto physics and the algorithms used to solve problems. However, in order tomake this as productive as possible, we will start off with a short descriptionof the hardware and then some discussion of the operating system.

2.1 Basic computer hardware

Apart from huge parallel supercomputers, all workstations you can buy todayare organized in a similar way (Figure 2.1).

The heart of the computer is the CPU (Central Processing Unit) controllingeverything in your workstation. Any disk I/O (Input/Output) or computationaltask is handled by the CPU. The speed at which this chip can executean instruction is measured in Hz (cycles per second) at several GHz. TheCPU needs places to store data and instructions. There are typically fourlevels of memory available: level I cache, level II cache, RAM (RandomAccess Memory) and swap space, the last being on the hard disk. The maindistinction between the different memory types is the speed.

The cache memory is located on the CPU chip.This CPU chip has two small memory areas (one for data and one for

instructions), called the level I caches (see below for further discussion),

5

6 Basics

Figure 2.1 Schematiclayout of a workstation.

memory

CPU

Internal disk

I/O

Graphics card

Network Interface

BUS

Internal Peripheral

Keyboard

Mouse

Printer

Screen

Ethernet/twisted pair

External Disk, Tapedrive

which are accessed at the full processor speed. The second cache, level II,acts as a fast storage for code or variables needed by the code. However,if the program is too large to fit into the second cache, the CPU will putsome of the code into the RAM. The communication between the CPU andthe RAM is handled by the Bus, which runs at a lower speed than the CPUclock. It is immediately clear that this poses a first bottleneck compared tothe speed the CPU would be able to handle. As we will discuss later, carefulprogramming can help in speeding up the execution of a program by reducingthe number of times the CPU has to read and write to RAM. If this additionalmemory is too small, a much more severe restriction will come into play,namely virtual memory or swap space. Virtual memory is an area on thedisk where the CPU can temporarily store code which does not fit into themain memory, calling it in when it is needed. However, the communicationspeed between the CPU and the virtual memory is now given by the speedat which a disk can do I/O operations.

The internal disk in Figure 2.1 is used for storing the operating systemand any application or code you want to keep. The disk size is measured ingigabytes (GB) and 18–20 GB disks are standard today for a workstation.Disk prices are getting cheaper all the time thus reducing the need for codewhich is optimized for size. However the danger also is that people cluttertheir hard disk and never clean it up.

Another important part of your system is the Input/Output (I/O) system,which handles all the physical interaction between you and the computer aswell as the communication between the computer and the external peripherals(e.g., printers). The I/O system responds to your keyboard or mouse but will

2.2 Software 7

also handle print requests and send them to the printer, or communicate withan external or internal tape or hard drive.

The last piece of hardware we want to describe briefly is the network inter-face card, which establishes communication between different computers overa network connection. Many home users connect to other computers through amodem, which runs over telephone lines. Recently cable companies have startedto offer the use of their cable lines for network traffic, allowing the use of fastercable modems. The telephone modem is currently limited to a maximum speedof 56 kB/s, which will be even slower if the line has a lot of interferences. In ourenvironment we are using an Ethernet network, which runs at 100 MB/s.

2.2 Software

Operating system

In getting your box to do something useful, you need a way to communicatewith your CPU. The software responsible for doing this is the operatingsystem or OS. The operating system serves as the interface between theCPU and the peripherals. It also lets you interact with the system. It usedto be that different OSs were tied to different hardware, for example VMSwas Digital Equipment’s (now Hewlett-Packard) operating system for theirVAX computers, Apple’s OS was running on Motorola chips and Windows3.1 or DOS was only running on Intel chips. This of course led to softwaredesigners concentrating on specific platforms, therefore seriously hamperingthe distribution of code to different machines. This has changed and todaywe have several OSs which can run on different platforms.

One of the first operating systems to address this problem was UNIX,developed at the AT&T Labs. Still a proprietary system, it at least enableddifferent computer manufacturers to get a variant of UNIX running on theirmachines, thus making people more independent in their choices of comput-ers. Various blends of UNIX appeared as shown in the following table:

Ultrix Digital Equipment Corporation (now HP)

HP True64 Unix (formerly OSF) HP

Solaris/SunOS SUN

AIX IBM

HP-Unix Hewlett-Packard

8 Basics

These systems are still proprietary and you cannot run HP-Unix on a Sunmachine and vice versa, even though for you as a user they look very similar.In the 1980s Linus Torwald started a project for his own amusement calledLinux, which was a completely free UNIX system covered under the GNUlicense. Since then many people have contributed to Linux and it is now amature and stable system. In the scientific community Linux is the fastestgrowing operating system, due to its stability and low cost and its ability torun on almost all computer platforms. You can either download Linux overthe internet from one of the sites listed in the appendix, or you can buy oneof the variants from vendors like Red Hat or SuSE. The differences in thesedistributions are in ease of installation, graphical interfaces and support fordifferent peripherals. However the kernel, the heart of the operating system,is the same for all.

Unlike other operating systems, when you get Linux, you also get thecomplete source code. Usually, you do not change anything in the codefor Linux, unless either you are very knowledgeable (but read the licenseinformation first) or you want to get into trouble really fast.

Two other advantages of Linux are that there is a lot of free applicationsoftware and it is a very stable system. The machines in our cluster run formonths without crashing or needing to be rebooted.

Applications and languages

This is the part the average user is most familiar with. Most users buy appli-cations out of the box, consisting of business software like spreadsheets, wordprocessors and databases. For us the most important issue is the programminglanguage. These are the languages you can use to instruct your computer todo certain tasks and are usually referred to as high level languages in contrastto assembly language.

We usually distinguish high level languages in the following way: inter-preted languages like Basic, Perl, awk and compiled ones like FORTRAN,FORTRAN90, C and C++. The distinction, however, is not clear cut; thereare C-interpreters and compiled versions of Perl. The interpreted languagesexecute every line the moment it is terminated by a carriage return and thenwait for the next line. In a way this is similar to your pocket calculator,where you do one operation after the next. This is a very handy way ofdoing some calculations but it will pose some serious restrictions, especiallywhen you try to solve more complex problems or you want to use libraries or

2.3 How does it work? 9

functions which have been previously written. Another disadvantage is theslow running of the program and the lack of optimization.

In a compiled language you first write your complete code (hopefullywithout error), gather all the necessary functions (which could be in libraries)and then have the computer translate your entire program into machinelanguage. Today’s compilers will not only translate your code but will alsotry to optimize the program for speed or memory usage. Another advantage ofthe compiler is that it can check for consistency throughout the program, andtry to catch some errors you introduced. This is similar to your sophisticatedword processor, which can catch spelling and even some grammar mistakes.However, as the spell checker cannot distinguish between two or too (bothwould be fine) or check whether what you have written makes sense, so thecompiler will not be able to ensure consistency in your program. We willdiscuss these issues further below, when we give our guidelines for goodprogramming practice.

After you have run your different routines through the compiler, the laststep is the linker or loader. This step will tie together all the different parts,reserve space in memory for variables, and bind any needed library to yourprogram. On Linux the last step is usually executed automatically whenyou invoke the compiler. The languages most used in scientific computing(especially in physics) are FORTRAN and C/C++. Traditionally FORTRANwas the language of choice, and still today there is a wealth of programsreadily available in FORTRAN libraries (e.g. CERN library, SLATEC,LAPACK). During the last decade, C/C++ has become more and moreimportant in physics, so that this book focuses on C++ (sigh!) and movesaway from FORTRAN. We are still convinced that FORTRAN is the betterlanguage for physics, but in order to get the newer FORTRAN90/95 compiler,one has to buy a commercial package, while the C and C++ compilers areavailable at no cost for Linux.

2.3 How does it work?

In Figure 2.2 we have outlined how the different layers on a UNIX work-station can be grouped logically. The innermost part is the kernel, whichcontrols the hardware, where the services are the part of the system whichinteracts directly with the kernel. Assembly language code will interact withthis system level. The utilities layer contains programs like rm (remove) orcp (copy) and the compilers. The user interface and program development

10 Basics

Figure 2.2 Schematiclayout of a workstation.

User Interface

Utilities & Tools

System Services

Kernel

Program Development F77, C++, nedit ...

Shell: Csh, bash

Assembler

Drivers

areas are where you will be working most. You can choose the particularshell you prefer, which then will be your interface to the lower levels. Eventhough you will be in an X-Window environment, you still have to use com-mand line input and write scripts, which will automate your tasks. This isdone in your chosen shell, and some of the commands will be different fordifferent shells. In the outermost shell you will have your applications, likecompiled programs.

Chapter 3Short introduction to Linux

Unix: The world’s first computer virusfrom The UNIX-Haters Handbook [4]

3.1 Getting started and logging in

We will try to jump-start you into the Linux environment. The first thing youhave to do is log into the system. Since Linux is a real multi-user system, theinteraction between you and the computer might be different than what youare used to from a Microsoft or Macintosh environment. You could be either atthe computer console or at a terminal, which is connected via a network to thecomputer. In either way, you will see a Windows-like screen which will dis-play a login screen, asking you for the username and the password. Assumingthat your system manager has set you up already, you will type in both, and aslong as you did not mistype anything you should now be in the computer. Incase you made a mistake in typing in either of the two items, the computer willnot let you in. (Note that Linux is case sensitive, so Emma is not the same asemma.) Depending on the setup of your computer, you will now be faced witha graphical user interface (GUI), the most common of these being either KDEor Gnome. Click on the small icon which resembles a terminal. This will bringa new window, which lets you type in commands, somewhat like the com-mand icon in DOS. If this is the first time you have logged into this account,you should change your password, especially if your system administratorhas assigned one to you. The normal Linux command for changing the pass-word is passwd, which you will have to type in. In our PC-farm environmentwe use the Network Information System, formerly known as the YP system,which has the password file for the whole cluster centralized. In this case youhave to type in yppasswd, then answer the questions for the old password (sonobody unauthorized can change yours) and give the new password twice:

passwdOld Password:New Password:New Password:

11

12 Short introduction to Linux

Make sure that your password is eight characters long and use special,lower and upper case characters. As an example of a bad password emma, aname comes to mind. However you can make this into a good password byusing $e5Mm%a!.

3.2 Getting help

Now that you are set up you can start using Linux. The first thing youwill probably do is cry for help. Linux, as every good UNIX system, isnot very good in responding to your needs for help, and provides you witha completely archaic system to learn about specific commands. (If you aregetting really disappointed with your lack of progress, log out for a while, getthe UNIX-Haters Handbook [4], and read some professionals’ opinions andfrustations about UNIX.) In Linux, to get help you use the man command,which will let you look up the specifics of a command. The man stands formanual and is organized in different sections. However, the man commandhas its own page, so the first thing you want to do is

man man

which will explain how to use the man system and what kind of argumentsand options the man command takes.

Most of the commands in Linux have options which change the behaviorof the command or give you additional information. One important commandis the man -k blabla, which will look up in the manual database anythingthat matches blabla. So in case you cannot remember the exact word youcan try to find it with the man -k command. But let us warn you, someof the commands have very unintuitive names like awk or cat. Anotherproblem with UNIX is that some of the commands have their own man pageslike wc, ls or awk. To add to the confusion, UNIX systems let you chooseyour preferred command language, called a shell. jobs or alias belong to aparticular shell and you have to read the complete man page for the particularshell you are using.

3.3 The filesystem, or where is everything?

In the following we will outline the typical Linux file system and how it isorganized. This should make it easier for you to understand the operatingsystem and learn how to find programs and resources on your computer.

3.4 Moving around in your system 13

/

/root / bin/etc /usr

/lib

/sputter = /home/klein/sputter

/kuhn

/home

/klein/include/local

Figure 3.1 The Linux filesystem.

As you can see from Figure 3.1, the top directory is called /. Everythingis referenced in respect to this directory or folder. The location which youwill be most concerned with is the /home/your username, which will be yourhome directory. You see a /home/klein, which would be the home directoryof Andi Klein. The home directory always has the same name as you havechosen for your username.

There are a few more directories worth mentioning, the first being /root.This is the home directory of your system administrator, who in Linux iscalled “root.” Root has special privileges such as the ability to shut downthe machine, delete files and create new user accounts. However, becausehe or she has such strong privileges, they can also do devastating things tothe system, like accidentally removing important system files and makingthe system inoperable. Every system adminstrator’s worst nightmare is thatsome malicious person can get access to the system with root privileges anderase the system.

The next directories we want to mention are the /usr/lib and /usr/include.Here you will find libraries and include files for various programs whichyou might want to use. You can either address these directories directly byspecifying their complete path every time you need something from thereor you can have the path included in your .login command file, which isexplained below.

3.4 Moving around in your system

When you log into your system, you usually land by default in your own so-called home directory. In our case this would be /home/klein. This is whereyou work and create your programs. By executing pwd, you can check the


full name of your home directory. pwd is a very helpful command, becauseit will always let you know where you currently are in the file system. Hereis the output of a typical pwd on our system:

/home/klein/comp_phys

telling us that we are in directory comp_phys, which is a subdirectory of thelogin directory /home/klein.

This is not very helpful yet, because you do not know how to movearound. Be patient, this will come in a short while. First, you should createsome subdirectories or folders. Tis will help you to keep your working areaorganized and lets you find programs or other important stuff again later.To create a directory type mkdir directory name. Typing cd dir, meaningchange directory to dir, you can go into this newly created subdirectory.

3.5 Listing your directory

Now that you know how to move around and create directories, it is timeto discuss how you list the content. This is done with ls which will giveyou a bare bones listing of almost all the files you have in your direc-tory (Figure 3.2).

In the second part of Figure 3.2 we have chosen the option -al which givesall the so-called hidden files, files which start with a ., and also lists the direc-tories. In addition it gives the ownership and the permissions rwx for each file.

A permission of w means that you can write to this file, or even moreimportantly, delete this file. So make sure that any file has only write per-mission in the first column, namely the permission for the owner. As wejust mentioned, you can delete your files, and the command to do this is rm

Figure 3.2 Listing ofdirectory from ls andls -al.

3.6 Creating your own files 15

filename. This is a command you should use carefully, because once youhave removed a file, it is gone. A safe way to prevent disaster is to aliasrm to rm -i, which will ask you for confirmation before a file goes into theblack hole. Here is a list of the commands covered so far:

ls list files and directories

ls -al list files with . and permissions

cd directory go to directory

cd go to home (login) directory

cd ∼ go to home (login) directory

cd .. change to parent directory of the current one

mkdir name create directory with name name

rm file delete file file (be careful)

pwd where am I?

3.6 Creating your own files

In order to create a file you can either type touch filename, which creates anempty file, or you can use an editor. Our preference is nedit, which, if thefile does not exist, will ask you first if you want to create it. nedit is an X-Windows editor, i.e., you need to be on a system with X running. Once neditis running you can click on the different menus on the window (Figure 3.3).

Figure 3.3 The nediteditor.


If you are only on a regular terminal, the editor to use is vi. For thisintroduction, the description of vi would be too long. If you want moreinformation on the vi command, get a book about UNIX; all of them have adescription of the different commands. There is also emacs, which can runon either a terminal or an X-Window system. Although this is an excellenteditor, it is fairly complicated to learn. Having created a file and saved it,say as test1.txt, you can then continue to work with it. One way to look atthe content of the file is to open it again with your editor, or by using theless or cat commands. For example:

less test1.txt

will list the content of the file one page at a time, and for longer files youcan use the up and down arrows to scroll forward or backward in your text.cat will immediately dump everything on your screen, which can be ratherannoying if the file consists of several hundred lines. A way out of thisconundrum is to use the pipeline mechanism of UNIX, which lets you “pipe”one command into the next one. You invoke the pipe with the vertical bar|, which separates the commands. For those of you who remember the DOSdays, that should be familiar. In our case, we want to pipe the cat commandinto the more command which stops after every page:

cat test1.txt | more

Let us assume that you want to rename this file. This is an important safetyfeature when you are developing a program. Once you have a running versionyou might want to expand your program. Instead of just editing your file,you should first make a backup of your working program and then changethe new version. (Remember, any time you leave the editor and save anychanges, the old file will be overwritten!) This way you can always go backto the old file in case the new one does not work. To make a copy of yourfile, use the cp command:

cp test1.txt test1.bak

In case you want to rename the program, you could also move it by typing:

mv test1.txt test1.bak

While in the first case you still have test1.txt, in the second you are actuallydeleting the file.

3.7 Doing some work 17

cp old new copies file old to new

mv old new renames file old to new

rm file deletes file

rm -i file deletes file, but asks for confirmation first

rmdir directory deletes a directory

cat file lists file content

cat file � more lists file content one page at a time

less file lists file content with cursor control

3.7 Doing some work

In order to tailor the system to your own preferences, first set up yourenvironment. In the c-shell you have two files which will control your session.The first one is the .login (hidden) file which is automatically executed everytime you log in. Here you can set or redefine environment variables or givethe search path. The environment variables are independent of the shell youare using; i.e., if you need to execute a different shell these variables willstay. In the following we give a sample .login file, which you can use as askeleton and add or change according to your own needs:

############Begin .login file#sample .login file# first set up what is in the path, i.e. where the system looksfor programs

if ($?path) thenset path=($HOME/bin $HOME/util /usr/bin/X11 /usr/local/bin$path .)

elseset path=($HOME/bin $HOME/util /usr/bin /usr/bin/X11/usr/local/bin /usr/lib .)

endif

setenv MANPATH /usr/local/pgqsl/man:/usr/share/man:

# here we set the prompt (works only with tcsh)set prompt="%B%U%m:%∼>"##This creates a buffer of 40 previous commands# which you can recallset history=40


# the alias section

alias netscape ‘/opt/netscape/netscape &’# the nexto command will only list directoriesalias lsd ‘ls\index{ls} -l | grep drwx’# here we set some environment variables

set correct=cmd # spell checking for command line Tcshset notify #tells\index{ls} me about finished background jobs#environment variablessetenv EDITOR /usr/bin/X11/nedit

#Setting up the root systemsetenv ROOTSYS $SOFT/root_newset path=($path $ROOTSYS/bin)setenv LD_LIBRARY_PATH $ROOTSYS/lib############# End of .login\index{.login} file

During this course, you will be writing your own programs. Linux has anexcellent C/C++ compiler: gcc or g++. You can also use the FORTRAN77compiler, which you invoke with g77. Here the help system is different;instead of man C you would type info gcc, or if Tk/Tcl is installed you canuse the graphical interface tkinfo. The info package lets you navigate aroundthe pages like you would on a Web page. There are several options for thecompiler worth mentioning here to get you started, which we show in thefollowing table.

Option Effect

-c compile only, create an object file .o

-o filename compile and link, name the executable filename

-Ox optimization level x, where x can be 0 or 1,2,3

-L for linker, gives the library path

-l for linker, gives the library name

-I gives path for include files

-g produces a debugging version

If you need to debug your program, you should turn off optimization g0and set the flag -g in your compilation. Now you can use the gnu debuggergdb or again for an X-window system ddd (Figure 3.4).

3.8 Good programming 19

Figure 3.4 The DDDdebugger window.

3.8 Good programming

One of the most common mistakes we see with our students is their rush togo and start “hacking in a program.” You can save yourself a lot of headacheand time if you think before you type. Here are a few guidelines to help youin being successful with your computer.

1. Do you really need a program for your task? This might sound silly, butconsider the following problem:

�∑

k=1

12kk

=?

Now you will immediately say: this is a series, I can easily do this witha computer. However, checking in the table of integrals, you will see that


this equals ln 2. So you just wasted a few perfectly good hours to writeand debug a program, instead of doing something fun. Not to mention thatyou are also wasting computer resources.

2. If the problem is more complicated, then the next question is: has anybodydone this before? Almost always the answer will be “yes” and probablybetter. So it might be worth spending some time either searching the mathlibraries on your system or looking around on the internet. This does notmean you do not have to understand the problem or the algorithm used,on the contrary, you have to examine the code description carefully to seewhether the particular solution will be adequate for you.

3. Okay, so you have finally convinced yourself that it must be done. Well,hold your horses, it is still too early to start typing your code in. Take apiece of paper and “design” your program. Write down the problem, theproposed solution and think about exceptions or areas where your solutionwill not be valid. For instance, make sure you will not get a division byzero or the log of a negative number. This will most likely involve checksat run time like if�x! = 0��y = a/x, but it could be a lot more complicated.The best way to do this is with a flow chart and pseudo code. In makingyour flow chart it is not as important to use the standard symbols for ifsand loops, as it is to be consistent. Once you have made a flow chart, youcan implement this in pseudo code, which merely phrases in English whatyou were trying to draw with the flow chart.

4. Once you have outlined your task, you can start your editor and write yourprogram. Here the important thing to remember is that we usually forgetvery quickly what we have done a few weeks or, even worse, a monthago. If you ever want to use a program or function again later, or modifyit, it is of the uttermost importance to comment as much as possible anduse a naming convention, which lends itself to clarity. One-letter variablesare generally bad because, first, you could only have a limited set and,second, the variables do not convey their meaning or usage.

3.9 Machine representation and precision

As a physicist trying to solve a problem or doing a measurement, you shouldalways try to understand the quality or precision of your result. As you havelearned in the introductory labs, if you quote a result you should include onlyas many digits as are significant where the least significant digit has been

3.9 Machine representation and precision 21

rounded off. So in an experiment it is your call to determine the precisionbased on your available instruments.

With computers you also have to be concerned with the precision of yourcalculation and with error propagation. Just because the computer printed outa result does not mean it is the correct one, even though a lot of people haveultimate confidence in computers and sometimes you hear statements like:our computer has calculated this� � � Of course apart from the rounding errorwe will discuss below, the program could also be just plain wrong.

One of the most important things to remember when you write computercode is:

Every computer has a limit on how small or large a numbercan be.

Let us have a closer look at this statement. A computer internally representsnumbers in binary form with so-called bits, where each bit represents a powerof 2, depending on its position. You can think of these bits as switches, whichare either open, representing a 0, or closed which means 1. If we take a hypo-thetical computer with 8 bits we could write a number in the following way

1 1 1 1 1 1 1 127 26 25 24 23 22 21 20

with the number 3 being represented by:

0000 0011

The highest number achievable on our computer would be 28 −1; i.e., allbits are set to 1. Clearly, if you now add one to this number the computercould not handle it and if you are lucky it will give you an error complainingabout an overflow. An even more serious problem with our computer isthat we do not have any negative numbers yet. The way this is handled bycomputers is to designate the bit all the way to the left, or the most significantbit, as the sign bit, leaving us with only 7 bits to express a number:

0111 1111

is now the highest number we can represent: 27 − 1 = 127. As you mighthave guessed zero is written with all 0s:

0000 0000


The negative numbers are expressed in what is called the twos complement:

+5 = 0000 0101−5 = 1111 1011

which you get by first forming the 1 complement (substituting a 1 for a 0 andvice versa) and then add 1. From this it is clear that the smallest number is

1000 0000

giving 27 = −128.

In this way a computer only needs to have instructions for addition. For aPentium III processor, which has 32 bit words the range is [−231� 231 −1].

Now the world of science would be rather limited if we could only dealwith integer numbers, so we also need floating point numbers. This is a littlebit more complicated, but will also illustrate the problem with precision inmore detail. The way to achieve a floating point number is to split up the 32bits into three blocks, the most significant one still being the sign bit. Thesecond block represents the exponent, and the third block is the mantissa.Here we show the number 0.5 represented as a so-called real number, thetypical default 32 bit or 4 byte representation.

0 1000 0000 100 0000 0000 0000 0000 0000 = 0�5signbit 8-bit exponent 23-bit mantissa (3.1)

This can be expressed in the following way:

xfloat = �−1�s ∗mantissa∗2exp−bias (3.2)

By expressing the exponent as an unsigned 8-bit word, we could only havepositive exponents. In order to circumvent this rather drastic limitation,the exponent contains a so-called bias which is not written, but implicitlysubtracted. This bias is 127, making the range of exponents [0 − 127 =−127� 255−127 = 128]. The most significant bit in the mantissa is the left-most bit, representing the value 1/2. From this discussion it is obvious thatthe precision of the machine is at the most:

1223

= 1�2∗10−7 (3.3)

The reason this is the best case has to do with how the computer will add twofloating point numbers. If you take the value 5 and you want to add 10−7 to

3.10 Exercises 23

it, you will still have only 5. In order to do this operation, the processor firsthas to make sure that the exponents for both numbers are the same, whichmeans it has to shift the bits in the mantissa to the right until both exponentsare the same. However, at 10−7 there are no places left for the bits to beright shifted, and the bits drop off; i.e., they are lost. You should always askyourself what is the best achievable precision of your program and whetherthis is sufficient. To increase the precision you can work in double precision,using 64 bits or two words. This will give you an accuracy of 10−15, certainlygood enough for most of your calculations.

3.10 Exercises

1. Make a subdirectory “Test” and create three files in your home directory,called “atest1,” “btest2” and “ctest3.” List the files alphabetically and inreverse order. Then move ctest3 to Test.

2. Still in your home directory, list the content of directory Test.3. Create a .login file and define an alias so that if you type lsd it will list

only directories (hint: use grep).4. Insert a command for your .login file, which will tell you on which host

you are logged in.5. Delete all the files and directories you created in the first exercise.6. Make a subdirectory in your home directory called “src.”7. Write a program which calculates the square root for any given number.8. Go into the src directory and create a program which will calculate the

area of a rectangle. Write the program in such a way that it asks you forinput of the two sides and gives you the output. Compile and run it. Checkfor correctness.

9. Modify your program such that in case you input two equal sides, it willgive you the radius of the outscribing circle.

Chapter 4Interpolation

An important part in a scientist’s life is the interpretation of measured dataor theoretical calculations. Usually when you do a measurement you willhave a discrete set of points representing your experiment. For simplic-ity, we assume your experiment to be represented by pairs of values: anindependent variable “x,” which you vary and a quantity “y,” which is themeasured value at the point x. As an illustration, consider a radioactivesource and a detector, which counts the number of decays. In order to deter-mine the half-life of this source, you would count the number of decaysN0�N1�N2� � � � �Nk at times t0� t1� t2� � � � � tk. In this case t would be yourindependent variable, which you hopefully would choose in such a way thatit is suitable for your problem. However, what you measure is a discreteset of pairs of numbers �tk�Nk� in the range of �t0� tk�. In order to extractinformation from such an experiment, we would like to be able to find ananalytical function which would give us N for any arbitrary chosen point t.But, sometimes trying to find an analytical function is impossible, or eventhough the function might be known, it is too time consuming to calculateor we might be only interested in a small local region of the independentvariable.

To illustrate this point, assume your radioactive source is 241Am, an �

emmiter. Its half-life is �1/2 = 430 years. Clearly you cannot determine thehalf-life by measuring it. Because it is very slowly decaying you probablywill measure the activity over a longer time period, say every Monday fora couple of months. After five months you would stop and look at thedata. One question you might want to answer is: what was the activity onWednesday of the third week? Because this day is inside your range of�t0� tk� you would use interpolation techniques to determine this value. If,on the other hand, you want to know the activity eight months from the endof your measurement, you would extrapolate to this point from the previous

25

26 Interpolation

series of measurements. The idea of interpolation is to select a function g�x�

such that g�xi� = fi for each data point i and that this function is a goodapproximation for any other x lying between the original data points. Butwhat can we consider as a good approximation to the original data if we donot have the original function? Because data points may be interpolated byan infinite number of functions we should have some criterion or a guidelineto select a reasonable function. In mathematics there are very many theoremson function analysis including interpolation with error analysis. As a rulethese methods are grounded on “smoothness” of the interpolated functions.But this would not work for functions such as the example given by Rungeof 1/�1 + 25x2� on the interval �−1�+1. Before we go into the discussionof interpolation techniques, we need to add a word of caution. Because youmeasure discrete points, you have to be very careful about the spacing ofyour independent variable. If these points are too far apart, you will looseinformation in between and your prediction from interpolation would betotally off. Figure 4.1 illustrates this point. Assuming you have made sixmeasurements at the indicated points, you will clearly miss the oscillatorybehavior of the function. In addition, judging from the points and taking intoaccount the error bars, a straight line is probably what you would assume forthe function’s behavior.

Figure 4.1 An example toillustrate the dangers ofinterpolation. 70 75 80 85 90 95 100 105 110

–4

–3

–2

–1

0

1

2

3

4

5

sin(x)/(90–x)

4.1 Lagrange interpolation 27

4.1 Lagrange interpolation

As a first step into interpolation we look at Lagrange interpolation. Thiswill help us to understand the principles of more complicated interpolationmethods like Neville’s algorithm. The method relies on the fact that in a finiteinterval [a,b] a function f�x� can always be represented by a polynomialP�x�. Our task will be to find this polynomial P�x�, from the set of points�xi� f�xi��. If we have just two such pairs, the interpolation is straightforwardand we are sure you have used it in some lab experiment before, looking uptabulated values and determining a point in between two listed values.

Let us take a look at the vapor pressure of 4He as a function of temperature(Figure 4.2). In the literature you would find tabulated values like this

Temperature [K] Vapor pressure [kPa]

2.3 6385122.7 1362182.9 186763.2 2825993.5 4040823.7 499945

Temperature [K]2 2.2 2.4 2.6 2.8 3 3.2 3.4 3.6 3.8 4

Vap

or P

ress

ure

[kP

a]

0

10

20

30

40

50

60

Helium-4 Vapor Pressure

Figure 4.2 4He vaporpressure as a function oftemperature.

28 Interpolation

and your task is to find the pressure at 30 K. The most straightforward wayis to do a linear interpolation between the two points. You would set uptwo equations

b = y2�x2�−y1�x1�

x2 −x1

(4.1)

a = y1�x1�−x1

y2�x2�−y1�x1�

x2 −x1

(4.2)

in order to solve:

y�x� = a+b ∗x (4.3)

Combining equations (4.1) and (4.2) and some reshuffling gives

y�x� = x−x2

x1 −x2

y1�x1�+ x−x1

x2 −x1

y2�x2� (4.4)

which is the equation for a line through �x1� y1�x1�� and �x2� y2�x2��. Usingthis interpolation we get a value of 21.871 kPa at 3 K, while the parameteri-zation gives 21.595 kPa.

In order to improve our result we could use a second degree polynomialand employ a quadratic interpolation. In this case our interpolation functionwould be:

y�x� = �x−x2��x−x3�

�x1 −x2��x1 −x3�y1�x1�+ �x−x1��x−x3�

�x2 −x1��x2 −x3�y2�x2�

+ �x−x1��x−x2�

�x3 −x1��x3 −x2�y3�x3� (4.5)

Using the points at 2.7, 2.9 and 3.2 K, our vapor pressure for 3 K is now21.671 kPa.

The next step would be to use four points and construct a third degree poly-nomial. In general, we can write for the interpolation in terms of a polynomial

P�x� =N∑

k=1

�k�x�f�xk� (4.6)

where

�k�x� =

N∏

l=1�=k

�x−xl�

N∏

l=1�=k

�xk −xl�

(4.7)

which is the Lagrange interpolation formula. This is a polynomial which hasthe values yk �k = 1� 2� 3� � � � �N � for xk �k = 1� 2� 3� � � � �N �.

4.2 Neville’s algorithm 29

4.2 Neville’s algorithm

Neville’s algorithm provides a good way to find the interpolating polynomial.In this method we are using linear interpolations between successive itera-tions. The degree of the polynomial will be just the number of iterations wehave done. Suppose you have five measurements of the vapor pressure Pi�Ti�

at five different temperatures Ti and you would like to find the vapor pressurefor an intermediate temperature T . The first iteration is to determine a linearpolynomial Pij between neighboring points for all the values we have:

P12 = 1T1 −T2

��T −T2�P�T1�− �T −T1�P�T2�� (4.8)

P23 = 1T2 −T3

��T −T3�P�T2�− �T −T2�P�T3�� (4.9)

P34 = 1T3 −T4

��T −T4�P�T3�− �T −T3�P�T4�� (4.10)

P45 = 1T4 −T5

��T −T5�P�T4�− �T −T4�P�T5�� (4.11)

The next iteration will again be an interpolation but now between theseintermediate points:

P123 = 1T1 −T3

��T −T3�P12 − �T −T1�P23� (4.12)

P234 = 1T2 −T4

��T −T4�P23 − �T −T2�P34� (4.13)

P345 = 1T3 −T5

��T −T5�P34 − �T −T3�P45� (4.14)

Our interpolation is now a polynomial of degree two. We will continue thisprocess for two more steps to end up with a fourth degree polynomial P12345

which is our final point. By including more points in your interpolation youwould increase the degree of your polynomial. The following diagram helpsto visualize this procedure:

T1 P�T1�

P12

T2 P�T2� P123

P23 P1234

T3 P�T3� P234 P12345

P34 P2345

T4 P�T4� P345

P45

T5 P�T5�

30 Interpolation

4.3 Linear interpolation

The idea of linear interpolation is to approximate data at a point x by astraight line passing through two data points xj and xj+1 closest to x:

g�x� = a0 +a1x (4.15)

where a0 and a1 are coefficients of the linear functions. The coefficients canbe found from a system of equations

g�xj� = fj = a0 +a1xj (4.16)

g�xj+1� = fj+1 = a0 +a1xj+1 (4.17)

Solving this system for a0 and a1 the function g�x� takes the form

g�x� = fj + x−xj

xj+1 −xj

(fj+1 −fj

)(4.18)

on the interval �xj� xj+1.It is clear that for the best accuracy we have to pick points xj and xj+1

closest to x. The linear interpolation (4.18) also may be written in a symmet-rical form

g�x� = fj

x−xj+1

xj −xj+1

+fj+1

x−xj

xj+1 −xj

(4.19)

Application of the linear interpolation for f�x� = sin�x2� is shown inFigure 4.3. We selected ten equidistant data points to present the original

Figure 4.3 Linearinterpolation forf�x� = sin�x2�.

0 1 2 3

–1.0

–0.5

0.0

0.5

1.0

f(x) = sin(x2)Data pointslinear interpolation

sin(

x2 )

x

4.4 Polynomial interpolation 31

function on the interval [0.0, 3.5]. Then the linear interpolation is applied tothe each xj and xj+1 interval. From the figure we may draw a conclusionthat the linear interpolation may work well for very smooth functions whenthe second and higher derivatives are small. We may improve the quality oflinear interpolation by increasing the number of data points xi on the interval.

It is worth noting that for each data interval one has a different set ofcoefficients, a0 and a1. This is the principal difference from data fitting wherethe same function, with the same coefficients, is used to fit the data pointson the whole interval �x1� xn.

4.4 Polynomial interpolation

Polynomial interpolation is a very popular method due, in part, to its simplicity

g�x� = a0 +a1x+a2x2 +· · ·+anx

n (4.20)

The condition that the polynomial g�x� passes through sample points fj�xj�

fj�xj� = g�xj� = a0 +a1xj +a2x2j +· · ·+anx

nj (4.21)

generates a system of n+1 linear equations to determine coefficients aj . Thenumber of data points minus one that are used for interpolation defines theorder of interpolation. Thus, linear (or two-point) interpolation is the firstorder interpolation.

In Chapter 9 we discuss how to solve a system of linear equations. How-ever, there is another way of obtaining coefficients for polynomial interpo-lation. Let us consider three-point (or second order) interpolation for threegiven points xj , fj with j� j +1� j +2:

fj = a0 +a1xj +a2x2j

fj+1 = a0 +a1xj+1 +a2x2j+1 (4.22)

fj+2 = a0 +a1xj+2 +a2x2j+2

This system may be solved analytically for the coefficients a0� a1� a2. Sub-stituting coefficients aj into equation (4.20), the interpolated function g�x�

may be written in the following symmetrical form

g�x� = fj

�x−xj+1��x−xj+2�

�xj −xj+1��xj −xj+2�+fj+1

�x−xj��x−xj+2�

�xj+1 −xj��xj+1 −xj+2�

+fj+2

�x−xj��x−xj+1�

�xj+2 −xj��xj+2 −xj+1�(4.23)

32 Interpolation

Comparing this result with equation (4.19) for linear interpolation one maywrite easily the interpolating polynomial of degree n through n+1 points

g�x� = f1

�x−x2��x−x3� · · · �x−xn+1�

�x1 −x2��x1 −x3� · · · �x1 −xn+1�

+f2

�x−x1��x−x3� · · · �x−xn+1�

�x2 −x1��x2 −x3� · · · �x2 −xn+1�

+· · ·+ fn+1

�x−x1��x−x2� · · · �x−xn�

�xn+1 −x1��xn+1 −x2� · · · �xn+1 −xn�(4.24)

This is Lagrange’s classical formula for polynomial interpolation.In Figure 4.4 you can see application of first, third, fifth and seventh order

polynomial interpolation to the same data set. It is clear that moving fromthe first order to the third and fifth order improves interpolated values tothe original function. However, the seventh order interpolation instead ofbeing closer to the function f�x� produces wild oscillations. This situationis not uncommon for high order polynomial interpolation. Thus the apparentsimplicity of the polynomial interpolation has a down side. There is a ruleof thumb: do not use high order interpolation. Fifth order may be consideredas a practical limit. If you believe that the accuracy of the fifth order inter-polation is not sufficient, then you should consider some other method ofinterpolation.

Figure 4.4 Polynomialinterpolation.

0 1 2 3–2.0

–1.5

–1.0

–0.5

0.0

0.5

1.0

1.5

2.0

2.5f(x)data points

cos(

3x)/

[0.4

+ (x

–2)2 ]

x

1-st order3-rd order5-th order7-th order

4.5 Cubic spline 33

4.5 Cubic spline

One of the principal drawbacks of the polynomial interpolation is relatedto the discontinuity of derivatives at data points xj . One way to improvepolynomial interpolation considerably is the spline interpolation.

The procedure for deriving coefficients of spline interpolations uses infor-mation from all data points, i.e., nonlocal information, to guarantee globalsmoothness in the interpolated function up to some order of derivatives.

The idea of spline interpolation is reminiscent of very old mechanicaldevices used by draftsmen to obtain a smooth shape. It is like securing a stripof elastic material (metal or plastic ruler) between knots (or nails). The finalshape is quite smooth (Figure 4.5).

Cubic splines are the most popular method. In this case the interpolated

function on the interval �xj� xj+1� is presented in the form

g�x� = fj +bj�x−xj�+ cj�x−xj�2 +dj�x−xj�

3 (4.25)

For each interval we need to have a set of three parameters: bj , cj and dj .Because there are �n − 1� intervals, one has to have 3n − 3 equations forderiving the coefficients for j = 1� � � � � n− 1. The fact that gj�xj� = fj�xj�

imposes �n−1� equations. Central to spline interpolation is the idea that the

0 1 2 3 4 5 6

–1.0

–0.5

0.0

0.5

1.0f(x)data points

spline

sin(

x)

x

1-st order

Figure 4.5 Splineinterpolation.

34 Interpolation

interpolated function g�x� has continuous first and second derivatives at eachof the n−2 interior points xj , i.e.:

g′j−1xj = g′

jxj (4.26)

g′′j−1xj = g′′

j xj (4.27)

These conditions impose 2�n− 2� equations resulting in n− 1 + 2�n− 2� =3n−5 equations for the coefficients. Thus two more conditions are needed.There are a few possibilities to fix two more conditions, for example, fornatural splines the second order derivatives are zero on boundaries. Solvinganalytically a system with many equations is straightforward but cumbersome.However, computers do this job very fast and efficiently. Because there aremany programs for cubic interpolation available, we recommend you use oneof those, instead of writing your own spline program. In Appendix B.1 andB.2 you will find a list of math libraries with spline programs, available onthe Web.

Generally, spline does not have advantages over polynomial interpolationwhen used for smooth, well behaved data, or when data points are close onthe x scale. The advantage of spline comes into the picture when dealingwith “sparse” data, when there are only a few points for smooth functions orwhen the number of points is close to the number of expected maximums.From Figure 4.5 you can see how well spline interpolation fits f�x� = sin�x�

on only six(!) data points.

4.6 Rational function interpolation

A rational function g�x� is a ratio of two polynomials:

g�x� = a0 +a1x+a2x2 +· · ·+anx

n

b0 +b1x+b2x2 +· · ·+bmxm

(4.28)

Often rational function interpolation is a more powerful method com-pared to polynomial interpolation. Rational functions may well inter-polate functions with poles, that is with zeros of the denominatorb0 +b1x+b2x

2 +· · ·+bmxm = 0. The procedure has two principal steps. Inthe first step we need to choose powers for the numerator and the denomina-tor, i.e., n and m. Plotting data on a logarithmic scale may help to evaluatethe power of the numerator for small x, if we have data in this region.Decreasing or increasing data for large x tells whether n < m or vice versa.

4.7 Exercises 35

It is difficult to say what would be the best combination of the powers n andm. You may need a few trials before coming to a conclusion. For example,our analysis tells us that the degree of the numerator is n = 2 and the degreeof the denominator is m = 1. Then:

g�x� = a0 +a1x+a2x2

b0 +b1x(4.29)

Now we know the number of parameters we need to find is five. However,we should fix one of the coefficients because only the ratio makes sense.If we choose, for example, b0 as a fixed number c, then we need four datapoints to solve a system of equations to find the coefficients

f�x1��c+b1x1� = a0 +a1x1 +a2x21

f�x2��c+b1x2� = a0 +a1x2 +a2x22

f�x3��c+b1x3� = a0 +a1x3 +a2x23

f�x4��c+b1x4� = a0 +a1x4 +a2x24

(4.30)

After a few rearrangements the system may be written in the traditional formfor a system of linear equations. Very stable and robust programs for rationalfunction interpolation can be found in many standard numerical libraries.

4.7 Exercises

1. Write a program that implements the first order (linear) interpolations.2. Write a program that implements n-point Lagrange interpolation. Treat n

as an input parameter.3. Apply the program to study the quality of the interpolation to functions

f�x� = sin�x2�, f�x� = esin�x� and f�x� = 02/��x − 32�2 + 004 initiallycalculated in 10 uniform points in the interval [0.0, 5.0]. Compare theresults with the cubic spline interpolation.

4. Use third and seventh order polynomial interpolation to interpolateRunge’s function:

f�x� = 11+25x2

at the n = 11 points xi = −10� −08� � � � � 08� 10. Compare the resultswith the cubic spline interpolation.

5. Study how the number of data points for interpolation affects the qualityof interpolation of Runge’s function in the example above.

Chapter 5Taking derivatives

In physics, one of the most basic mathematical tasks is differentiation. Thefirst laws of physics you encountered were Newton’s laws, where the firstand second laws are differential equations relating derivatives to a function.Another every day life derivative is velocity, and there are many more youencounter on a daily basis. In this chapter we will discuss taking deriva-tives numerically.

5.1 General discussion of derivatives with computers

The most important statement one can make about taking derivatives withcomputers is:

Don’t

if you can avoid it.To illuminate this rather drastic statement, let us start with a function f�x�

and the definition of the derivatives with respect to x:

df�x�

dx= lim

h→0

f�x+h�−f�x�

h(5.1)

There are two issues with this equation: the fact that this is defined for h → 0and that you subtract two numbers from each other which only differ bya small amount. Clearly the second point is easy to address. As we haveseen in Chapter 3, the computer has only a finite representation and whenyour difference is smaller than 10−7 this is then for the computer equal tozero. However, be careful; the standard for C and C++ is to convert floatinto double for any arithmetic operation. As is pointed out in NumericalRecipes [5] this implicit conversion is “sheer madness.” The result still willcome out as a float, but now using much longer CPU time because the naturalprocessor word is 32 bit and implicitly you calculate in 64 bit.

37

38 Taking derivatives

5.2 Forward difference

Let us now start with the first example of taking the derivative numerically,the so-called forward derivative. The starting point for deriving this methodis, as in many other cases as well, the Taylor series. Any function f�x� canbe written as a Taylor series around x+h:

f�x+h� = f�x�+ h

1!f′�x�+ h2

2! f ′′�x�+ h3

3! f ′′′�x�+· · · (5.2)

We can solve this equation for f ′�x� which results in:

f ′�x� = 1h

(f�x+h�−f�x�− h2

2! f ′′�x�− h3

3! f ′′′�x�−· · ·)

(5.3)

or in a more intuitive form

f ′�x� = f�x+h�−f�x�

h−(

h

2!f′′�x�+ h2

3! f ′′′�x�+· · ·)

(5.4)

As you can see, the first term is very close to our definition of the derivative,but the h → 0 is missing. This is the term which we are going to use asour approximation, and in doing so we are introducing an error of order h.Question: in what case do we have a smaller error? So our forward differencederivative can be written as:

f ′�x� �f�x+h�−f�x�

h(5.5)

You can display this graphically as approximating the function with a straightline between two x-values. Clearly we have chosen too large a step, but thishelps in illustrating the point (Figure 5.1).

5.3 Central difference and higher order methods

A much improved version can be derived by using an additional point leadingto a three-point formula. We again start with the Taylor series, but insteadof using only x and x +h, we will also employ x −h. This means we willexpand our function around f�x� for f�x + h� and f�x − h�. This leads totwo equations:

f�x+h� = f�x�+ h

1!f′�x�+ h2

2! f ′′�x�+ h3

3! f ′′′�x�+· · · (5.6)

f�x−h� = f�x�− h

1!f′�x�+ h2

2! f ′′�x�− h3

3! f ′′′�x�+· · · (5.7)

5.3 Central difference and higher order methods 39

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.2

0.4

0.6

0.8

1

sin(x) Figure 5.1 The functionsin(x) and theapproximation to thederivative with theforward difference.

We still want to solve for f ′�x� but, first we subtract equation (5.6) from(5.7) to get:

f�x+h�−f�x−h� = 2h

1!f′�x�+2

h3

3! f ′′′�x�+odd terms (5.8)

which then leads to

f ′�x� = f�x+h�−f�x−h�

2h+��h2� (5.9)

where ��h2� stands for the lowest order of the remainder. The advantage ofthis method is that all the terms odd in h drop out and we have increased theaccuracy by one order.

Clearly we do not have to stop here. Instead of one point on each sideof x, we can take several, for instance at x − 2h�x − h�x + h, and x + 2h.Instead of a three-point formula we are now using a five-point formula whichimproves the accuracy now to ��h4� shown in equation (5.10):

f ′�x� = 112h

�f�x−2h�−8f�x−h�+8f�x+h�−f�x+2h��+��h4� (5.10)

However, from the point of view of computational methods, two remarksare necessary. First, if the function is not known outside some boundaries,but you have to determine the derivative at the boundaries, you have toextrapolate either the function or the derivative. Second, we have written

40 Taking derivatives

equation (5.10) grouped together in a mathematically intuitive way. Forcomputational purposes it is better to do the following:

f1 = f�x−2h�+8f�x+h�

f2 = 8f�x−h�+f�x+2h�

f ′�x� = 112h

�f1 −f2�

In this way you reduce the number of subtractions by one, therefore reducingthe problem of subtractive cancellation.

5.4 Higher order derivatives

A closer look at equations (5.6) and (5.7) also reveals the way to get thesecond derivative. Instead of subtracting them we will add the two equations,therefore getting rid of f ′�x� and calculating f ′′�x�. This leads to:

f ′′�x� = f�x+h�+f�x−h�−2f�x�

h2+��h2� (5.11)

which is a three-point formula for the second derivative.

5.5 Exercises

1. Write a program to calculate sin and cos and determine the forward dif-ferentiation.

2. Do the same, but use central difference.3. Plot all derivatives and compare with the analytical derivative.4. The half-life t1/2 of 60Co is 5.271 years. Write a program which calcu-

lates the activity as a function of time and amount of material. Designyour program in such a way that you could also input different radioac-tive materials.

5. Write a program which will calculate the first and second derivatives forany function you give.

Chapter 6Numerical integration

6.1 Introduction to numerical integration

In the previous chapter we discussed derivatives and their importance inphysics. Now we need to address the inverse, namely integration. A lot ofproblems in physics are described by derivatives, for example the free fall.In a real-world experiment you would be standing on top of a large towerwith a stop watch and you would drop an object, determining the accelerationg from the time it takes the object to hit the ground and the known height h

of the tower. This clearly involves two integrations:

�f =∫ t

0g dt (6.1)

h =∫ t

0gt dt (6.2)

This of course is a problem you would not consider solving numerically, butit illustrates the need for integration in physics. More complicated problemsinvolve determining the normalization constant for a wavefunction, whichyou find from:

∫ +�

−��x��2 dx = 1 (6.3)

Again, it is helpful to look at the definition of an integral, especially the caseof definite integrals, which we will be dealing with. According to Riemann1

a function is integrable if the limit I

∫ b

af�x� dx = lim

�x→0

n−1∑i=0

f�xi��xi (6.4)

1 Bernhard Riemann, 1826–1866, German mathematician

41

42 Numerical integration

exists, where the interval has been divided into n−1 slices. Again, we see thebrute force approach clearly: let us just take the computer, divide this intervalinto many slices with a small �x and be done with it. This is pretty much whatwe will do in the next chapter, and for a quick and “dirty” answer it is just fine.

6.2 The simplest integration methods

The rectangular and trapezoid integration

First we start with the rectangular method. The idea behind this is to dividethe interval �a� b into N intervals, equally spaced by h = xi − xi−1, andapproximate the area under the curve by simple rectangular strips. The widthof the strips will be h, while the height is given by the average of the functionin this interval. We could take:

fi�x� = fi�x�+fi−1�x�

2(6.5)

Or to speed up the calculation, we could use the fact that for a slowly varyingfunction the value f�xi−1/2� at xi −h/2 is a good approximation to fi�x�. Inthis case our integral is expressed as:

∫ b

af�x� dx = h

N∑t=1

fi−1/2 (6.6)

which is known as the rectangular method (Figure 6.1). Clearly, the smalleryou make the width of the rectangles the smaller the error becomes, as longas you are not dominated by the finite machine representation. Naturally theprice you pay is increased computational time.

A better approach is to employ a trapezoid for the areas which will representthe integral of the function. We are still using N slices, but instead ofusing the function value in the middle, we are now determining the areaby using a trapezoid, which consists of the four points �xi� 0�� xi� f�xi��

�xi+1� f�xi+1�� xi+1� 0�. The area of this trapezoid is then:

I�xi�xi+1 = hf�xi�+f�xi+1�

2(6.7)

= h

2�f�xi�+f�xi+1��

One of the things to note here is that the values inside the boundaries areused twice, while the function values at the boundaries a and b are each usedonly once. Keeping this in mind, now we can write the trapezoidal rule as:∫ b

af�x� dx = h

(12

f�x0 = a�+f�x0 +h�+· · ·+ 12

f�xN = b�

)(6.8)

6.2 The simplest integration methods 43

50 60 70 80 90 100 110 120 1300

0.2

0.4

0.6

0.8

1

sin(x*3.1416 / 180.)

50 60 70 80 90 100 110 120 1300

0.2

0.4

0.6

0.8

1

sin(x*3.1416 / 180.)

Figure 6.1 The integral ofsin(x) with therectangular (top) andtrapezoidal(bottom) methods.

The Simpson method

In Simpson’s2 method we are approximating the function in each inter-val by a parabola. To derive the method we are going back to Taylorand doing a Taylor expansion of the integral around the midpoint xi in aninterval �xi−1� xi+1. This looks rather complicated at first, but the reasonfor this will become immediately clear once we have a closer look at theexpansion:

xi+1∫xi−1

f�x� dx =f�xi�

xi+1∫xi−1

dx+ f ′�xi�

1!xi+1∫

xi−1

�x−xi� dx

+ f ′′�xi�

2!xi+1∫

xi−1

�x−xi�2 dx+ f ′′′�xi�

3!xi+1∫

xi−1

�x−xi�3 dx

+· · ·+ f �n��xi�

n!xi+1∫

xi−1

�x−xi�n dx

(6.9)

2 Thomas Simpson, 1710–1761, English mathematician.


Because we have taken the expansion symmetrically around xi all the termswith odd derivatives will drop out because their integral is 0 and we areleft with:

xi+1∫xi−1

f�x� dx = f�xi�∗2h+f ′′�xi�∗ 2h3

3∗2! +��h5f �4��xi�� (6.10)

We end up with a result which is of order h5f �4�. However, we now haveto deal with the second derivative. For this we use equation (5.11) from theprevious chapter, which we substitute into equation (6.10) to get:

xi+1∫xi−1

f�x� dx =2hf�xi�+ h

3�f�xi−1�−2f�xi�+f�xi+1��+ ��h5f �4��xi��

=h

(13

f�xi−1�+ 43

f�xi�+ 13

f�xi+1�

)+ ��h5f �4��xi��

(6.11)

As the last step we extend this for the whole interval �a� b to end up with:

∫ b

af�x� dx = h

3�f�a�+4f�a+h�+2f�a+2h�+4f�a+3h�+· · ·+f�b�� (6.12)

Remember, we are using three points for our integration. This requires thatthe number of intervals is even and therefore the number of points is odd.

6.3 More advanced integration

Gaussian integration with Legendre polynomials

We will now turn our attention to a more sophisticated technique of inte-gration. All the previous methods can be expressed with the following gen-eral approximation:

∫ b

af�x� dx =

N∑i=1

f�xi�wi (6.13)

where wi is the corresponding weight for f�xi�. In the case of the trape-zoidal method the weights are (h/2� h/2), and in the case of the Simpsonmethod the weights are �h/3� 4h/3� h/3�. In addition, the points xi are sep-arated equidistantly. These integrations are also known as quadrature, andwe can consider them as approximations of the integrands by polynomials ineach slice. The next form, Gauss–Legendre integration, will approximate theintegrand with a polynomial over the whole interval. The fundamental idea

6.3 More advanced integration 45

behind this method lies in the fact that we can express a function f�x� in aninterval �a� b in terms of a complete set of polynomials Pl,

f�x� =n∑

l=0

lPl (6.14)

As you will see below, it is convenient to choose orthogonal polynomials,and the first such set we will use are the Legendre3 polynomials, which havethe following property:

∫ +1

−1Pn�x�Pm�x� = 2

2m+1�n�m (6.15)

Because these polynomials are only defined in �−1� 1, we have to changeour integration limits from �a� b → �−1� 1 by the following substitution:

x → b−a

2x+ b+a

2(6.16)

leading to

∫ b

af�x� dx → b−a

2

∫ +1

−1f

(b−a

2x+ b+a

2

)dx (6.17)

For some functions, other polynomials will be more convenient, and you willhave to remap your integral limits to those for which the polynomials aredefined. Because this remapping is trivial, for the following discussion, wewill assume that we are only interested in the integral in �−1� 1.

As the starting point for our discussion we will assume that the functionf�x� can be approximated by a polynomial of order 2n−1� f�x� ≈ p2n−1�x�.If this is a good representation we can express our integral with:

∫ +1

−1f�x� dx =

n∑i=1

wif�xi� (6.18)

In other words, knowing the n abscissas xi and the corresponding weightvalues wi we can compute the integral. This means that we have to find outhow to determine these quantities. In the first step we will decompose p2n−1

into two terms

p2n−1�x� = pn−1�x�Pn�x�+ rn−1 (6.19)

3 Adrien-Marie Legendre, 1752–1833, French mathematician.


where Pn is a Legendre polynomial of order n, while pn−1 and rn−1 arepolynomials of order �n− 1�. Expressing pn−1 by Legendre polynomials aswell, we get

∫ +1

−1p2n−1�x� dx =

n−1∑k=0

k

∫ +1

−1Pk�x�Pn�x� dx+

∫ +1

−1rn−1�x� dx (6.20)

Because of the orthogonality, the first term vanishes and we are left with:∫ +1

−1p2n−1�x� dx =

∫ +1

−1rn−1�x� dx (6.21)

Furthermore, we know that Pn has n zeros in �−1�+1, which we will labelx1� x2� � � � � xn. For these points we have the relation:

p2n−1�xi� = r2n−1�xi� (6.22)

Expressing rn−1�x� with Legendre polynomials

rn−1�x� =n−1∑k=0

wkPk�x� (6.23)

and using equation (6.22) we finally get:

p2n−1�xi� =n−1∑k=0

wkPk�xi� (6.24)

The Pk�xi� are the values of the Legendre polynomials of order k, up tok = n−1 evaluated at the roots xi of the Legendre polynomial Pn�x�.If we write this out for k = 2 we get:

p2n−1�x1� = w0P0�x1�+w1P1�x1�+w2P2�x1�

p2n−1�x2� = w0P0�x2�+w1P1�x2�+w2P2�x2�

p2n−1�x3� = w0P0�x3�+w1P1�x3�+w2P2�x3�

which we can write conveniently in matrix form:

p2n1�x1� � � � � xn� =

⎛⎜⎜⎝

P0�x1� P1�x1� � � � Pk�x1�

· · · · · · · · · · · · · · · · · · · · · · · · · · ·P0�xn� P1�xn� · · · Pk�xn�

⎞⎟⎟⎠

⎛⎜⎜⎜⎜⎝

w0

w1

wk

⎞⎟⎟⎟⎟⎠ (6.25)

In order to solve for the unknown wk we have to invert the matrix P:

wk =n∑

i=1

p2n−1�xi� P−1�ik (6.26)

6.3 More advanced integration 47

Finally, after all this mathematical hoopla we get back to our integral:

∫ +1

−1f�x� dx =

∫ +1

−1p2n−1�x� dx =

n−1∑k=0

wk

∫ +1

−1Pk�x� dx (6.27)

Using the fact that Pk=0�x� = 1, we can multiply this equation with P0�x�

and use the orthogonality condition to get

∫ +1

−1p2n−1�x� dx = 2w0 = 2

n∑i=1

p2n−1�xi� P−1�i0 (6.28)

since∫ +1

−1Pk�x�P0�x� dx = 2

2k+1�k�0 (6.29)

from the properties of the Legendre polynomials. Up to now we have assumedthat f�x� is exactly represented by a polynomial, while in most cases this isan approximation:

p2n−1�xi� ≈ f�xi� (6.30)

Using equation (6.30) we can express our integral as

∫ +1

−1f�x� dx =

n∑i=1

f�xi�wi (6.31)

with

wi = 2 P−1�i0 (6.32)

The wi is the weight of f�xi� at the zeros of Pn�x� and can be calculated.However, these values are also tabulated, for example by Abramowitz andStegun in the Handbook of Mathematical Functions [6]. The following tablelists the values for n = 4 and n = 5.

Order Abscissa xi Weight wi

n = 4 ±0.339981043584856 0.652145154862546

±0.861136311594053 0.347854845137454

n = 5 ±0.000000000000000 0.568888888888889

±0.538469310105683 0.478628670499366

±0.906179845938664 0.236926885056189


Gaussian integration with Laguerre polynomials

The drawback of the Gauss–Legendre integration is that the limits have tobe finite, because the Legendre polynomials are only defined on �−1� 1.However, many times you will encounter integrals of the form:

∫ �

0f�x� dx (6.33)

or

∫ �

−�f�x� dx (6.34)

One type of integral commonly encountered in physics is of the type:

∫ �

0e−xf�x� dx (6.35)

This integral can be calculated using Gauss–Laguerre quadrature, where weare using Laguerre polynomials, which are orthogonal polynomials in theregion [0, �].

∫ �

0e−xf�x� dx =

N∑k=1

f�xk�Wk (6.36)

where the xk are the zeros of the Laguerre polynomials and the Wk are thecorresponding weights and are tabulated in [6]. Another way to look at theintegration in [0, �] is

∫ �

0f�x� dx =

∫ �

0exe−xf�x� dx

=N∑

k=1

w�xk� exk f�xk� (6.37)

which is also tabulated in Abramowitz and Stegun [6]. You can also findthese values at:

http://www.efunda.com/math/num_integration/num_int_gauss.cfm.

Many integrals involving Gaussian distributions are solvable by usingHermitian polynomials H�x�, which are defined in �−��

∫ �

−�f�x� dx ≈

N∑k=1

wkf�xk� (6.38)

6.4 Exercises 49

and integrals of the form

∫ 1

−1

f�x� dx√1−x2

(6.39)

can be solved with Chebyshev polynomials:

∫ 1

−1

f�x� dx√1−x2

≈N∑

k=1

wkf�xk� (6.40)

where in both cases the xk are the roots of the respective polynomials.

6.4 Exercises

1. Using your sin program, write a new program which integrates

f�x� =∫ �

0sin�x� dx

with N intervals, where N = 4� 8� 16� 256 and 1024 and compare theresult for the trapezoid and Simpson methods.

2. Write a general use function or class in which you can give the functionand integration limits.

3. Use your created function to solve the problem of a projectile with airresistance to determine the horizontal and vertical distances as well as thecorresponding velocities as a function of time. This is a problem whichyou have to outline carefully before you attack it.

4. Use Laguerre integration to calculate the Stefan–Boltzmann constant.

Chapter 7Solution of nonlinear equations

From school we know how to find the roots of a quadratic equation ax2 +bx+ c = 0, namely the solution x = �−b ±√

b2 −4ac�/2a. The problem offinding the roots of a quadratic equation is a particular case of nonlinearequations f�x� = 0. The function f�x� can be a polynomial, transcendental, ora combination of different functions, like f�x� = exp�x� ln�x�−cos�x� = 0. Inscience we encounter many forms of nonlinear equations besides the quadraticones. As a rule, it is difficult or not feasible to find analytic solutions.A few lines of computer code can find numerical solutions instantly. Youmay already have some experience with solving nonlinear equations with aprogrammable graphical calculator. In this chapter we consider a few simplebut powerful methods for solving nonlinear equations f�x� = 0. However,the simplicity may be deceptive. Without proper understanding, even simplemethods may lead you to poor or wrong solutions.

In the following, let f�x� be a function of a single real variable x, and wewill look for a real root on an interval �a� b�.

7.1 Bisection method

The bisection method is the simplest but most robust method. This methodnever fails. Let f�x� be a continuous function on �a� b� that changes signbetween x = a and x = b, i.e. f�a�f�b� < 0. In this case there is at least onereal root on the interval �a� b�. The bisectional procedure begins with dividingthe initial interval �a� b� into two equal intervals with the middle point atx1 = �a+b�/2. There are three possible cases for the product of f�a�f�x1�

f�a�f�x1�

⎧⎪⎨⎪⎩

< 0 there is a root in �a� x1�

> 0 there is a root in �x1� b�

= 0 then x1 is the root

51

52 Solution of nonlinear equations

We can repeat the bisectional procedure for a new interval where the functionf�x� has a root. On each bisectional step we reduce by two the interval wherethe solution occurs. After n steps the original interval �a� b� will be reduced tothe interval �b−a�/2n. The bisectional procedure is repeated until �b−a�/2n

is less than the given tolerance.

7.2 Newton’s method

Newton’s method exploits the derivatives f ′�x� of the function f�x� to accel-erate convergence for solving f�x� = 0 with the required tolerance. A contin-uous function f�x� around the point x may be expanded in a Taylor series:

f�x� = f�x0�+ �x−x0�f′�x0�+ �x−x0�

2 f ′′�x0�

2! + �x−x0�3 f ′′′�x0�

3! + · · · (7.1)

Suppose x is the solution for f�x� = 0. If we keep the first two terms in ourTaylor series, we obtain:

f�x� = 0 = f�x0�+ �x−x0�f′�x0� (7.2)

Whence it follows that:

x = x0 − f�x0�

f ′�x0�(7.3)

Because equation (7.2) corresponds to a linear approximation over �x−x0�,we need more than one iteration to find the root. And the next iteration is:

xk+1 = xk − f�xk�

f ′�xk�(7.4)

Usually the procedure requires only a few iterations to obtain the given toler-ance. However, the method has three weak points: (i) a very slow approach tothe solution when f ′�x� → 0 around the root; (ii) difficulty with local minima,leading to the next iteration value xk+1 being far away; (iii) lack of conver-gence for asymmetric functions f�a+x� = −f�a−x�. We recommend usingNewton’s method when computer time is an issue and you know that themethod will work. Otherwise, the bisection method will be the safer choice.

7.3 Method of secants

The secant method is a variation of Newton’s method when the evaluationof derivatives is difficult. The derivative f ′ of the continuum function f�x�

at point x0 can be presented by

f ′�x0� = f�x0�−f�x1�

x0 −x1

(7.5)

7.5 Exercises 53

and equation (7.4) becomes

xk+1 = xk − �xk −xk−1�f�xk�

fk −fk−1

(7.6)

Therefore, one has to select two initial points to start the iterative procedure.The convergence of the secant method is somewhere between Newton’smethod and the bisection method.

7.4 Brute force method

All the methods above are designed to find a single root on an interval [a,b].However, the following situations are possible: (i) there are two or more rootson the interval [a,b]; (ii) there is one root, but the function does not changesign, as in the equation x2 − 2x + 1 = 0; (iii) there are no roots at all; (iv)there is a singularity, so that the function does change sign but the equationdoes not have a root. The bisection method in the last case will converge onthe singularity if the interval has no roots.

The brute force method is a good approach for dealing with multiple roots.You split the original interval [a,b] into smaller intervals with a stepsize h,applying some of the methods mentioned above to each subinterval. Careshould be taken in selecting the size of h. If the interval is too large we maymiss multiple zeros. On the other hand, choosing too small a step may resultin time consuming calculations. A graphical analysis of the equation mayhelp determine the most reasonable size for h.

7.5 Exercises

1. Write a program that implements the bisection method to find roots onthe interval [a,b].

2. Apply the program developed above to find a root between x = 0 andx = 4 for f�x� = exp�x� ln�x�− cos�x� = 0.

3. Develop a program that can solve a nonlinear equation with New-ton’s method.

4. Compare the effectiveness of the bisection method and Newton’s methodfor the equation x3 −2x−2 = 0 which has a single root between x = −4and x = 2.

54 Solution of nonlinear equations

5. Find the real zero of x2 −2x+1 = 0 on �−5�+5�.6. Write a program that implements the brute force method together with

the bisection method for subintervals to solve f�x� = cos�2x�−0�4x = 0on �−4�0�+6�5�.

Chapter 8Differential equations

8.1 Introduction

Some of the most fundamental and frequently occurring mathematical prob-lems in physics use differential equations. Whereas “regular” equations havenumbers as a solution, the solution to a differential equation is a functionwhich satisfies the equation. Because this is such an important topic, we willdiscuss several approaches to solving these equations.

Probably the first differential equation you encountered is Newton’s sec-ond law:

F = ddt

�mv� = mdvdt

= mr (8.1)

This is a second order differential equation (because you have a secondderivative), which you can integrate provided the force F and the initialconditions are known. Two other common differential equations are theSchrodinger equation and the equation describing a harmonic oscillator.

Generally, differential equations are equations in which there is a function,derivatives of the function, and independent variables:

y′ +2xy = 0 (8.2)

In this case y is the function you are trying to find, y′ is its first derivative,and x is the independent variable. One solution is y = e−x2

which you cantest by substituting into equation (8.2).

8.2 A brush up on differential equations

Before we continue, here is a quick tour of the various equations called dif-ferential equations. There are two different classes: the ordinary differential

55

56 Differential equations

equations (ODE) which contain functions of only one independent variable,and partial differential equations (PDE) having functions of several indepen-dent variables. An example of the latter is the time dependent Schrödingerequation, where the wave function ��r� t� is a function of the space coor-dinates r and the time t. In this book we will limit ourselves to ordinarydifferential equations.

ODEs can themselves be grouped into subcategories according to theirorder n and whether they are linear. In a linear equation, the function and itsderivatives appear only in their first power and are not multiplied with eachother. To illustrate this point the following equation

x4y′′′ = y (8.3)

is linear while

x4y′′′y′ = y (8.4)

and

x4y′′′ = y2 (8.5)

are not, because they contain the products y′′′y′ and y2 respectively.However, all three equations are third order, because the highest derivative

appearing is y′′′.The most general form for an nth order ordinary differential equation is:

F�x� y� y′� y′′� � � � � y�n�� = 0 (8.6)

When solving differential equations one has the further distinction betweenthe general and a particular solution. We illustrate this with another example:

y′ = y (8.7)

The general solution of equation (8.7) is:

y = C ex (8.8)

where C is an arbitrary constant, while y = 2ex or y = ex would each bea particular solution of equation (8.7). Finding the general solution is alsocalled the “integration” of the equation.

8.3 Introduction to the simple and modified Euler methods 57

The last two items to discuss are the distinction between homogenous andnonhomogeneous equations. In homogenous equations, each term containseither the function or its derivatives, but no other function of the independentvariable. A good example of this kind is the simple harmonic oscillator:

md2x�t�

dt2+kx�t� = 0 (8.9)

This is a linear (no higher power of x�t�� x′′�t� or multiplications), sec-ond order homogeneous differential equation, well known from introductoryphysics. Also note that we now have x as a function of t. However, addingan external force F�t�, as in the driven harmonic oscillator, leads to a non-homogeneous equation:

md2x�t�

dt2+kx�t� = F0 cos �t (8.10)

In physics we frequently use second order differential equations. Deter-mining what the initial conditions are is the first step in solving a problem.The reason for this is that every nth order ODE has n integrating constants,which have to be known in order to solve the system. These constants can beeither the initial values, as mentioned above, or boundary conditions, whichwill be determined at the integration limits. Examples are the initial positionand velocity of a particle subject to a known force, or the boundaries of thepotential well in the Schrödinger equation, respectively.

8.3 Introduction to the simple and modified Eulermethods

Let us start with the simplest approach to solving an ODE, namely the Eulermethod. This is more for instructional purposes than any serious attempt tosolve ODEs, especially in the case of the simple method. However, it is agood introduction to the chapter, because approaches like the Runge–Kuttaformalism are really just sophisticated adaptations of the Euler method. As astarting point we look at the approximation for the derivative:

y′�x0� ≈ y�x0 +h�−y�x0�

�x0 +h�− �x0�(8.11)

We of course are interested in solving for y�x0 +h�, requiring us to rewritethe equation as

y�x0 +h� ≈ y�x0�+hy′�x0� (8.12)


We have just calculated the next point y�x0 + h� using the previous pointy�x0�. In other words, if we know the derivative at x0 and the step size h,we predict the value of y at x0 + h. You might also recognize this as thefirst two terms of the Taylor series. Clearly, only keeping the first two termswill not give us a very precise solution. You can guess from this that errorpropagation can become a problem.

8.4 The simple Euler method

As an example we will begin our discussion with a first order ODE

xy′ +y = 0 (8.13)

which is solvable analytically. If we rewrite this equation as

xdy

dx= −y (8.14)

the following steps become quite clear:

dy

y= −dx

x(8.15)

∫ dy

y= −

∫ dx

x(8.16)

ln y = − ln x+C (8.17)

which results in

y = C

x(8.18)

In Figure 8.1 the analytical solution is drawn.

We will now discuss how to solve this equation with numerical tools,

starting with the simple Euler method. Using equation (8.12) and the fact that

y′ = − y

x(8.19)

8.4 The simple Euler method 59

10–1

1

Euler solution

1

Analytical solution

0.5 1 1.5 2 2.5 3 3.5 4 4.5 50.5 1 1.5 2 2.5 3 3.5 4 4.5 50.5 1 1.5 2 2.5 3 3.5 4 4.5 50

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

2.2

Difference

Figure 8.1 Simple Euler,analytical solution andratio of Euler over theanalytical solution for astep size of 0.01.

we arrive at

y�x0 +h� = y�x0�−hy�x0�

x0

(8.20)

In this case we start from a point x0, divide the interval between x0 and xf

into N equidistant steps of size h and calculate y at x0� x0 +h�x0 +2h� � � � �

x0 +Nh.As you can see from Figure 8.1, the simple Euler approach does give a

satisfactory result (depending on your standards); the difference between thenumerical and analytical solutions is just a few percent. Part of the reasonfor the difference is that the step size in the program was rather coarse, only0.01. If you choose a smaller step size, the Euler solution will more closelyapproximate the analytical result. However, the finer you make the step size,the more computing time you will need.


To show you how this method can fail very quickly, we will now discussthe standard problem of a mass m on a spring with spring constant k. Thephysical problem to be solved is:

mx = −kx (8.21)

This is a second order linear homogeneous differential equation, and thereforerequires two initial values to solve. We will choose �0�t = 0� and x0�t = 0�.To proceed we will make use of the fact that any nth order linear differ-ential equation can be reduced to n coupled linear differential equations.(Notice the boldface used to write “coupled.” This is to remind you thatyou have to solve these equations simultaneously; you cannot treat them asindependent problems.)

y�n� = f�x� y� y′� y′′� � � � � y�n−1�� (8.22)

can by the following substitution

u1 = y′

u2 = y′′

un−1 = y�n−1�

be cast into a system of n linear equations

y′ = u1

u′1 = u2

u′2 = u3

u′n−2 = un−1

u′n−1 = f�x� y�u1� u2� � � � � un−1�

Now back to our problem of the harmonic oscillator. Because of the secondderivative we will then have two equations, one for

u1 = dx

dt

and one for

u2 = d�

dt= d2x

dt2

8.4 The simple Euler method 61

which are now both linear. Writing this for the harmonic oscillator we endup with:

mdx

dt= mv (8.23)

md�

dt= −kx (8.24)

In order to show that this is not just a mathematical shuffling around wehave kept the m in equations (8.23) and (8.24) to make the connection forthe more advanced student to the Hamiltonian equations, which are:

mdxi

dt= pi (8.25)

dpi

dt= Fi (8.26)

Again, we already know the result of this problem. If �0�t = 0� = 0 andx0�t = 0� = A we have:

x�t� = A cos��t� (8.27)

��t� = −�A sin��t� (8.28)

� =√

k

m(8.29)

We will use conservation of energy to check our numerical solution later.

In the next step we have set up the two equations as they appear for the

numerical solution with m = k = 1. Using equations (8.12), (8.23) and (8.24)

and writing t for h, the first one becomes

x�t +t� = x�t�+t��t� (8.30)

while the second is

��t +t� = ��t�−tx�t� (8.31)

In Appendix D you will find the source code for the program.As you can see in Figure 8.2, this simple approach fails miserably. Even

though we get the expected oscillating result, a closer look reveals that the


Figure 8.2 Harmonicoscillator solution usingthe simple Euler model.

0 2 4 6 8 10 12 14 16 18 20–10

–8

–6

–4

–2

0

2

4

6

8

10Simple Euler

EnergyVelocityPosition

amplitudes are growing with time, violating conservation of energy. Thiswould be a perfect perpetuum mobile, with the energy always increasing.This failure is caused by our truncation of the Taylor series.

8.5 The modified Euler method

We can improve our results by being slightly more sophisticated. In thesimple Euler method we used the derivative calculated at the beginning ofthe interval (i.e., at x) to predict the value of y at x + h. The modifiedEuler method estimates the solution by using the derivative at the midpointbetween x and x + h, similar to taking the central difference described inChapter 5. Again we start the discussion with the first problem encountered,equation (8.13)

xy′ +y = 0

y�x0 +h� = y�x0�+hy′�x0 +h/2� (8.32)

We will first determine y at the midpoint by writing

y

(x0 + h

2

)= y�x0�+ h

2y′�x0� (8.33)

8.5 The modified Euler method 63

and then insert this value into our equation for y′, effectively propagating

this now over the whole interval:

y�x0 +h� = y�x0�+hy�x0 +h/2�

x0 +h/2(8.34)

In Figure 8.3 the results from the halfpoint method are shown. As you cansee the agreement is now excellent.

Let us now apply this improved method to the harmonic oscillator. Firstwe calculate the values for x and v at the midpoint of the interval using thesimple Euler method:

�mid = �

(t + t

2

)= ��t�− t

2x�t� (8.35)

xmid = x

(t + t

2

)= x�t�+ t

2��t� (8.36)

Using the velocity and position at these points we can then propagate oursystem forward to t +t:

��t +t� = ��t�−txmid (8.37)

x�t +t� = x�t�+t�mid (8.38)

1

mid point euler solution

1

analytical solution difference

0.5 1 1.5 2 2.5 3 3.5 4 4.5 50.5 1 1.5 2 2.5 3 3.5 4 4.5 50.5 1 1.5 2 2.5 3 3.5 4 4.5 50

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Figure 8.3 Modified Eulermethod, analyticalsolution and ratio ofEuler over the analyticalsolution for a step sizeof 0.01.


To implement this in your code, use your previous code with the simpleEuler method and add two lines for the midpoint calculation. Replace

v_new[loop]=v_old-step*x_old; // calculate the forwardvelocityX_new[loop]=x_old+step*v_old; // calculate the forwardposition

with the following new code segment:

V_mid=v_old-step/2.*x_old; // calculate the midpoint\index{midpoint} velocityx_mid=x_old+step/2.*v_old; // calculate the midpoint\index{midpoint} positionv_new[loop]=v_old-step*x_mid; // calculate the forwardvelocityx_new[loop]=x_old+step*v_mid; // calculate the forwardposition

In Figure 8.4 you can see the result of the new method. Neither velocitynor position diverge, and energy is now conserved. This is much better butstill not perfect. A closer look reveals that for either a large step size or aloop over many periods, the energy still diverges (Figure 8.5).

Figure 8.4 Harmonicoscillator with themodified Euler modelusing two periods. 0 2 4 6 8 10 12 14 16 18 20

–6

–4

–2

0

2

4

6Modified Euler

PositionVelocityEnergy

8.6 Runge–Kutta method 65

0 100 200 300 400 500 600

–15

–10

–5

0

5

10

15

Spring with modified Euler Method Figure 8.5 Harmonicoscillator with themodified Euler model.The step size has beenchosen to be 0.3 insteadof 0.1 and the number ofperiods is now 50. Notehow the amplitudeincreases and theenergy diverges.

8.6 Runge–Kutta method

It is now time to introduce the most widely used method to solve differentialequations numerically: the fourth order Runge–Kutta (RK) method. This isan improvement of the modified Euler method, which is itself a second orderRunge–Kutta method. In the following we will use a slightly more formalisticapproach. To understand this better, we start with the Taylor series for afunction (where we write yn for y�xn� and xn = x+nh)

yn+1 � y�xn +h� = yn +hy′n + h2

2y′′

n +��h3� (8.39)

Just keeping the first two terms, we have the simple Euler method which hasan error of order h2. Writing

y′n = f�xn� yn� (8.40)

i.e., the derivative of y at xn. We can express the simple method as:

yn+1 = yn +hf�xn� yn� (8.41)

This is the expression for the simple method, which as you can see has anerror of order h2. The next step is to get to the midpoint method. In order todo this we now assume we know yn+1/2 and expand this in a Taylor seriesaround x−h/2, giving us yn:

yn = yn+1/2 − h

2y′

n+1/2 + h2

2y′′

n+1/2 +��h3� (8.42)


As you can see we have kept one more term in the series, therefore gaining

in precision. The trick now is to write a second expansion for our function,

but this time around x+h/2:

yn+1 = yn+1/2 + h

2y′

n+1/2 + h2

2y′′

n+1/2 +��h3� (8.43)

This will then allow us to get rid of the unknown y′′n+1/2 by subtracting

equation (8.42) from (8.43) which results in:

yn+1 = yn +hy′n+1/2 +��h3� (8.44)

This way the term quadratic in h has canceled out. However, we do notreally know yn+1/2. We will calculate y′

n+1/2 using the simple Euler methodto approximate this midpoint:

y′n+1/2 = yn + h

2y′

n (8.45)

= f

(xn+1/2� yn + h

2f�xn� yn�

)

= f�xn+1/2� yn+1/2�

Now we can use this result to start writing down an iterative algorithm, whichrequires you to use the known derivative at xn (namely k1), to calculate fromthis the midpoint derivative at k2

k1 = f�xn� yn� (8.46)

k2 = f

(xn + h

2� yn + hk1

2

)(8.47)

and finally propagate this through the whole interval:

yn+1 = yn +hk2 (8.48)

We have just re-derived the modified Euler method, but now in terms of whatis called a second order Runge–Kutta method (the method is called nth orderif its error is ��hn+1�). If we iterate these steps a few more times, we can


cancel higher and higher orders of h. In practice people usually do this up tofourth or fifth order, which is shown in the next equations (fourth order):

k1 = f�xn� yn� (8.49)

k2 = f

(xn + h

2� yn + hk1

2

)(8.50)

k3 = f

(xn + h

2� yn + hk2

2

)(8.51)

k4 = f�xn +h�yn +hk3� (8.52)

with

yn+1 = yn +h

(k1

6+ k2

3+ k3

3+ k4

6

)+��h5� (8.53)

This is the most straightforward Runge–Kutta calculation and very useful forquick integrations of ODEs, especially if the function is smooth and slowlyvarying, and if computer time is not an issue.

Again, we will use our harmonic oscillator as an example hopefully toshed more light on this method. We start with our two equations, replacingour second order equation with the two linear equations (8.23) and (8.24).In the following, for instructional purposes, we have chosen a slightly morecumbersome way to program the task. Later, in the adaptive step controlprogram, you will find the elegant solution with function calls for determiningthe derivatives.

The first thing we have to do is determine the derivatives f�tn� xn� andf�tn� �n� for x and � (x is now the dependent variable and t is the independentone). We have to find (x�t�� t�). From dx/dt = � we get

f�tn� xn� = �n (8.54)

kx1 = f�tn� xn� (8.55)

xtemp = xn + h

2kx1 (8.56)

and similarly for �

f�tn� �n� = −xn (8.57)

kv1 = f�tn� �n� (8.58)

�temp = �n + h

2k�1 (8.59)


Having now found our first two derivatives kx1 and k�1, we need to find thenext pair kx2 and k�2 for t +h/2. The temporary values xtemp and �temp arethe values of the derivatives at this point, so we write:

f

(tn + h

2� xn + kx1

2

)= �temp (8.60)

kx2 = f

(tn + h

2� xn + kx1

2

)(8.61)

xtemp = xn + h

2kx2 (8.62)


f

(tn + h

2� �n + k�1

2

)= −xtemp (8.63)

k�2 = f

(tn + h

2� �n + k�1

2

)(8.64)

�temp = �n + h

2k�2 (8.65)

The third step then becomes:

f

(tn + h

2� xn + kx2

2

)= �temp (8.66)

kx3 = f

(tn + h

2� xn + kx2

2

)(8.67)

xtemp = xn +hkx3 (8.68)


f

(tn + h

2� �n + k�2

2

)= −xtemp (8.69)

k�3 = f

(tn + h

2� �n + k�2

2

)(8.70)

�temp = �n +hk�3 (8.71)

And finally the last two derivatives are just

kx4 = �temp (8.72)

k�4 = −xtemp (8.73)


The complete solution for both variables x and � becomes:

x�t +h� = x�t�+ h

6�kx1 +2kx2 +2kx3 +kx4� (8.74)

��t +h� = ��t�+ h

6�k�1 +2k�2 +2k�3 +2k�4� (8.75)

The new values at t +h now become the starting values for the next loopand you go back to the top. The following is a little code snippet for thisproblem.

while (time <100.){kx1 = v_old;x_temp = x_old+step ∗kx1/2.;

kv1 = -x_old;v_temp = v_old+step∗kv1/2.;

kx2 = v_temp;x_temp = x_old+step/2∗kx2;kv2 = -x_temp;v_temp = v_old+step/2∗kv2;

kx3 = v_temp;x_temp = x_old+step∗kx3;kv3 = -x_temp;v_temp = v_old+step∗kv3;

kx4 = v_temp;kv4 = -x_temp;

x_new = x_old+1./6.∗(kx1+2∗kx2+2∗kx3+kx4)∗step;v_new=v_old+1./6.∗(kv1+2∗kv2+2∗kv3+kv4)∗step;

time=time+step;

v_old=v_new;x_old=x_new;loop=++loop;

}

One of the remaining tasks is to estimate the quality of your answer.You could run your code with two different step sizes and see whether thedifference is within acceptable limits. If you find that this is not the case,


usually you would make your step size smaller and try again, as we didin the modified Euler method. However, when the function is not slowlyvarying, then it will be more productive to optimize your algorithm. Themethods so far described used the same step size over the full interval, anapproach which can be potentially wasteful in computer time. A region wherethe function is rapidly varying needs a much smaller step size than a regionwhere the function is slowly varying. One way out of this predicament isto use an algorithm which has a step size that is automatically adjusted fordifferent regions.

8.7 Adaptive step size Runge–Kutta

The traditional and also most intuitive way to vary the step size adaptivelyis the step doubling technique. The procedure involves first calculating thenext point, y1, with a step size of h and then a second point, y2, with stepsize h/2. You then take the two calculated values y1 and y2 and compare thedifference = y2 −y1 with a predefined acceptable error �. If is less than�, we can use step size h, otherwise we would repeat this with smaller stepsizes h/2� h/4, until we find a satisfactory . However, this method requirescalculating the derivatives at both f�x+h�y +h� and f�x+h/2� y +h/2�, arather time consuming process.

An improvement to this approach was introduced by Fehlberg [7], whofound a fifth order Runge–Kutta method, using six function evaluations. Hisimprovement came from the fact that if he took a different combination of thesix function evaluations, he got an additional fourth order method. Comparingthe two solutions by requiring that their difference is smaller than a givenvalue � you will get a measure of the error resulting from the truncation.If this difference is larger than desired, the program reduces the step sizeautomatically and recalculates the function values at the new points in x.

In the following, we outline how to make an estimate for the step size. Foran nth order Runge–Kutta solution yn,1 the deviation from the exact result isof the order ��hn+1�, and similarly for an �n+ 1�th order solution yn+1, weget ��hn+2�. If we take the difference between the two solutions

yn −yn+1 = ≈ hn+1 < � (8.76)

1 In this context yn and yn+1 represent an nth and �n+1�th order solution and not the nth or�n+1�th derivative.

8.7 Adaptive step size Runge–Kutta 71

for small h. If our difference is smaller than � we want to use a larger step

size for the next interval, one which would produce a new = �. This leads

us to an estimate for a new step size hnew from

∣∣∣∣ new

current

∣∣∣∣ =∣∣∣∣ �

current

∣∣∣∣ ≈(

hnew

hcurrent

)1/n

(8.77)

This leads to

hnew = hcurrent

∣∣∣∣ �

current

∣∣∣∣1/n

(8.78)

If the difference is larger than �, calculated from equation (8.78), hnew will be

smaller than hcurrent and we would go back and use the new hnew as the step

size. However, you have to guard against the possibility that hnew becomes

smaller than the least significant figure in x. This would cause an infinite

loop. To prevent this you have to put a check in your program. In addition,

because we have derived this equation with a lot of “..approximately..” we

should also put a safety factor in it, which in effect makes the reduction for

hnew not as large as equation (8.78) would allow. Commonly a factor of 0.9

is used, leading to:

hnew = 09hcurrent

∣∣∣∣ �

current

∣∣∣∣1/n

(8.79)

Today you can choose between many variations, but all follow the samerecipe for a kth order Runge–Kutta:

k1 = hf�xn� yn� (8.80)

k2 = hf�xn +a2h�yn +b21k1� (8.81)

kk+1 = hf�xn +akh� yn +�kj=1bk+1jkn� (8.82)

From this you can then calculate the kth and �k−1�th order with

yn+1 = yn +k+1∑j=1

cjkj kth order (8.83)

yn+1 = yn +k+1∑j=1

c∗j kj �k−1�th order (8.84)


the difference being in the cj and c∗j . Below is a table for fourth and fifth

order solutions [8].

i ai bij ci c∗i

1 37378

282527648

2 15

15 0 0

3 310

340

940

250621

1857548384

4 35

310 − 9

1065

125594

1352555296

5 1 − 1154

52 − 70

27 − 3527 0 277

14336

6 78

163155296

175512

57513824

44275110592

2534096

5121771

14

8.8 The damped oscillator

In this section we will discuss the oscillator, but now we will also have aretarding force. We will use this problem to clarify the use of the Runge–Kuttamethod with adaptive step size control. As you know, the simple harmonicoscillator, also called a free oscillator, is an oversimplification of the realworld. This ideal device, once started, would continue to run infinitely. In thereal world, however, we always have energy dissipated because of frictionallosses, and eventually the motion dies out. A reasonable assumption is thatthe frictional force resisting the motion of a body is some power of thevelocity of this body, i.e.,

F ∝ −b��n�v (8.85)

where the minus sign expresses the fact that the force is directed in theopposite direction to the movement. This leads to the following equation fora mass m moving under the influence of a restoring force −kx and a retardingforce −bx

mx+bx+kx = 0 (8.86)

where we have assumed that the retarding force is linear in the velocity.Rewriting this as

x+2 x+�20x = 0 (8.87)

8.8 The damped oscillator 73

we express the problem in the more familiar form with = b/2m the dampingparameter, and �2

0 = k/m the angular frequency of the system.First we will solve this analytically. Differential equations of the type

y′′ +ay′ +by = 0 (8.88)

can always be solved by making the Ansatz:

y = erx (8.89)

y′ = rerx y′′ = r2erx (8.90)

Using this in equation (8.88) leads to the following algebraic equation:

r2 + ra+b = 0 (8.91)

with the solution

r = −a

2±√

a2 −4b

4(8.92)

If the two roots r1 and r2 are not identical, then

y = c1er1x + c2er2x (8.93)

is the general solution. In the case r1 = r2 = r it can be shown that xerx is alsoa solution, and because erx and xerx are linearly independent (the Wronskiandeterminant does not vanish),

y = c1erx + c2xerx (8.94)

will be the general solution. Now, going back to our damped harmonicoscillator, our general solution of equation (8.87) becomes:

x�t� = e− t[A1e

√ 2−�2

0t +A2e−√

2−�20t]

(8.95)

Application of the adaptive step size Runge–Kuttamethod

To demonstrate the use of the adaptive step size Runge–Kutta method, we usethe damped oscillator and explain the functions rk4_stepper and rk4_step inmore detail. Both routines are general use routines we have written and are not


limited to second order ODEs. The user, in this case you, has to provide twoprograms: one program supplies the input (e.g., initial conditions, starting stepsize), and the second program supplies the two linear differential equations.

Because the damped oscillator is a second order ODE, we will need toseparate this equation into two first order differential equations. This can beachieved by setting,

y�0� = x

y�1� = dy�0�

dt= x

dy�1�

dt= x

where we now have to integrate two equations simultaneously at differentlocations in t. To set up our problem for the form used in the Runge–Kuttamechanism we use

f0 = x (8.96)

f1 = −�2x−2 x (8.97)

This is the heart of the routine calculating the derivative at various pointsin time t and will be called from rk4_step.

#include "damp.h"void deri (int nODE\index{ODE}, double x, double y[],double f[])// calculate the derivatives\index{derivatives} for the dampedoscillator{f[0]=y[1];f[1]=-omega2∗y[0]-2∗beta∗y[1];return;}

Now we come to the central pieces of the code, rk4_stepper andrk4_step. The first is used to control the step size in t and will changethe size depending on the precision of the solution found by rk4_step.rk4_step propagates the solution forward by one time slice and returns theintegrated solutions for both equations as well as the differences between therespective fifth and sixth solutions. As we have seen in equation (8.76), thisis a measure of the deviation from the “true” result. The stepper routine will


then determine whether we have achieved the given precision or whether weneed to reduce the step size. If the step size has to be reduced, rk4_stepis called again with the smaller step. However, there is the problem that wecould burn up a lot of computing time by continually reducing our step sizeand trying again and again until we reach the desired precision. Therefore,the program checks how much the step size has been reduced and exits whenit has become smaller than a factor of 10. This means that you either have todecrease your desired precision or make the starting step size smaller. If youchoose too small a step size, the program will increase your step size, but onlyto a given limit. rk4_stepper also opens a file where the integrated valueswill be written away. Here is the function rk4_stepper with additionaldocumentation, especially in reference to our damped oscillator problem:

void rk4_stepper(double y_old[], int nODE, double xstart,

double xmax, double hstep, double eps, int &nxstep)

The next section has to do with the declarations:

y_old[] array of starting values for y

nODE the degree of the ODE, in our case 2

xstart starting value, here begin in time

xmax end of time

hstep the beginning step size

eps chosen precision

nxstep returns the number of steps taken

void rk4_step(double ∗, double ∗, double ∗, int,double,double);

double heps; // the product of the step size and the chosen errordouble yerrmax=.99; // the max error in a step,int i=0;double const REDUCE=-.22 ; // reduce stepsize powerdouble esmall; // the lower limit of precision, if the result issmaller//than this we increase the step sizedouble ydiff[nODE];double y[nODE];double hnew; // new step sizedouble x0;


double xtemp; // temporary x for rk4\_stepdouble step_lolim; // lower limit for step sizedouble step_hilim; // upper limit for step size

// here we create a file for storing the calculated functionsofstream out_file;out_file.open("rk4.dat");out_file.setf(ios::showpoint | ios::scientific | ios::right);x0=xstart;for(i=0 ; i<=nODE-1 ; i++){// store starting values

out_file << x0 << " "<<y_old[i]<<"\n";}esmall=eps/50.;heps=eps∗hstep;

step_lolim=hstep/10.; // the step size should not go lowerthan 10%step_hilim=hstep∗10.; // We don’t want larger step sizefor (i=0 ; i<=nODE-1 ; i++){y[i]=y_old[i];}

And here comes the loop, where we break out once the error is smallerthan the preset precision. You can see that this is checked for every ODE toensure that all integrals reach the required accuracy. If the step size is toolarge, it is reduced and the step is repeated.

for( ; ; ){yerrmax=.99;for( ; ; ){xtemp=x0+hstep;rk4_step(y,ydiff,y_old,nODE,hstep,xtemp);for(i=0;i<=nODE-1;i++){yerrmax=max(yerrmax,heps/ydiff[i]);}


if(yerrmax > 1.) break;hstep=.9∗ hstep∗ pow(yerrmax,REDUCE);

heps=hstep∗ eps;// error if step size gets too lowif(hstep<step_lolim){cout << "rk4_stepper: lower step limit reached; trylower starting"<< "step size \n";

cout << "I will terminate now \n";exit(0);}}

This section will call rk4_step until the planned precision has been achievedor will terminate if hstep becomes smaller than step_lolimit. Becausethis function integrates many ODEs, every integral has to be checked forprecision:

yerrmax=max(yerrmax,heps/ydiff[i]);

And, finally, the last part of the routine where x is propagated by hstep

and the data are stored in the file.

// go further by one step// first check if our step size is too smallif (yerrmax>1./esmall)hstep=hstep∗ 2.;

// set upper limit for step sizeif(hstep>step_hilim){hstep=step_hilim;}for(i=0 ; i<=nODE-1 ; i++){y_old[i]=y[i];

// store data in file rk4.datout_file << xtemp << " "<<y[i]<<"\n";}

// x0 +=hstep;x0=xtemp;nxstep=nxstep+1;if(x0>xmax)


{cout << nxstep;out_file.close(); // close data filereturn;

}}return;

This leaves us to discuss the routine which actually calculates the integralfor one step, rk4_step:

void rk4_step(double ∗ y,double ∗ ydiff,double ∗y_old,int nODE,double step, double x0)

y array of the calculated fifth order integral

ydiff array of difference between fifth and sixth orders

y_old array of starting values for this step

x0 the current x value for y, in our case the time

The next block has to do with setting up the constants for the Runge–Kuttamethod, and defining the f-arrays, where the differential will be calculated.

void deri (int , double ,double ∗, double ∗);// user supplied routine// which calculates derivative// setup the constants first// the coefficients for the steps in xdouble const a2=1./5. ,a3=3./10., a4=3./5., a5=1. ,a6=7./8. ;// coefficients for the y valuesdouble const b21=1./5. ;double const b31=3./40. , b32=9./40. ;double const b41=3./10. , b42=-9./10., b43=6./5. ;double const b51=-11./54. , b52=5./2. , b53=-70./20. ,b54=-35./27.;double const b61=1631./55296. , b62=175./312. ,b63=575./13824. ;double const b64=44275./110592. , b65=253./1771.;//

coefficients for y(n-th order)double const c1=37./378. , c2=0. , c3=250./621., c4=125./594.,c5=0., c6=512./1771.;

// coefficients for y(n+1-th order)double const cs1=2825./27648., cs2=0., cs3=18575./48384.,


cs4=13525./55296., cs5=277./14336., cs6 = 1./4. ;

// the calculated values fdouble f[nODE] , f1[nODE] , f2[nODE] , f3[nODE], f4[nODE],

f5[nODE];

// the x valuedouble x;double yp[nODE];int i;

In the last section, first the derivatives are calculated and then the equationscorresponding to (8.84) shown above.

// here starts the calculation of the RK\index{RK} parametersderi (i,x,y,f);

for(i=0; i<=nODE-1;i++) // 1. step{y[i]=y_old[i]+b21∗step∗f[i];}x=x0+ a2∗step;

deri (i,x,y,f1);

for (i=0; i<=nODE-1;i++) //2. step{y[i]=y_old[i]+b31∗step∗f[i]+b32∗step∗f1[i];}

x=x0+ a3∗step;

deri (i,x,y,f2);for(i=0; i<=nODE-1;i++) //3. step{y[i]=y_old[i]+b41∗step∗f[i]+b42∗step∗f1[i]+b43∗step∗f2[i];}

x=x0+ a4∗step;

deri (i,x,y,f3);for (i=0; i<=nODE-1;i++) //4. step{y[i]=y_old[i]+b51∗step∗f[i]+b52∗step∗f1[i]+b53∗step∗f2[i]+b54∗step∗f3[i];


}x=x0+ a5∗step;

deri (i,x,y,f4);for (i=0; i<=nODE-1;i++) //5. step{y[i]=y_old[i]+b61∗step∗f[i]+b62∗step∗f1[i]+b63∗step∗f2[i]+b64∗ step∗f3[i]+b65∗step∗f4}

x=x0+ a6∗ step;

deri (i,x,y,f5);for (i=0; i<=nODE-1;i++) //6. step{y[i]=y_old[i]+step∗(c1∗f[i]+c2∗f1[i]+c3∗f2[i]+c4∗f3[i]+c5∗f4[i]+c6∗f5[i]);yp[i]=y_old[i]+step∗(cs1∗f[i]+cs2∗f1[i]+cs3∗f2[i]+cs4∗f3[i]+cs5∗f4[i]+cs6∗f5[i]); ydiff[i]=fabs (yp[i]-y[i]);}return;

}

0 2 4 6 8 10 12 14 16 18 20 22

–3–2–1012345

Position

0ω2 > β2

0 2 4 6 8 10 12 14 16 18 20 22

–4

–3

–2

–10

1

2

3

Velocity

Figure 8.6 Result from the Runge–Kutta program for the underdamped case.

8.9 Exercises 81

0 2 4 6 8 10 12 14 16 18 20 220

1

2

3

4

5

Position

=β2ω2

0 2 4 6 8 10 12 14 16 18 20 22–2

–1.5

–1

–0.5

0

Velocity

0

Figure 8.7 Result from theRunge–Kutta program forthe critically dampedcase.

One could have optimized this code by calculating yp[i] and taking thedifference using y[i] and constants, where the difference has already beentaken into account. However, for clarity and readability we prefer this method.

Now we are at the point were we can test our program and check whetherthe result is correct. In the first step we would run this code without anydamping and check that the results for the position and velocity are indeedsinusoidal. We show here only the results for the cases of underdamped�2

0 > 2 (Figure 8.6) and critically damped �20 = 2 (Figure 8.7) oscillators.

8.9 Exercises

1. Write a program which uses the simple Euler method to solve for

y′ = y

x− x2

y2

Use y�1� = 1 and step sizes of h = 005� 01 and 0.20 for 0 < x < 3√

e.2. Modify your program for the same problem but using the modified

Euler method.3. Use the adaptive step size program to solve the two dimensional harmonic

oscillator:

F = −kr


Use different initial conditions for x(0) and y(0) and plot x versus y.Modify your program in such a way that the angular frequencies in �x

and �y are different and plot x versus y again.4. Write a program which uses fourth order Runge–Kutta to solve the problem

of a projectile with air resistance to determine the horizontal and verticaldistances as well as the corresponding velocities as a function of time.This is a problem which you have to outline carefully before you attackit. The program should take initial muzzle velocity and inclination angleas input.

Chapter 9Matrices

Matrices are among the most important mathematical objects in physics. Twoprincipal computational problems are associated with matrices: linear systemsof equations and eigenvalue problems.

9.1 Linear systems of equations

A set of simultaneous linear equations can be written in the form

a11x1 +a12x2 +· · ·+a1nxn = b1

a21x1 +a22x2 +· · ·+a2nxn = b2

· · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·am1x1 +am2x2 +· · ·+amnxn = bm

(9.1)

where xj� j = 1� � � � � n is a set of unknowns, bi, i = 1� � � � �m are theright-hand side coefficients, ai�j are the coefficients of the system. Threecases are possible: m > n� m = n, and m < n. If the number of equationsm is larger than the number of unknowns n, the system of equations isoverdetermined. This case is quite common in data processing. When m < n

the system of equations is underdetermined and cannot be solved. In thissection we will consider the case where m = n, the number of unknowns isequal to the number of equations. It is this case that corresponds to solvingproblems in physics from “first principles.”

Using matrix notation the system (9.1) for the n×n case can be written as⎛⎜⎜⎜⎝

a11 a12 · · · a1n

a21 a22 · · · a2n

· · · · · · · · · · · ·an1 an2 · · · ann

⎞⎟⎟⎟⎠

⎛⎜⎜⎜⎝

x1

x2

· · ·xn

⎞⎟⎟⎟⎠ =

⎛⎜⎜⎜⎝

b1

b2

· · ·bn

⎞⎟⎟⎟⎠ (9.2)

83

84 Matrices

or

A ·x = b (9.3)

Systems of linear equations with the nonzero right-hand side coefficients bi

have a unique solution when the determinant of the matrix A is not equal tozero. If all the coefficients bi are equal to zero, then the solution exists if,and only if, the determinant of the matrix A is zero.

9.2 Gaussian elimination

For systems with a small number of equations, the analytic solutions can beeasily found. In school you were already solving linear systems with twoequations using the elimination technique. For example, expressing the firstunknown x1 from the first equation and substituting in the second equationgives a single equation with one unknown x2. Because there is no suchoperator as elimination either in C++ or in FORTRAN we should trans-late this procedure to an appropriate numerical method for solving systemsof linear equations. For clarity we will consider a system with three linearequations:

a11x1 +a12x2 +a13x3 = b1

a21x1 +a22x2 +a23x3 = b2

a31x1 +a32x2 +a33x3 = b3

(9.4)

Generalization for n equations is then straightforward. Let us subtract the firstequation, multiplied by the coefficient a21/a11, from the second equation,and multiplied by the coefficient a31/a11 from the third equation. The system(9.4) will transform into the following

a11x1 +a12x2 +a13x3 = b1

0+a′22x2 +a′

23x3 = b′2

0+a′32x2 +a′

33x3 = b′3

(9.5)

where the new coefficients a′ij = aij − a1jai1/a11 and b′

i = bi − b1ai1/a11,i = 2� � � � � n� j = 1� � � � � n. We may notice that the unknown x1 has beeneliminated from the last two equations. If we ignore the first equation, then

9.2 Gaussian elimination 85

the last two equations form a system of two equations with two unknowns.Repeating the same operation for the last two equations gives

a11x1 +a12x2 +a13x3 = b1

0+a′22x2 +a′

23x3 = b′2

0+0+a′′33x3 = b′′

3

(9.6)

with a′′ij = a′

ij −a′2ja

′i2/a′

22 and b′′i = b′

i −b′2a

′i2/a′

22, i = 3� � � � � n� j = 2� � � � � n.For a system with n equations we repeat the procedure of this forwardelimination n− 1 times. From the last equation follows x3 = b′′

3/a′′33. Doing

back substitution we will find x2 and then x1. This direct method to findsolutions for a system of linear equations by successive eliminations is knownas Gaussian elimination.

This method appears to be very simple and effective. However, in ourconsideration we missed several important points: zero diagonal elements,round-off errors, ill conditioned systems, and computational time. What if oneof the diagonal elements is zero? The answer is clear: the procedure will fail.Nevertheless, the system may have a unique solution. The problem may besolved by interchanging the rows of the system, pushing elements which arezero off the diagonal. This is called the partial pivoting procedure. Moreover,reordering the system in a way when a11 > a22 > a33 > · · · > ann wouldincrease the efficiency of this method. This is the issue of complete pivoting.

For each new elimination, the Gaussian method employs results from theprevious ones. This procedure accumulates round-off errors and thus for largesystems you may get a wrong numerical solution. It is highly recommended tocheck your solutions by direct substitution of x1� x2� � � � � xn into the originalsystem of linear equations. How can we reduce the round-off errors? Usually,complete pivoting may be very efficient. Besides scaling, multiplication ofthe ith equation by a constant ci may also help in improving accuracy.

Another possible disaster in the waiting is the case of ill conditioned sys-tems. For such systems a small change in coefficients will produce largechanges in the result. In particular, this situation occurs when the determinantfor A is close to zero. Then the solution may be very unstable. An additionalissue is time. As the number of equations in the system increases, the com-putation time grows nonlinearly. Systems with hundreds, even thousands, ofequations are common in physics. And you may face a problem of waiting forweeks, if not years, to get an output from your computer. Sometimes iterativemethods can help to increase speed, but generally they are less accurate.

86 Matrices

9.3 Standard libraries

At this moment you may be overwhelmed by terminology such as: “pivoting,”“scaling,” “round-off errors,” etc. How much experience and time do weneed to understand the basic variations of just Gaussian elimination? Howabout other methods? Jordan elimination, LU decomposition and singularvalue decomposition are all widely used techniques. In addition, there arenumerous methods for special forms of equations: symmetrical, tridiagonal,sparse systems. We believe that the most powerful method to solve mostsystems of linear equation is the use of “standard libraries.” In Appendix B.1and B.2 you will find a list of some of the most popular libraries on theWeb with many robust programs for solving systems of linear equations.Specifically, the LAPACK library is a very large linear algebra packagewith hundreds of programs. However, you have to be careful in selecting aprogram that is right for your specific system of linear equations.

9.4 Eigenvalue problem

The eigenvalue problem is the key to structure calculations for quantum

systems in atomic, molecular, nuclear and solid state physics. Mathematically,

the eigenvalue problem may be written as

A ·x = �x (9.7)

or

a11x1 +a12x2 +· · ·+a1nxn = �x1

a21x1 +a22x2 +· · ·+a2nxn = �x2

· · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·an1x1 +an2x2 +· · ·+annxn = �xn

(9.8)

The equations above look like a linear system of equations (9.1). However,there is a substantial difference between the eigenvalue problem and thelinear system of equations. For the eigenvalue problem the coefficients � areunknown and solutions for the system (9.8) exist only for specific values of�. These values are called eigenvalues.

9.4 Eigenvalue problem 87

Regrouping terms in the system (9.8) leads to

�a11 −��x1 +a12x2 +· · ·+a1nxn = 0

a21x1 + �a22 −��x2 +· · ·+a2nxn = 0

· · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·an1x1 +an2x2 +· · ·+ �ann −��xn = 0

(9.9)

Introducing a unit matrix I, which is defined as

I =

⎛⎜⎜⎝

1 0 · · · 00 1 · · · 0· · · · · · · · · · · ·0 0 · · · 1

⎞⎟⎟⎠ (9.10)

one can rewrite the system of linear equations (9.9) in the following form:

�A−�I� ·x = 0 (9.11)

Solutions for the system (9.11) exist if, and only if, the determinant of thematrix A−�I is zero

det�A−�I� = 0 (9.12)

For an (n×n) matrix the equation above would give a polynomial in � ofdegree n

cn�n + cn−1�

n−1 +· · ·+ c1�+ c0 = 0 (9.13)

The coefficients c are determined through the matrix elements aij by the defini-tion for the matrix determinant. This polynomial equation is known as the char-acteristic equation of the matrix A. Roots of this equation would give therequired eigenvalues. In physics, we often deal with either symmetric aij = aji

or Hermitian aij = a∗ji matrices (a∗ stands for the complex conjugate element).

It is important to know that all the eigenvalues for these matrices are real.In Chapter 7 we discussed methods for solving nonlinear equations. For

matrices with small n these methods may be applied to finding the eigen-values � from equation (9.13). Once we have determined the eigenvalues,we may solve the system of linear equations (9.9) to find a set of solutionsx = �x1� x2� � � � � xn� for each value of �. These solutions are called the eigen-vectors. For Hermitian matrices, the eigenvectors corresponding to distincteigenvalues are orthogonal.

88 Matrices

In general, the scheme above for solving the eigenvalue problem looks verystraightforward. However, this scheme becomes unstable as the size of thematrix increases. The standard libraries have many robust and stable computerprograms for solving the eigenvalue problem. In particular, programs basedon the Faddeev–Leverrier method are very popular and successful in atomicand molecular structure calculations. The Lanczos algorithm is a good choicefor large and sparse matrices which are common in many-body problems.

9.5 Exercises

1. Write a program that implements the Gaussian elimination method forsolving a system of linear equations. Treat n as an input parameter.

2. Apply the program to solve the set of equations⎛⎜⎝

2 3 −2

1 −6 4

4 −1 6

⎞⎟⎠

⎛⎜⎝

x1

x2

x3

⎞⎟⎠ =

⎛⎜⎝

2

4

6

⎞⎟⎠

3. Compare accuracy of the Gaussian elimination method with a programfrom a standard library for solutions of the following system of equations:

⎛⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎝

112

13

14

12

13

14

15

13

14

15

16

14

15

16

17

⎞⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠

⎛⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎝

x1

x2

x3

x4

⎞⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠

=

⎛⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎜⎝

4

3

2

1

⎞⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎟⎠

4. Write a program that calculates eigenvalues for an n×n matrix.Implement the program to find all eigenvalues of the matrix:

⎛⎜⎝

1 2 3

2 2 −2

3 −2 1

⎞⎟⎠

Using a program from standard libraries find all eigenvalues and eigen-vectors of the matrix above. Compare the results with your programfor eigenvalues

Chapter 10Random processes andMonte Carlo simulation

10.1 Random processes in science

In science as well as in everyday life we constantly encounter situations andprocesses which are stochastic in nature, i.e., we cannot predict from theobservation of one event how the next event with the same initial startingconditions will come out. Say you have a coin which you throw in the air.The only prediction about the outcome you can make is to say that you havea 50% chance that the coin will land on its tail. After recording the first event,which was a head, you toss it up again. The chance that the coin will landon its tail is still 50%. As every statistics teacher has taught you, you need toexecute these throws a large number of times, to get an accurate distribution.The same is true when you roll a die. The only prediction you can makeis that in a large number of throws each number has a probability of 1/6.Now assume that you have 10 dice, and you are to record all distributions in10 000 throws. Of course probability theory will predict the outcome for anycombination you wish to single out; for example, the chances to get all “1s” inone throw (1/6). However, actually trying to do this experiment would keepyou busy for quite some time (and in addition you would not learn anythingnew). This is where the so-called “Monte Carlo” simulation technique comesinto play. The idea is that you let the computer throw the coin for you andalso record the outcome, which you can then plot. In this scheme you simplyneed a program which generates randomly a variable, which can have onlytwo values, say 1 and 0. Having the computer do this 10 000 times and recordthe outcome each time will give you the proper distribution. But, how do youwrite a program which creates a random number? And how do you knowthat such a number is truly random?

89

90 Random processes and Monte Carlo simulation

10.2 Random number generators

Before we discuss how to create random numbers with computers, let usmake the following announcement:

There is no true computer-generated random number!

A computer has only a finite precision because of the limited number ofbits it has to represent a number. Eventually the sequence of numbers from thegenerator will repeat itself. Say you record all the numbers produced by thecomputer and carefully look at the series. At some point you will notice thatthe numbers repeat themselves in exactly the same sequence. This is calledthe period of the generator. The following example illustrates why you haveto be concerned about this. Assume you write a program, which simulatesa simple process like rolling one die and recording the outcome. In order tohave around 10 000 occurrences for each possible outcome, therefore givingyou a statistical error of 1%, you would have to have the computer generate60 000 events. If your random number generator had a period of only 10 000,you actually would have only the statistical accuracy corresponding to 10 000and not 60 000, because you just repeat the same sequence six times. Thetricky part, however, is that unless you check your generator for the lengthof the period, you will not be aware of this problem.

Other issues which you have to be concerned about are uniformness overthe range of numbers and possible correlations between numbers in thesequence. The ideal random number generator should distribute its numbersevenly over an interval. Otherwise you will bias your simulation with thedistribution you have originally generated.

The most widely known random number generators (RNG) are the so-called linear congruential generators and are based on a congruence rela-tionship given by:

Ii+1 = �aIi + c� mod�m� (10.1)

where m is the modulus and a and c are constants chosen by the programmer.This recursive relation generates the next random integer from the previousone. Herein lies one of the problems of this sequence. If by chance the newnumber Ii+1 has already shown up before, the whole sequence repeats and youhave reached the end of the period. Another problem we want to mention in

10.2 Random number generators 91

Figure 10.1 The resultsfrom a unsatisfactoryrandom numbergenerator.

passing is the problem of correlations. If the constants in the generator are notcarefully selected, you will end up with a situation where you could predict alater random number by looking at a previous one. Below is a small routinewhich demonstrates the problems of a simple random number generator:

int rand_ak(int &random){// the generator I_(i+1)=aI_(i)+c (mod m)

int a=65;int c=319;int m=65537;

random=(a∗random+c) % m;

return (random);

}

The main program creates eight different series of random numbers. Thesecan be plotted independently (like the upper two panels in Figure 10.1) ordisplayed one variable against one from a different series (see lower leftpanel). In the lower right panel we show a three-dimensional plot of threevariables. Already in the one dimensional case the problems are clearlyrecognizable. There are fluctuations in the distributions which are way outside


Figure 10.2 The resultsfrom the Linux randomnumber generator.

of any statistical variation, indicating that some numbers show up more oftenthan others. Graphically, the most intriguing is the three dimensional case.There are clear lines visible, which reflect a strong correlation among thethree random numbers.

An acceptable generator will fill a three dimensional cube completelyhomogeneous as is the case in the next figure 10.2:

This distribution was generated from the Linux generator rand3 (describedin man pages rand(3)). The number generated is divided by RAND_MAX(the largest generated number; provided by the system), so that the intervalis now from zero to one.

10.3 The random walk

To illuminate the power of simulations with the computer we start with avery simple problem, namely the random walk in one dimension and thengradually add more complexity to it. This is closely related to the phenomenaof diffusion.

Suppose you live in a one dimensional world and you only can take stepseither to the left or to the right, like on a very narrow mountain path. Thestep length in either direction is the same and you choose randomly if you

10.3 The random walk 93

–40 –30 –20 –10 0 10 20 30 400

5000

10000

15000

20000

25000

30000

35000

htempNent = 500000

RMS = 7.087

x

Mean = – 0.004896

Figure 10.3 The positiondistribution for 5000walks and each 100steps.

want to go left or right. This is also referred to as the drunkard’s walk. Thequestion then is, where will you be after N steps?

x = ml (10.2)

where l is the length of the step and m is such that −N ≤ m ≤ N . Becausethe probability for stepping to the right is the same as stepping to the left, themost likely position for several walks will be at x = 0 and the distributionfor the positions will be binomial.

The following program has 5000 different walks in it and each walkhas 100 steps (Figure 10.3). This program is also a nice illustration forthe power of ROOT trees, which will allow you to analyze the differentwalks.

// rnd_walk1.cxx// Random walk in one dimension// ak 4/30/2002#include <iostream>#include <fstream>#include <math.h>#include <iomanip>#include <stdio.h>#include <stdlib.h>#include <fcntl.h>


#include "TROOT.h"#include "TTree.h" // we will use a tree to storeour values

#include "TApplication.h"#include "TFile.h"

int rand(void);void srand(unsigned int);

void main(int argc, char **argv){

//structure for our random walk variablesstruct walk_t { int x; // the position after a stepint nstep; // the step numberint left; // number of steps to the leftint right; // number of steps to the rightint jloop; // number of outer loops};

walk_t walk;

int i_loop; //inner loopint j_loop; //outer loopint jloop_max=5000; // the max number of different trials

unsigned int seed = 68910 ; // here is the startingvalue or seed

int loop_max=100; // the maximum number of steps

double rnd;

TROOT root("hello", "computational physics");

// initialize rootTApplication theApp("App", &argc, argv);

//open output fileTFile *out_file = new TFile("rnd_walk68910.root","RECREATE", "example of random random walk"); //create root file

// Declare treeTTree*ran_walk = new TTree("ran_walk", "tree withrandom \index{random} walk variables");

ran_walk->Branch("walk",&walk.x,"x/I:nstep/I:left/I:right/I:jloop/I");

// set seed value

10.3 The random walk 95

srand(seed);

// the outer loop, trying the walk jloop timesfor(j_loop=0;j_loop < jloop_max ;j_loop= j_loop+1){

walk.x=0;walk.nstep=0;walk.left=0;walk.right=0;walk.jloop=j_loop+1;

for(i_loop=0;i_loop < loop_max ;i_loop= i_loop+1){

// here we get the steprnd=double(rand())/double(RAND_MAX);

if((rnd-.5)<=0.){walk.x=walk.x-1;walk.left=walk.left+1;}else

{walk.x=walk.x+1;

walk.right=walk.right+1;}walk.nstep=walk.nstep+1;// fill the treeran_walk->Fill();

}}

out_file->Write();

}

To analyze the ROOT file rnd_walk68910.root you have created,you start up ROOT and read in the file:

Welcome to the ROOTroot [0] TFile*f = new TFile ("rnd_walk68910.root")root [1] TTree*mytree = ran_walkroot [2] mytree->ls()OBJ: TTree ran_walk tree with random\indexrandom walkvariables : 0**********************************************************


*Tree :ran_walk : tree with random\indexrandom walkvariables

*Entries : 500000 : Total = 10018021 bytes File Size =2852491 *

* : : Tree compression factor = 3.52 ************************************************************ Br 0 :walk : x/I:nstep/I:left/I:right/I:jloop/I ** Entries : 500000 : Total Size= 10011931 bytes *

File Size = 2846401 ** Baskets : 313 : Basket Size= 32000 bytes *

Compression= 3.52 **........................................................*root [4]

With the mytree->Print() statement you get a list of the tree and thebranches, in our case only one branch, called “walk” with several leaves. Thenext command will draw the positions for all the different walks, representinga binomial distribution:

root�4�mytree− > Print��

Now in order to look at the different walks, say for instance walk number 5,we set the condition jloop==5, on the analysis:

Figure 10.4 The x

distribution as a functionof the number of stepsfor trial number 5. 0 20 40 60 80 100

–4

–2

0

2

4

6

8

x:nstep { jloop==5}

10.4 Random numbers for nonuniform distributions 97

root [5] mytree->Draw("x:nstep", "jloop==5", "P")

This gives the random walk for the fifth trial (Figure 10.4).In Appendix D.3 you will find a program which describes the random walk

in two dimensions.

10.4 Random numbers for nonuniform distributions

A lot of situations in physics and science require random numbers whichare not distributed uniformly. For example, a typical beam from a particleaccelerator has an energy distribution around its mean energy, which in somecases can be approximated by a Gaussian distribution. The challenge then isto generate random numbers which follow this probability. In addition, younow have to deal with intervals which are different from [0, 1). There areseveral different options to produce a specific distribution with a uniformrandom number generator.

Acceptance and rejection method

This is probably the easiest method to implement, however it might not be thefastest. This procedure is related to the Monte Carlo integration technique.Suppose you want to generate numbers which follow a certain distributionf�x�, with x � �a� b�. The most straightforward way to achieve this is tothrow all events away which are outside of f�x� in a two dimensional plane.If, for example, your distribution is f�x� = sin�x� in [0, 180), you woulduse as your x variable a uniformly distributed number between 0 and 180.As your y-coordinate for your plane you would use a second independentvariable between 0 and 1. If your random generator is uniform, then initiallythis plane is filled uniformly in the two dimensions. If you now reject all the�x� y� points where y > f�x� (in our case sin(x)), the remaining accepted x

will be distributed according to sin (x) (see Appendix D.4).As an example of this we will discuss the Drell–Yan process in high energy

physics [9]. A nucleon, proton or neutron, is made up of what is calledthree valence quarks, and a “sea” of quark–anti-quark pairs, which you canconsider as virtual excitations, the same way as a photon can be describedby an electron–positron pair. As you might know, there exist six differentquarks and the corresponding anti-quarks, but for the case of the nucleon wewill only deal with the u and d quarks. Quarks have fractional charge, the u


quark has Qu = 2/3 and the d has Qd = −1/3. It is immediately clear thatthe proton consists of two u and one d quarks because the total charge isQp = 1. Similarly, the neutron has two d and one u quarks. If you combinea quark and an anti-quark together, they will annihilate into a virtual photon,which then can decay either back into two quarks or into two leptons, likean electron–positron or muon–anti-muon pair.

The Drell–Yan process is exactly this annihilation of a quark–anti-quarkpair into a heavy virtual photon, which then decays into a �−�+ pair.

In order to study this reaction you collide two proton beams of highenergy with each other and during the interaction one of the quarks from oneproton will annihilate with an anti-quark from the “sea” of the other. Thecross-section for this process can be written down in the following way:

d2

x1x2

= constant∗ 1M2

∑i

e2i

[fA

i �x1�fBi �x2�+ f A

i �x1�fBi �x2�

](10.3)

Even though this formula looks somewhat intimidating it is actually ratherstraightforward. The first terms to be defined are the x1 and x2. The sub-scripts 1, 2 refer to the two quarks from the beams. The x is the Bjorkenx, which expresses the fraction of the momentum of the nucleon carried bythe respective quarks. Because a nucleon has three valence quarks, each ofthem will carry on average a third of the nucleon momentum. However, asfor a nucleon bound in a nucleus, the quark has a momentum distribution,which can be between 0 (at rest) and 1 (carrying all the momentum). InFigure 10.5 we show the parameterization found from experiments [10] forthe three different quark types:

uv�x� = 213√

x�1−x�28 (10.4)

dv�x� = 126√

x�1−x�38 (10.5)

gsea = 027�1−x�81 (10.6)

In Equation (10.3) the fi corresponds to the two valence distributions,while f i represents the sea quark distribution. The term e2

i stems from thefractional charge of the quarks and the index i goes from 1 to 3 for the threevalence quarks which can be involved. The last variable to be explained isM2. This is the square of the invariant mass of the produced di-lepton pairand is a function of the total energy S and the momentum fractions of theparticipating quarks x1 and x2:

M2 = Sx1x2 (10.7)

10.4 Random numbers for nonuniform distributions 99

x-Bjorken

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5u-quark

d-quark

sea quark

quark structure functions Figure 10.5 The quarkmomentum distributionsfor the u, d and the seaof quarks.

In the program drell_yan1.cxx, the Drell–Yan cross-section is calcu-lated using the rejection method and a beam energy of 25 GeV. The result isshown in Figure 10.6.

Inversion method

Suppose you want to simulate some decay process, like the decay of anensemble of radioactive nuclei. In the case of a nuclear decay, we could forinstance choose the decay time at random from the given decay law

f�t� = exp�−�t� (10.8)

This means we would like to choose a variable which follows an exponentialdistribution. In this case there is a much more efficient and faster methodavailable, namely the inverse transform method. We will briefly describe thisin the following section.

The usual situation is that we have a uniform distribution x available in theinterval [0, 1). The probability of finding a number in the interval dx aroundx is then given by

p�x� dx = dx for 0 ≤ x < 1

= 0 otherwise (10.9)


Figure 10.6 The Drell–Yancross-section as afunction of theinvariant mass.

5 6 7 8 9 10 11 12

10–8

10–7

Drell Yan

Invariant Mass [GeV]

x 1x 2

d2 σ

pp → μ+μ– +X

The probability has to be normalized, i.e.,∫ −�

�p�x� dx = 1 (10.10)

Our goal is to obtain a distribution P�y� dy = e−y dy, which implies that wehave to find the proper transformation. If we require that P�y� dy has thesame normalization as p�x� we can write

P�y� dy = p�x� dx (10.11)

This together with Equations (10.9) and (10.10) allows us to write

e−y = x (10.12)

which solving for y leads to:

y�x� = − ln�x� (10.13)

This is a very fast and efficient way to produce an exponentially decayingdistribution. This method, however, will work only if the function we aretrying to use for a distribution has an inverse:

x = F�y� =∫

P�y� dy

y = F−1�x�

10.5 Monte Carlo integration 101

Additional distributions

Other distributions, which are important in physics, are the normal or Gaus-sian, the Lorentz, and the Landau distributions. In Handbook of MathematicalFunctions [6] you will find these and other examples. In addition, ROOTprovides most of the common distributions and will give you any randomdistribution according to a user defined function.

// Define function to be sampledTF1 *f1=new TF1("f1", "exp(-x)",0.,1.);

//x_random is now distributed according to exp(-x)x_random=f1.GetRandom();

10.5 Monte Carlo integration

This section deals more with the mathematical application of random num-bers than with a physical one. We describe how Monte Carlo methods canbe applied to the integration of complicated functions. These procedures areespecially useful when you are trying to integrate multi-dimensional inte-grals.

The best way to understand the principle of Monte Carlo integration isby using a geometrical description. Imagine you are charged with the taskof determining the area of the polygon in Figure 10.7. One way would beto create little squares and count how many squares can be put into theshape. However, suppose you have a shotgun, which with every shot youtake, sprays the whole area of the rectangle homogeneously and, in addition,you know how many pellets in total came out from one round. In orderto determine the area, you count how many pellets went into the polygonarea and compare them with the total number of pellets in the completearea. Clearly a rectangular spray from a shotgun is not very likely, but acircular area is probably a good approximation. The important part is thatthe enclosing area is easy to calculate and the inside is completely contained

Figure 10.7 A complicatedshape bounded by arectangular box.


in this outer shape. The most commonly used example demonstrating thisis to determine the value of �. The area of a circle is �r2, which for acircle of unit radius is equal to �. To compute this with the Monte Carlointegration method, you limit yourself to one quarter of the unit circle, andthrow two random variables for x and y. Any time y is less than

√�1−x2�

you are inside the circle and count this as a hit. The following code fragmentillustrates this:

for(i_loop=0;i_loop < i_loop_max ;i_loop= i_loop+1){

x=double(rand())/double(RAND_MAX);y=double(rand())/double(RAND_MAX);

if(y<=sqrt(1-pow(x,2))){hit=hit+1; // the point is within the circle}

}pi_result=4*hit/i_loop_max; // We have only used one quarter

Figure 10.8 Thedistribution of the valueof � for 1000 throws(upper) and 1 000 000throws (lower panel).

2.6 2.8 3 3.2 3.4 3.6 3.8 40

5

10

15

20

25

30

35

40

distribution of pi

3.05 3.1 3.15 3.2 3.250

20406080

100120140160180

histoNent = 1000 Mean = 3.142RMS = 0.001688Chi2 / ndf = 10.08 / 13Constant = 177.8 ± 7.042 Mean = 3.142 ± 5.321e-05 Sigma = 0.001667 ± 3.961e-05

distribution of pi

histoNent = 1000 Mean = 3.141RMS = 0.05094Chi2 / ndf = 76.42 / 71Constant = 31.04 ± 1.291 Mean = 3.142 ± 0.001609 Sigma = 0.04772 ± 0.001262

10.6 Exercises 103

Before we continue, we need to discuss how to estimate the accuracy anderror of the Monte Carlo integration. In Figure 10.8 we have plotted thecalculation of � for two different cases. Both panels evaluated �. However,in the upper panel each integral was determined from 1000 random points,while in the lower panel we used 1 000 000 points. (Note the different scalesfor the axes.) The distribution has a Gaussian shape, which is often the casefor random processes.

If you take a closer look at the for both Gaussian fits, you will noticethat lower ∼ upper/

√�1000�. The standard deviation can be written as

=√

1N

f 2�xi�− � 1N

f�xi��2

N(10.14)

which immediately shows how ∝ N−1/2. In order to reduce the dispersionof our integral by 2 we need to throw four times as many points.

10.6 Exercises

1. Pi-mesons are unstable particles with a rest mass of m� ≈ 140 MeV/c2.Their lifetime in the rest system is � = 26−8 s. If their kinetic energyis 200 MeV, write a program which will simulate how many pions willsurvive after having traveled 20 m. Start with an initial sample of 108

pions. Assume that the pions are monoenergetic.2. Modify your program in such a way that the pions are not monoener-

getic, but have a Gaussian energy distribution around 200 MeV with = 50 MeV.

3. Use Monte Carlo integration to calculate∫ 1

0

ln�x�

1−xdx

References

[1] Rene Brun and Fons Rademakers, ROOT – an object oriented data analysis frame-work, Proceedings AIHENP’96 Workshop, Lausanne, September 1996, Nucl.Instrum. Methods Phys. Res. A 389 (1997) 81–86.

[2] Brian W. Kernighan and Dennis M. Ritchie, The C Programming Language.Englewood Cliffs, NJ: Prentice-Hall, 1978.

[3] Bjarne Stroustrup, The C++ Programming Language. New York: Addison-Wesley, 1986.

[4] S. Garfinkel, D. Weise and S. Strassmann, editors, The UNIX-Haters Handbook,San Mateo, CA: IDG Books, 1994.

[5] W. H. Press, S. A. Teukolsky, W. T. Vetterlin and B. P. Flannery, NumericalRecipes in C, p. 25. Cambridge: Cambridge University Press, 2nd edition, 1995.

[6] M. Abramowitz and I. A. Stegun, Handbook of Mathematical Functions, NewYork: Dover Publications, 1964.

[7] E. Fehlberg, Low order classical Runge–Kutta formulas with step-size controland their application to some heat transfer problems, NASA TR R-315, 1969.

[8] J. R. Cash and A. H. Karp, ACM Trans. Math. Software 16 (1990) 201.[9] S. Drell and T. M. Yan, Phys. Rev. Lett. 25 (1970) 316.

[10] D. Antreasyan et al., Phys. Rev. Lett. 48 (1982) 302.[11] ROOT Users’ Guide, CERN, 2001.

105

Appendix AThe ROOT system

A.1 What is ROOT

Root is a very powerful data analysis package which was developed and writ-ten at CERN, the European high energy accelerator lab. It consists of a libraryof C++ classes and functions for histogramming, data manipulation (like fit-ting and data reduction) and storage tools. There are two ways to use ROOT.One way is by calling the appropriate functions from your own program.The second way relies on a C++ interpreter and a graphical user interface,allowing you to display and manipulate data interactively. Originally writtenfor high energy physics, it is today also widely used in other fields, includingastrophysics, neural network applications, and financial institutions on WallStreet.

A.2 The ROOT basics

ROOT in the word of its authors is an “object oriented framework” [11].Instead of having to write your own graphics routines, or modify existingprograms to suit your needs, a framework provides you with functions androutines which are already well tested. ROOT, having been developed byphysicists for physics problems, gives you a large class of routines whichyou can just use “out of the box.”

One of the main advantages of ROOT is that it is running and supportedon all the major UNIX platforms, as well as on Windows XP and MacOS.This widespread availability makes it the perfect choice, reducing the chancesof having to learn a new package every time you change computers. Thisappendix is a short introduction. In order to take advantage of the full potentialof the ROOT package, we refer you to the ROOT users manual, which youcan find on the ROOT Web page http://root.cern.ch/root/RootDoc.html. Also

107

108 Appendix A

check out the main page which has lots of useful links. The most importantof these is the Reference Guide, which has a listing of all the classes anddocumentation on how to use them.

The only way of learning a new language or program is to use it. Thisappendix heavily relies on small exercises and problems, which you solveeither in the lab or at home with your computer. The first thing you have todo is find out whether the ROOT environment is defined or whether you haveto define it (assuming that it has been installed; to obtain the distribution,look in Appendix B.3). By typing echo $ROOTSYS, the system will tellyou whether this environment variable has been defined. Once you haveconvinced yourself that ROOT indeed is defined, we can explore the system.In this course, we will use ROOT with the graphical interface, reducing to aminimum the need for calls to the ROOT libraries. In most of the programs,the only reference to ROOT will be writing to a ROOT file and saving it.Once your program has finished (hopefully successfully), you will then startup ROOT.

A.3 The first steps

This section introduces you to writing your output to a ROOT file, using asimple program, which will calculate the sin of an angle.

// program to calculate the sin for a given interval and step size// this is the first step to using ROOT.# include "iostream.h"# include "math.h"

void main(){double xmin, xmax; // the lower and upper bounds of the intervaldouble xval; //the value where we calculate the sindouble step; // the step size for the angle incrementdouble result;double const deg_rad = 3.141592653589793/180.; // converts deg toradians

cout << "This programs calculates the sin for a range of angles \nYou have to give the lower and upper boundaries for the angles \nand the step size \n";

cout << "give lower and upper angle limit";cin >> xmin >> xmax;

The ROOT system 109

cout << "Give step size";cin >> step;

// here we loop over the angles

step = step*deg_rad; // convert the step size into radiansxmin = xmin*deg_rad;xmax = xmax*deg_rad;

for (xval = xmin; xval<=xmax; xval+=step){result = sin(xval);cout << "The sin of" << xval/deg_rad <<"degrees is" <<result <<"\n";

}}

This is a very simple program and will output the values for sin(x) ontoyour screen. This is not very useful if you want to see a whole list of values.The first thing you could do would be to redirect the output from the programinto a file; i.e., sin1 ¿ sin.dat and then plot this output file with plottingsoftware.

In the next step we will take this program and modify it in such a waythat it will send the output to a ROOT file. In ROOT, because it deals withobjects, these are also the quantities you write to a file. You can save anyobject you created in your program or your interactive section to a file, andlater open this file again with either a program or the interactive version.However, before you can use these features you have to initialize the ROOTsystem in your program. You will also need to include the header filesbelonging to the specific class you want to use. There are two different waysto get the output into a file. One way is to use a histogram from the onedimensional histogram class TH1 and using the TGraph class.

Creating a histogram file

In this example we have used the TH1 class to create a histogram.

// program to calculate the sin for a given interval and step size// this is the modified sin1 program now using the ROOTsystem.

# include "iostream.h"# include "math.h"

110 Appendix A

// Here starts the block of include files for ROOT//**************************************************************//#include "TROOT.h" // the main include file for the root system#include "TFile.h" // include file for File I/O in root#include "TH1.h" // To create histograms//**************************************************************//

void main(){double xmin, xmax; // the lower and upper bounds of the intervaldouble xval; //the value where we calculate the sindouble step; // the step size for the angle incrementdouble result;double const deg_rad = 3.141592653589793/180.; // converts deg toradiansint nbin; // number of bins for histogram

//******************** ROOT***************************************TROOT root ("hello", "the sine problem"); //initialize rootTFile *out_file = new TFile("sin_histo.root", "RECREATE","sin_histo");// The RECREATE option, will create or overwrite the file if it alreadyexists.////****************ROOT*******************************************

cout << "This program calculates the sin for a range of angles \nYou have to give the lower and upper boundaries for the angles \nand the step size \n";

cout << "give lower and upper angle limit";cin >> xmin >> xmax;

cout << "Give step size";cin >> step;

// In order to define a histogram, we need to know how many bins we willhave.

// we calculate the number automatically from the upper and lowerlimits and divide by

// the step size.

nbin = abs(static_cast<int>((xmax-xmin)/step)) +1;

// Now define a pointer to a new histogram, which is defined for double

TH1D *hist1 = new TH1D("hist1", "sin(x)", nbin, xmin, xmax);


The ROOT system 111

step = step*deg_rad; // convert the step size into radiansxmin=xmin*deg_rad;xmax=xmax*deg_rad;

for(xval=xmin; xval<=xmax; xval+=step){result=sin(xval);

//Here we fill the bins of the histogram;hist1->Fill(xval/deg_rad,result);

}

//And last we need to write the histogram to disk and close the file

hist1->Write();out_file->Close();

}

As you can see, by adding a few statements, dealing with the number ofbins and the ROOT classes, we are now ready to use this program to creategraphical output. However, we need to change the simple compile commandto include the ROOT libraries. Instead of a simple:

g++ -o sin1 sin1.cxx

as in the previous example, we have to tell the compiler what needs to beincluded, and where the necessary files can be found. A simple script to dothis, called make_sin2 is shown in the next paragraph (this script uses themake facility):

# simple scriptfile to compile sin2# First define the ROOT stuffROOTCFLAGS = $(shell root-config --cflags)ROOTLIBS = $(shell root-config --libs)ROOTGLIBS = $(shell root-config --glibs)CXXFLAGS += $(ROOTCFLAGS)LIBS = $(ROOTLIBS)GLIBS = $(ROOTGLIBS)

sin2:g$++$ -o sin2 sin2.cxx $ (ROOTCFLAGS) $ (ROOTGLIBS) $ (ROOTGLIBS)

Note: g++ is indented by a TAB, which is required by the make facil-ity, and belongs to the target sin2. To execute this command you wouldtype

112 Appendix A

make -f make_sin2 sin2

This will then produce an executable called sin2.

Creating a file with a graph

In this example we are taking advantage of the TGraph class in root. Ina graph you create pairs of x and y values and then plot them. You firstdecide how many points you want to include in your graph, dimension x andy accordingly, and simply create a new object. Be careful that you are nottrying to create more points than you have dimensioned.

// program to calculate the sin for a given interval and step size// this is the modified sin1 program now using the ROOT system.// Contrary to sin2 this one uses graphs instead of histograms.

# include "iostream.h"# include "math.h"

// Here starts the block of include files for ROOT//*************************************************************//#include "TROOT.h" // the main include file for the root system#include "TFile.h" // include file for File I/O in root#include "TGraph.h" // To create a graphics//*************************************************************//

void main(){int array=100;// In order for the graph to be used we need to have an array of x and y.// We dimension it for 100, but then have to make sure later on that we// are not running out of boundary of the array

double xmin, xmax; // the lower and upper bounds of the intervaldouble xval1 [array]; //the value where we calculate the sindouble step; // the step size for the angle incrementdouble xval;double result[array];

double const deg_rad = 3.141592653589793/180.; // converts deg toradiansint nbin; // number of bins for histogram

The ROOT system 113

//******************* ROOT**********************************TROOT root("hello", "the sine problem"); //initialize rootTFile * out_file = new TFile ("sin_graph.root", "RECREATE","sin_graph");// The RECREATE option, will create or overwrite the file if it alreadyexists.////*********************** ROOT************************************

cout << " This programs calculates the sin for a range of angles \nYou have to give the lower and upper boundaries for the anglesand the step size \n";

cout << " give lower and upper angle limit ";cin >> xmin >> xmax;

cout << " Give step size ";cin >> step;

// Here we will check that we do not have more points than we defined inthe array.

nbin = abs(static_cast<int>((xmax-xmin)/step)) +1;if(nbin>array){

cout << " array size too small \n";out_file->Close();exit;}

// Now define a pointer to a new histogram, which is defined for double


step = step*deg_rad; // convert the step size into radiansxmin=xmin* deg_rad;xmax=xmax* deg_rad;array=0;

for( xval=xmin ; xval<=xmax ;xval+=step){

result[array] = sin(xval);xval1[array]=xval/deg_rad;++array;

}

// Here we create the graphTGraph * graph =new TGraph(array,xval1,result);

114 Appendix A

//And last we need to write the graph to disk and close the filegraph->Write();out_file->Close();}

Again you have to modify your compile script.

A.4 Lab ROOT

In this lab section we will try to show you how easy it is to calcu-late and plot functions with ROOT. Now that you have created two files,sin_histo.root and sin_graph.root, we can explore ROOT. Thefirst thing you need to do is get ROOT up: root. This will bring you to thecommand line environment.

*************************************

* *

* W E L C O M E to R O O T *

* *

* Version 3.01/00 9 May 2001 *

* You are welcome to visit our Web site *

* http://root.cern.ch *

* *

*************************************

Compiled with thread support.CINT/ROOT C/C++ Interpreter version 5.14.83, Apr 5 2001Type ? for help. Commands must be C++ statements.Enclose multiple statements between { }.root [0]

The first thing you want to create is a so-called canvas, which will be theplace where you will see all your plots. Because ROOT brings you into aC-interpreter, you execute commands as you would in your C++ program.

The ROOT system 115

The command for creating the canvas is

TCanvas ∗tc = new TCanvas(“tc,” “my first canvas”)

which will produce a plotting area in the upper left corner. To get control

over your files, you should also start a browser, which will display the

content of your directories with icons.

TBrowser ∗tb = new TBrowser(“tb,” “my browser”,500,500,500,400)

where we have given it x and y coordinates on screen and width and height

of the browser. Now we are ready to draw our functions. In the browser,

double click on one of the .root files, go to the ROOT files directory and

double click on it again. It will then plot the chosen function.

116 Appendix A

A.5 Exercises

1. Modify the sin program in such a way that the amplitude of the sin decaysfrom max to 0.1 over three full cycles. Choose either the histogram orgraph version.

2. Do the same in ROOT.3. Create a two dimensional plot of sin(x) versus cos(x).4. Create a Gaussian distribution and plot it.

Appendix BFree scientific libraries

In the past, the dominant computer language for scientific computing wasFORTRAN. This created a wealth of libraries with well tested routines, whichaddress almost all numerical problems covered in this book. However, usingthese libraries “out of the box,” without understanding their limitations andalgorithms, is bound to get you into trouble one day. Because we are usingC++, you will want to know how to call these FORTRAN routines from aC++ program. This will allow you to use these routines without having torewrite them in C++.

B.1 LAPACK

LAPACK: Linear Algebra PACKage.

LAPACK is written in Fortran77 and provides routines for solving systems of

simultaneous linear equations, least-squares solutions of linear systems of

equations, eigenvalue problems, and singular value problems. The associated

matrix factorizations (LU, Cholesky, QR, SVD, Schur, generalized Schur) are

also provided, as are related computations such as reordering of the Schur

factorizations and estimating condition numbers. Dense and banded matrices are

handled, but not general sparse matrices. In all areas, similar functionality is

provided for real and complex matrices, in both single and double precision.

From the Website http://www.netlib.org/lapack/index.html, where the wholepackage can be downloaded. There is now also a C++ version available,called LAPACK++.

117

118 Appendix B

B.2 SLATEC

SLATEC is another very useful package which contains general pur-pose mathematical and statistical programs. You will find this athttp://www.netlib.org/slatec/index.html.

B.3 Where to obtain ROOT

An important aspect about ROOT to remember is the developer’s philosophy:“Release early and release often.” This requires you to check periodicallyon the main website (http://root.cern.ch) for new releases. You can eitherdownload the package in compiled form, or you can get the source code andbuild the system yourself.

Appendix CFORTRAN and C++

Before we can discuss how to call FORTRAN functions from C++, we needfirst to look at some of the differences.

1. The most important difference is that FORTRAN passes variables byreference, while C++, like C, passes by value. This in turn means thatany FORTRAN routine expects to get a reference, so you have to makesure your program passes the variables appropriately.

2. When you use arrays in your program, you have to take into account thatthe first array element in FORTRAN is a(1), while in C++ it would bea[0]. Multi-dimensional arrays are stored differently in memory. ArrayA(3,5) in FORTRAN would be A(1,1), A(2,1), A(3,1), A(1,2), while inC++ the corresponding arrangement would be A(0,0), A(0,1), A(0,2),A(0,3), A(0,4), A(1,0) and so on.

3. C++ is a strongly typed language; if you do not define a variable yourprogram will not compile. FORTRAN has by default an implicit typeconvention: any variable which starts with letters i through n is an integer,while any other variable is a real variable. Unless the programmer hasused the “implicit none” statement any FORTRAN compiler will adhereto the standards.

4. FORTRAN is case insensitive, while C++ clearly distinguishes betweenDummy and dummy.

5. Some compilers add an underscore to the name when they compile theprogram. You can list the contents of your library with ar -t somelib.a tosee whether this is the case. Your compiler will usually have a switch (atleast in UNIX) which is -nosecondunderscore.

6. How to call FORTRAN from C++ is strongly dependent on the C++compiler you use. So if the Linux/g++ recipe does not work for you, youhave to experiment.

119

120 Appendix C

C.1 Calling FORTRAN from C++The following shows an example of calling a routine from the SLATECpackage, namely a Gaussian integration routine using Legendre polynomials.Only the statements which have to deal with this call are given:

// code snippet for function to integrate with gauss quadrature// uses dgaus8.f from lapack.// ak 9/2000//#include <iostream.h>#include <fstream.h>#include <math.h>#include <iomanip.h>#include <stdio.h>

extern "C" double dgaus8_(double (*func)(double*), double*,double*, double* ,double*, int*); // This program uses theslatec double precision gaussian

//integration. You pass it a pointer to the function you// want to integrate.

main(){double integral=0.; // the calculated integraldouble x_low; //lowerdouble x_high; //upper limits

int n_point; // number of integration pointsdouble const deg2rad=3.14159265/180.; // converts degrees intoradiansdouble err=1.e-15; // tolerated errorint ierr=0;...dgaus8_(&func,&x_low,&x_high,&err,&integral,&ierr);

// compile and link// g++ -o gaus_int gaus_int.cxx -lslatec -llapack˜-lg2c

Note the underscore at dgaus8_ and how all the variables are passed aspointers. At the end are included the compile and link commands for ourLinux system. It links against SLATEC and LAPACK as well as libg2c.a.SLATEC calls some routines in LAPACK and the g2c library, which is theFORTRAN run time library.

Appendix DProgram listings

D.1 Simple Euler

Spring.cxx

// program to calculate the mass on a spring problem.// m=k=1; therefore a=-x// with the simple euler method// ak 9/7/00// all in double precision#include <iostream.h>#include <fstream.h>#include <math.h>#include <iomanip.h>#include <stdio.h>#include "TROOT.h"#include "TApplication.h"#include "TCanvas.h"#include "TLine.h"#include "TGraph.h"TROOT root ("hello", "Spring Problem"); // initialize root

int main (int argc, char **argv){

double v_new[100]; // the forward value calculateddouble v_old=5.; // the previous value; for the firstcalculation

// the boundary conditiondouble x_new[100]; // the forward x-valuedouble x_old=0.;// the x value from previousdouble energy[100]; // the energy from mv**2/2+ kx**2/2double ti[100]; // array to hold the timedouble step =0.1; // the step size for the euler methoddouble time=0. ;// the time

int loop = 0; // the loop counter

// first the root stuffTApplication theApp("App", &argc, argv);

121

122 Appendix D

TCanvas *c = new TCanvas ("c", "Hyperbola", 600, 600);//* create Canvas

// create the graphs //

while (loop <100)

{V_new[loop]=v_old-step*x_old; // calculate theforward velocityx_new[loop]=x_old+step*v_old; // calculate theforward positionenergy[loop]=pow(x_new[loop],2)/2+pow(v_new[loop],2)/2.; // energy conservationenergy[loop]=energy[loop]/5. ;// rescaled forplotting purposes

time=time+step; // move forward in time bythe stepv_old=v_new[loop]; // the new value becomes nowthe oldx_old=x_new[loop];ti[loop]=time;time=time+step; // move forward in time bythe steploop=loop+1;}

TGraph *root1 = new TGraph(100,ti,x_new);TGraph *root2 = new TGraph(100,ti,v_new);TGraph *root3 = new TGraph(100,ti,energy);

root1->SetTitle("Spring with simple EulerMethod");

root1->SetLineColor(1);root1->Draw("AC");root2->SetLineWidth(2);root2->SetLineColor(2);root2->Draw("C");root2->SetLineWidth(2);root3->SetLineColor(4);root3->Draw("C");

theApp.Run();

}

// compile g++ -o spring spring.cxx with root libraries

Program listings 123

D.2 Runge–Kutta program

rk4.cxx

// function to calculate step through ODE by// 4th and 5th order Runge Kutta Fehlberg.// calculates both orders and gives difference back// ak 2/2/2001// all in double precision#include <iostream.h>#include <fstream.h>#include <math.h>#include <iomanip.h>#include <stdio.h>void deri( double x, double y[], double f[], double omega2,double gam);double omega2;double gam;int len=2;

double y[2],y_old[2];double f0[2], f1[2], f2[2], f3[2], f4[2], f[2];

main( ){

// constants for the RKF methoddouble const a0 =.25, a1=3./8. , a2=12./13. ;double const b10 =.25 ;double const b20 =3./32. , b21 = 9./32. ;double const b30 =1932./2197. , b31 = -7200./2197. ,b32 = 7296./2197. ;double const b40 =439./216. , b41 = -8. ,b42=3680./513., b43=-845./4104.,

double const r1 = 25./216. , r2 = 1408./2565. ,r3 = 2197./4104. , r4=-1./5. ;// end of constants

// user inputdouble spring=0.; // spring constantdouble mass=0.; // massdouble damp=0.; // dampingdouble v_0=0.; // initial condition for velocitydouble x_0=0.; // initial condition for amplitudedouble step=0.; // stepsizechar filename[80]; // plotfile// end user input

124 Appendix D

double y[2],y_old[2];double e_old, e_new;double f0[2], f1[2], f2[2] , f3[2] , f4[2], f[2];double result = 0;double x=0 , x0;int len=2;

***********************************************************// here we deal with the user input

cout << "give the filename for output:";cin >> filename;

// Now open the file

ofstream out_file(filename) ;// Open fileout_file.setf(ios::showpoint);// keep decimal point

cout << "give the step size:";cin >> step ;cout << "mass and damping";cin >> mass >> damp;cout << "spring constant";cin >> spring;cout << "initial position and velocity";cin >> x_0 >> v_0 ;omega2=spring/mass;gam=damp/(mass*2.);

***********************************************************// input is finished// let’s do the calculation

// use the initial conditions;x=0.;y[0]=x_0;y[1]=v_0;y_old[0]=y[0];y_old[1]=y[1];

while(x<20.){

// here starts the loopx0=x;deri (x0, &y[0], &f[0], omega2,gam);x=x0+a0*step;


y[0]=y_old[0]+b10*step*f[0];y[1]=y_old[1]+b10*step*f[1];

deri (x, &y[0], &f1[0], omega2,gam);x=x0+a1*step;

y[0]=y_old[0]+b20*step*f[0]+b21*step*fl[0];y[1]=y_old[1]+b20*step*f[1]+b21*step*fl[1];

deri (x, &y[0], &f2[0], omega2,gam);x=x0+a2*step;

y[0]=y_old[0]+b30*step*f[0]+b31*step*fl[0]+b32*step*f2[0];y[1]=y_old[1]+b30*step*f[1]+b31*step*fl[1]+b32*step*f2[1];

deri (x, &y[0], &f3[0], omega2,gam);x=x0+step;

y[0]=y_old[0]+b40*step*f[0]+b41*step*fl[0]+b42*step*f2[0]+b43*step*f3[0];y[1]=y_old[1]+b40*step*f[1]+b41*step*fl[1]+b42*step*f2[1]+b43*step*f3[1];

deri (x, &y[0], &f4[0], omega2,gam);y[0]=y_old[0]+step*(r1*f[0]+r2*f2[0]+r3*f3[0]+r4*f4[0]);y[1]=y_old[1]+step*(r1*f[1]+r2*f2[1]+r3*f3[1]+r4*f4[1]);

// cout << x << " " << y[0] <<" " << y[1] << "\ n";out_file << x << " " <<y[1] <<"\ n";y_old[0]=y[0];y_old[1]=y[1];}

// cout << f[0] << " " <<f[1] <<"\ n";// cout << f1[0] << " "<<fl[1] <<"\ n";

out_file.close();

}

void deri( double x, double y[ ], double f[ ], double omega2,double gam){

f[0]=y[1];f[1]=-omega2*y[0]-2*gam*y[1];return;

}// compile g++ damp.cxx

126 Appendix D

rk4_step.cxx

// Runge Kutta with adaptive step size control#include <iostream.h>

void rk4_step( double *y, double *ydiff, double *y_old,int nODE, double step, double x0 )

{

void deri ( int , double , double *, double *);// user supplied routine// which calculates derivative// setup the constants first

//the coefficients for the steps in xdouble const a2=1./5. , a3=3./10., a4=3./5., a5=1. ,a6=7./8. ;

// coefficents for the y valuesdouble const b21=1./5. ;double const b31=3./40. , b32=9./40. ;double const b41=3./10. , b42=-9./10., b43=6.5. ;double const b51=-11./54. , b52=5./2. , b53=-70./20. ,b54=-35./27.;double const b61=1631./55296. , b62=175./312. ,b63=575./13824. ;double const b64=44275./110592. , b65=253./1771. ;

// coefficents for y(n-th order)double const c1=37./378. , c2=0. , c3=250./621.,c4=125./594., c5=0., c6=512./1771. ;

// coefficents for y(n+1-th order)double const cs1=2825./27648. , cs2=0. ,cs3=18575./48384., cs4=13525./55296. ,cs5=277./14336., cs6 = 1./4. ;

// the calculated values fdouble f[nODE] , f1[nODE] , f2[nODE] , f3[nODE] ,f4[nODE] , f5[nODE];

// the x valuedouble x;double yp[nODE];int i;

// here starts the calculation of the RK parametersderi (i,x,y,f);


for(i=0; i<=nODE-1;i++) // 1. step{

y[i]=y_old[i]+b21*step*f[i];}x=x0+ a2*step;

deri(i,x,y,f1);for(i=0; i<nODE-1;i++) //2. step{y[i]=y_old[i]+b31*step*f[i]+b32*step*f1[i];

}x=x0+ a3*step;

deri(i,x,y,f2);for(i=0; i<=nODE-1;i++) //3. step{y[i]=y_old[i]+b41*step*f[i]+b42*step*f1[i]+b43*step*f2[i];

}x=x0+ a4*step;

deri (i,x,y,f3);for(i=0; i<=nODE-1;i++) //4. step{y[i]=y_old[i]+b51*step*f[i]+b52*step*fl[i]+b53*step*f2[i]+b54*step*f3[i];

}x=x0+ a5*step;

deri (i,x,y,f4);for(i=0; i<=nODE-1;i++) //5. step{y[i]=y_old[i]+b61*step*f[i]+b62*step*f1[i]+b63*step*f2[i]+b64*step*f3[i]+b65*step*f4[i];}x=x0+ a6*step;

deri (i,x,y,f5);for(i=0; i<=nODE-1;i++) //6. step{y[i]=y_old[i]+step*(c1*f[i]+c2*f1[i]+c3*f2[i]+c4*f3[i]+c5*f4[i]+c6*f5[i]);yp[i]=y_old[i]+step*(cs1*f[i]+cs2*f1[i]+cs3*f2[i]+cs4*f3[i]+cs5*f4[i]+cs6*f5[i]);ydiff[i]=fabs (yp[i]-y[i]);

}return;

}

128 Appendix D

rk4_stepper.cxx

// routine rk4_stepper// adaptive step size Runge Kutta ODE solver// uses rk4_step#include <iostream.h>#include <fstream.h>#include <math.h>#include <minmax.h>#include "TROOT.h"#include "TApplication.h"#include "TCanvas.h"#include "TLine.h"#include "TPaveLabel.h"#include "TRandom.h"#include "TH1.h"#include "TH2,h"#include "TH3.h"#include "TPad.h"

void rk4_stepper(double y_old [], int nODE, double xstart,double xmax, double hstep, double eps,int&nxstep)

{void rk4_step (double *, double *, double *, int ,double ,double);double heps; // the product of the step size and thechosen errordouble yerrmax=.99; // the max error in a step,int i=0;double const REDUCE=-.22 // reduce stepsize powerdouble esmall ; // the lower limit of precision, if theresult is smaller than this we increase the step sizedouble ydiff[nODE];double y[nODE];double hnew; // new step sizedouble x0;double xtemp; // temporary x for rk4_stepdouble step_lolim; // lower limit for step sizedouble step_hilim; // upper limit for step size

// here we create a file for storing the calculatedfunctionsofstream out_file;out_file.open ("rk4.dat");


out_file.setf(ios::showpoint | ios ::scientific |ios::right);

x0=xstart;for(i=0 ; i<=nODE-1 ; i++){// store starting values

out_file << x0 << " "<<Y_old[i]<<"\ n";}

esmall=eps/50.;heps=eps*hstep;

step_lolim=hstep/10.; // the step size should not golower than 10 %step_hilim=hstep*10.; // We don’t want larger step size

for(i=0; i<=nODE-1;i++){y[i]=y_old[i];}for( ; ; ){yerrmax=.99;for( ; ; )

{xtemp=x0+hstep;rk4_step(y,ydiff,y_old,nODE, hstep,xtemp);for(i=0; i<=nODE-1;i++){yerrmax=max(yerrmax, heps/ydiff[i]);

}if (yerrmax > 1.) break;hstep=.9*hstep*pow (yerrmax, REDUCE);

// error if step size gets too lowif (hstep<step_lolim){

cout << "rk4_stepper: lower steplimit reacher; trylower starting"

<< "step size\ n" ;cout << "I will terminate now \ n";exit(0)

}}

130 Appendix D

// go further by one step// first check if our step size is toosmall

if (yerrmax>1./esmall)hstep=hstep*2.;

// set upper limit for step sizeif (hstep>step_hilim) {hstep=step_hilim;}for (i=0 ; i<=nODE-1 ; i++){

y_old[i]=y[i];// store data in file rk4.dat

out_file << xtemp << ""<<y[i]<<"\ n";

}// x0 += hstep;

x0=xtemp;nxstep=nxstep+1;if(x0>xmax){cout << nxstep;out_file.close(); // close data filereturn ;

}}return;

}


D.3 Random walk in two dimensions

rnd_walk2.cxx

// rnd_walk2.cxx// Random walk in two dimension// ak 4/30/2002#include <iostream>#include <fstream>#include <math.h>#include <iomanip>#include <stdio.h>#include <stdlib.h>#include <fcnt1.h>

#include "TROOT.h"#include "TTree.h" // we will use a tree to store our values#include "TApplication.h"#include "TFile.h"

int rand(void);void Srand(unsigned int);

void main(int argc, char ** argv){

//structure for our random walk variablesstruct walk_t { double x; // the position after a step

double y; //int nstep; // the step numberint jloop; // number of outer loops};

walk_t walk;

int i_loop; //inner loopint j_loop; //outer loopint jloop_max=5000; // the max number ofdifferent trials

unsigned int seed = 557 ; // here is the starting valueor seed

int loop_max=100; // the maximum number of steps

double rnd1;double rnd2;double y_temp; // temporary variable for y

132 Appendix D

TROOT root ("hello", "computational physics");

// initialize rootTApplication theApp("App", &argc, argv);

//open output fileTFile *out_file = new TFile("rnd_walk557_2.root","RECREATE","example of random random walk");// create root file

// Declare treeTTree *ran_walk = new TTree("ran_walk","tree withrandom walk variables");ran_walk->Branch("walk",&walk.x, "x/D:y/D:nstep/I:jloop/I");

// set seed valuesrand(seed);

// the outer loop, trying the walk jloop timesfor (j_loop=0;j_loop < jloop_max ; j_loop= j_loop+1){

walk.x=0.;walk.y=0.;walk.nstep=0;walk.jloop=j_loop+1;

for(i_loop=0; i_loop < loop_max ;i_loop= i_loop+1){// here we get the step

rnd1=double(rand())/double(RAND_MAX);rnd1=2*rnd1-1.;walk.x=walk.x+rnd1;

if(rnd1*rnd1>1.) rnd1=1.; //safety forsquare rootY_temp=sqrt(1.-rnd1*rnd1);

rnd2=double(rand())/double(RAND_MAX);if((rnd2-.5)<0.){walk.y=walk.y-y_temp;}else{walk.y=walk.y+y_temp;}


walk.nstep=walk.nstep+1;// fill the treeran_walk->Fill();

}}

out_file->write();}

134 Appendix D

D.4 Acceptance and rejection method with sin(x)distribution

rnd_accept.cxx

// rnd_invert.cxx// Random number using the acceptance / inversion method// this simple program uses the sin function as the probability// distribution// ak4/30/2002#include <iostream>#include <fstream>#include <math.h>#include <stdio.h>#include <stdlib.h>#include <fcnt1.h>

#include "TROOT.h"#include "TTree.h" // we will use a tree to store our values#include "TApplication.h"#include "TFile.h"#include "TFl.h"

int rand(void);void srand(unsigned int);

void main(int argc, char ** argv){

double x_low=0.; // lower limit of our distributiondouble x_high=180.; // the upper limitdouble deg_rad=3.14159265/180.; //converts degrees inrads

//structure for our distribution variablesstructure dist_t { doubt x_throw; // the thrown value

double x_acc; // the accepted valuedouble y_throw;double y_acc;};

dist_t dist;

int i_loop; //inner loop

unsigned int seed = 68910 ; // here is the startingvalue or seedint loop_max=1000000; // the maximumnumber of steps


double rnd;

TROOT root ("hello", "computational physics");

// initialize rootTApplication theApp(*App*, &argc, argv);

//open output fileTFile *out_fill = new TFile("rnd_acc.root","RECREATE", "A distribution following a sine");// create root file

// Declare treeTTree *dist_tree = new TTree("dist_tree","treewith rejection");

dist_tree->Branch("dist", &dist.x_throw,"x_throw/D:x_acc/ D:y_throw/D:y_acc");

// set seed valuesrand(seed);

for(i_loop=0;i_loop < loop_max= i_loop=i_loop+1){// step 1: throw x between 0 and 180dist.x_throw=x_low+double(rand())/double(RAND_MAX)*(x_high-x_low)*deg_rad;

//step 2: create a random variable in Y between 0 and 1dist.y_throw=1.*double(rand())/double(RAND_MAX); // from 0,1

//step 3: Check it f(x)>y and if true acceptif(sin(dist.x_throw)>dist,y_throw){dist.x_acc=dist.x_throw/deg_rad;dist.y_acc=dist.y_throw;

else// these are the rejected ones.

{dist.x_acc=-999;dist.y_acc=-999;}dist_three->Fill();

}out_file->Write();

}

Index

Apple, 2,7awk, 8, 12bias, 22, 90bisectional, 51–2

Borland, 3

C++, 2–4, 9, 18, 37, 84,107, 119

cache, 5–6cat, 16–17central difference, 38–9Chebyshev, 49compiler, 3, 9, 18, 111, 119cp, 9, 16–17CPU, 5–6, 37

derivatives, 37–40, 74,79, 126

differential equations, 55–81directory, 14–15, 115

Euler method, 57–64, 65–70

Fehlberg, 70file system, 12–13fit, 31, 34, 103floating point number, 22FORTRAN, 3–4, 8, 9, 18,

84, 119

Gauss–Legendre, 44, 48Gaussian integration, 44, 49

GNU, 3, 8, 18

harmonic oscillator, 57, 60–1,62, 64, 65, 72–3

help, 12, 18, 114info, 18

Lagrange interpolation, 27–8Laguerre, 48–91LAPACK, 3, 9, 117less, 16–17linear interpolation, 28, 30, 31Linux, 2–3, 11–23, 92.login, 11, 13, 14, 23ls, 12, 14–15, 18, 95

man, 12, 18mantissa, 22–3memory, 5–6, 119Microsoft, 2, 3, 11midpoint, 62–6mkdir, 14, 15, 123–30modified Euler, 62–5Monte Carlo, 89–103mv, 16, 55, 61, 121

nedit, 10, 15, 18Newton’s method, 52, 53nonlinear equations, 51–3

ODE, 56–7, 67, 74, 75–80

password, 11–12pipe, 16polynomial, 31–2, 44–9

precision,20–3, 66, 74–7, 90,121, 123, 128

pwd, 13–14, 15

quadratic equation, 51

RAM, 5–6

random, 89–103

random numbers, 89–92,97–101

rational function, 34–5

Red Hat, 81

rejection method, 97–9,134–5

rm, 9, 14–15, 17

Runge–Kutta, 65–72, 73–4

secant, 52–3

shell, 10, 12, 17, 111

simple Euler, 58–62, 121–2

simulation, 89–103

splines, 33–4

Stroustrup, 3

Taylor, 38, 43, 52, 58, 65

Taylor series, 38, 52, 58,62, 65

terminal, 11, 16

X-Windows, 10, 16, 18

137

Date post:	19-May-2018
Category:	Documents
Upload:	voliem
View:	223 times
Download:	4 times

This page intentionally left blank - Julian Oliver · · 2009-10-01Introducing students to...

Documents