Numerical analysisFrom Wikipedia, the free encyclopediaJump to: navigation, search

Babylonian clay tablet BC 7289 (c. 1800–1600 BC) with annotations. The approximation of the square root of 2 is four sexagesimal figures, which is about six decimal figures. 1 + 24/60 + 51/602 + 10/603 = 1.41421296...[1] Image by Bill Casselman.[2]

Numerical analysis is the study of algorithms that use numerical approximation (as opposed to general symbolic manipulations) for the problems of continuous mathematics (as distinguished from discrete mathematics).

One of the earliest mathematical writings is the Babylonian tablet BC 7289, which gives a

sexagesimal numerical approximation of , the length of the diagonal in a unit square. Being able to compute the sides of a triangle (and hence, being able to compute square roots) is extremely important, for instance, in carpentry and construction.[3]

Numerical analysis continues this long tradition of practical mathematical calculations. Much

like the Babylonian approximation of , modern numerical analysis does not seek exact answers, because exact answers are often impossible to obtain in practice. Instead, much of numerical analysis is concerned with obtaining approximate solutions while maintaining reasonable bounds on errors.

Numerical analysis naturally finds applications in all fields of engineering and the physical sciences, but in the 21st century, the life sciences and even the arts have adopted elements of scientific computations. Ordinary differential equations appear in the movement of heavenly bodies (planets, stars and galaxies); optimization occurs in portfolio management; numerical linear algebra is important for data analysis; stochastic differential equations and Markov chains are essential in simulating living cells for medicine and biology.

Before the advent of modern computers numerical methods often depended on hand interpolation in large printed tables. Since the mid 20th century, computers calculate the required functions instead. The interpolation algorithms nevertheless may be used as part of the software for solving differential equations.

Contents

[hide]

1 General introduction o 1.1 History o 1.2 Direct and iterative methods

1.2.1 Discretization and numerical integration o 1.3 Discretization

2 The generation and propagation of errors o 2.1 Round-off o 2.2 Truncation and discretization error o 2.3 Numerical stability and well-posed problems

3 Areas of study o 3.1 Computing values of functions o 3.2 Interpolation, extrapolation, and regression o 3.3 Solving equations and systems of equations o 3.4 Solving eigenvalue or singular value problems o 3.5 Optimization o 3.6 Evaluating integrals o 3.7 Differential equations

4 Software 5 See also 6 Notes 7 References 8 External links

[edit] General introduction

The overall goal of the field of numerical analysis is the design and analysis of techniques to give approximate but accurate solutions to hard problems, the variety of which is suggested by the following.

Advanced numerical methods are essential in making numerical weather prediction feasible.

Computing the trajectory of a spacecraft requires the accurate numerical solution of a system of ordinary differential equations.

Car companies can improve the crash safety of their vehicles by using computer simulations of car crashes. Such simulations essentially consist of solving partial differential equations numerically.

Hedge funds (private investment funds) use tools from all fields of numerical analysis to calculate the value of stocks and derivatives more precisely than other market participants.

Airlines use sophisticated optimization algorithms to decide ticket prices, airplane and crew assignments and fuel needs. This field is also called operations research.

Insurance companies use numerical programs for actuarial analysis.

The rest of this section outlines several important themes of numerical analysis.

[edit] History

The field of numerical analysis predates the invention of modern computers by many centuries. Linear interpolation was already in use more than 2000 years ago. Many great mathematicians of the past were preoccupied by numerical analysis, as is obvious from the names of important algorithms like Newton's method, Lagrange interpolation polynomial, Gaussian elimination, or Euler's method.

To facilitate computations by hand, large books were produced with formulas and tables of data such as interpolation points and function coefficients. Using these tables, often calculated out to 16 decimal places or more for some functions, one could look up values to plug into the formulas given and achieve very good numerical estimates of some functions. The canonical work in the field is the NIST publication edited by Abramowitz and Stegun, a 1000-plus page book of a very large number of commonly used formulas and functions and their values at many points. The function values are no longer very useful when a computer is available, but the large listing of formulas can still be very handy.

The mechanical calculator was also developed as a tool for hand computation. These calculators evolved into electronic computers in the 1940s, and it was then found that these computers were also useful for administrative purposes. But the invention of the computer also influenced the field of numerical analysis, since now longer and more complicated calculations could be done.

[edit] Direct and iterative methods

Direct vs iterative methods

Consider the problem of solving

3x3+4=28

for the unknown quantity x.

Direct method

3x3 + 4 = 28.Subtract 4 3x3 = 24.Divide by 3 x3 = 8.Take cube roots x = 2.For the iterative method, apply the bisection method to f(x) = 3x3 - 24. The initial values are a = 0, b = 3, f(a) = -24, f(b) = 57.

Iterative methoda b mid f(mid)

0 3 1.5 -13.8751.5 3 2.25 10.17...1.5 2.25 1.875 -4.22...1.875 2.25 2.0625 2.32...We conclude from this table that the solution is between 1.875 and 2.0625. The algorithm might return any number in that range with an error less than 0.2.

[edit] Discretization and numerical integration

In a two hour race, we have measured the speed of the car at three instants and recorded them in the following table.

Time0:20 1:00 1:40

km/h 140 150 180A discretization would be to say that the speed of the car was constant from 0:00 to 0:40, then from 0:40 to 1:20 and finally from 1:20 to 2:00. For instance, the total distance traveled in the first 40 minutes is approximately (2/3h x 140 km/h)=93.3 km. This would allow us to estimate the total

distance traveled as 93.3 km + 100 km + 120 km = 313.3 km, which is an example of numerical integration (see below) using a Riemann sum, because displacement is the integral of velocity.

Ill posed problem: Take the function f(x) = 1/(x − 1). Note that f(1.1) = 10 and f(1.001) = 1000: a change in x of less than 0.1 turns into a change in f(x) of nearly 1000. Evaluating f(x) near x = 1 is an ill-conditioned problem.

Well-posed problem: By contrast, the

function is continuous and so evaluating it is well-posed, at least for x being not close to zero.

Direct methods compute the solution to a problem in a finite number of steps. These methods would give the precise answer if they were performed in infinite precision arithmetic. Examples include Gaussian elimination, the QR factorization method for solving systems of linear equations, and the simplex method of linear programming. In practice, finite precision is used and the result is an approximation of the true solution (assuming stability).

In contrast to direct methods, iterative methods are not expected to terminate in a number of steps. Starting from an initial guess, iterative methods form successive approximations that converge to the exact solution only in the limit. A convergence test is specified in order to decide when a sufficiently accurate solution has (hopefully) been found. Even using infinite precision arithmetic these methods would not reach the solution within a finite number of steps (in general). Examples include Newton's method, the bisection method, and Jacobi iteration. In computational matrix algebra, iterative methods are generally needed for large problems.

Iterative methods are more common than direct methods in numerical analysis. Some methods are direct in principle but are usually used as though they were not, e.g. GMRES and the conjugate gradient method. For these methods the number of steps needed to obtain the exact solution is so large that an approximation is accepted in the same manner as for an iterative method.

[edit] Discretization

Furthermore, continuous problems must sometimes be replaced by a discrete problem whose solution is known to approximate that of the continuous problem; this process is called discretization. For example, the solution of a differential equation is a function. This function must be represented by a finite amount of data, for instance by its value at a finite number of points at its domain, even though this domain is a continuum.

[edit] The generation and propagation of errors

It has been suggested that this article or section be merged into Discretization error. (Discuss)

The study of errors forms an important part of numerical analysis. There are several ways in which error can be introduced in the solution of the problem.

[edit] Round-off

Round-off errors arise because it is impossible to represent all real numbers exactly on a machine with finite memory (which is what all practical digital computers are).

[edit] Truncation and discretization error

Truncation errors are committed when an iterative method is terminated or a mathematical procedure is approximated, and the approximate solution differs from the exact solution. Similarly, discretization induces a discretization error because the solution of the discrete problem does not coincide with the solution of the continuous problem. For instance, in the iteration in the sidebar to compute the solution of 3x3 + 4 = 28, after 10 or so iterations, we conclude that the root is roughly 1.99 (for example). We therefore have a truncation error of 0.01.

Once an error is generated, it will generally propagate through the calculation. For instance, we have already noted that the operation + on a calculator (or a computer) is inexact. It follows that a calculation of the type a+b+c+d+e is even more inexact.

What does it mean when we say that the truncation error is created when we approximate a mathematical procedure. We know that to integrate a function exactly requires one to find the sum of infinite trapezoids. But numerically one can find the sum of only finite trapezoids, and hence the approximation of the mathematical procedure. Similarly, to differentiate a function, the differential element approaches to zero but numerically we can only choose a finite value of the differential element.

[edit] Numerical stability and well-posed problems

Numerical stability is an important notion in numerical analysis. An algorithm is called numerically stable if an error, whatever its cause, does not grow to be much larger during the calculation. This happens if the problem is well-conditioned, meaning that the solution changes by only a small amount if the problem data are changed by a small amount. To the contrary, if a problem is ill-conditioned, then any small error in the data will grow to be a large error.

Both the original problem and the algorithm used to solve that problem can be well-conditioned and/or ill-conditioned, and any combination is possible.

So an algorithm that solves a well-conditioned problem may be either numerically stable or numerically unstable. An art of numerical analysis is to find a stable algorithm for solving a well-posed mathematical problem. For instance, computing the square root of 2 (which is roughly 1.41421) is a well-posed problem. Many algorithms solve this problem by starting with

an initial approximation x1 to , for instance x1=1.4, and then computing improved guesses x2, x3, etc... One such method is the famous Babylonian method, which is given by xk+1 = xk/2 + 1/xk. Another iteration, which we will call Method X, is given by xk + 1 = (xk

2−2)2 + xk.[4] We have calculated a few iterations of each scheme in table form below, with initial guesses x1 = 1.4 and x1 = 1.42.

Babylonian Babylonian Method X Method Xx1 = 1.4 x1 = 1.42 x1 = 1.4 x1 = 1.42x2 = 1.4142857... x2 = 1.41422535... x2 = 1.4016 x2 = 1.42026896x3 = 1.414213564... x3 = 1.41421356242... x3 = 1.4028614... x3 = 1.42056...

... ...x1000000 = 1.41421... x28 = 7280.2284...

Observe that the Babylonian method converges fast regardless of the initial guess, whereas Method X converges extremely slowly with initial guess 1.4 and diverges for initial guess 1.42. Hence, the Babylonian method is numerically stable, while Method X is numerically unstable.

Numerical stability is affected by the number of the significant digits the machine keeps on, if we use a machine that keeps on the first four floating-point digits,a good example on loss of significance these two equivalent functions

if we compare the results of

and

by looking to the two above results, we realize that loss of significance which is also called Subtractive Cancelation has a huge effect on the results, even though both functions are equivalent; to show that they are equivalent simply we need to start by f(x) and end with g(x), and so

the true value for the result is 11.174755...which is exactly g(500)=11.1748 after rounding the result to 4 decimal digitsnow imagine that you use tens of terms like these functions in your program, your error will increase as you proceed in the program, unless you use the suitable formula of the two functions each time you evaluate either f(x), or g(x), the choice is dependent on the parity of x .

The example is taken from Mathew; Numerical methods using matlab , 3rd ed.

[edit] Areas of study

The field of numerical analysis is divided into different disciplines according to the problem that is to be solved.

[edit] Computing values of functions

Interpolation: We have observed the temperature to vary from 20 degrees Celsius at 1:00 to 14 degrees at 3:00. A linear interpolation of this data would conclude that it was 17 degrees at 2:00 and 18.5 degrees at 1:30pm.

Extrapolation: If the gross domestic product of a country has been growing an average of 5% per year and was 100 billion dollars last year, we might extrapolate that it will be 105 billion dollars this year.

Regression: In linear regression, given n points, we compute a line that passes as close as possible to those n

points.

Optimization: Say you sell lemonade at a lemonade stand, and notice that at $1, you can sell 197 glasses of lemonade per day, and that for each increase of $0.01, you will sell one less lemonade per day. If you could charge $1.485, you would maximize your profit, but due to the constraint of having to charge a whole cent amount, charging $1.49 per glass will yield the maximum income of $220.52 per day.

Differential equation: If you set up 100 fans to blow air from one end of the room to the other and then you drop a feather into the wind, what happens? The feather will follow the air currents, which may be very complex. One approximation is to measure the speed at which the air is blowing near the feather every second, and advance the simulated feather as if it were moving in a straight line at that same speed for one second, before measuring the wind speed again. This is called the Euler method for solving an ordinary differential equation.

One of the simplest problems is the evaluation of a function at a given point. The most straightforward approach, of just plugging in the number in the formula is sometimes not very efficient. For polynomials, a better approach is using the Horner scheme, since it reduces the necessary number of multiplications and additions. Generally, it is important to estimate and control round-off errors arising from the use of floating point arithmetic.

[edit] Interpolation, extrapolation, and regression

Interpolation solves the following problem: given the value of some unknown function at a number of points, what value does that function have at some other point between the given points?

Extrapolation is very similar to interpolation, except that now we want to find the value of the unknown function at a point which is outside the given points.

Regression is also similar, but it takes into account that the data is imprecise. Given some points, and a measurement of the value of some function at these points (with an error), we want to determine the unknown function. The least squares-method is one popular way to achieve this.

[edit] Solving equations and systems of equations

Another fundamental problem is computing the solution of some given equation. Two cases are commonly distinguished, depending on whether the equation is linear or not. For instance, the equation 2x + 5 = 3 is linear while 2x2 + 5 = 3 is not.

Much effort has been put in the development of methods for solving systems of linear equations. Standard direct methods, i.e., methods that use some matrix decomposition are Gaussian elimination, LU decomposition, Cholesky decomposition for symmetric (or hermitian) and positive-definite matrix, and QR decomposition for non-square matrices. Iterative methods such as the Jacobi method, Gauss–Seidel method, successive over-relaxation and conjugate gradient method are usually preferred for large systems.

Root-finding algorithms are used to solve nonlinear equations (they are so named since a root of a function is an argument for which the function yields zero). If the function is differentiable and the derivative is known, then Newton's method is a popular choice. Linearization is another technique for solving nonlinear equations.

[edit] Solving eigenvalue or singular value problems

Several important problems can be phrased in terms of eigenvalue decompositions or singular value decompositions. For instance, the spectral image compression algorithm [5] is based on the singular value decomposition. The corresponding tool in statistics is called principal component analysis.

[edit] Optimization

Main article: Optimization (mathematics)

Optimization problems ask for the point at which a given function is maximized (or minimized). Often, the point also has to satisfy some constraints.

The field of optimization is further split in several subfields, depending on the form of the objective function and the constraint. For instance, linear programming deals with the case that both the objective function and the constraints are linear. A famous method in linear programming is the simplex method.

The method of Lagrange multipliers can be used to reduce optimization problems with constraints to unconstrained optimization problems.

[edit] Evaluating integrals

Main article: Numerical integration

Numerical integration, in some instances also known as numerical quadrature, asks for the value of a definite integral. Popular methods use one of the Newton–Cotes formulas (like the midpoint rule or Simpson's rule) or Gaussian quadrature. These methods rely on a "divide and conquer" strategy, whereby an integral on a relatively large set is broken down into integrals on smaller sets. In higher dimensions, where these methods become prohibitively expensive in terms of computational effort, one may use Monte Carlo or quasi-Monte Carlo methods (see Monte Carlo integration), or, in modestly large dimensions, the method of sparse grids.

[edit] Differential equations

Main articles: Numerical ordinary differential equations and Numerical partial differential equations

Numerical analysis is also concerned with computing (in an approximate way) the solution of differential equations, both ordinary differential equations and partial differential equations.

Partial differential equations are solved by first discretizing the equation, bringing it into a finite-dimensional subspace. This can be done by a finite element method, a finite difference method, or (particularly in engineering) a finite volume method. The theoretical justification of these methods often involves theorems from functional analysis. This reduces the problem to the solution of an algebraic equation.

[edit] Software

Main articles: List of numerical analysis software and Comparison of numerical analysis software

Since the late twentieth century, most algorithms are implemented in a variety of programming languages. The Netlib repository contains various collections of software routines for numerical

problems, mostly in Fortran and C. Commercial products implementing many different numerical algorithms include the IMSL and NAG libraries; a free alternative is the GNU Scientific Library.

There are several popular numerical computing applications such as MATLAB, S-PLUS, LabVIEW, and IDL as well as free and open source alternatives such as FreeMat, Scilab, GNU Octave (similar to Matlab), IT++ (a C++ library), R (similar to S-PLUS) and certain variants of Python. Performance varies widely: while vector and matrix operations are usually fast, scalar loops may vary in speed by more than an order of magnitude.[6][7]

Many computer algebra systems such as Mathematica also benefit from the availability of arbitrary precision arithmetic which can provide more accurate results.

Also, any spreadsheet software can be used to solve simple problems relating to numerical analysis.

Error analysisFrom Wikipedia, the free encyclopediaJump to: navigation, search

This article does not cite any references or sources.Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. (September 2009)

Error analysis is the study of kind and quantity of error that occurs, particularly in the fields of applied mathematics (particularly numerical analysis), applied linguistics and statistics.

Contents

[hide]

1 Error analysis in numerical modeling 2 Error analysis in language teaching 3 Error analysis in molecular dynamics simulation 4 See also 5 References 6 External links

[edit] Error analysis in numerical modeling

In numerical simulation or modeling of real systems, error analysis is concerned with the changes in the output of the model as the parameters to the model vary about a mean.

For instance, in a system modeled as a function of two variables . Error analysis deals with the propagation of the numerical errors in and (around mean values and ) to error in (around a mean ).[1]

In numerical analysis, error analysis comprises both forward error analysis and backward error analysis. Forward error analysis involves the analysis of a function which is an approximation (usually a finite polynomial) to a function to determine the bounds on the error in the approximation; i.e., to find such that . Backward error analysis involves the analysis of the approximation function , to determine the bounds on the parameters such that the result

Numerical analysis is the area of mathematics and computer science that creates, analyzes, and implements algorithms for solving numerically the problems of continuous mathematics.  Such problems originate generally from real-world applications of algebra, geometry, and calculus, and they involve variables which vary continuously. These problems occur throughout the natural sciences, social sciences, medicine, engineering, and business. Beginning in the 1940's, the growth in power and availability of digital computers has led to an increasing use of realistic mathematical models in science, medicine, engineering, and business; and numerical analysis of increasing sophistication has been needed to solve these more accurate and complex mathematical models of the world. The formal academic area of numerical analysis varies from highly theoretical mathematical studies to computer science issues involving the effects of computer hardware and software on the implementation of specific algorithms.

Contents

[hide]

1 Areas of numerical analysis o 1.1 Systems of linear and nonlinear equations o 1.2 Approximation theory o 1.3 Numerical solution of differential and integral equations

2 Some common viewpoints and concerns in numerical analysis 3 Development of numerical methods

o 3.1 Numerical solution of systems of linear equations o 3.2 Numerical solution of systems of nonlinear equations o 3.3 Numerical methods for solving differential and integral equations

4 References 5 See also

Areas of numerical analysis

A rough categorization of the principal areas of numerical analysis is given below, keeping in mind that there is often a great deal of overlap between the listed areas. In addition, the numerical solution of many mathematical problems involves some combination of some of these

areas, possibly all of them. There are also a few problems which do not fit neatly into any of the following categories.

Systems of linear and nonlinear equations

Numerical solution of systems of linear equations. This refers to solving for in the

equation with given matrix and column vector . The most important case

has a square matrix. There are both direct methods of solution (requiring only a finite number of arithmetic operations) and iterative methods (giving increased accuracy with each new iteration). This topic also includes the matrix eigenvalue problem for a square

matrix , solving for and in the equation . Numerical solution of systems of nonlinear equations. This refers to rootfinding problems

which are usually written as with a vector with components and a

vector with components. The most important case has .

Optimization . This refers to minimizing or maximizing a real-valued function . The

permitted values for can be either constrained or unconstrained. The

'linear programming problem' is a well-known and important case; is linear, and

there are linear equality and/or inequality constraints on .

Approximation theory

Use computable functions to approximate the values of functions that are not easily computable or use approximations to simplify dealing with such functions. The most popular

types of computable functions are polynomials, rational functions, and piecewise versions of them, for example spline functions. Trigonometric polynomials are also a very useful choice.

Best approximations. Here a given function is approximated within a given finite-dimensional family of computable functions. The quality of the approximation is expressed by a functional, usually the maximum absolute value of the approximation error or an integral involving the error. Least squares approximations and minimax approximations are the most popular choices.

Interpolation. A computable function is to be chosen to agree with a given at a

given finite set of points . The study of determining and analyzing such interpolation

functions is still an active area of research, particularly when is a multivariate polynomial.

Fourier series. A function is decomposed into orthogonal components based on a

given orthogonal basis , and then is approximated by using only the largest of such components. The convergence of Fourier series is a classical area of mathematics, and it is very important in many fields of application. The development of the Fast Fourier Transform in 1965 spawned a rapid progress in digital technology. In the 1990s wavelets became an important tool in this area.

Numerical integration and differentiation. Most integrals cannot be evaluated directly in terms of elementary functions, and instead they must be approximated numerically. Most functions can be differentiated analytically, but there is still a need for numerical differentiation, both to approximate the derivative of numerical data and to obtain approximations for discretizing differential equations.

Numerical solution of differential and integral equations

These equations occur widely as mathematical models for the physical world, and their numerical solution is important throughout the sciences and engineering.

Ordinary differential equations . This refers to systems of differential equations in which the unknown solutions are functions of only a single variable. The most important cases are initial value problems and boundary value problems, and these are the subjects of a number of textbooks. Of more recent interest are 'differential-algebraic equations', which are mixed systems of algebraic equations and ordinary differential equations. Also of recent interest are 'delay differential equations', in which the rate of change of the solution depends on the state of the system at past times.

Partial differential equations . This refers to differential equations in which the unknown solution is a function of more than one variable. These equations occur in almost all areas of engineering, and many basic models of the physical sciences are given as partial differential equations. Thus such equations are a very important topic for numerical analysis. For example, the Navier-Stokes equations are the main theoretical model for the motion of fluids, and the very large area of 'computational fluid mechanics' is concerned with solving numerically these and other equations of fluid dynamics

Integral equations. These equations involve the integration of an unknown function, and linear equations probably occur most frequently. Some mathematical models lead directly to integral equations; for example, the radiosity equation is a model for radiative heat transfer. Another important source of such problems is the reformulation of partial differential equations, and such reformulations are often called 'boundary integral equations'.

Some common viewpoints and concerns in numerical analysis

Most numerical analysts specialize in small sub-areas of the areas listed above, but they share some common concerns and perspectives. These include the following.

The mathematical aspects of numerical analysis make use of the language and results of linear algebra, real analysis, and functional analysis.

If you cannot solve a problem directly, then replace it with a 'nearby problem' which can be solved more easily. This is an important perspective which cuts across all types of mathematical problems. For example, to evaluate a definite integral numerically, begin by approximating its integrand using polynomial interpolation or a Taylor series, and then integrate exactly the polynomial approximation.

All numerical calculations are carried out using finite precision arithmetic, usually in a framework of floating-point representation of numbers. What are the effects of using such finite precision computer arithmetic? How are arithmetic calculations to be carried out? Using finite precision arithmetic will affect how we compute solutions to all types of problems, and it forces us to think about the limits on the accuracy with which a problem can be solved numerically. Even when solving finite systems of linear equations by direct numerical methods, infinite precision arithmetic is needed in order to find a particular exact solution.

There is concern with 'stability', a concept referring to the sensitivity of the solution of a given problem to small changes in the data or the given parameters of the problem. There are two aspects to this. First, how sensitive is the original problem to small changes in the data of the problem? Problems that are very sensitive are often referred to as 'ill-conditioned' or 'ill-posed', depending on the degree of sensitivity. Second, the numerical method should not introduce additional sensitivity that is not present in the original mathematical problem being solved. In developing a numerical method to solve a problem, the method should be no more sensitive to changes in the data than is true of the original mathematical problem.

There is a fundamental concern with error, its size, and its analytic form. When approximating a problem, a numerical analyst would want to understand the behaviour of the error in the computed solution. Understanding the form of the error may allow one to minimize or estimate it. A 'forward error analysis' looks at the effect of errors made in the solution process. This is the standard way of understanding the consequences of the approximation errors that occur in setting up a numerical method of solution, e.g. in numerical integration and in the numerical solution of differential and integral equations. A 'backward error analysis' works backward in a numerical algorithm, showing that the approximating numerical solution is the exact solution to a perturbed version of the original mathematical problem. In this way the stability of the original problem can be used to explain possible difficulties in a numerical method. Backward error analysis has been especially important in understanding the behaviour of numerical methods for solving linear algebra problems.

In order to develop efficient means of calculating a numerical solution, it is important to understand the characteristics of the computer being used. For example, the structure of the computer memory is often very important in devising efficient algorithms for large linear algebra problems. Also, parallel computer architectures lead to efficient algorithms only if the algorithm is designed to take advantage of the parallelism.

Numerical analysts are generally interested in measuring the efficiency of algorithms. What is the cost of a particular algorithm? For example, the use of Gaussian elimination

to solve a linear system containing equations will require approximately

arithmetic operations. How does this compare with other numerical methods for solving this problem? This topic is a part of the larger area of 'computational complexity'.

Use information gained in solving a problem to improve the solution procedure for that problem. Often we do not fully understand the characteristics of a problem, especially very complicated and large ones. Such a solution process is sometimes referred to as being an 'adaptive procedure', and it can also be viewed as a feedback process.

Development of numerical methods

Numerical analysts and applied mathematicians have a variety of tools which they use in developing numerical methods for solving mathematical problems. An important perspective, one mentioned earlier, which cuts across all types of mathematical problems is that of replacing the given problem with a 'nearby problem' which can be solved more easily. There are other perspectives which vary with the type of mathematical problem being solved.

Numerical solution of systems of linear equations

Linear systems arise in many of the problems of numerical analysis, a reflection of the approximation of mathematical problems using linearization. This leads to diversity in the characteristics of linear systems, and for this reason there are numerous approaches to solving linear systems. As an example, numerical methods for solving partial differential equations often lead to very large 'sparse' linear systems in which most coefficients are zero. Solving such sparse systems requires methods that are quite different from those used to solve more moderate sized 'dense' linear systems in which most coefficients are non-zero.

There are 'direct methods' and 'iterative methods' for solving all types of linear systems, and the method of choice depends on the characteristics of both the linear system and on the computer hardware being used. For example, some sparse systems can be solved by direct methods, whereas others are better solved using iteration. With iteration methods, the linear system is sometimes transformed to an equivalent form that is more amenable to being solved by iteration; this is often called 'pre-conditioning' of the linear system.

With the matrix eigenvalue problem , it is standard to transform the matrix to a simpler form, one for which the eigenvalue problem can be solved more easily and/or cheaply. A favorite choice are 'orthogonal transformations' because they are a simple and stable way to

convert the given matrix . Orthogonal transformations are also very useful in transforming other problems in numerical linear algebra. Of particular importance in this regard is the least squares solution of over-determined linear systems.

The linear programming problem was solved principally by the 'simplex method' until new approaches were developed in the 1980s, and it remains an important method of solution. The simplex method is a direct method that uses tools from the numerical solution of linear systems.

Numerical solution of systems of nonlinear equations

With a single equation , and having an initial estimate of the root , approximate

by its tangent line at the point . Find the root of this tangent line as an

approximation to the root of the original equation . This leads to 'Newton's iteration method',

Other linear and higher degree approximations can be used, and these lead to alternative iteration methods. An important derivative-free approximation of Newton’s method is the 'secant method'.

For a system of nonlinear equations for a solution vector in , we approximate by

its linear Taylor approximation about the initial estimate . This leads to Newton's method for nonlinear systems,

in which denotes the Jacobian matrix, of order for .

In practice, the Jacobian matrix for is often too complicated to compute directly; instead the partial derivatives in the Jacobian matrix are approximated using 'finite differences'. This leads to a 'finite difference Newton method'. As an alternative strategy and in analogy with the development of the secant method for the single variable problem, there is a similar rootfinding iteration method for solving nonlinear systems. It is called 'Broyden’s method' and it uses finite difference approximations of the derivatives in the Jacobian matrix, avoiding the evaluation of

the partial derivatives of .

Numerical methods for solving differential and integral equations

With such equations, there are usually at least two general steps involved in obtaining a nearby problem from which a numerical approximation can be computed; this is often referred to as 'discretization' of the original problem. The given equation will have a domain on which the unknown function is defined, perhaps an interval in one dimension and maybe a rectangle, ellipse, or other simply connected bounded region in two dimensions. Many numerical methods begin by introducing a mesh or grid on this domain, and the solution is to be approximated using this grid. Following this, there are several common approaches.

One approach approximates the equation with a simpler equation defined on the mesh. For example, consider approximating the boundary value problem

Introduce a set of mesh points , , with for some given . Approximate the boundary value problem by

The second derivative in the original problem has been replaced by a numerical approximation to the second derivative. The new problem is a finite system of nonlinear equations, presumably amenable to solution by known techniques. The solution to this new problem is , and it is

defined on only the mesh points .

A second approach to discretizing differential and integral equations is as follows. Choose a finite-dimensional family of functions, denoted here by , with which to approximate the

unknown solution function . Write the given differential or integral equation as ,

with a function for any function , perhaps over a restricted class of functions . The

numerical method consists of selecting a function such that is a small function in some sense. The various ways of doing this lead to 'Galerkin methods', 'collocation methods', and 'least square methods'.

Yet another approach is to reformulate the equation as an optimization problem. Such reformulations are a part of the classical area of mathematics known as the 'calculus of variations', a subject that reflects the importance in physics of minimization principles. The well-known 'finite element method' for solving elliptic partial differential equations is obtained in this way, although it often coincides with a Galerkin method.

The approximating functions in are often chosen as piecewise polynomial functions which are polynomial over the elements of the mesh chosen earlier. Such methods are sometimes called 'local methods'. When the approximating functions are defined without reference to a grid, then the methods are sometimes called 'global methods' or 'spectral methods'. Examples of such are sets of polynomials or trigonometric functions of some finite degree or less.

With all three approaches to solving a differential or integral equations, the intent is that the

resulting solution be close to the desired solution . The business of theoretical numerical analysis is to analyze such an algorithm and investigate the size of .

References

For an historical account of early numerical analysis, see

Herman Goldstine. A History of Numerical Analysis From the 16th Through the19th Century, Springer-Verlag, New York, 1977.

For a current view of numerical analysis as taught at the undergraduate level, see

Cleve Moler. Numerical Computing with MATLAB, SIAM Pub., Philadelphia, 2004.

For a current view of numerical analysis as taught at the advanced undergraduate or beginning graduate level, see

Alfio Quarteroni, Riccardo Sacco, and Fausto Saleri. Numerical Mathematics, Springer-Verlag, New York, 2000.

Christoph W. Ueberhuber. Numerical Computation: Vol. 1: Methods, Software, and Analysis, Vol. 2: Methods, Software, and Analysis, Springer-Verlag, New York, 1997.

For one perspective on a theoretical framework using functional analysis for studying many problems in numerical analysis, see

Kendall Atkinson and Weimin Han. Theoretical Numerical Analysis: A Functional Analysis Framework, 2nd ed., Springer-Verlag, New York, 2005.

As references for numerical linear algebra, see

Gene Golub and Charles Van Loan. Matrix Computations, 3rd ed., Johns Hopkins University Press, 1996.

Nicholas Higham. Accuracy and Stability of Numerical Algorithms, SIAM Pub., Philadelphia, 1996.

For an introduction to practical numerical analysis for solving ordinary differential equations, see

Lawrence Shampine, Ian Gladwell, Skip Thompson. Solving ODEs with Matlab, Cambridge University Press, Cambridge, 2003.

For information on computing aspects of numerical analysis, see

Michael Overton. Numerical computing with IEEE floating point arithmetic, SIAM Pub., Philadelphia, 2001.

Suely Oliveira and David Stewart. Writing Scientific Software: A Guide to Good Style, Cambridge University Press, Cambridge, 2006.

Introduction

Numerical analysis involves the study of methods of computing numerical data. In many problems this implies producing a sequence of approximations; thus the questions involve the rate of convergence, the accuracy (or even validity) of the answer, and the completeness of the response. (With many problems it is difficult to decide from a program's termination whether other solutions exist.) Since many problems across mathematics can be reduced to linear algebra, this too is studied numerically; here there are significant problems with the amount of time necessary to process the initial data. Numerical solutions to differential equations require the determination not of a few numbers but of an entire function; in particular, convergence must be judged by some global criterion. Other topics include numerical simulation, optimization, and graphical analysis, and the development of robust working code.

Numerical linear algebra topics: solutions of linear systems AX = B, eigenvalues and eigenvectors, matrix factorizations. Calculus topics: numerical differentiation and integration, interpolation, solutions of nonlinear equations f(x) = 0. Statistical topics: polynomial approximation, curve fitting.

History

Applications and related fields

For papers involving machine computations and programs in a specific mathematical area, See Section --04 in that area. This includes computational issues in group theory, number theory, geometry, statistics, and so on; for each of these fields there are software packages or libraries of code which are discussed on those index pages. (On the other hand, most results of numerical integration, say, are in this section rather than Measure and Integration; topics in optimization are in section 65K rather than Operations Research.)

For calculations of a combinatorial nature and for graph-theoretic questions such as the traveling salesman problem or scheduling algorithms, see Combinatorics. (These are distinguished by the discrete nature of the solution sought.) Portions of that material -- particularly investigations into the complexity of the algorithms -- is also treated in Computer Science.

General issues of computer use, such as system organization and methodology, or artificial intelligence, are certainly in computer science. Topics in computer algebra or symbolic calculation are treated separately.

Issues concerning limitations of specific hardware or software are not strictly speaking part of mathematics at all but often illustrate some of the issues addressed in numerical analysis. Some of these can be seen in examples seen below.

Applications of numerical analysis occur throughout the fields of applied (numerical) mathematics, in particular in the fields of physics (sections 70-86). Many of these areas including subheading e.g. for finite element methods (which are primarily treated here in 65L - 65P).

There are also applications to areas typically considered part of pure mathematics; for example, there is substantial work done on the roots of 26C:Polynomials and rational functions.

This area is undergirded by the areas of analysis. See for example Real analysis or Complex analysis for general topics of convergence.

Types of Errors in Numerical Analysis

By Tom Lutzenberger, eHow Contributor

updated: February 24, 2010

I want to do this!

The abacus: an early math calculator

In the world of math, the practice of numerical analysis is well known for focusing on algorithms as they are used to solve issues in continuous math. The practice is familiar territory for engineers and those who work with physical science, but it is beginning to expand further into liberal arts areas as well. This can be seen in astrology, stock portfolio analysis, data analysis and medicine. Part of the application of numerical analysis involves the use of errors. Specific errors are sought out and applied to arrive at mathematical conclusions.

The Round-Off Error

1. The round-off error is used because it a representation of every number as a real number is not possible. So rounding is introduced adjust for this situation. A round-off error, represents the numerical amount between what a figure actually is versus its closest real number value, depending on how the round is applied. For instance, rounding to the nearest whole number means you round up or down to what is the closest whole figure. So if your result is 3.31 then you would round to 3. Rounding the highest amount would be a bit different. In this approach, if your figure is 3.31, your rounding would be to 4. In terms of numerical analysis the round-off error is an attempt to identify what the rounding distance is when it comes up in algorithms. It's also known as a quantization error.

The Truncation Error

2. A truncation error occurs when approximation is involved in numerical analysis. The error factor is related to how much the approximate value is a variance from the actual value in a formula or math result. For example, take the formula of 3 times 3 plus 4. The calculation equals 28. Now, break it down and the root is close to 1.99. The truncation error value is equal to 0.01.

The Discretization Error

3. As a type of truncation error, the discretization error focuses on how much a discrete math problem is not consistent with a continuous math problem.

Numerical Stability Errors

4. If an error stays at one point in an algorithm and doesn't aggregate further as the calculation continues, then it is considered a numerically stable error. This happens when the error

causes only a very small variation in the formula result. If the opposite occurs, and the error propagates bigger as the calculation continues, then it is considered numerically unstable.

Conclusion

5. Math errors, unlike the inference of their name, come in useful in statistics, computer programming, advanced mathematics and much more. The error evaluation provides significantly useful information, especially when probability is required.

Read more: Types of Errors in Numerical Analysis | eHow.com http://www.ehow.com/list_6022901_types-errors-numerical-analysis.html#ixzz14hmctaaH