Iterative Refinement Methods for
Christopher Thomas Lenard
A thesis submitted for the degree of Doctor of Philosophy of the Australian National University
in September of the year One Thousand Nine Hundred and Eighty Nine.
I declare that, except where otherwise stated, this thesis is my own work and
was not carried out jointly with others.
U1005913 Text Box
Really this should be entitled “Thank you’s” because I want to do more than just
acknowledge the help I have received in the course of this thesis, and before.
Thank you to my supervisor Dr R.S. Anderssen for always finding time to help me
despite his hectic and busy work. The old adage “If you want something done, ask a busy
man” applies to Bob more than just about anyone else I know. I always came away wiser
from the many discussions I had with Bob about the work in this thesis. The following
pages owe a lot to his guidance. He has always been patient, and given much good advice
over the years; not only did I learn a lot about mathematics from Bob, but a lot about life.
Someone once quipped that our mathematics department is “user friendly”; I can’t
express it better than that. Thank you to everyone in the Department of Mathematics and
the Centre for Mathematical Analysis for making such a comfortable and enjoyable place to
Thank you also to Professor B.V. Limaye and Dr M.T. Nair for their enthusiasm
about mathematics, and their interest in my work. I profitted from conversations with them
about various topics in this thesis, about mathematics in general, and not a few other things
Too many friends to mention individually were always encouraging, but thank you to
all of them.
The Australian Government kindly provided me with a Commonwealth Postgraduate
Finally, but of course most importantly, thank you to my Parents for giving me life
and succour, and for their sacrifices throughout my life. To them I dedicate this thesis.
The subject of this thesis is the numerical solution of eigenproblems from the point of view
of iterative refinement. On the whole, we will be concerned with linear, symmetric
problems, but occasionally we will make forays into non-linearity and non-symmetry.
The initial goal was to develop a better understanding of Rayleigh quotient iteration
(RQI) and its numerical performance. Along the way it was necessary to look at a variety
of methods proposed for the iterative refinement of eigenelements to see what relationships,
if any, they have with RQI. As a consequence we identified a natural progression from
algebraic (discrete) methods to continuous methods, some of which have direct discrete
Chapter 1 provides an overview of eigenproblems and some of the main methods for
their numerical solution. Particular emphasis is given to two of the key players which will
be found throughout the thesis; namely, inverse iteration and the Rayleigh quotient. In
Chapter 2, these are combined to form the Rayleigh quotient iteration; a method with
remarkable convergence properties (at least for normal, compact operators). The first part
of the chapter, Sections 1 to 4, examine RQI, what its properties are, the way it works, and
what it does in terms of minimizing naturally occuring functionals. Section 5 completes
the chapter by using Taylor’s series to show why RQI is such a special process. Not many
numerical procedures are cubically convergent, and the obvious ploy of using the first three
terms of the Taylor’s series to get such fast convergence only results in very inelegant
iterations when applied to the eigenproblem. Although it must be said that while the
evaluation of the second differential of an arbitrary (vector valued) function is in general
quite daunting, and the rewards are probably outweighed by the costs, the functions one
would expect in the eigenproblem yield second differentials which are quite simple.
Chapter 3 is a bridge between inverse iteration in the first two chapters, and
continuous methods in Chapter 4. The link is established through the
Rayleigh-Schrödinger series which is the motivation behind Rayleigh-Schrödinger iteration
and its several variants. Essentially these are inverse iterations, but using generalized
inverses which come in as reduced resolvents. For the self-adjoint case, the iterations
follow a particularly nice pattern that is reminiscent of the error squaring
(superconvergence) property of the Rayleigh quotient. As with RQI, the iterations have a
natural interpretation in terms of minimizing functionals. In this chapter, Section 2 is an
inset giving a novel way of arriving at the iteration based on matrix calculus.
The derivation of the Rayleigh-Schrödinger series itself, however, is as a homotopy
method for getting from a known eigenpair of a perturbed operator to an eigenpair of the
unperturbed operator. One way of tackling homotopies is via differential equations, and so
in Chapter 4 we turn our attention to these matters.
The discussion in Chapter 4 is based on continuous analogues of discrete processes
which have their genesis in the discovery that the QR algorithm is closely related to the
Toda flow. Many discrete methods follow the solution trajectory of a differential equation,
either exactly or approximately. For example, Newton’s iteration can be thought of as
Euler’s method applied to a particular initial value problem. Other methods though, like the
QR algorithm, produce iterates that are exactly on the solution curve, so that one can think
of the continuous method as an interpolation of the discrete iteration.
Finally Chapter 5 stands apart in the sense that it does not directly continue on from
continuous methods; however, inverse iteration does plays the central role. The main idea
is to build up information from the traces of a matrix, its powers, and its inverse powers,
which can then be used to approximate eigenvalues. Here, Laguerre’s method for finding
the roots of a polynomial is shown to be connected with the (standard) method of traces
applied to matrices (or integral operators).
Declaration page i
1 Introductory comments 1
2 The Rayleigh quotient 4
3 Natural and inverse iteration 11
4 Standard methods - tried and trusted 22
2 Rayleigh Quotient Iteration
1 Introduction to Rayleigh quotient iteration 36
2 Rayleigh quotient iteration for compact, normal operators 37
3 Projected Rayleigh quotient iteration 47
4 Inverse iteration and optimization 48
5 Taylor’s series and the eigenproblem - or why Rayleigh quotient iteration is so special 56
3 Rayleigh-Schrödinger Iteration and Series
1 Review of Rayleigh-Schrödinger series and associated iterations 68
2 Matrix calculus 74
3 Rayleigh-Schrödinger series as homotopy : finding the nearest eigenvalue 80
4 Homotopy and Continuous Methods
1 Path following : homotopies and differential equations 88
2 Continuous Newton, power and inverse iteration 92
3 Isospectral flows and matrix factorizations 94
4 Rayleigh quotient iteration analogue and interpolated RQI 102
1 The method of traces 107
2 Laguerre’s method 107
3 Other trace type methods 111
1 .1 Introductory comments
As Ian Stewart succinctly states, “Everything in the universe vibrates.” (Stewart (1988)),
which can be taken to mean that eigenvalue equations are the underpinnings of the
mathematics of the universe. From the fundamental structure of space itself to the everyday
macroscopic world, things that vibrate are modelled by an eigenvalue equation.
Modem physics, although by now it is ‘classical’, such as Schrödinger’s equation, is
largely vibrational in nature; post-modernism as exemplified by string theory is also
vibrational. “... a single string possesses many possible energies of vibration. The goal of
the string picture of reality is to attribute each force and elementary particle species of
Nature to a different vibrational state of a single string. The lowest-energy vibration should
be associated with gravity, the weakest force, whilst the more energetic excitations of the
string may give rise to the other forces and particles.” (Barrow (1988))
Whatever the physical system, the mathematical model is usually a continuous
eigenvalue problem, either a differential or integral equation; more rarely is the underlying
problem algebraic. Although, algebraic eigenproblems do arise in their own right in such
areas as optimal control.
Solving a continuous eigenproblem by analytic means is not possible in all but the
simplest cases, and even algebraic problems wh