INGRID DAUBECHIESRutgers University andAT&T Belt Laboratories

Ten Lectures onWavelets

SOCIETY FOR INDUSTRIAL AND APPLIED MATHEMATICS

PHILADELPHIA, PENNSYLVANIA 1992

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

CBMS-NSF REGIONAL CONFERENCE SERIESIN APPLIED MATHEMATICS

A series of lectures on topics of current research interest in applied mathematics under the directionof the Conference Board of the Mathematical Sciences, supported by the National ScienceFoundation and published by SIAM.

GARRETF BIRKHOr-F, The Numerical Solution of Elliptic Equations

D. V. LINDLEY, Bayesian Statistics, A Review

R. S. VARGA, Functional Analysis and Approximation Theory in Numerical Analysis

R. R. BAHADUR, Some Limit Theorems in Statistics

PATRICK BILLINGSLEY, Weak Convergente of Measures: Applications in Probability

J. L. Lioxs, Some Aspects of the Optimal Control of Distributed Parameter Systems

ROGER PENROSE, Techniques of Differential Topology in Relativity

HERMAN CHERNOFF, Sequential Analysis and Optimal Design

J. DURBIN, Distribution Theory for Tests Based on the Sample Distribution Function

SOL I. RUBINOW, Mathematical Problems in the Biological Sciences

P. D. LAx, Hyperbolic Systems of Conservation Laws and the Mathematical Theory

of Shock Waves

L J. SCHOENBERG, Cardinal Spline Interpolation

IVAN SINGER, The Theory of Best Approximation and Functional Analysis

WERNER C. RHEINBOLDT, Methods of Solving Systems of Nonlinear Equations

HANS F. WEINBERGER, Variational Methods for Eigenvalue Approximation

R. TYRRELL ROCKAFELLAR, Conjugate Duality and Optimization

SIR JAMES LIGHTHILL, Mathematical Biofluiddynamics

GERARD SALTON, Theory of Indexing

CATHLEEN S. MORAWETZ, Notes on Time Decay and Scattering for Some Hyperbolic Problems

F. HOPPENSTEADT, Mathematical Theories of Populations: Demographics, Genetics

and Epidemics

RICHARD ASKEY, Orthogonal Polynomials and Special Functions

L. E. PAYNE, Improperly Posed Problems in Partial Differential Equations

S. RoSEN, Lectures on the Measurement and Evaluation of the Performance

of Computing Systems

HERBERT B. KELLER, Numerical Solution of Two Point Boundary Value Problems

J. P. LASALLE, The Stability of Dynamical Systems - Z. ARTSTEIN, Appendix A: Limiting

Equations and Stability of Nonautonomous Ordinary Differential Equations

D. GOTTLIEB AND S. A. ORSZAG, Numerical Analysis of Spectral Methods: Theory

and Applications

PETER J. HUBER, Robust Statistical Procedures

HERBERT SOLOMON, Geometric Probability

FRED S. ROBERTS, Graph Theory and Its Applications to Problems of Society

JURis HARTMANIS, Feasible Computations and Provable Complexity Properties

ZOHAR MANNA, Lectures on the Logic of Computer Programming

ELL[s L. JOHNSON, Integer Programming: Facets, Subadditivity, and Duality for Group

and Semi-Group Problems

SHMUEL WINOGRAD, Arithmetic Complexity of Computations

J. F. C. KINGMAN, Mathematics of Genetic Diversity

MORTON E. GURTIN, Topics in Finite Elasticity

THOMAS G. KURTZ, Approximation of Population Processes

(continued on inside back cover)

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

JERROLD E. MARSDEN, Lectures on Geometric Methods in Mathematical Physics

BRADLEY EFRON, The Jackknife, the Bootstrap, and Other Resampling Plans

M. WOODROOFE, Nonlinear Renewal Theory in Sequential Analysis

D. H. SATTINGER, Branching in the Presence of Symmetry

R. TÉMAM, Navier-Stokes Equations and Nonlinear Functional Analysis

MIKLbs CSÖRGO, Quantile Processes with Statistical Applications

J. D. BUCKMASTER AND G. S. S. LUDFORD, Lectures on Mathematical Combustion

R. E. TARJAN, Data Structures and Network Algorithms

PAUL WALTMAN, Competition Models in Population Biology

S. R. S. VARADHAN, Large Deviations and Applications

KiYosi ITó, Foundations of Stochastic Differential Equations in Infinite Dimensional Spaces

ALAN C. NEWELL, Solitons in Mathematics and Physics

PRANAB KLIMAR SEN, Theory and Applications of Sequential Nonparametrics

LnSZLÓ Lovász, An Algorithmic Theory of Numbers, Graphs and Convexity

E. W. CHENEY, Multivariate Approximation Theory: Selected Topics

JOEL SPENCER, Ten Lectures on the Probabilistic Method

PAUL C. FIFE, Dynamics of Interhal Layers and Diffusive Interfaces

CHARLES K. CHUi, Multivariate Splines

HERBERT S. WILF, Combinatorial Algorithms: An Update

HENRY C. TUCKWELL, Stochastic Processes in the Neurosciences

FRANK H. CLARKE, Methods of Dynamic and Nonsmooth Optimization

ROBERT B. GARDNER, The Method of Equivalente and Its Applications

GRACE WAHBA, Spline Models for Observational Data

RICHARD S. VARGA, Scientific Computation on Mathematical Problems and Conjectures

INGRID DAUBECHIES, Ten Lectures on Wavelets

STEPHEN F. MCCORMICK, Multilevel Projection Methods for Partial Differential Equations

HARALD NIEDERREITER, Random Number Generation and Quasi-Monte Carlo Methods

JOEL SPENCER, Ten Lectures on the Probabilistic Method, Second Edition

CHARLES A. MICCHELLI, Mathematical Aspects of Geometrie Modeling

ROGER Tí MAM, Navier—Stokes Equations and Nonlinear Functional Analysis, Second Edition

GLENN SHAFER, Probabilistic Expert Systems

PETER J. HUBER, Robust Statistical Procedures, Second Edition

J. MICHAEL STEELE, Probability Theory and Combinatorial Optimization

WERNER C. RHEINBOLDT, Methods for Solving Systems of Nonlinear Equations, Second Edition

J. M. CUSHING, An Introduction to Structured Population Dynamics

TAI-PING Liu, Hyperbolic and Viscous Conservation Laws

MICHAEL RENARDY, Mathematical Analysis of Viscoelastic Flows

GÉRARD CORNUÉJOLS, Combinatorial Optimization: Packing and CoveringIRENA LASIECKA, Mathematical Control Theory of Coupled PDEs

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

Copyright 1992 by the Society for Industrial and Applied Mathematics

All rights reserved. No part of this book may be reproduced, stored, or transmitted in anymanner without the written permission of the Publisher. For information, write the Societyfor Industrial and Applied Mathematics, 3600 University City Science Center, Philadelphia,Pennsylvania 19104-2688.

Second Printing 1992Third Printing 1994Fourth Printing 1995Fifth Printing 1997Sixth Printing 1999Seventh Printing 2002Eighth Printing 2004Ninth Printing 2006

Library of Congress Cataloging-in-Publication Data

Daubechies, Ingrid.Ten lectures on wavelets / Ingrid Daubechies.

p. cm. — (CBMS-NSF regional conference series in appliedmathematics ; 61)

Includes bibliographical references and index.ISBN-13: 978-0-898712-74-2ISBN 10: 0-89871-274-21. Wavelets (Mathematics)—Congresses. I. Title. II. Series.

QA403.3.D38 1992515.2433—dc20 92-13201

n

is a registered trademark.

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

To my mother, who gave me the will to be independent.

To my father, who stimulated my interest in science.

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

Contents

vii INTRODUCTION

xi PRELIMINARIES AND NOTATION

1 CHAPTER 1: The What, Why, and How of Wavelets

17 CHAPTER 2: The Continuous Wavelet Transform

53 CHAPTER 3: Discrete Wavelet Transforms: Frames

107 CHAPTER 4: Time-Frequency Density and Orthonormal Bases

129 CHAPTER 5: Orthonormal Bases of Wavelets and Multiresolution Analysis

167 CHAPTER 6: Orthonormal Bases of Compactly Supported Wavelets

215 CHAPTER 7: More About the Regularity of Compactly Supported Wavelets

251 CHAPTER 8: Symmetry for Compactly Supported Wavelet Bases

289 CHAPTER 9: Characterization of Functional Spaces by Means of Wavelets

313 CHAPTER 10: Generalizations and Tricks for Orthonormal Wavelet Bases

341 REFERENCES

353 SUBJECT INDEX

355 AUTHOR INDEX

v

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

Introd uction

Wavelets are a relatively recent development in applied mathematics. Theirname itself was coined approximately a decade ago (Morlet, Arens, Fourgeau,and Giard (1982), Morlet (1983), Grossmann and Morlet (1984)); in the last tenyears interest in them has grown at an explosive rate. There are several rea-sons for their present success. On the one hand, the concept of wavelets can beviewed as a synthesis of ideas which originated during the last twenty or thirtyyears in engineering (subband coding), physics (coherent states, renormalizationgroup), and pure mathematics (study of Calderón—Zygmund operators). As aconsequence of these interdisciplinary origins, wavelets appeal to scientists andengineers of many different backgrounds. On the other hand, wavelets are a fairlysimple mathematical tool with a great variety of possible applications. Alreadythey have led to exciting applications in signal analysis (sound, images) (someearly references are Kronland-Martinet, Morlet and Grossmann (1987), Mallat(1989b), (1989c); more recent references are given later) and numerical analy-sis (fast algorithms for integral transforms in Beylkin, Coifman, and Rokhlin(1991)); many other applications are being studied. This wide applicability alsocontributes to the interest they generate.

This book contains ten lectures I delivered as the principal speaker at theCBMS conference on wavelets organized in June 1990 by the Mathematics De-partment at the University of Lowell, Massachusetts. According to the usualformat of the CBMS conferences, other speakers (G. Battle, G. Beylkin, C. Chui,A. Cohen, R. Coifman, K. Gröchenig, J. Liandrat, S. Mallat, B. Torrésani,and A. Willsky) provided lectures on their work related to wavelets. Moreover,three workshops were organized, on applications to physics and inverse problems(chaired by B. DeFacio), group theory and harmonic analysis (H. Feichtinger),and signal analysis (M. Vetterli). The audience consisted of researchers activein the field of wavelets as well as of mathematicians and other scientists andengineers who knew little about wavelets and hoped to learn more. This secondgroup constituted the largest part of the audience. I saw it as my task to providea tutorial on wavelets to this part of the audience, which would then be a solidgrounding for more recent work exposed by the other lecturers and myself. Con-sequently, about two thirds of my lectures consisted of "basic wavelet theory,"

vii

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

viii INTRODUCTION

the other third being devoted to more recent and unpublished work. This divi-sion is reflected in the present write-up as well. As a result, I believe that thisbook will be useful as an introduction to the subject, to be used either for indi-vidual reading, or for a seminar or graduate course. None of the other lecturesor workshop papers presented at the CBMS conference have been incorporatedhere. As a result, this presentation is biased more toward my own work than theCBMS conference was. In many instances I have included pointers to referencesfor further reading or a detailed exposition of particular applications, comple-menting the present text. Other books on wavelets published include Waveletsand Time Frequency Methods (Combes, Grossmann, and Tchamitchian (1987)),which contains the proceedings of the International Wavelet Conference held inMarseille, France, in December 1987, Ondelettes, by Y. Meyer (1990) (in French;English translation expected soon), which contains a mathematically more ex-panded treatment than the present lectures, with fewer forays into other fieldshowever, Les Ondelettes en 1989, edited by P. G. Lemarié (1990), a collection oftalks given at the Université Paris XI in the spring of 1989, and An Introductionto Wavelets, by C. K. Chui (1992b), an introduction from the approximationtheory viewpoint. The proceedings of the International Wavelet Conference inMay 1989, held again in Marseille, are due to come out soon (Meyer (1992)).Moreover, many of the other contributors to the CBMS conference, as well assome wavelet researchers who could not attend, were invited to write an essayon their wavelet work; the result is the essay collection Wavelets and their Ap-plications (Ruskai et al. (1992)), which can be considered a companion book tothis one. Another wavelet essay book is Wavelets: A Tutorial in Theory andApplications, edited by C. K. Chui (1992c); in addition, I know of several otherwavelet essay books in preparation (edited by J. Benedetto and M. Frazier, an-other by M. Barlaud), as well as a monograph by M. Holschneider; there wasa special wavelet issue of IEEE Trans. Inform. Theory in March of 1992; therewill be another one, later in 1992, of Constructive Approximation Theory, andone in 1993, of IEEE Trans. Sign. Proc. In addition, several recent books in-clude chapters on wavelets. Examples are Multirate Systems and Filter Banksby P. P. Vaidyanathan (1992) and Quantum Physics, Relativity and ComplexSpacetime: Towards a New Synthesis by G. Kaiser (1990). Readers interestedin the present lectures will find these books and special issues useful for manydetails and other aspects not fully presented here. It is moreover clear that thesubject is still developing rapidly.

This book more or less follows the path of my lectures: each of the ten chap-ters stands for one of the ten lectures, presented in the order in which theywere delivered. The first chapter presents a quick overview of different aspectsof the wavelet transform. It sketches the outlines of a big fresco; subsequentchapters then fill in more detail. From there on, we proceed to the continu-ous wavelet transform (Chapter 2; with a short review of bandlimited functionsand Shannon's theorem), to discrete but redundant wavelet transforms (frames;Chapter 3) and to a general discussion of time-frequency density and the possibleexistence of orthonormal bases (Chapter 4). Many of the results in Chapters 2-4 can be formulated for the windowed Fourier transform as well as the wavelet

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

INTRODUCTION ix

transform, and the two cases are presented in parallel, with analogies and differ-ences pointed out as we go along. The remaining chapters all focus on orthonor-mal bases of wavelets: multiresolution analysis and a first general strategy forthe construction of orthonormal wavelet bases (Chapter 5), orthonormal basesof compactly supported wavelets and their link to subband coding (Chapter 6),sharp regularity estimates for these wavelet bases (Chapter 7), symmetry forcompactly supported wavelet bases (Chapter 8). Chapter 9 shows that orthonor-mal bases are "good" bases for many functional spaces where Fourier methodsare not well adapted. This chapter is the most mathematical of the whole book;most of its material is not connected to the applications discussed in other chap-ters, so that it can be skipped by readers uninterested in this aspect of wavelettheory. I included it for several reasons: the kind of estimates used in the proofare very important for harmonic analysis, and similar (but more complicated)estimates in the proof of the "T(1)"-theorem of David and Journé have turnedout to be the groundwork for the applications to numerical analysis in the workof Beylkin, Coifman, and Rokhlin (1991). Moreover, the Calderón—Zygmundtheorem, explained in this chapter, illustrates how techniques using differentscales, one of the forerunners of wavelets, were used in harmonic analysis longbefore the advent of wavelets. Finally, Chapter 10 sketches several extensionsof the constructions of orthonormal wavelet bases: to more than one dimension,to dilation factors different from two (even noninteger), with the possibility ofbetter frequency localization, and to wavelet bases on a finite interval insteadof the whole line. Every chapter concludes with a section of numbered "Notes,"referred to in the text of the chapter by superscript numbers. These containadditional references, extra proofs excised to keep the text flowing, remarks, etc.

This book is a mathematics book: it states and proves many theorems. Italso presupposes some mathematical background. In particular, I assume thatthe reader is familiar with the basic properties of the Fourier transform andFourier series. I also use some basic theorems of measure and integration theory(Fatou's lemma, dominated convergence theorem, Fubini's theorem; these canbe found in any good book on real analysis). In some chapters, familiarity withbasic Hilbert space techniques is useful. A list of the basic notions and theoremsused in the book is given in the Preliminaries.

The reader who finds that he or she does not know all of these prerequisitesshould not be dismayed, however; most of the book can be followed with just thebasic notions of Fourier analysis. Moreover, I have tried to keep a very pedes-trian pace in almost all the proofs, at the risk of boring some mathematicallysophisticated readers. I hope therefore that these lecture notes will interest peo-ple other than mathematicians. For this reason I have often shied away fromthe "Definition—Lemma—Proposition—Theorem—Corollary" sequence, and I havetried to be intuitive in many places, even if this meant that the exposition be-came less succinct. I hope to succeed in sharing with my readers some of theexcitement that this interdisciplinary subject has brought into my scientific life.

I want to take this opportunity to express my gratitude to the many peoplewho made the Lowell conference happen: the CBMS board, and the MathematicsDepartment of the University of Lowell, in particular Professors G. Kaiser and

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

x INTRODUCTION

M. B. Ruskai. The success of the conference, which unexpectedly turned out tohave many more participants than customary for CBMS conferences, was due inlarge part to its very efficient organization. As experienced conference organizerI. M. James (1991) says, "every conference is mainly due to the efforts of asingle individual who does almost all the work"; for the 1990 Wavelet CBMSconference, this individual was Mary Beth Ruskai. I am especially grateful toher for proposing the conference in the first place, for organizing it in sucha way that I had a minimal paperwork load, while keeping me posted aboutall the developments, and for generally being the organizational backbone, nosmall task. Prior to the conference I had the opportunity to teach much of thismaterial as a graduate course in the Mathematics Department of the Universityof Michigan, in Ann Arbor. My one-term visit there was supported jointly bya Visiting Professorship for Women from the National Science Foundation, andby the University of Michigan. I would like to thank both institutions for theirsupport. I would also like to thank all the faculty and students who sat in onthe course, and who provided feedback and useful suggestions. The manuscriptwas typeset by Martina Sharp, who I thank for her patience and diligence, andfor doing a wonderful job. I wouldn't even have attempted to write this bookwithout her. I am grateful to Jefl Lagarias for editorial comments. Several peoplehelped me spot typos in the galley proofs, and I am grateful to all of them; Iwould like to thank especially Pascal Auscher, Gerry Kaiser, Ming-Jun Lai, andMartin Vetterli. All remaining mistakes are of course my responsiblity. I alsowould like to thank Jim Driscoll and Sharon Murrel for heiping me prepare theauthor index. Finally, I want to thank my husband Robert Calderbank for beingextremely supportive and committed to our two-career-track with family, eventhough it occasionally means that he as well as I prove a few theorems less.

Ingrid DaubechiesRutgers University

andAT&T Bell Laboratories

In subsequent printings minor mistakes and many typographical errors havebeen corrected. I am grateful to everybody who helped me to spot them. I havealso updated a few things: some of the previously unpublished references haveappeared and some of the problems that were listed as open have been solved. Ihave made no attempt to include the many other interesting papers on waveletsthat have appeared since the first printing; in any case, the list of references wasnot and is still not meant as a complete bibliography of the subject.

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

Preliminaries and Notation

This preliminary chapter fixes notation conventions and normalizations. It alsostates some basic theorems that will be used later in the book. For those lessfamiliar with Hilbert and Banach spaces, it contains a very brief primer. (Thisprimer should be used mainly as a reference, to come back to in those instanceswhen the reader comes across some Hilbert or Banach space language that sheor he is unfamiliar with. For most chapters, these concepts are not used.)

Let us start by some notation conventions. For x E IR, we write `xj for thelargest integer not exceeding x,

[xj=max{nE Z; n<x}.

For example, [3/2] = 1, [-3/2] = —2, [-2] = —2. Similarly, [xl is the smallestinteger which is larger than or equal to x.

If a—>0 (or oo), then we denote by O(a) any quantity that is bounded by aconstant times a, by o(a) any quantity that tends to 0 (or oo) when a does.

The end of a proof is always marked with a.; for clarity, many remarks orexamples are ended with a ❑ .

In many proofs, C denotes a "generic" constant, which need not havethe same value throughout the proof. In chains of inequalities, I often useC, C', C", • • • or Cl, C2 , C3 ,••• to avoid confusion.

We use the following convention for the Fourier transform (in one dimension):

(Ff) ( ) = f() = 1 f d e.f (x) . (0.0.1)

With this normalization, one has

IIflILa = 11f 11L2 ,

if(^)I < (27r)-112 I1fIILl ,

where1/p

11 f ILP = I(^

J dx I.f (x)V'} (0.0.2)

xi

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

xii PRELIMINARIES AND NOTATION

Inversion of the Fourier transform is then given by

f (x) = 1 f 00 g ez£x (Ff) (^) = (pf ) V (x)27r .

(0.0.3)

9(x) =

Strictly speaking, (0.0.1), (0.0.3) are well defined only if f, respectively .Ff, areabsolutely integrable; for general L2-functions f, e.g., we should define .F f viaa limiting process (see also below). We will implicitly assume that the adequatelimiting process is used in all cases, and write, with a convenient abuse of no-tation, formulas similar to (0.0.1) and (0.0.3) even when a limiting process isunderstood.

A standard property of the Fourier transform is:

r ( dt

_f) = (i (Ff)(),

hence

f dx If (e) (x)I 2 <00 ' .' Jd II2t If(^)I 2 < oo ,

with the notation f(t) = d f.If a function f is compactly supported, i.e., f(x) = 0 if x < a or x > b,

where —oo < a < b < oo, then its Fourier transform f() is well defined also forcomplex , and

bf(S)I < (27r)-1/2 1. dx e(Im£)x

If(x)Ia

1eb (Im £) if Im > 0

< (2ir)— / 2 Ilf IIL 1ea (Im^) if Im < 0.

If f is moreover infinitely differentiable, then the same argument can be appliedto f (P) , leading to bounds on If(e)I. For a C°° function f with support [a, b]there exist therefore constants CN so that the analytic extension of the Fouriertransform of f satisfies

N ( eb if Im >0f(^)I <—cN(l+lil) Jll ea if Im <0.

(0.0.4)

Conversely, any entire function which satisfies bounds of the type (0.0.4) for allN E N is the analytic extension of the Fourier transform of a C°° function withsupport in [a, b]. This is the Paley—Wiener theorem.

We will occasionally encounter (tempered) distributions. These are linearmaps T from the set S(R) (consisting of all C functions that decay faster thanany negative power (1 + IxI) —N ) to C, such that for all m, n E N, there existsC„,m for which

IT(f)I < Cn,m sup I( 1 + Ixl)n f(-()

xER

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

PRELIMINARIES AND NOTATION xiii

holds, for all f E S(R). The set of all such distributions is called S'(R). Anypolynomially bounded function F can be interpreted as a distribution, withF(f) = f dx F(x) f(x). Another example is the so-called "b-function" of Dirac,S(f) = f(0). A distribution T is said to be supported in [a, b] if T (f) = 0 forall functions f the support of which has empty intersection with [a, b]. One candefine the Fourier transform .FT or T of a distribution T by T (f) = T (f) (ifT is a function, then this coincides with our earlier definition). There exists aversion of the Paley—Wiener theorem for distributions: an entire function T(e)is the analytic extension of the Fourier transform of a distribution T in S'(R)supported in [a, b] if and only if, for some N E N, CN > 0,

/ bIm£

IT( )ICCN( 1 +l^^)N { eaIm£ lm <0.

The only measure we will use is Lebesgue measure, on R and R". We willoften denote the (Lebesgue) measure of S by ISI; in particular, I[a, bil = b — a(where b> a).

Well-known theorems from measure and integration theory which we will useinclude

Fatou's lemma. If f, _> 0, f(x) —> f(x) almost everywhere (i.e., the setof points where pointwise convergence fails has zero measure with respect toLebesgue measure), then

J dx f(x) < limsup J dx fn (x) .

In particular, if this lim sup is finite, then f is integrable.

(The lim sup of a sequence is defined by

limsup c = lim [sup {ak; k > n}]n—oo —

every sequence, even if it does not have a limit (such as a„ = (-1)' ), has alim sup (which may be oo); for sequences that converge to a limit, the lim supcoincides with the limit.)

Dominated convergence theorem. Suppose f. (x) —> f(x) almost every-where. If fn (x)l <_ g(x) for all n, and f dx g(x) < oo, then f is integrable,and f dxf(x)= lim fdxfn(x).

Fubini's theorem. If f dx[ f dy I f (x, y) I] < oo, then

f dx f dYf(xY) = J dx {

f dy [

dx f (x, y)]

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

xiv PRELIMINARIES AND NOTATION

i.e., the order of the integrations can be permuted.In these three theorems the domgin of integration can be any measurable

subset of IR (or R 2 for F^lbini).When Hilbert spaces are used, they are usually denoted by h, unless they

already have a name. We will follow the mathematician's convention and usescalar products which are linear in the first argument:

(\1u1 + )2u2, v) = )t1(ui, V) + 1\z(u2, v)•

As usual, we have(v, u) = (u, v)

where á denotes the complex conjugate of a, and (u, u) >_ 0 for all u E 7-l. Wedefine the norm I I u I I of u by

IIuII 2 = (u, u) . (0.0.5)

In a Hilbert space, IIuil = 0 implies u = 0, and all Cauchy sequences (withrespect to II II) have limits within the space. (More explicitly, if un E 1-1 and if

IS, - u„611 becomes arbitrarily small if n, m are large enough—i.e., for all e > 0,there exists no, depending on e, so that Ilun - um 11 <_ e if n, m _> no-, then thereexists u E 7-1 so that the u, tend to u for n-+oo, i.e., limn^,0 Ilu - unll = 0 .)

A standard example of such a Hilbert space is L 2 (R), with

(f, g) = Jdx f(x) 9(x)

Here the integration runs from -oo to oo; we will often drop the integrationbounds when the integral runs over the whole real line.

Another example is e2 (Z), the set of all square summable sequences of com-plex numbers indexed by integers, with

00

(c,d)_ cndn.

Again, we will often drop the limits on the summation index when we sumover all integers. Both L2 (R) and £2 (Z) are infinite-dimensional Hilbert spaces.Even simpler are finite-dimensional Hilbert spaces, of which C k is the standardexample, with the scalar product

k

(u, v)j=1

for u = (u1, ... , uk), v = (vl, ... , vk) E Ck .Hilbert spaces always have orthonormal bases, i.e., there exist families of

vectors en in l{(en, em) = sn,m

and

IIu112 I(u,en)I2

xiv PRELIMINARIES AND NOTATION

i.e., the order of the integrations can be permuted.In these three theorems the domain of integration can be any measurable

subset of R (or lR.2 for Fubini).When Hilbert spaces are used, they are usually denoted by 'H, unless they

already have a name. We will follow the mathematician's convention and usescalar products which are linear in the first argument:

(AIUI + A2U2, v) = Al(UI, v) + A2(U2, v) .

As usual, we have(v,U) = (u,v) ,

where it denotes the complex conjugate of a, and (u,u) 2: 0 for all U E 'H. Wedefine the norm Iluli of u by

lIull 2 = (u,u) . (0.0.5)

In a Hilbert space, lIuli = 0 implies u = 0, and all Cauchy sequences (withrespect to II II) have limits within the space. (More explicitly, if Un E 'H and iflIun - urn II becomes arbitrarily small if n, m are large enough-Le., for all 10 > 0,there exists no, depending on 10, so that Ilun - urn II ~ 10 if n, m 2: no-, then thereexists u E 1i so that the Un tend to u for n---+oo, i.e., limn-H Xl Ilu - unll = 0.)

A standard example of such a Hilbert space is L2 (lR.), with

(I,g) = Jdx f(x) g(x) .

Here the integration runs from -00 to OOj we will often drop the integrationbounds when the integral runs over the whole real line.

Another example is P2(Z), the set of all square summable sequences of com­plex numbers indexed by integers, with

00

(c,d)= L endn .

n=-oo

Again, we will often drop the limits on the summation index when we sumover all integers. Both L 2(lR.) and P2(Z) are infinite-dimensional Hilbert spaces.Even simpler are finite-dimensional Hilbert spaces, of which Ck is the standardexample, with the scalar product

k

(u,v) = L Uj iij ,j=l

for u = (UI,'" ,Uk), v = (VI,'" ,Vk) E Ck.Hilbert spaces always have orthonormal bases, i.e., there exist families of

vectors en in 'H

and

n

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

PRELIMINARIES AND NOTATION xv

for all u E X (We only consider separable Hilbert spaces, i.e., spaces in whichorthonormal bases are countable.) Examples of orthonormal bases are the Her-mite functions in L2 (R), the sequences e„ defined by (e^) = 6, , with n, j E Zin £2 (Z) (i.e., all entries but the nth vanish), or the k vectors e 1 ,•• , ek in Ck

defined by (ee) m = 6e,m , with 1 < t, m < k. (We use Kronecker's symbol 6 withthe usual meaning: 62 ,E = 1 if i = j, 0 if i # j.)

A standard inequality in a Hilbert space is the Cauchy—Schwarz inequality,

I(v,w)I <— huil Ilwll , (0.0.6)

easily proved by writing (0.0.5) for appropriate linear combinations of v and w.In particular, for f, g E L 2 (R), we have

1/ l /

2f dx f(x) 9(x) <— (fdX If (x) l 2) 1

2 (fdx 19(x)12)

and for c = (cl )fEz, d = (dn ).fEz E tz (Z),

1/2 1/2

cn <_ E Icn1 2 Idn12n n n

A consequence of (0.0.6) is

uil = sup I(u,v)I = sup I(u,v)I . (0.0.7)v, IlvII< 1 v, IlvII= 1

"Operators" on are linear maps from h to another Hilbert space, often *-1itself. Explicitly, if A is an operator on h, then

A(i\1u1 + í)2u2) =) 1Au1 + .X2Au2 .

An operator is continuous if Au — Av can be made arbitrarily small by makingu — v small. Explicitly, for all e > 0 there should exist 6 (depending on e) sothat Ilu — vll <_ 6 implies IIAu — Avll <_ E. If we take v = 0, e = 1, then wefind that, for some b> 0, I I Au I I <_ 1 if I I u I I <_ b. For any w E H we can define

w^ = fl.- 1 w; clearly II w' II < b and therefore II Aw hl = II Aw' I) < b-1II w hl• IfIiAwll/Ilwll (w # 0) is bounded, then the operator A is called bounded. We havejust seen that any continuous operator is bounded; the reverse is also true. Thenorm II A I I of A is defined by

All = sup IlAull/IIull = sup IlAull . (0.0.8)uEx, IluII^O 11111=1

It immediately follows that, for all u E x,

flAull <— IIAII IluliOperators from ?i to C are called "linear functionals." For bounded linear

functionals one has R.iesz' representation theorem: for any P: fl-C, linear and

PRELIMINARIES AND NOTATION xv

for all u E H, (We only consider separable Hilbert spaces, i.e., spaces in whichorthonormal bases are countable.) Examples of orthonormal bases are the Her­mite functions in L2(1R), the sequences en defined by (en)j = Dn,j, with n,j E Zin £2(Z) (i.e., all entries but the nth vanish), or the k vectors ell" . ,ek in Ck

defined by (e~.)m = Dl,m, with 1 s £, m s k. (We use Kronecker's symbol Dwiththe usual meaning: Di,j = 1 if i = j, 0 if i =F j.)

A standard inequality in a Hilbert space is the Cauchy-Schwarz inequality,

!(v,w)I ::; Ilvll IIwll , (0.0.6)

easily proved by writing (0.0.5) for appropriate linear combinations of v and w.In particular, for f, iJ E L2(1R), we have

IIdx f(x) g(x)1 s (I dx If(XW) 1/2 (I dx Ig(x)12) 1/2 ,

A consequence of (0.0.6) is

lIull = supv,lIvll9

l(u,v)1 = supv, IIvll=1

I(u,v)! . (0.0.7)

"Operators" on 'H are linear maps from 'H to another Hilbert space, often Hitself. Explicitly, if A is an operator on 'H, then

An operator is continuous if Au - Av can be made arbitrarily small by makingu - v small. Explicitly, for all E > 0 there should exist {j (depending on E) sothat Ilu - vII ::; D implies IIAu - Avll ::; E. If we take v = 0, E = 1, then wefind that, for some b > 0, IIAull ::; 1 if lIull ::; b. For any w E 1i we can define

w' = 1I~lIw; clearly Ilw'lI ::; b and therefore IIAwll = ¥ IIAw'lI ::; b- 11Iwll. IfIIAw11/11w11 (w =F 0) is bounded, then the operator A is called bounded. We havejust seen that any continuous operator is bounded; the reverse is also true. Thenorm IIAII of A is defined by

IIAII = supuE'H., lIull,eo

IIAull/llull = sup IIAull·lIull=1

(0.0.8)

It immediately follows that, for all u E H,

IIAul1 ::; IIAII Ilull .

Operators from 'H to C are called "linear functionals." For bounded linearfunctionals one has Riesz' representation theorem: for any £: 1i-C, linear and

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

xvi PRELIMINARIES AND NOTATION

bounded, i.e., 11(u)I < Cllull for all u E ?<, there exists a unique v£ E h so that1(u) = (u,vi) —

An operator U from lij to 412 is an isometry if (Uv, Uw) = (v, w) for allv, w E 7- 1; U is unitary if moreover UI<, = 9-12, i.e., every element v2 E fl2 canbe written as v2 = Uv1 for some vl E Wl. If the en constitute an orthonormalbasis in hl, and U is unitary, then the Ue„, constitute an orthonormal basis inf2. The reverse is also true: any operator that maps an orthonormal basis toanother orthonormal basis is unitary.

A set D is called dense in fl if every u E 7-1 can be written as the limit ofsome sequence of u„ in D. (One then says that the closure of D is all of h. Theclosure of a set S is obtained by adding to it all the v that can be obtained aslimits of sequences in S.) If Av is only defined for v E D, but we know that

IlAvII<Cllvll for allvED, (0.0.9)

then we can extend A to all of H "by continuity." Explicitly: if u E 9-1, findu„ E D so that lim„—, un = u. Then the u„ are necessarily a Cauchy sequence,and because of (0.0.9), so are the Aug ; the Au„ have therefore a limit, which wecall Au (it does not depend on the particular sequence un that was chosen).

One can also deal with unbounded operators, i.e., A for which there existsno finite C such that IlAull <_ Cllull holds for all u E H. It is a fact of life thatthese can usually only be defined on a dense set D in fl, and cannot be extendedby the above trick (since they are not continuous). An example is d^ in L 2 (R),where we can take D = Co (R), the set of all infinitely differentiable functionswith compact support, for D. The dense set on which the operator is defined iscalled its domgin.

The adjoint A* of a bounded operator A from a Hilbert space ?ij to a Hilbertspace 7-l2 (which may be Hl itself) is the operator from fl2 to H l defined by

(ul, A u2) = (Aui, u2 ) ,

which should hold for all u l E lil, u2 E l2. (The existence of A* is guaranteedby Riesz' representation theorem: for fixed U2, we can define a linear functional£ on all by £(u1 ) = (Aul , uz). It is clearly bounded, and corresponds thereforeto a vector v so that (ui, v) = £(ui). It is easy to check that the correspondenceu2 —>v is linear; this defines the operator A*.) One has

A*II = IIAII, IIA*AII = IIAI1 2

If A* = A (only possible if A maps H to itself), then A is called self-adjoint. Ifa self-adjoint operator A satisfies (Au, u) >_ 0 for all u E h, then it is called apositive operator; this is often denoted A > 0. We will write A > B if A — B isa positive operator.

Trace-class operators are special operators such that ^ 1(Ae„, e„)1 is finite

for all orthonormal bases in h. For such a trace-class operator, E„ (Ae,,,, en) isindependent of the chosen orthonormal basis; we call this sum the trace of A,

tr A = (Aen, en ) .

xvi PRELIMINARIES AND NOTATION

bounded, i.e., Il(u)1 :::; Gllull for all u E 'H, there exists a unique Vi E 'H so thatl(u) = (U,Vi)'

An operator U from 'H1 to 'H2 is an isometry if (Uv, Uw) = (v, w) for allv, w E 'H1; U is unitary if moreover U'H1 = 'H2 , i.e., every elemento, E 'H2 canbe written as V2 = UV1 for some V1 E 'H1. If the en constitute an orthonormalbasis in 'Hl, and U is unitary, then the Ue., constitute an orthonormal basis in'H2 • The reverse is also true: any operator that maps an orthonormal basis toanother orthonormal basis is unitary.

A set D is called dense in 'H if every u E 'H can be written as the limit ofsome sequence of Un in D. (One then says that the closure of D is all of 'H. Theclosure of a set S is obtained by adding to it all the v that can be obtained aslimits of sequences in S.) If Av is only defined for v E D, but we know that

IIAvl1 :::; Gllvll for all v ED, (0.0.9)

then we can extend A to all of 'H "by continuity." Explicitly: if u E 'H, findUn E D so that limn ......oo Un = u. Then the Un are necessarily a Cauchy sequence,and because of (0.0.9), so are the Aun; the AUn have therefore a limit, which wecall Au (it does not depend on the particular sequence Un that was chosen).

One can also deal with unbounded operators, i.e., A for which there existsno finite Gsuch that IIAul1 :::; Gllull holds for all u E 'H. It is a fact of life thatthese can usually only be defined on a dense set D in 'H, and cannot be extendedby the above trick (since they are not continuous). An example is 1x in L2(1R),

where we can take D = GO'(IR), the set of all infinitely differentiable functionswith compact support, for D. The dense set on which the operator is defined iscalled its domain.

The adjoint A* of a bounded operator A from a Hilbert space 'H1 to a Hilbertspace 1i2 (which may be 1i 1 itself) is the operator from 1i2 to 1i1 defined by

(Ul,A*U2) = (Aul, U2) ,

which should hold for all U1 E 'Hl, U2 E 'H2. (The existence of A* is guaranteedby Riesz' representation theorem: for fixed U2, we can define a linear functionall on 'H1 by i(U1) = (AUl,U2). It is clearly bounded, and corresponds thereforeto a vector v so that (Ul,v) = l(U1). It is easy to check that the correspondenceU2 --+v is linear; this defines the operator A* .) One has

IIA*II = IIAII, IIA*All = IIAII2•

If A* = A (only possible if A maps 'H to itself), then A is called self-adjoint. Ifa self-adjoint operator A satisfies (Au, u) ::::: 0 for all u E 'H, then it is called apositive operator; this is often denoted A ::::: O. We will write A ::::: B if A - B isa positive operator.

Trace-class operators are special operators such that Ln I(Aen, en)I is finitefor all orthonormal bases in 'H. For such a trace-class operator, Ln (Aen, en) isindependent of the chosen orthonormal basis; we call this sum the trace of A,

tr A = L (Aen, en} .n

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

PRELIMINARIES AND NOTATION xvii

If A is positive, then it is sufficient to check whether > n (Aen , en ) is finite foronly one orthonormal basis; if it is, then A is trace-class. (This is not true fornon-positive operators!)

The spectrum v(A) of an operator A from 1 -í to itself consists of all the.. E C such that A — ) Id (Id stands for the identity operator, Id u = u) doesnot have a bounded inverse. In a finite-dimensional Hilbert space, v(A) consistsof the eigenvalues of A; in the infinite-dimensional case, v(A) contains all theeigenvalues (constituting the point spectrum) but often contains other )¼ as well,constituting the continuous spectrum. (For instance, in L2 (R), multiplication off(x) with sinirx has no point spectrum, but its continuous spectrum is [-1, 1].)The spectrum of a self-adjoint operator consists of only real numbers; the spec-trum of a positive operator contains only non-negative numbers. The spectralradius p(A) is defined by

p(A) = sup {IXI; ) E (A)}.

It has the properties

p(A) 5 IIAII and p(A) = lim hlAnll i ^nn--.00

Self-adjoint operators can be diagonalized. This is easiest to understand iftheir spectrum consists only of eigenvalues (as is the case in finite dimensions).One then has

a(A)={;n; nEN},

with a corresponding orthonormal family of eigenvectors,

Aen = An en .

It then follows that, for all u E 7-1,

Au = (Au, enen = (u, Aen)en = )'.n(u, enenn n n

which is the "diagonalization" of A. (The spectral theorem permits us to gen-eralize this if part (or all) of the spectrum is continuous, but we will not need itin this book.) If two operators commute, i.e., ABu = BAu for all u E 7-1, thenthey can be diagonalized simultaneously: there exists an orthonormal basis suchthat

Aen = an en and Ben = ,Qnen .

Many of these properties for bounded operators can also be formulated for un-bounded operators: adjoints, spectrum, diagonalization all exist for unboundedoperators as well. One has to be very careful with domains, however. For in-stance, generalizing the simultaneous diagonalization of commuting operatorsrequires a careful definition of commuting operators: there exist pathologicalexamples where A, B are both defined on a domain D, where AB and BA bothmake sense on D and are equal on D, but where A and B nevertheless are not

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

xviii PRELIMINARIES AND NOTATION

simultaneously diagonalizable (because D was chosen "too small"; see, e.g., Reedand Simon (1971) for an example). The proper definition of commuting for un-bounded self-adjoint operators uses associated bounded operators: Hl and H2commute if their associated unitary evolution operators commute. For a self-adjoint operator H, the associated unitary evolution operators Ut are defined asfollows: for any v E D, the domain of H (beware: the domain of a self-adjointoperator is not just any dense set on which H is well defined), UTV is the solutionv(t) at time t = T of the differential equation

idt v(t) = Hv(t) ,

with initial condition v(0) = v.Banach spaces share many properties with but are more general than Hilbert

spaces. They are linear spaces equipped with a norm (which need not be andgenerally is not derived from a scalar product), complete with respect to thatnorm (i.e., all Cauchy sequences converge; see above). Some of the conceptswe reviewed above for Hilbert spaces also exist in Banach spaces; e.g., boundedoperators, linear functionals, spectrum and spectral radius. An example of aBanach space that is not a Hilbert space is LP(IR), the set of all functions f on Rsuch that 11f II LP (see (0.0.2)) is finite, with 1 _< p < oo, p # 2. Another exampleis L°° (R), the set of all bounded functions on R, with IIf II L°° = supXER I f (x) I.The dual E* of a Banach space E is the set of all bounded linear functionalson E; it is also a linear space, which comes with a natural norm (defined as in(0.0.8)), with respect to which it is complete: E* is a Banach space itself. In thecase of the Lp-spaces, 1 <p < oo, it turn out that elements of L4 , where p andq are related by p' + q- ' = 1, define bounded linear functionals on LP. Indeed,one has Hölder's inequality,

Jdx f(x) 9(x)1 <_ IIfIILP II9IIL9 .

It turn out that all bounded linear functionals on LP are of this type, i.e.,(LP)* = L . In particular, L2 is its own dual; by R.iesz' representation theorem(see above), every Hilbert space is its own dual. The adjoint A* of an operatorA from El to E2 is now an operator from E2 to E, defined by

(A*€2)(vi) = £2(Avi) •

There exist different types of bases in Banach spaces. (We will again onlyconsider separable spaces, in which bases are countable.) The en consti-tute a Schauder basis if, for all v E E, there exist unique p n E C so thatv = limNy 0 En 1 p.e. (i.e., 11v - ^n 1 µne„ II-^0 as N-*oo). The uniquenessrequirement of the µn forces the en to be linearly independent, in the sense thatno en can be in the closure of the linear span of all the others, i.e., there exist no

so that e limN_,^ >N # e In a Schauder basis, the orderingn = m=1, m n rym m• éiof the en may be important. A basis is called unconditional if in addition itsatisfies one of the following two equivalent properties:

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php

PRELIMINARIES AND NOTATION xix

• whenever E p en E E, it follows that > I µn len E E;n n

• if E µH en E E, and en, = ±1, randomly chosen for every n, thenn

/(.n onen E E.n

For an unconditional basis, the order in which the basis vectors are taken doesnot matter. Not all Banach spaces have unconditional bases: L l (R) and L°° (IR)do not.

In a Hilbert space 9-1, an unconditional basis is also called a Riesz basis. ARiesz basis can also be characterized by the following equivalent requirement:there exist a > 0, ,Q < oo so that

aIIuII 2 < E I(u, en)l' <2 , (0.0.10)n

for all u E h. If A is a bounded operator with a bounded inverse, then Amaps any orthonormal basis to a Riesz basis. Moreover, all Riesz bases can beobtained as such images of an orthonormal basis. In a way, Riesz bases are thenext best thing to an orthonormal basis. Note that the inequalities in (0.0.10)are not sufficient to guarantee that the en constitute a Riesz basis: the en alsoneed to be linearly independent!

Dow

nloa

ded

02/2

1/13

to 1

31.1

70.6

.51.

Red

istr

ibut

ion

subj

ect t

o SI

AM

lice

nse

or c

opyr

ight

; see

http

://w

ww

.sia

m.o

rg/jo

urna

ls/o

jsa.

php