Applied and Numerical Harmonic Analysis Series Editor John J. Benedetto University of Maryland Editorial Advisory Board Akram Aldroubi NIH, Biomedical Engineering/ Instrumentation Ingrid Daubechies Princeton University Christopher Heil Georgia Institute of Technology James McClellan Georgia Institute of Technology Michael Unser NIH, Biomedical Engineering/ Instrumentation M. Victor Wickerhauser Washington University Douglas Cochran Arizona State University Hans G. Feichtinger University of Vienna MuratKunt Swiss Federal Institute of Technology, Lausanne Wim Sweldens Lucent Technologies Bell Laboratories Martin Vetterli Swiss Federal Institute of Technology, Lausanne
Transcript
Applied and Numerical Harmonic Analysis
Series Editor John J. Benedetto University of Maryland
All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+ Business Media, LLC), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former are not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.
SPIN 10875334
ISBN 978-1-4612-6500-9 ISBN 978-0-8176-8224-8 (eBook) DOI 10.1007/978-0-8176-8224-8
Typeset by the author.
9 8 7 6 5 4 3 2 I
Series Preface
The Applied and Numerical Harmonic Analysis ( ANHA) book series aims to provide the engineering, mathematical, and scientific communities with significant developments in harmonic analysis, ranging from abstract harmonic analysis to basic applications. The title of the series reflects the importance of applications and numerical implementation, but richness and relevance of applications and implementation depend fundamentally on the structure and depth of theoretical underpinnings. Thus, from our point of view, the interleaving of theory and applications and their creative symbiotic evolution is axiomatic.
Harmonic analysis is a wellspring of ideas and applicability that has flourished, developed, and deepened over time within many disciplines and by means of creative cross-fertilization with diverse areas. The intricate and fundamental relationship between harmonic analysis and fields such as signal processing, partial differential equations (PDEs), and image processing is reflected in our state of the art ANHA series.
Our vision of modern harmonic analysis includes mathematical areas such as wavelet theory, Banach algebras, classical Fourier analysis, timefrequency analysis, and fractal geometry, as well as the diverse topics that impinge on them.
For example, wavelet theory can be considered an appropriate tool to deal with some basic problems in digital signal processing, speech and image processing, geophysics, pattern recognition, biomedical engineering, and turbulence. These areas implement the latest technology from sampling methods on surfaces to fast algorithms and computer vision methods. The underlying mathematics of wavelet theory depends not only on classical Fourier analysis, but also on ideas from abstract harmonic analysis, including von Neumann algebras and the affine group. This leads to a study of the Heisenberg group and its relationship to Gabor systems, and of the metaplectic group for a meaningful interaction of signal decomposition methods. The unifying influence of wavelet theory in the aforementioned topics illustrates the justification for providing a means for centralizing and disseminating information from the broader, but still focused, area of harmonic analysis. This will be a key role of ANHA. We intend to publish with the scope and interaction that such a host of issues demands.
Along with our commitment to publish mathematically significant works at the frontiers of harmonic analysis, we have a comparably strong commitment to publish major advances in the following applicable topics in
vi Series Preface
which harmonic analysis plays a substantial role:
Antenna theory Biomedical signal processing
Digital signal processing Fast algorithms
Gabor theory and applications I mage processing
Numerical partial differential equations
Prediction theory Radar applications Sampling theory
Spectral estimation Speech processing Time-frequency and time-scale analysis Wavelet theory
The above point of view for the ANHA book series is inspired by the history of Fourier analysis itself, whose tentacles reach into so many fields.
In the last two centuries Fourier analysis has had a major impact on the development of mathematics, on the understanding of many engineering and scientific phenomena, and on the solution of some of the most important problems in mathematics and the sciences. Historically, Fourier series were developed in the analysis of some of the classical PDEs of mathematical physics; these series were used to solve such equations. In order to understand Fourier series and the kinds of solutions they could represent, some of the most basic notions of analysis were defined, e.g., the concept of "function." Since the coefficients of Fourier series are integrals, it is no surprise that Riemann integrals were conceived to deal with uniqueness properties of trigonometric series. Cantor's set theory was also developed because of such uniqueness questions.
A basic problem in Fourier analysis is to show how complicated phenomena, such as sound waves, can be described in terms of elementary harmonics. There are two aspects of this problem: first, to find, or even define properly, the harmonics or spectrum of a given phenomenon, e.g., the spectroscopy problem in optics; second, to determine which phenomena can be constructed from given classes of harmonics, as done, for example, by the mechanical synthesizers in tidal analysis.
Fourier analysis is also the natural setting for many other problems in engineering, mathematics, and the sciences. For example, Wiener's Tauberian theorem in Fourier analysis not only characterizes the behavior of the prime numbers, but also provides the proper notion of spectrum for phenomena such as white light; this latter process leads to the Fourier analysis associated with correlation functions in filtering and prediction problems, and these problems, in turn, deal naturally with Hardy spaces in the theory of complex variables.
Nowadays, some of the theory of PDEs has given way to the study of Fourier integral operators. Problems in antenna theory are studied in terms of unimodular trigonometric polynomials. Applications of Fourier analysis abound in signal processing, whether with the Fast Fourier transform (FFT), or filter design, or the adaptive modeling inherent in time-
Series Preface vii
frequency-scale methods such as wavelet theory. The coherent states of mathematical physics are translated and modulated Fourier transforms, and these are used, in conjunction with the uncertainty principle, for dealing with signal reconstruction in communications theory. We are back to the raison d'etre of the ANHA series!
John J. Benedetto Series Editor
University of Maryland College Park
To Khadija, Jakob; and Karen
Contents
Preface
1 Frames in Finite-dimensional Inner Product Spaces 1.1 Some basic facts about frames
xvii
1 2
1.2 Frame bounds and frame algorithms 10 1.3 Frames in en . . . . . . . . . . . . . 14 1.4 The discrete Fourier transform . . . . 19 1.5 Pseudo-inverses and the singular value decomposition 23 1.6 Finite-dimensional function spaces . 28 1. 7 Exercises . . . . . . . . . . . . . . 32
2 Infinite-dimensional Vector Spaces and Sequences 2.1 Sequences ............ . 2.2 Banach spaces and Hilbert spaces 2.3 L2 (IR) and e2 (N) .... 2.4 The Fourier transform 2.5 Operators on L2 (IR) 2.6 Exercises ..... .
3 Bases 3.1 Bases in Banach spaces ..... . 3.2 Bessel sequences in Hilbert spaces
4 Bases and their Limitations 79 4o1 Gabor systems and the Balian-Low Theorem 82 402 Bases and wavelets 83 4o3 General shortcomings 0 86
5 Frames in Hilbert Spaces 87 501 Frames and their properties 88 502 Frame sequences 0 0 0 0 92 5o3 Frames and operators 0 0 0 93 5.4 Frames and bases ..... 96 505 Characterization of frames 101 506 The dual frames 0 0 111 507 Tight frames 0 0 0 0 0 0 0 0 115 508 Continuous frames .... 115 5o9 Frames and signal processing 0 117 5010 Exercises 0 0 0 0 0 0 0 0 0 119
6 Frames versus Riesz Bases 123 601 Conditions for a frame being a Riesz basis 123 602 Riesz frames and near-Riesz bases 0 0 0 0 126 603 Frames containing a Riesz basis ..... 126 6.4 A frame which does not contain a basis 128 605 A moment problem 134 606 Exercises 0 0 0 0 0 136
7 Frames of Translates 137 701 Sequences in JRd ...... 138 702 Frames of translates 0 0 0 0 0 140 703 Frames of integer-translates 147 7.4 Irregular frames of translates 0 153 705 The sampling problem 156 706 Frames of exponentials 157 707 Exercises 0 0 0 0 0 0 163
8.3 Necessary conditions 8.4 Sufficient conditions . 8.5 The Wiener space W 8.6 Special functions .. 8.7 General shift-invariant systems. 8.8 Exercises . . . . . . . . . . . . .
Contents xiii
174 176 187 190 192 198
9 Selected Topics on Gabor Frames 201 9.1 Popular Gabor conditions . . . . . . . . . . . . . . . . . 202 9.2 Representations of the Gabor frame operator and duality 204 9.3 The duals of a Gabor frame 208 9.4 The Zak transform . . 215 9.5 Tight Gabor frames . . . 219 9.6 The lattice parameters . 222 9.7 Irregular Gabor systems 226 9.8 Applications of Gabor frames 230 9.9 Wilson bases . 232 9 .1 0 Exercises . . . . . .
10 Gabor Frames in £2 (Z) 10.1 Translation and modulation on £2 (Z) 10.2 Discrete Gabor systems through sampling 10.3 Gabor frames in cL ....... . 10.4 Shift-invariant systems .... . 10.5 Frames in £2 (Z) and filter banks 10.6 Exercises ............ .
A.3 Integration . . . . . . . . . . . . . . A.4 Some special normed vector spaces. A.5 Operators on Banach spaces A.6 Operators on Hilbert spaces A. 7 The pseudo-inverse . . A.8 Some special functions A.9 B-splines A.lO Notes .
List of symbols
References
Index
Contents XV
405 406 407 408 410 412 413 416
419
421
437
Preface
Frames have fascinated me since day one. Every student in mathematics
learns about bases in vector spaces, allowing one to represent each element
in a convenient and unique way. One day in 1990, Henrik Stetkrer, who was
my masters thesis advisor, showed me the definition of a frame and told me
that a frame is some kind of "overcomplete basis": one can also represent
each element in the vector space via a frame, but the representation might
not be unique. I was really surprised: how come that the question in e.g.,
linear algebra always was how to extract a basis from an overcomplete set,
and one never got the idea that overcompleteness by itself could be useful?
A search on Mathematical Reviews or Zentralblatt shows only a few titles
of books or articles concerning frames published before 1991; among those
we mention the original paper by Duffin and Schaeffer (121], the excellent
book by Young [279], and the important papers by Daubechies, Grossmann
and Meyer (108), Daubechies (105], and Heil and Walnut (172). Now, just ten years later, hundreds of papers have the word frame in the title, and
perhaps a thousand discuss one or more results. Today, no single book can
treat all the important and interesting results that have been published. The aim of this book is to present parts of the modern theory for bases
and frames in Hilbert spaces in a way that the material can be used in a
graduate course, as well as by professional readers. For use in a graduate
course, a number of exercises is included; they appear at the end of each
chapter. The number of exercises give a hint of the level of the chapter:
there are many exercises in the introductory chapters, but only few in the
advanced chapters. In the same spirit, almost all results in the introductory chapters appear with full proofs; in the advanced chapters several results
xviii Preface
are presented without proofs. We believe it is more useful to state a large number of results in a common framework than to see detailed proofs of significantly fewer statements; this feature also makes the book useful as a reference.
The content can be split naturally into three parts: Chapters 1-6 describe the theory on an abstract level, Chapters 7-14 describe explicit constructions in L2-spaces, and Chapters 15-17 deal with selected research topics.
In Chapters 1-6 almost all results concern frames in general Hilbert spaces. The goal is an almost complete treatment of the known results for frames. For the explicit constructions in L2 ( -1r, 1r) and L2 (JR), which appear in Chapters 7-14, the situation is different. For this part, I was forced to concentrate on selected parts of the theory. Since we are mainly interested in overcomplete systems, the theory presented in these chapters is part of a larger picture, and the reader will certainly benefit from knowledge of the background. Chapter 7 connects to the theory for nonharmonic Fourier series, cf. the book [279] by Young. Gabor frames arise naturally in the context of time-frequency analysis, and the book by Grochenig [153] will clarify the role of Chapters 8-9 in time-frequency analysis. Finally, the role of wavelets is highlighted in the classic book [106] by Daubechies, which also gives the motivation for the study of frames arising from multiscale methods in Chapters 13-14. Technically, we do not rely on any of these books (only at a few places will we refer to results from them without proof), but they put the results of frame studies in the right perspective. Chapters 7-14 are also influenced by the fact that the material is used in several areas of applied mathematics; the reader will observe that although this is a book about mathematics, those chapters concentrate on applicable ways to construct frames rather than on abstract characterizations.
Let us describe the chapters in more detail. Chapter 1 presents basic results in finite-dimensional vector spaces with an inner product. This enables a reader with a basic knowledge of linear algebra to understand the idea behind frames without the technical complications in infinite-dimensional spaces. Many of the topics from the rest of the book are presented here, so Chapter 1 can also serve as an introduction to the later chapters.
Chapter 2 collects some definitions and conventions concerning infinitedimensional vector spaces. Special attention is given to the Hilbert space L2 (JR) and operators hereon. We expect the reader to be familiar with this material; the chapter is too short to be considered as an introduction to the subject.
Chapter 3 describes the classical theory for bases in Hilbert spaces and Banach theory. The examples in this chapter are chosen so they lead naturally to the constructions in Chapters 7-14.
Chapter 4 highlights some of the limitations on the properties one can obtain from bases, and thus provides motivation for considering generalizations of bases.
Preface xix
Chapters 5-6 contain the core material about frames and Riesz bases. Chapter 5 is classical, but Chapter 6 contains several results published in the last five years. The interplay between frames and bases is discussed in detail in Chapter 6, and we also discuss frames that become bases when a certain number of elements are deleted.
Chapters 7-14 deal with frames having a special structure. A central part deals with various sufficient conditions for existence of those frames. The most fundamental frames, namely frames of exponentials in L 2 ( -1r, 1r) and frames of translates in L2 (IR), are discussed in Chapter 7. If one wants to consider frames in L2 (IR), these frames easily lead to Gabor frames, which is the subject of Chapters 8-10. Wavelet frames are introduced in Chapter 11, and sufficient conditions to find them are given for arbitrary dilation parameter a > 1 and translation parameter b > 0. Some results concerning irregular wavelet frames are also presented there. Chapter 12 specializes to the important case a= 2, b = 1, which has attracted much attention during the past ten years. Constructions via multiscale methods are the focus in Chapters 13-14.
In Chapter 15, the question is stability of frames, i.e., whether a set of elements close to a frame is itself a frame. Since real-life measurements are never completely exact, this question is very important for applications.
Chapter 16 presents methods for the approximation of the inverse frame operator using finite subsets of the frame. Since every application of frame theory has to be performed in a finite-dimensional vector space, this topic is also of fundamental importance for applications.
Chapter 17 deals with extensions and generalizations of the material from the previous chapters. Expansions in Banach spaces and their relationship to frames in Hilbert spaces are discussed, as well as frames appearing via integrable group representations. The latter subject gives a unified description of the frames from Chapters 8-11.
Finally, an Appendix collects several basic results for easy reference. It also contains material on pseudo-inverse operators and splines which is not expected to be known in advance, and therefore is treated in more detail.
For the purpose of a graduate course, we mention that if students have a good background in functional analysis, they can skip Chapter 1 and parts of Chapters 2-3. Chapter 4 is important as motivation and Chapter 5 is also core material. But after covering these chapters, a course can continue in several ways. One possibility is to follow a theoretical track, and consider the relationship between frames and bases in more detail; this could be followed by a study of one of the three final chapters. Another possibility is to continue with constructions of exponential frames and Gabor frames, or wavelet frames. If wavelets are chosen as the subject, it is worth noticing that the four wavelet chapters are almost independent of each other.
This book presents frames and Riesz bases from the functional analytic point of view, and we concentrate on their mathematical aspects. However, as demonstrated by several papers by Daubechies and others, frames are
xx Preface
very useful in several areas of applied mathematics, including signal processing and image processing. But this part of the story should be told by the people who are directly involved in it, and we will only sketch a few applications.
It is a pleasure to thank the many colleagues and students who helped in the process of writing this book. The starting point was seventy pages of notes, which were written jointly with Tor ben Klint Jensen, who was at that time a masters student. My original idea was to write a book concentrating on frames in general Hilbert spaces; I am very happy that Thomas Strohmer and an anonymous reviewer suggested that I further go into detail with wavelet and Gabor systems. Their ideas added more than a hundred pages to the book, and extended the scope considerably. Very useful suggestions for adding material were also given by Hans Feichtinger.
Alexander Lindner read a large part of the final manuscript and proposed many improvements. Elena Cordero, Niklas Grip, Per Christian Hansen, Reza Mahdavi, John Rassias, Henrik Stetkrer, and Diana Stoeva read parts of the book and helped to spot mistakes; I am very grateful to all of them.
I am thankful to the Department of Mathematics at the Technical University of Denmark for providing me with the excellent working conditions that made it possible to concentrate on the book for two semesters. In addition, a large part of the book was written during a stay at the research group NuHAG at the University of Vienna. It is a pleasure to thank the group leader, Hans Feichtinger, and the members of NuHAG for their support.
I am thankful to John Benedetto for inviting me to write this book, and I thank the staff at Birkhauser, especially Tom Grasso and Ann Kostant, for their assistance and support. Thanks are also given to Elizabeth Loew from Texniques, who helped with Latex problems.
Finally, I acknowledge support from the WAVE-program, sponsored by the Danish Science Foundation.
Ole Christensen Kgs. Lyngby, Denmark September 2002
