Cornerstones

Series EditorsCharles L. Epstein, University of Pennsylvania, PhiladelphiaSteven G. Krantz, University of Washington, St. Louis

Advisory BoardAnthony W. Knapp, State University of New York at Stony Brook, Emeritus

Anthony W. Knapp

Basic Real Analysis

Along with a companion volumeAdvanced Real Analysis

BirkhäuserBoston • Basel • Berlin

Anthony W. Knapp81 Upper Sheep Pasture RoadEast Setauket, NY 11733-1729U.S.A.e-mail to: aknapp@math.sunysb.eduhttp://www.math.sunysb.edu/˜ aknapp/books/basic.html

Cover design by Mary Burgess.

Mathematics Subject Classicification (2000): 28-01, 26-01, 42-01, 54-01, 34-01

Library of Congress Cataloging-in-Publication DataKnapp, Anthony W.

Basic real analysis: along with a companion volume Advanced real analysis / AnthonyW. Knapp

p. cm. – (Cornerstones)Includes bibliographical references and index.ISBN 0-8176-3250-6 (alk. paper)

1. Mathematical analysis. I. Title. II. Cornerstones (Birkhäuser)

QA300.K56 2005515–dc22 2005048070

ISBN-10 0-8176-3250-6 eISBN 0-8176-4441-5 Printed on acid-free paper.ISBN-13 978-0-8176-3250-2

Advanced Real Analysis ISBN 0-8176-4382-6Basic Real Analysis and Advanced Real Analysis (Set) ISBN 0-8176-4407-5

c©2005 Anthony W. KnappAll rights reserved. This work may not be translated or copied in whole or in part without the writtenpermission of the publisher (Birkhäuser Boston, c/o Springer Science+Business Media Inc., 233 SpringStreet, New York, NY 10013, USA) and the author, except for brief excerpts in connection with reviewsor scholarly analysis. Use in connection with any form of information storage and retrieval, electronicadaptation, computer software, or by similar or dissimilar methodology now known or hereafter developedis forbidden.The use in this publication of trade names, trademarks, service marks and similar terms, even if they arenot identified as such, is not to be taken as an expression of opinion as to whether or not they are subject toproprietary rights.

Printed in the United States of America. (MP)

9 8 7 6 5 4 3 2 1 SPIN 10934074

www.birkhauser.com

To Susan

and

To My Real-Analysis Teachers:

Salomon Bochner, William Feller, Hillel Furstenberg,

Harish-Chandra, Sigurdur Helgason, John Kemeny,

John Lamperti, Hazleton Mirkil, Edward Nelson,

Laurie Snell, Elias Stein, Richard Williamson

CONTENTS

Preface xiDependence Among Chapters xivGuide for the Reader xvList of Figures xviiiAcknowledgments xixStandard Notation xxi

I. THEORY OF CALCULUS IN ONE REAL VARIABLE 1

1. Review of Real Numbers, Sequences, Continuity 22. Interchange of Limits 133. Uniform Convergence 154. Riemann Integral 265. Complex-Valued Functions 416. Taylor’s Theorem with Integral Remainder 437. Power Series and Special Functions 448. Summability 539. Weierstrass Approximation Theorem 5810. Fourier Series 6111. Problems 78

II. METRIC SPACES 82

1. Definition and Examples 832. Open Sets and Closed Sets 913. Continuous Functions 954. Sequences and Convergence 975. Subspaces and Products 1026. Properties of Metric Spaces 1057. Compactness and Completeness 1088. Connectedness 1159. Baire Category Theorem 11710. Properties of C(S) for Compact Metric S 12111. Completion 12712. Problems 130

vii

viii Contents

III. THEORY OF CALCULUS IN SEVERAL REAL VARIABLES 1351. Operator Norm 1352. Nonlinear Functions and Differentiation 1393. Vector-Valued Partial Derivatives and Riemann Integrals 1464. Exponential of a Matrix 1485. Partitions of Unity 1516. Inverse and Implicit Function Theorems 1527. Definition and Properties of Riemann Integral 1618. Riemann Integrable Functions 1669. Fubini’s Theorem for the Riemann Integral 16910. Change of Variables for the Riemann Integral 17111. Problems 179

IV. THEORY OF ORDINARY DIFFERENTIAL EQUATIONSAND SYSTEMS 1831. Qualitative Features and Examples 1832. Existence and Uniqueness 1873. Dependence on Initial Conditions and Parameters 1944. Integral Curves 1995. Linear Equations and Systems, Wronskian 2016. Homogeneous Equations with Constant Coefficients 2087. Homogeneous Systems with Constant Coefficients 2118. Series Solutions in the Second-Order Linear Case 2189. Problems 226

V. LEBESGUE MEASURE AND ABSTRACTMEASURE THEORY 2311. Measures and Examples 2312. Measurable Functions 2383. Lebesgue Integral 2414. Properties of the Integral 2455. Proof of the Extension Theorem 2536. Completion of a Measure Space 2627. Fubini’s Theorem for the Lebesgue Integral 2658. Integration of Complex-Valued and Vector-Valued Functions 2749. L1, L2, L∞, and Normed Linear Spaces 27910. Problems 289

VI. MEASURE THEORY FOR EUCLIDEAN SPACE 2961. Lebesgue Measure and Other Borel Measures 2972. Convolution 3063. Borel Measures on Open Sets 3144. Comparison of Riemann and Lebesgue Integrals 318

Contents ix

VI. MEASURE THEORY FOR EUCLIDEAN SPACE (Continued)

5. Change of Variables for the Lebesgue Integral 3206. Hardy–Littlewood Maximal Theorem 3277. Fourier Series and the Riesz–Fischer Theorem 3348. Stieltjes Measures on the Line 3399. Fourier Series and the Dirichlet–Jordan Theorem 34610. Distribution Functions 35011. Problems 352

VII. DIFFERENTIATION OF LEBESGUE INTEGRALSON THE LINE 357

1. Differentiation of Monotone Functions 3572. Absolute Continuity, Singular Measures, and

Lebesgue Decomposition 3643. Problems 370

VIII. FOURIER TRANSFORM IN EUCLIDEAN SPACE 373

1. Elementary Properties 3732. Fourier Transform on L1, Inversion Formula 3773. Fourier Transform on L2, Plancherel Formula 3814. Schwartz Space 3845. Poisson Summation Formula 3896. Poisson Integral Formula 3927. Hilbert Transform 3978. Problems 404

IX. L p SPACES 409

1. Inequalities and Completeness 4092. Convolution Involving L p 4173. Jordan and Hahn Decompositions 4184. Radon–Nikodym Theorem 4205. Continuous Linear Functionals on L p 4246. Marcinkiewicz Interpolation Theorem 4277. Problems 436

X. TOPOLOGICAL SPACES 441

1. Open Sets and Constructions of Topologies 4412. Properties of Topological Spaces 4473. Compactness and Local Compactness 4514. Product Spaces and the Tychonoff Product Theorem 4585. Sequences and Nets 4636. Quotient Spaces 471

x Contents

X. TOPOLOGICAL SPACES (Continued)

7. Urysohn’s Lemma 4748. Metrization in the Separable Case 4769. Ascoli–Arzelà and Stone–Weierstrass Theorems 47710. Problems 480

XI. INTEGRATION ON LOCALLY COMPACT SPACES 485

1. Setting 4852. Riesz Representation Theorem 4903. Regular Borel Measures 5044. Dual to Space of Finite Signed Measures 5095. Problems 517

XII. HILBERT AND BANACH SPACES 520

1. Definitions and Examples 5202. Geometry of Hilbert Space 5263. Bounded Linear Operators on Hilbert Spaces 5354. Hahn–Banach Theorem 5375. Uniform Boundedness Theorem 5436. Interior Mapping Principle 5457. Problems 549

APPENDIX 553

A1. Sets and Functions 553A2. Mean Value Theorem and Some Consequences 559A3. Inverse Function Theorem in One Variable 561A4. Complex Numbers 563A5. Classical Schwarz Inequality 563A6. Equivalence Relations 564A7. Linear Transformations, Matrices, and Determinants 565A8. Factorization and Roots of Polynomials 568A9. Partial Orderings and Zorn’s Lemma 573A10. Cardinality 577

Hints for Solutions of Problems 581Selected References 637Index of Notation 639Index 643

PREFACE

This book and its companion volume Advanced Real Analysis systematicallydevelop concepts and tools in real analysis that are vital to every mathematician,whether pure or applied, aspiring or established. The two books together containwhat the young mathematician needs to know about real analysis in order tocommunicate well with colleagues in all branches of mathematics.The books are written as textbooks, and their primary audience is students who

are learning the material for the first time and who are planning a career in whichthey will use advanced mathematics professionally. Much of the material in thebooks corresponds to normal course work. Nevertheless, it is often the case thatcore mathematics curricula, time-limited as they are, do not include all the topicsthat one might like. Thus the book includes important topics that may be skippedin required courses but that the professional mathematician will ultimately wantto learn by self-study.The content of the required courses at each university reflects expectations of

what students needbefore beginning specialized study andworkon a thesis. Theseexpectations vary from country to country and from university to university. Evenso, there seems to be a rough consensus aboutwhatmathematics a plenary lecturerat a broad international or national meeting may take as known by the audience.The tables of contents of the two books represent my own understanding of whatthat degree of knowledge is for real analysis today.

Key topics and features of Basic Real Analysis are as follows:

• Early chapters treat the fundamentals of real variables, sequences and seriesof functions, the theory of Fourier series for the Riemann integral, metricspaces, and the theoretical underpinnings of multivariable calculus and ordi-nary differential equations.

• Subsequent chapters develop the Lebesgue theory in Euclidean and abstractspaces, Fourier series and the Fourier transform for the Lebesgue integral,point-set topology, measure theory in locally compact Hausdorff spaces, andthe basics of Hilbert and Banach spaces.

• The subjects of Fourier series and harmonic functions are used as recurringmotivation for a number of theoretical developments.

• The development proceeds from the particular to the general, often introducingexamples well before a theory that incorporates them.

xi

xii Preface

• More than 300 problems at the ends of chapters illuminate aspects of thetext, develop related topics, and point to additional applications. A separate55-page section “Hints for Solutions of Problems” at the end of the book givesdetailed hints for most of the problems, together with complete solutions formany.

Beyond a standard calculus sequence in one and several variables, the mostimportant prerequisite for using Basic Real Analysis is that the reader alreadyknow what a proof is, how to read a proof, and how to write a proof. Thisknowledge typically is obtained from honors calculus courses, or from a coursein linear algebra, or from a first junior-senior course in real variables. In addition,it is assumed that the reader is comfortablewith amodest amount of linear algebra,including row reduction of matrices, vector spaces and bases, and the associatedgeometry. A passing acquaintance with the notions of group, subgroup, andquotient is helpful as well.Chapters I–IV are appropriate for a single rigorous real-variables course and

may be used in either of two ways. For students who have learned about proofsfrom honors calculus or linear algebra, these chapters offer a full treatment of realvariables, leaving out only the more familiar parts near the beginning—such aselementary manipulations with limits, convergence tests for infinite series withpositive scalar terms, and routine facts about continuity and differentiability. Forstudents who have learned about proofs from a first junior-senior course in realvariables, these chapters are appropriate for a second such course that begins withRiemann integration and sequences and series of functions; in this case the firstsection of Chapter I will be a review of some of the more difficult foundationaltheorems, and the course can conclude with an introduction to the Lebesgueintegral from Chapter V if time permits.Chapters V through XII treat Lebesgue integration in various settings, as well

as introductions to the Euclidean Fourier transform and to functional analysis.Typically this material is taught at the graduate level in the United States, fre-quently in one of threeways: ThefirstwaydoesLebesgue integration inEuclideanand abstract settings and goes on to consider the Euclidean Fourier transform insome detail; this corresponds to Chapters V–VIII. A second way does Lebesgueintegration in Euclidean and abstract settings, treats L p spaces and integration onlocally compact Hausdorff spaces, and concludes with an introduction to Hilbertand Banach spaces; this corresponds to Chapters V–VII, part of IX, and XI–XII.A third way combines an introduction to the Lebesgue integral and the EuclideanFourier transform with some of the subject of partial differential equations; thiscorresponds to some portion of Chapters V–VI and VIII, followed by chaptersfrom the companion volume Advanced Real Analysis.In my own teaching, I have most often built one course around Chapters I–IV

and another around Chapters V–VII, part of IX, and XI–XII. I have normally

Preface xiii

assigned the easier sections of Chapters II and X as outside reading, indicatingthe date when the lectures would begin to use that material.More detailed information about how the book may be used with courses may

be deduced from the chart “Dependence among Chapters” on page xiv and thesection “Guide to the Reader” on pages xv–xvii.The problems at the ends of chapters are an important part of the book. Some

of them are really theorems, some are examples showing the degree to whichhypotheses can be stretched, and a few are just exercises. The reader gets noindication which problems are of which type, nor of which ones are relativelyeasy. Each problem can be solved with tools developed up to that point in thebook, plus any additional prerequisites that are noted.

Two omissions from the pair of books are of note. One is any treatment ofStokes’s Theorem and differential forms. Although there is some advantage,when studying these topics, in having the Lebesgue integral available and inhaving developed an attitude that integration can be defined by means of suitablelinear functionals, the topic of Stokes’s Theorem seems to fit better in a bookabout geometry and topology, rather than in a book about real analysis.The other omission concerns the use of complex analysis. It is tempting to try

to combine real analysis and complex analysis into a single subject, but my ownexperience is that this combination does not work well at the level of Basic RealAnalysis, only at the level of Advanced Real Analysis.Almost all of the mathematics in the two books is at least forty years old, and I

make no claim that any result is new. The books are a distillation of lecture notesfrom a 35-year period of my own learning and teaching. Sometimes a problem atthe end of a chapter or an approach to the exposition may not be a standard one,but no attempt has been made to identify such problems and approaches. In thereverse direction it is possible that my early lecture notes have directly quotedsome source without proper attribution. As an attempt to rectify any difficultiesof this kind, I have included a section of “Acknowledgements” on pages xix–xxof this volume to identify the main sources, as far as I can reconstruct them, forthose original lecture notes.I amgrateful toAnnKostant andStevenKrantz for encouraging this project and

for making many suggestions about pursuing it, and to Susan Knapp and DavidKramer for helping with the readability. The typesetting was by AMS-TEX, andthe figures were drawn with Mathematica.I invite corrections and other comments from readers. I plan to maintain a list

of known corrections on my own Web page.A. W. KNAPP

May 2005

DEPENDENCE AMONG CHAPTERS

Below is a chart of the main lines of dependence of chapters on prior chapters.The dashed lines indicate helpful motivation but no logical dependence. Apartfrom that, particular examples maymake use of information from earlier chaptersthat is not indicated by the chart.

I, II, III in order

V IV

VI

VIII VII X

IX

IX.6 XI

XII

xiv

GUIDE FOR THE READER

This section is intended to help the reader find out what parts of each chapter aremost important and how the chapters are interrelated. Further information of thiskind is contained in the abstracts that begin each of the chapters.The book pays attention to certain recurring themes in real analysis, allowing

a person to see how these themes arise in increasingly sophisticated ways. Ex-amples are the role of interchanges of limits in theorems, the need for certainexplicit formulas in the foundations of subject areas, the role of compactness andcompleteness in existence theorems, and the approach of handling nice functionsfirst and then passing to general functions.All of these themes are introduced in Chapter I, and already at that stage they

interact in subtle ways. For example, a natural investigation of interchanges oflimits in Sections 2–3 leads to the discovery of Ascoli’s Theorem, which is afundamental compactness tool for proving existence results. Ascoli’s Theoremis proved by the “Cantor diagonal process,” which has other applications tocompactness questions and does not get fully explained until Chapter X. Theconsequence is that, no matter where in the book a reader plans to start, everyonewill be helped by at least leafing through Chapter I.

The remainder of this section is an overview of individual chapters and groupsof chapters.Chapter I. Every section of this chapter plays a role in setting up matters

for later chapters. No knowledge of metric spaces is assumed anywhere in thechapter. Section 1will be a review for anyonewhohas already had a course in real-variable theory; the section shows how compactness and completeness addressall the difficult theorems whose proofs are often skipped in calculus. Section 2begins the development of real-variable theory at the point of sequences and seriesof functions. It contains interchange results that turn out to be special cases ofthe main theorems of Chapter V. Sections 8–9 introduce the approach of handlingnice functions before general functions, and Section 10 introduces Fourier series,which provided a great deal of motivation historically for the development of realanalysis and are used in this book in that same way. Fourier series are somewhatlimited in the setting of Chapter I because one encounters no class of functions,other than infinitely differentiable ones, that corresponds exactly to some class ofFourier coefficients; as a result Fourier series, with Riemann integration in use,

xv

xvi Guide for the Reader

are not particularly useful for constructing new functions from old ones. Thisdefect will be fixed with the aid of the Lebesgue integral in Chapter VI.Chapter II. Now that continuity and convergence have been addressed on

the line, this chapter establishes a framework for these questions in higher-dimensional Euclidean space and other settings. There is no point in ad hocdefinitions for each setting, and metric spaces handle many such settings at once.Chapter X later will enlarge the framework from metric spaces to “topologicalspaces.” Sections 1–6 of Chapter II are routine. Section 7, on compactnessand completeness, is the core. The Baire Category Theorem in Section 9 is notused outside of Chapter II until Chapter XII, and it may therefore be skippedtemporarily. Section 10 contains the Stone–Weierstrass Theorem, which is afundamental approximation tool. Section 11 is used in some of the problems butis not otherwise used in the book.Chapter III. This chapter does for the several-variable theory what Chapter I

has done for the one-variable theory. Themain results are the Inverse and ImplicitFunction Theorems in Section 6 and the change-of-variables formula for multipleintegrals in Section 10. The change-of-variables formula has to be regarded asonly a preliminary version, since what it directly accomplishes for the changeto polar coordinates still needs supplementing; this difficulty will be repaired inChapter VI with the aid of the Lebesgue integral. Section 4, on exponentialsof matrices, may be skipped if linear systems of ordinary differential equationsare going to be skipped in Chapter IV. Some of the problems at the end of thechapter introduce harmonic functions; harmonic functions will be combined withFourier series in problems in later chapters to motivate and illustrate some of thedevelopment.Chapter IV provides theoretical underpinnings for the material in a traditional

undergraduate course in ordinary differential equations. Nothing later in the bookis logically dependent on Chapter IV; however, Chapter XII includes a discussionof orthogonal systems of functions, and the examples of these that arise in ChapterIV are helpful as motivation. Some people shy away from differential equationsand might wish to treat Chapter IV only lightly, or perhaps not at all. The subjectis nevertheless of great importance, and Chapter IV is the beginning of it. Aminimal treatment of Chapter IV might involve Sections 1–2 and Section 8, allof which visibly continue the themes begun in Chapter I.Chapters V–VI treat the core of measure theory—including the basic conver-

gence theorems for integrals, the development of Lebesgue measure in one andseveral variables, Fubini’s Theorem, the metric spaces L1 and L2 and L∞, andthe use of maximal theorems for getting at differentiation of integrals and othertheorems concerning almost-everywhere convergence. In Chapter V Lebesguemeasure in one dimension is introduced right away, so that one immediately hasthe most important example at hand. The fundamental Extension Theorem for

Guide for the Reader xvii

gettingmeasures to bedefinedonσ -rings andσ -algebras is statedwhenneededbutis provedonly after the basic convergence theorems for integrals havebeenproved;the proof in Sections 5–6 may be skipped on first reading. Section 7, on Fubini’sTheorem, is a powerful result about interchange of integrals. At the same timethat it justifies interchange, it also constructs a “double integral”; consequentlythe section prepares the way for the construction in Chapter VI of n-dimensionalLebesguemeasure from 1-dimensional Lebesguemeasure. Section 10 introducesnormed linear spaces along with the examples of L1 and L2 and L∞, and it goeson to establish some properties of all normed linear spaces. Chapter VI fleshesout measure theory as it applies to Euclidean space in more than one dimension.Of special note is the Lebesgue-integration version in Section 5 of the change-of-variables formula for multiple integrals and the Riesz–Fischer Theorem inSection 7. The latter characterizes square-integrable periodic functions by theirFourier coefficients and makes the subject of Fourier series useful in constructingfunctions. Differentiation of integrals in approached in Section 6 of Chapter VIas a problem of estimating finiteness of a quantity, rather than its smallness; thedevice is the Hardy–Littlewood Maximal Theorem, and the approach becomes aroutine way of approaching almost-everywhere convergence theorems. Sections8–10 are of somewhat less importance and may be omitted if time is short;Section 10 is applied only in Section IX.6.Chapters VII–IX are continuations of measure theory that are largely indepen-

dent of each other. Chapter VII contains the traditional proof of the differentiationof integrals on the line via differentiation of monotone functions. No later chapteris logically dependent on Chapter VII; the material is included only because of itshistorical importance and its usefulness as motivation for the Radon–NikodymTheorem in Chapter IX. Chapter VIII is an introduction to the Fourier transformin Euclidean space. Its core consists of the first four sections, and the rest may beconsidered as optional if Section IX.6 is to be omitted. Chapter IX concerns L p

spaces for 1 ≤ p ≤ ∞; only Section 6 makes use of material from Chapter VIII.Chapter X develops, at the latest possible time in the book, the necessary part

of point-set topology that goes beyond metric spaces. Emphasis is on productand quotient spaces, and on Urysohn’s Lemma concerning the construction ofreal-valued functions on normal spaces.Chapter XI contains one more continuation of measure theory, namely special

features ofmeasures on locally compactHausdorff spaces. It provides an examplebeyond L p spaces in which one can usefully identify the dual of a particularnormed linear space. These chapters depend on Chapter X and on the first fivesections of Chapter IX but do not depend on Chapters VII–VIII.Chapter XII is a brief introduction to functional analysis, particularly toHilbert

spaces, Banach spaces, and linear operators on them. The main topics are thegeometry of Hilbert space and the three main theorems about Banach spaces.

LIST OF FIGURES

1.1. Approximate identity 59

1.2. Fourier series of sawtooth function 65

1.3. Dirichlet kernel 69

2.1. An open set centered at the origin in the hedgehog space 88

2.2. Open ball contained in an intersection of two open balls 92

4.1. Graphs of solutions of some first-order ordinary differentialequations 185

4.2. Integral curve of a vector field 199

4.3. Graph of Bessel function J0(t) 225

6.1. Construction of a Cantor function F 343

7.1. Rising Sun Lemma 358

xviii

ACKNOWLEDGMENTS

The author acknowledges the sources below as themain ones he used in preparingthe lectures from which this book evolved. Any residual unattributed directquotations in the book are likely to be from these.

The descriptions below have been abbreviated. Full descriptions of the booksand Stone article may be found in the section “Selected References” at theend of the book. The item “Feller’s Functional Analysis” refers to lectures byWilliam Feller at Princeton University for Fall 1962 and Spring 1963, and theitem “Nelson’s Probability” refers to lectures by Edward Nelson at PrincetonUniversity for Spring 1963.

This list is not to be confused with a list of recommended present-day readingfor these topics; newer books deserve attention.

CHAPTER I. Rudin’s Principles of Mathematical Analysis, Zygmund’sTrigonometric Series.

CHAPTER II. Feller’s Functional Analysis, Kelley’s General Topology,Stone’s “A generalized Weierstrass approximation theorem.”

CHAPTER III. Rudin’s Principles of Mathematical Analysis, Spivak’sCalculus on Manifolds.

CHAPTER IV. Coddington–Levinson’s Theory of Ordinary DifferentialEquations, Kaplan’s Ordinary Differential Equations.

CHAPTER V. Halmos’s Measure Theory, Rudin’s Principles of MathematicalAnalysis.

CHAPTER VI. Rudin’s Principles of Mathematical Analysis, Rudin’s Real andComplex Analysis, Saks’s Theory of the Integral, Spivak’sCalculus onManifolds,Stein–Weiss’s Introduction to Fourier Analysis on Euclidean Spaces.

CHAPTER VII. Riesz–Nagy’s Functional Analysis, Zygmund’s TrigonometricSeries.

CHAPTER VIII. Stein’s Singular Integrals and Differentiability Properties ofFunctions, Stein–Weiss’s Introduction to Fourier Analysis on Euclidean Spaces.

CHAPTER IX. Dunford–Schwartz’s Linear Operators, Feller’s FunctionalAnalysis, Halmos’s Measure Theory, Saks’s Theory of the Integral, Stein’sSingular Integrals and Differentiability Properties of Functions.

xix

xx Acknowledgments

CHAPTER X. Kelley’s General Topology, Nelson’s Probability.

CHAPTER XI. Feller’s Functional Analysis, Halmos’s Measure Theory,Nelson’s Probability.

CHAPTER XII. Dunford–Schwartz’s Linear Operators, Feller’s FunctionalAnalysis, Riesz–Nagy’s Functional Analysis.

APPENDIX. For Sections 1, 9, 10: Dunford–Schwartz’s Linear Operators,Hayden–Kennison’s Zermelo–Fraenkel Set Theory, Kelley’s General Topology.

STANDARD NOTATION

Item Meaning

#S or |S| number of elements in S∅ empty set{x ∈ E | P} the set of x in E such that P holdsEc complement of the set EE ∪ F, E ∩ F, E − F union, intersection, difference of sets⋃

α Eα,⋂

α Eα union, intersection of the sets EαE ⊆ F, E ⊇ F E is contained in F , E contains FE × F, ×s∈S Xs products of sets(a1, . . . , an), {a1, . . . , an} ordered n-tuple, unordered n-tuplef : E → F, x → f (x) function, effect of functionf ◦ g, f ∣∣E composition of f following g, restriction to Ef ( · , y) the function x → f (x, y)f (E), f −1(E) direct and inverse image of a setδi j Kronecker delta: 1 if i = j , 0 if i �= j(nk

)binomial coefficient

n positive, n negative n > 0, n < 0Z, Q, R, C integers, rationals, reals, complex numbersmax (and similarly min) maximum of finite subset of a totally ordered set∑or∏

sum or product, possibly with a limit operationcountable finite or in one-one correspondence with Z[x] greatest integer ≤ x if x is realRe z, Im z real and imaginary parts of complex zz̄ complex conjugate of z|z| absolute value of z1 multiplicative identity1 or I identity matrix or operatordim V dimension of vector spaceRn , Cn spaces of column vectorsdet A determinant of AAtr transpose of Adiag(a1, . . . , an) diagonal matrix∼= is isomorphic to, is equivalent to

xxi

CHAPTER I

Theory of Calculus in One Real Variable

Abstract. This chapter, beginning with Section 2, develops the topic of sequences and seriesof functions, especially of functions of one variable. An important part of the treatment is anintroduction to the problem of interchange of limits, both theoretically and practically. This problemplays a role repeatedly in real analysis, but its visibility decreases as more and more results aredeveloped for handling it in various situations. Fourier series are introduced in this chapter and arecarried along throughout the book as a motivating example for a number of problems in real analysis.Section 1 makes contact with the core of a first undergraduate course in real-variable theory.

Some material from such a course is repeated here in order to establish notation and a point of view.Omitted material is summarized at the end of the section, and some of it is discussed in a little moredetail in an appendix at the end of the book. The point of view being established is the use of definingproperties of the real number system to prove the Bolzano–Weierstrass Theorem, followed by theuse of that theorem to prove some of the difficult theorems that are usually assumed in a one-variablecalculus course. The treatment makes use of the extended real-number system, in order to allow supand inf to be defined for any nonempty set of reals and to allow lim sup and lim inf to be meaningfulfor any sequence.Sections 2–3 introduce the problem of interchange of limits. They show how certain concrete

problems can be viewed in this way, and they give a way of thinking about all such interchanges ina common framework. A positive result affirms such an interchange under suitable hypotheses ofmonotonicity. This is by way of introduction to the topic in Section 3 of uniform convergence andthe role of uniform convergence in continuity and differentiation.Section 4 gives a careful development of the Riemann integral for real-valued functions of one

variable, establishing existence of Riemann integrals for bounded functions that are discontinuousat only finitely many points, basic properties of the integral, the Fundamental Theorem of Calculusfor continuous integrands, the change-of-variables formula, and other results. Section 5 examinescomplex-valued functions, pointing out the extent to which the results for real-valued functions inthe first four sections extend to complex-valued functions.Section 6 is a short treatment of the version of Taylor’s Theorem in which the remainder is given

by an integral. Section 7 takes up power series and uses them to define the elementary transcendentalfunctions and establish their properties. The power series expansion of (1+x)p for arbitrary complexp is studied carefully. Section 8 introduces Cesàro and Abel summability, which play a role in thesubject of Fourier series. A converse theorem to Abel’s theorem is used to exhibit the function |x | asthe uniform limit of polynomials on [−1, 1]. The Weierstrass Approximation Theorem of Section 9generalizes this example and establishes that every continuous complex-valued function on a closedbounded interval is the uniform limit of polynomials.Section 10 introduces Fourier series in one variable in the context of the Riemann integral. The

main theorems of the section are a convergence result for continuously differentiable functions,Bessel’s inequality, the Riemann–Lebesgue Lemma, Fejér’s Theorem, and Parseval’s Theorem.

1

2 I. Theory of Calculus in One Real Variable

1. Review of Real Numbers, Sequences, Continuity

This section reviews some material that is normally in an undergraduate coursein real analysis. The emphasis will be on a rigorous proof of the Bolzano–Weierstrass Theorem and its use to prove some of the difficult theorems that areusually assumed in a one-variable calculus course. We shall skip over some easieraspects of an undergraduate course in real analysis that fit logically at the end ofthis section. A list of such topics appears at the end of the section.The system of real numbersRmay be constructed out of the system of rational

numbersQ, and we take this construction as known. The formal definition is thata real number is a cut of rational numbers, i.e., a subset of rational numbers thatis neither Q nor the empty set, has no largest element, and contains all rationalnumbers less than any rational that it contains. The idea of the construction isas follows: Each rational number q determines a cut q∗, namely the set of allrationals less than q. Under the identification of Q with a subset of R, the cutdefining a real number consists of all rational numbers less than the given realnumber.The set of cuts gets a natural ordering, given by inclusion. In place of ⊆, we

write ≤. For any two cuts r and s, we have r ≤ s or s ≤ r , and if both occur,then r = s. We can then define in the expected way. The positivecuts r are those with 0∗ < r , and the negative cuts are those with r < 0∗.Once cuts and their ordering are in place, one can go about defining the usual

operations of arithmetic and proving that R with these operations satisfies thefamiliar associative, commutative, and distributive laws, and that these interactwith inequalities in the usual ways. The definitions of addition and subtractionare easy: the sum or difference of two cuts is simply the set of sums or differencesof the rationals from the respective cuts. For multiplication and reciprocals onehas to take signs into account. For example, the product of two positive cutsconsists of all products of positive rationals from the two cuts, as well as 0 and allnegative rationals. After these definitions and the proofs of the usual arithmeticoperations are complete, it is customary to write 0 and 1 in place of 0∗ and 1∗.An upper bound for a nonempty subset E of R is a real number M such that

x ≤ M for all x in E . If the nonempty set E has an upper bound, we can take thecuts that E consists of and form their union. This turns out to be a cut, it is anupper bound for E , and it is ≤ all upper bounds for E . We can summarize thisresult as a theorem.

Theorem 1.1. Any nonempty subset E of R with an upper bound has a leastupper bound.

The least upper bound is necessarily unique, and the notation for it is supx∈E xor sup {x | x ∈ E}, “sup” being an abbreviation for the Latin word “supremum,”

1. Review of Real Numbers, Sequences, Continuity 3

the largest. Of course, the least upper bound for a set E with an upper boundneed not be in E ; for example, the supremum of the negative rationals is 0, whichis not negative.A lower bound for a nonempty set E ofR is a real numberm such that x ≥ m

for all x ∈ E . If m is a lower bound for E , then−m is an upper bound for the set−E of negatives of members of E . Thus −E has an upper bound, and Theorem1.1 shows that it has a least upper bound supx∈−E x . Then−x is a greatest lowerbound for E . This greatest lower bound is denoted by infy∈E y or inf {y | y ∈ E},“inf” being an abbreviation for “infimum.” We can summarize as follows.

Corollary 1.2. Any nonempty subset E ofRwith a lower bound has a greatestlower bound.

A subset ofR is said to be bounded if it has an upper bound and a lower bound.Let us introduce notation and terminology for intervals of R, first treating thebounded ones.1 Let a and b be real numbers with a ≤ b. The open intervalfrom a to b is the set (a, b) = {x ∈ R | a < x < b}, the closed interval isthe set [a, b] = {x ∈ R | a ≤ x ≤ b}, and the half-open intervals are the sets[a, b) = {x ∈ R | a ≤ x < b} and (a, b] = {x ∈ R | a < x ≤ b}. Each of theabove intervals is indeed bounded, having a as a lower bound and b as an upperbound. These intervals are nonempty when a < b or when the interval is [a, b]with a = b, and in these cases the least upper bound is b and the greatest lowerbound is a.Open sets in R are defined to be arbitrary unions of open bounded intervals,

and a closed set is any set whose complement inR is open. A set E is open if andonly if for each x ∈ E , there is an open interval (a, b) such that x ∈ (a, b) ⊆ E .In this case we of course have a < x < b. If we put � = min{x − a, b − x},then we see that x lies in the subset (x − �, x + �) of (a, b). The open interval(x − �, x + �) equals {y ∈ R ∣∣ |y − x | < �}. Thus an open set in R is any set Esuch that for each x ∈ E , there is a number � > 0 such that {y ∈ R ∣∣ |y− x | < �}lies in E . A limit point x of a subset F of R is a point of R such that anyopen interval containing x meets F in a point other than x . For example, the set[a, b) ∪ {b+ 1} has [a, b] as its set of limit points. A subset of R is closed if andonly if it contains all its limit points.Now let us turn to unbounded intervals. To provide notation for these, we shall

make use of two symbols+∞ and−∞ thatwill shortly be defined to be “extendedreal numbers.” If a is in R, then the subsets (a, +∞) = {x ∈ R | a < x},(−∞, a) = {x ∈ R | x < a}, (−∞, +∞) = R, [a, +∞) = {x ∈ R | a ≤ x},and (−∞, a] = {x ∈ R | x ≤ a} are defined to be intervals, and they are allunbounded. The first three are open sets of R and are considered to be open

1Bounded intervals are called “finite intervals” by some authors.

4 I. Theory of Calculus in One Real Variable

intervals, while the last three are closed sets and are considered to be closedintervals. Specifically the middle set R is both open and closed.One important consequence of Theorem 1.1 is the archimedean property of

R, as follows.

Corollary 1.3. If a and b are real numbers with a > 0, then there exists aninteger n with na > b.

PROOF. If, on the contrary, na ≤ b for all integers n, then b is an upper boundfor the set of allna. LetM be the least upper boundof the set {na | n is an integer}.Using that a is positive, we find that a−1M is a least upper bound for the integers.Thus n ≤ a−1M for all integers n, and there is no smaller upper bound. However,the smaller number a−1M − 1 must be an upper bound, since saying n ≤ a−1Mfor all integers is the same as saying n−1 ≤ a−1M−1 for all integers. We arriveat a contradiction, and we conclude that there is some integer n with na > b.

The archimedean property enables one to see, for example, that any twodistinct real numbers have a rational number lying between them. We provethis consequence as Corollary 1.5 after isolating one step as Corollary 1.4.

Corollary 1.4. If c is a real number, then there exists an integer n such thatn ≤ c < n + 1.PROOF. Corollary 1.3 with a = 1 and b = c shows that there is an integer M

with M > c, and Corollary 1.3 with a = 1 and b = −c shows that there is aninteger m with m > −c. Then −m < c < M , and it follows that there exists agreatest integer n with n ≤ c. This n must have the property that c < n + 1, andthe corollary follows.

Corollary 1.5. If x and y are real numbers with x < y, then there exists arational number r with x < r < y.

PROOF. By Corollary 1.3 with a = y − x and b = 1, there is an integer Nsuch that N (y − x) > 1. This integer N has to be positive. Then 1N < y − x .By Corollary 1.4 with c = Nx , there exists an integer n with n ≤ Nx < n + 1,hence with nN ≤ x < n+1N . Adding the inequalities nN ≤ x and 1N < y − x yieldsn+1N < y. Thus x ≤ nN < n+1N < y. Since nN < 2n+12N < n+1N , the rational numberr = 2n+12N has the required properties.

A sequence in a set S is a function from a certain kind of subset of integers intoS. It will be assumed that the set of integers is nonempty, consists of consecutiveintegers, and contains no largest integer. In particular the domain of any sequenceis infinite. Usually the set of integers is either all nonnegative integers or all

1. Review of Real Numbers, Sequences, Continuity 5

positive integers. Sometimes the set of integers is all integers, and the sequencein this case is often called “doubly infinite.” The value of a sequence f at theinteger n is normally written fn rather than f (n), and the sequence itself may bedenoted by an expression like { fn}n≥1, in which the outer subscript indicates thedomain.A subsequence of a sequence f with domain {m,m+1, . . . } is a composition

f ◦ n, where f is a sequence and n is a sequence in the domain of f such thatnk < nk+1 for all k. For example, if {an}n≥1 is a sequence, then {a2k}k≥1 is thesubsequence in which the function n is given by nk = 2k. The domain of asubsequence, by our definition, is always infinite.A sequence an in R is convergent, or convergent in R, if there exists a real

number a such that for each � > 0, there is an integer N with |an − a| < �for all n ≥ N . The number a is necessarily unique and is called the limitof the sequence. Depending on how much information about the sequence isunambiguous, we may write limn→∞ an = a or limn an = a or lim an = a oran → a. We also say an tends to a as n tends to infinity or∞.A sequence in R is called monotone increasing if an ≤ an+1 for all n in the

domain, monotone decreasing if an ≥ an+1 for all n in the domain, monotoneif it is monotone increasing or monotone decreasing.

Corollary 1.6. Any bounded monotone sequence in R converges. If thesequence is monotone increasing, then the limit is the least upper bound of theimage in R of the sequence. If the sequence is monotone decreasing, the limit isthe greatest lower bound of the image.

REMARK. Often it is Corollary 1.6, rather than the existence of least upperbounds, that is taken for granted in an elementary calculus course. The reasonis that the statement of Corollary 1.6 tends for calculus students to be easier tounderstand than the statement of the least upper bound property. Problem 1 at theend of the chapter asks for a derivation of the least-upper-bound property fromCorollary 1.6.

PROOF. Suppose that {an} is monotone increasing and bounded. Let a =supn an , the existence of the supremum being ensured by Theorem 1.1, and let� > 0 be given. If there were no integer N with aN > a− �, then a− � would bea smaller upper bound, contradiction. Thus such an N exists. For that N , n ≥ Nimplies a − � < aN ≤ an ≤ a < a + �. Thus n ≥ N implies |an − a| < �.Since � is arbitrary, limn→∞ an = a. If the given sequence {an} is monotonedecreasing, we argue similarly with a = infn an .

In working with sup and inf, it will be quite convenient to use the notationsupx∈E x evenwhen E is nonempty but not bounded above, and to use the notation

6 I. Theory of Calculus in One Real Variable

infx∈E x evenwhen E is nonempty but not bounded below. We introduce symbols+∞ and−∞, plus and minus infinity, for this purpose and extend the definitionsof supx∈E x and infx∈E x to all nonempty subsets E of R by taking

supx∈E

x = +∞ if E has no upper bound,

infx∈E

x = −∞ if E has no lower bound.

To work effectively with these new pieces of notation, we shall enlarge R to aset R∗ called the extended real numbers by defining

R∗ = R ∪ {+∞} ∪ {−∞}.

An ordering onR∗ is defined by taking−∞ < r < +∞ for every member r ofRand by retaining the usual ordering within R. It is immediate from this definitionthat

infx∈E

x ≤ supx∈E

x

if E is any nonempty subset ofR. In fact, we can enlarge the definitions of infx∈E xand supx∈E x in obvious fashion to include the case that E is any nonemptysubset of R∗, and we still have inf ≤ sup. With the ordering in place, we canunambiguously speak of open intervals (a, b), closed intervals [a, b], and half-open intervals [a, b) and (a, b] in R∗ even if a or b is infinite. Under ourdefinitions the intervals of R are the intervals of R∗ that are subsets of R, even ifa or b is infinite. If no special mention is made whether an interval lies in R orR∗, it is usually assumed to lie in R.The next step is to extend the operations of arithmetic to R∗. It is important

not to try to make such operations be everywhere defined, lest the distributivelaws fail. Letting r denote any member of R and a and b be any members of R∗,we make the following new definitions:

Multiplication: r(+∞) = (+∞)r =⎧⎨⎩

+∞ if r > 0,0 if r = 0,−∞ if r < 0,

r(−∞) = (−∞)r =⎧⎨⎩

−∞ if r > 0,0 if r = 0,+∞ if r < 0,

(+∞)(+∞) = (−∞)(−∞) = +∞,(+∞)(−∞) = (−∞)(+∞) = −∞.

1. Review of Real Numbers, Sequences, Continuity 7

Addition: r + (+∞) = (+∞) + r = +∞,r + (−∞) = (−∞) + r = −∞,

(+∞) + (+∞) = +∞,(−∞) + (−∞) = −∞.

Subtraction: a − b = a + (−b) whenever the right side is defined.Division: a/b = 0 if a ∈ R and b is ±∞,

a/b = b−1a if b ∈ R with b �= 0 and a is ±∞.The only surprise in the list is that 0 times anything is 0. This definition will beimportant to us when we get to measure theory, starting in Chapter V.It is now a simple matter to define convergence of a sequence inR∗. The cases

that need addressing are that the sequence is inR and that the limit is+∞ or−∞.We say that a sequence {an} inR tends to+∞ if for any positive number M , thereexists an integer N such that an ≥ M for all n ≥ N . The sequence tends to −∞if for any negative number −M , there exists an integer N such that an ≤ −Mfor all n ≥ N . It is important to indicate whether convergence/divergence of asequence is being discussed inR or inR∗. The default setting isR, in keepingwithstandard terminology in calculus. Thus, for example, we say that the sequence{n}n≥1 diverges, but it converges in R∗ (to +∞).With our new definitions every monotone sequence converges in R∗.For a sequence {an} inR or even inR∗, we now introduce members lim supn an

and lim infn an of R∗. These will always be defined, and thus we can apply theoperations lim sup and lim inf to any sequence in R∗. For the case of lim supwe define bn = supk≥n ak as a sequence in R∗. The sequence {bn} is monotonedecreasing. Thus it converges to infn bn in R∗. We define2

lim supn

an = infnsupk≥n

ak

as a member of R∗, and we define

lim infn

an = supninfk≥n

ak

as a member of R∗. Let us underscore that lim sup an and lim inf an always exist.However, one or both may be ±∞ even if an is in R for every n.Proposition 1.7. The operations lim sup and lim inf on sequences {an} and

{bn} in R∗ have the following properties:(a) if an ≤ bn for all n, then lim sup an ≤ lim sup bn and lim inf an ≤

lim inf bn ,

2The notation lim was at one time used for lim sup, and lim was used for lim inf.

8 I. Theory of Calculus in One Real Variable

(b) lim inf an ≤ lim sup an ,(c) {an} has a subsequence converging in R∗ to lim sup an and another sub-

sequence converging in R∗ to lim inf an ,(d) lim sup an is the supremum of all subsequential limits of {an} in R∗, and

lim infn is the infimum of all subsequential limits of {an} in R∗,(e) if lim sup an < +∞, then lim sup an is the infimum of all extended real

numbers a such that an ≥ a for only finitely many n, and if lim inf an >−∞, then lim inf an is the supremum of all extended real numbers a suchthat an ≤ a for only finitely many n,

(f) the sequence {an} in R∗ converges in R∗ if and only if lim inf an =lim sup an , and in this case the limit is the common value of lim inf an andlim sup an .

REMARK. It is enough to prove the results about lim sup, since lim inf an =− lim sup(−an).PROOFS FOR lim sup.(a) From al ≤ bl for all l, we have al ≤ supk≥n bk if l ≥ n. Hence supl≥n al ≤

supk≥n bk . Then (a) follows by taking the limit on n.(b) This follows by taking the limit on n of the inequality infk≥n ak ≤ supk≥n ak .(c) We divide matters into cases. The main case is that a = lim sup an is in R.

Inductively, for each l ≥ 1, choose N ≥ nl−1 such that | supk≥N ak − a| < l−1.Then choose nl > nl−1 such that |anl − supk≥N ak | < l−1. Together theseinequalities imply |anl − a| < 2l−1 for all l, and thus liml→∞ anl = a. Thesecond case is that a = lim sup an equals +∞. Since supk≥n ak is monotonedecreasing in n, we must have supk≥n ak = +∞ for all n. Inductively for l ≥ 1,we can choose nl > nl−1 such that anl ≥ l. Then liml→∞ anl = +∞. Thethird case is that a = lim sup an equals −∞. The sequence bn = supk≥n ak ismonotone decreasing to −∞. Inductively for l ≥ 1, choose nl > nl−1 such thatbnl ≤ −l. Then anl ≤ bnl ≤ −l, and liml→∞ anl = −∞.(d) By (c), lim sup an is one subsequential limit. Let a = limk→∞ ank be an-

other subsequential limit. Put bn = supl≥n al . Then {bn} converges to lim sup anin R∗, and the same thing is true of every subsequence. Since ank ≤ supl≥nk al =bnk for all k, we can let k tend to infinity and obtain a = limk→∞ ank ≤limk→∞ bnk = lim sup an .(e) Since lim sup an < +∞, we have supk≥n ak < +∞ for n greater than or

equal to some N . For this N and any a > supk≥N ak , we then have an ≥ a onlyfinitely often. Thus there exists a ∈ R such that an ≥ a for only finitely many n.On the other hand, if a′ is a real number< lim sup an , then (c) shows that an ≥ a′for infinitely many n. Hence

lim sup an ≤ inf {a | an ≥ a for only finitely many a}.

1. Review of Real Numbers, Sequences, Continuity 9

Arguing by contradiction, suppose that< holds in this inequality, and let a′′ be areal number strictly in between the two sides of the inequality. Then supk≥n ak <a′′ for n large enough, and so an ≥ a′′ only finitely often. But then a′′ is in the set

{a | an ≥ a for only finitely many a},

and the statement that a′′ is less than the infimum of this set gives a contradiction.(f) If {an} converges inR∗, then (c) forces lim inf an = lim sup an . Conversely

suppose lim inf an = lim sup an , and let a be the common value of lim inf an andlim sup an . The main case is that a is inR. Let � > 0 be given. By (e), an ≥ a+�only finitely often, and an ≤ a − � only finitely often. Thus |an − a| < � forall n sufficiently large. In other words, lim an = a as asserted. The other casesare that a = +∞ or a = −∞, and they are completely analogous to each other.Suppose for definiteness that a = +∞. Since lim inf an = +∞, the monotoneincreasing sequence bn = infk≥n ak converges in R∗ to +∞. Given M , chooseN such that bn ≥ M for n ≥ N . Then also an ≥ M for n ≥ N , and an convergesin R∗ to +∞. This completes the proof.

With Proposition 1.7 as a tool, we can nowprove theBolzano–Weierstrass The-orem. The remainder of the section will consist of applications of this theorem,showing that Cauchy sequences in R converge in R, that continuous functionson closed bounded intervals of R are uniformly continuous, that continuousfunctions on closed bounded intervals are bounded and assume their maximumand minimum values, and that continuous functions on closed intervals take onall intermediate values.

Theorem 1.8 (Bolzano–Weierstrass). Every bounded sequence in R has aconvergent subsequence with limit in R.

PROOF. If the given bounded sequence is {an}, form the subsequence noted inProposition 1.7c that converges in R∗ to lim sup an . All quantities arising in theformation of lim sup an are in R, since {an} is bounded, and thus the limit is in R.

A sequence {an} in R is called a Cauchy sequence if for any � > 0, thereexists an N such that |an − am | < � for all n and m that are ≥ N .

EXAMPLE. Every convergent sequence in R with limit in R is Cauchy. In fact,let a = lim an , and let � > 0 be given. Choose N such that n ≥ N implies|an − a| < �. Then n,m ≥ N implies

|an − am | ≤ |an − a| + |a − am | < � + � = 2�.

Hence the sequence is Cauchy.

10 I. Theory of Calculus in One Real Variable

In the above example and elsewhere in this book, we allow ourselves the luxuryof having our final bound come out as a fixed multiple M� of �, rather than �itself. Strictly speaking, we should have introduced �′ = �/M and aimed for�′ rather than �. Then our final bound would have been M�′ = �. Since thetechnique for adjusting a proof in this way is always the same, we shall not addthese extra steps in the future unless there would otherwise be a possibility ofconfusion.This convention suggests a handy piece of terminology—that a proof as in the

above example, in which M = 2, is a “2� proof.” That name conveys a great dealof information about the proof, saying that one should expect two contributionsto the final estimate and that the final bound will be 2�.

Theorem 1.9 (Cauchy criterion). Every Cauchy sequence in R converges to alimit in R.

PROOF. Let {an} be Cauchy in R. First let us see that {an} is bounded. Infact, for � = 1, choose N such that n,m ≥ N implies |an − am | < 1. Then|am | ≤ |aN | + 1 for m ≥ N , and M = max{|a1|, . . . , |aN−1|, |aN | + 1} is acommon bound for all |an|.Since {an} is bounded, it has a convergent subsequence {ank }, say with limit

a, by the Bolzano–Weierstrass Theorem. The subsequential limit has to satisfy|a| ≤ M within R∗, and thus a is in R.Finally let us see that lim an = a. In fact, if � > 0 is given, choose N such

that nk ≥ N implies |ank − a| < �. Also, choose N ′ ≥ N such that n,m ≥ N ′implies |an − am | < �. If n ≥ N ′, then any nk ≥ N ′ has |an − ank | < �, andhence

|an − a| ≤ |an − ank | + |ank − a| < � + � = 2�.This completes the proof.

Let f be a function with domain an interval and with range in R. The intervalis allowed to be unbounded, but it is required to be a subset of R. We saythat f is continuous at a point x0 of the domain of f within R if for each� > 0, there is some δ > 0 such that all x in the domain of f that satisfy|x − x0| < δ have | f (x) − f (x0)| < �. This notion is sometimes abbreviated aslimx→x0 f (x) = f (x0). Alternatively, one may say that f (x) tends to f (x0) asx tends to x0, and one may write f (x) → f (x0) as x → x0.Amathematically equivalent definition is that f is continuous at x0 if whenever

a sequence has xn → x0 within the domain interval, then f (xn) → f (x0). Thislatter version of continuity will be shown in Section II.4 to be equivalent to theformer version, given in terms of continuous limits, in greater generality than justfor R, and thus we shall not stop to prove the equivalence now. We say that f iscontinuous if it is continuous at all points of its domain.

1. Review of Real Numbers, Sequences, Continuity 11

We say that the a function f as above is uniformly continuous on its domainif for any � > 0, there is some δ > 0 such that | f (x) − f (x0)| < � whenever xand x0 are in the domain interval and |x− x0| < δ. (In other words, the conditionfor the continuity to be uniform is that δ can always be chosen independently ofx0.)

EXAMPLE. The function f (x) = x2 is continuous on (−∞, +∞), but it isnot uniformly continuous. In fact, it is not uniformly continuous on [1, +∞).Assuming the contrary, choose δ for � = 1. Thenwemust have ∣∣(x+ δ2 )2−x2∣∣ < 1for all x ≥ 1. But ∣∣(x + δ2 )2 − x2∣∣ = δx + δ24 ≥ δx , and this is ≥ 1 for x ≥ δ−1.Theorem 1.10. A continuous function f from a closed bounded interval [a, b]

into R is uniformly continuous.

PROOF. Fix � > 0. For x0 in the domain of f , the continuity of f at x0 meansthat it makes sense to define

δx0(�) = min{1, sup

{δ′ > 0

∣∣∣∣ |x − x0| < δ′ and x in the domainof f imply | f (x) − f (x0)| < �}}

.

If |x − x0| < δx0(�), then | f (x) − f (x0)| < �. Put δ(�) = infx0∈[a,b] δx0(�).Let us see that it is enough to prove that δ(�) > 0. If x and y are in [a, b] with|x − y| < δ(�), then |x − y| < δ(�) ≤ δy(�). Hence | f (x) − f (y)| < � asrequired.Thus we are to prove that δ(�) > 0. If δ(�) = 0, then, for each integer

n > 0, we can choose xn such that δxn (�) <1n . By the Bolzano–Weierstrass

Theorem, there is a convergent subsequence, say with xnk → x ′. Along thissubsequence, δxnk (�) → 0. Fix k large enough so that |xnk − x ′| < 12δx ′( �2 ). Then| f (xnk ) − f (x ′)| < �2 . Also, |x − xnk | < 12δx ′( �2 ) implies

|x − x ′| ≤ |x − xnk | + |xnk − x ′| < 12δx ′( �2 ) + 12δx ′( �2 ) = δx ′( �2 ),so that | f (x) − f (x ′)| < �2 and

| f (xnk ) − f (x)| ≤ | f (xnk ) − f (x ′)| + | f (x ′) − f (x)| < �2 + �2 = �.Consequently our arbitrary large fixed k has δxnk ≥ 12δx ′( �2 ), and the sequence{δxnk (�)} cannot be tending to 0.Theorem 1.11. A continuous function f from a closed bounded interval [a, b]

into R is bounded and takes on maximum and minimum values.

PROOF. Let c = supx∈[a,b] f (x) in R∗. Choose a sequence xn in [a, b]with f (xn) increasing to c. By the Bolzano–Weierstrass Theorem, {xn} has aconvergent subsequence, say xnk → x ′. By continuity, f (xnk ) → f (x ′). Thenf (x ′) = c, and c is a finite maximum. The proof for a finite minimum is similar.

12 I. Theory of Calculus in One Real Variable

Theorem 1.12 (Intermediate Value Theorem). Let a < b be real numbers,and let f : [a, b] → R be continuous. Then f , in the interval [a, b], takes on allvalues between f (a) and f (b).

REMARK. The proof below, which uses the Bolzano–Weierstrass Theorem,does not make absolutely clear what aspects of the structure of R are essential tothe argument. A conceptually clearer proof will be given in Section II.8 and willbring out that the essential property of the interval [a, b] is its “connectedness”in a sense to be defined in that section.

PROOF. Let f (a) = α and f (b) = β, and let γ be between α and β. We mayassume that γ is in fact strictly between α and β. Possibly by replacing f by− f , we may assume that also α < β. Let

A = {x ∈ [a, b] | f (x) ≤ γ } and B = {x ∈ [a, b] | f (x) ≥ γ }.These sets are nonempty, since a is in A and b is in B, and f is bounded asa result of Theorem 1.11. Thus the numbers γ1 = sup { f (x) | x ∈ A} andγ2 = inf { f (x) | x ∈ B} are well defined and have γ1 ≤ γ ≤ γ2.If γ1 = γ , thenwe can find a sequence {xn} in A such that f (xn) converges to γ .

Using the Bolzano–Weierstrass Theorem, we can find a convergent subsequence{xnk } of {xn}, say with limit x0. By continuity of f , { f (xnk )} converges to f (x0).Then f (x0) = γ1 = γ , and we are done. Arguing by contradiction, we maytherefore assume that γ1 < γ . Similarly we may assume that γ < γ2, but we donot need to do so.Let � = γ2 − γ1, and choose, by Theorem 1.10 and uniform continuity, δ > 0

such that |x1 − x2| < δ implies | f (x1) − f (x2)| < � whenever x1 and x2 bothlie in [a, b]. Then choose an integer n such that 2−n(b − a) < δ, and considerthe value of f at the points pk = a + k2−n(b − a) for 0 ≤ k ≤ 2−n . Sincepk+1 − pk = 2−n(b − a) < δ, we have | f (pk+1) − f (pk)| < � = γ2 − γ1.Consequently if f (pk) ≤ γ1, then

f (pk+1) ≤ f (pk) + | f (pk+1) − f (pk)| < γ1 + (γ2 − γ1) = γ2,and hence f (pk+1) ≤ γ1. Now f (p0) = f (a) = α ≤ γ1. Thus induction showsthat f (pk) ≤ γ1 for all k ≤ 2−n . However, for k = 2n , we have p2−n = b, andf (b) = β ≥ γ > γ1, and we have arrived at a contradiction.Further topics. Here a number of other topics of an undergraduate course in real-variable theory

fit well logically. Among these are countable vs. uncountable sets, infinite series and tests for theirconvergence, the fact that every rearrangement of an infinite series of positive terms has the samesum, special sequences, derivatives, the Mean Value Theorem as in Section A2 of the appendix,and continuity and differentiability of inverse functions as in Section A3 of the appendix. We shallnot stop here to review these topics, which are treated in many books. One such book is Rudin’sPrinciples of Mathematical Analysis, the relevant chapters being 1 to 5. In Chapter 2 of that book,only the first few pages are needed; they are the ones where countable and uncountable sets arediscussed.

2. Interchange of Limits 13

2. Interchange of Limits

Let {bi j } be a doubly indexed sequence of real numbers. It is natural to ask forthe extent to which

limilimjbi j = lim

jlimibi j ,

more specifically to ask how to tell, in an expression involving iterated limits,whether we can interchange the order of the two limit operations. We can viewmatters conveniently in terms of an infinite matrix⎛⎝ b11 b12 · · ·b21 b22

.... . .

⎞⎠ .The left-hand iterated limit, namely limi limj bi j , is obtained by forming the limitof each row, assembling the results, and then taking the limit of the row limitsdown through the rows. The right-hand iterated limit, namely limj limi bi j , isobtained by forming the limit of each column, assembling the results, and thentaking the limit of the column limits through the columns. If we use the particularinfinite matrix ⎛⎜⎜⎜⎜⎝

1 1 1 1 · · ·0 1 1 1 · · ·0 0 1 1 · · ·0 0 0 1 · · ·...

. . .

⎞⎟⎟⎟⎟⎠ ,thenwe see that the first iterated limit depends only on the part of thematrix abovethe main diagonal, while the second iterated limit depends only on the part of thematrix below the main diagonal. Thus the two iterated limits in general have noreason at all to be related. In the specific matrix that we have just considered,they are 1 and 0, respectively. Let us consider some examples along the samelines but with an analytic flavor.

EXAMPLES.

(1) Let bi j = ji + j . Then limi limj bi j = 1, while limj limi bi j = 0.

(2) Let Fn be a continuous real-valued function onR, and suppose that F(x) =lim Fn(x) exists for every x . Is F continuous? This is the same kind of question.

It asks whether limt→x F(t)?= F(x), hence whether

limt→x limn→∞ Fn(t)

?= limn→∞ limt→x Fn(t).

14 I. Theory of Calculus in One Real Variable

If we take fk(x) = x2

(1+ x2)k for k ≥ 0 and define Fn(x) =∑n

k=0 fk(x), then

each Fn is continuous. The sequence of functions {Fn} has a pointwise limitF(x) =

∑∞k=0

x2

(1+ x2)k . The series is a geometric series, and we can easilycalculate explicitly the partial sums and the limit function. The latter is

F(x) ={0 if x = 01+ x2 if x �= 0.

It is apparent that the limit function is discontinuous.

(3) Let { fn} be a sequence of differentiable functions, and suppose that f (x) =lim fn(x) exists for every x and is differentiable. Is lim f ′n(x) = f ′(x)? Thisquestion comes down to whether

limn→∞ limt→x

fn(t) − fn(x)t − x

?= limt→x limn→∞

fn(t) − fn(x)t − x .

An example where the answer is negative uses the sine and cosine functions,which are undefined in the rigorous development until Section 7 on power series.

The example has fn(x) = sin nx√n

for n ≥ 1. Then limn fn(x) = 0, so thatf (x) = 0 and f ′(x) = 0. Also, f ′n(x) =

√n cos nx , so that f ′n(0) =

√n does

not tend to 0 = f ′(0).Yet we know many examples from calculus where an interchange of limits is

valid. For example, in calculus of two variables, the first partial derivatives ofnice functions—polynomials, for example—can be computed in either order withthe same result, and double integrals of continuous functions over a rectangle canbe calculated as iterated integrals in either order with the same result. Positivetheorems about interchanging limits are usually based on some kind of uniformbehavior, in a sense that we take up in the next section. A number of positiveresults of this kind ultimately come down to the following general theorem aboutdoubly indexed sequences that are monotone increasing in each variable. InSection 3 we shall examine the mechanism of this theorem closely: the proofshows that the equality in question is supi supj bi j = supj supi bi j and that itholds because both sides equal supi, j bi j .

Theorem 1.13. Let bi j be members ofR∗ that are≥ 0 for all i and j . Supposethat bi j is monotone increasing in i , for each j , and is monotone increasing in j ,for each i . Then

limilimjbi j = lim

jlimibi j ,

with all the indicated limits existing in R∗.

3. Uniform Convergence 15

PROOF. Put Li = limj bi j and L ′j = limi bi j . These limits exist in R∗, sincethe sequences in question are monotone. Then Li ≤ Li+1 and L ′j ≤ L ′j+1, andthus

L = limiLi and L

′ = limjL ′j

both exist in R∗. Arguing by contradiction, suppose that L < L ′. Then we canchoose j0 such that L ′j0 > L . Since L

′j0

= limi bi j0 , we can choose i0 such thatbi0 j0 > L . Then we have L < bi0 j0 ≤ Li0 ≤ L , contradiction. Similarly theassumption L ′ < L leads to a contradiction. We conclude that L = L ′.

Corollary 1.14. If al j are members of R∗ that are ≥ 0 and are monotoneincreasing in j for each l, then

limj

∑l

al j =∑l

limjal j

in R∗, the limits existing.

REMARK. This result will be generalized by the Monotone ConvergenceTheorem when we study abstract measure theory in Chapter V.

PROOF. Put bi j =∑i

l=1 al j in Theorem 1.13.

Corollary 1.15. If ci j are members of R∗ that are ≥ 0 for all i and j , then∑i

∑j

ci j =∑j

∑i

ci j

in R∗, the limits existing.

REMARK. This result will be generalized by Fubini’s Theorem when we studyabstract measure theory in Chapter V.

PROOF. This follows from Corollary 1.14.

3. Uniform Convergence

Let us examine more closely what is happening in the proof of Theorem 1.13, inwhich it is proved that iterated limits can be interchanged under certain hypothesesof monotonicity. One of the iterated limits is L = limi limj bi j , and the claim isthat L is approached as i and j tend to infinity jointly. In terms of a matrix whose

16 I. Theory of Calculus in One Real Variable

entries are the various bi j ’s, the pictorial assertion is that all the terms far downand to the right are close to L:⎛⎜⎜⎜⎝ · · · · · ·

· · · All terms hereare close to L

⎞⎟⎟⎟⎠ .To see this claim, let us choose a row limit Li0 that is close to L and then take anentry bi0 j0 that is close to Li0 . Then bi0 j0 is close to L , and all terms down and tothe right from there are even closer because of the hypothesis of monotonicity.To relate this behavior to something uniform, suppose that L < +∞, and let

some � > 0 be given. We have just seen that we can arrange to have |L−bi j | < �whenever i ≥ i0 and j ≥ j0. Then |Li − bi j | < � whenever i ≥ i0, providedj ≥ j0. Also, we have limj bi j = Li for i = 1, 2, . . . , i0−1. Thus |Li −bi j | < �for all i , provided j ≥ j ′0, where j ′0 is some larger index than j0. This is thenotion of uniform convergence that we shall define precisely in a moment: anexpression with a parameter ( j in our case) has a limit (on the variable i in ourcase) with an estimate independent of the parameter. We can visualize matters asin the following matrix:

j j ′0

i

(· · ·

All terms hereare close to Lion all rows.

).

The vertical dividing line occurs when the column index j is equal to j ′0, and allterms to the right of this line are close to their respective row limits Li .Let us see the effect of this situation on the problem of interchange of limits.

The above diagram forces all the terms in the shaded part of

( · · · · · ·· · · //////

)to

be close to one number if lim Li exists, i.e., if the row limits are tending to alimit. If the other iterated limit exists, then it must be this same number. Thusthe interchange of limits is valid under these circumstances.Actually, we can get by with less. If, in the displayed diagram above, we

assume that all the column limits L ′j exist, then it appears that all the columnlimits with j ≥ j ′0 have to be close to the Li ’s. From this we can deduce that thecolumn limits have a limit L ′ and that the row limits Li must tend to the limitof the column limits. In other words, the convergence of the rows in a suitableuniform fashion and the convergence of the columns together imply that both

3. Uniform Convergence 17

iterated limits exist and they are equal. We shall state this result rigorously asProposition 1.16, which will become a prototype for applications later in thissection.Let S be a nonempty set, and let f and fn , for integers n ≥ 1, be functions

from S to R. We say that fn(x) converges to f (x) uniformly for x in S if forany � > 0, there is an integer N such that n ≥ N implies | fn(x) − f (x)| < � forall x in S. It is equivalent to say that supx∈E | fn(x) − f (x)| tends to 0 as n tendsto infinity.

Proposition 1.16. Let bi j be real numbers for i ≥ 1 and j ≥ 1. Suppose that(i) Li = limj bi j exists in R uniformly in i , and(ii) L ′j = limi bi j exists in R for each j .

Then

(a) L = limi Li exists in R,(b) L ′ = limj L ′j exists in R,(c) L = L ′,(d) the double limit on i and j of bi j exists and equals the common value of

the iterated limits L and L ′, i.e., for each � > 0, there exist i0 and j0 suchthat |bi j − L| < � whenever i ≥ i0 and j ≥ j0,

(e) L ′j = limi bi j exists in R uniformly in j .REMARK. In applications we shall sometimes have additional information,

typically the validity of (a) or (b). According to the statement of the proposition,however, the conclusions are valid without taking this extra information as anadditional hypothesis.

PROOF. Let � > 0 be given. By (i), choose j0 such that |bi j − Li | < � for alli whenever j ≥ j0. With j ≥ j0 fixed, (ii) says that |bi j − L ′j | < � whenever i is≥ some i0 = i0( j). For j ≥ j0 and i ≥ i0( j), we then have

|Li − L ′j | ≤ |Li − bi j | + |bi j − L ′j | < � + � = 2�.If j ′ ≥ j0 and i ≥ i0( j ′), we similarly have |Li − L ′j ′ | < 2�. Hence if j ≥ j0,j ′ ≥ j0, and i ≥ max{i0( j), i0( j ′)}, then

|L ′j − L ′j ′ | ≤ |L ′j − Li | + |Li − L ′j ′ | < 2� + 2� = 4�.In other words, {L ′j } is a Cauchy sequence. By Theorem 1.9, L ′ = limj L ′j existsin R. This proves (b).Passing to the limit in our inequality, we have |L ′j − L ′| ≤ 4� when j ≥ j0

and in particular when j = j0. If i ≥ i0( j0), then we saw that |bi j0 − Li | < �and |bi j0 − L ′j0 | < �. Hence i ≥ i0( j0) implies

|Li − L ′| ≤ |Li − bi j0 | + |bi j0 − L ′j0 | + |L ′j0 − L ′| < � + � + 4� = 6�.

18 I. Theory of Calculus in One Real Variable

Since � is arbitrary, L = limi Li exists and equals L ′. This proves (a) and (c).Since limi Li = L , choose i1 such that |Li −L| < � whenever i ≥ i1. If i ≥ i1

and j ≥ j0, we then have

|bi j − L| ≤ |bi j − Li | + |Li − L| < � + � = 2�.

This proves (d).Let i1 and j0 be as in the previous paragraph. We have seen that |L ′j−L ′j ′ | < 4�

for j ≥ j0. By (b), |L ′j −L ′| ≤ 4� whenever j ≥ j0. Hence (c) and the inequalityof the previous paragraph give

|bi j − L ′j | ≤ |bi j − L| + |L − L ′| + |L ′ − L ′j | < 2� + 0+ 4� = 6�

whenever i ≥ i1 and j ≥ j0. By (b), choose j1 ≥ j0 such that |bi j − L ′j | < 6�whenever i ∈ {1, . . . , i1−1} and j ≥ j1. Then j ≥ j1 implies |bi j − L ′j | < 6�for all i whenever j ≥ j1. This proves (e).

In checking for uniform convergence, we often do not have access to explicitexpressions for limiting values. One device for dealing with the problem is auniform version of the Cauchy criterion. Let S be a nonempty set, and let { fn}n≥1be a sequence of functions from S toR. We say that { fn(x)} is uniformlyCauchyfor x ∈ S if for any � > 0, there is an integer N such that n ≥ N and m ≥ Ntogether imply | fn(x) − fm(x)| < � for all x in S.

Proposition 1.17 (uniform Cauchy criterion). A sequence { fn} of functionsfrom a nonempty set S to R is uniformly Cauchy if and only if it is uniformlyconvergent.

PROOF. If { fn} is uniformly convergent to f , we use a 2� argument, just asin the example before Theorem 1.9: Given � > 0, choose N such that n ≥ Nimplies | fn(x) − f (x)| < �. Then n ≥ N and m ≥ N together imply

| fn(x) − fm(x)| ≤ | fn(x) − f (x)| + | f (x) − fm(x)| < � + � = 2�.

Thus { fn} is uniformly Cauchy.Conversely suppose that { fn} is uniformly Cauchy. Then { fn(x)} is Cauchy for

each x . Theorem 1.9 therefore shows that there exists a function f : S → R suchthat limn fn(x) = f (x) for each x . We prove that the convergence is uniform.Given � > 0, choose N , as is possible since { fn} is uniformly Cauchy, such thatn ≥ N and m ≥ N together imply | fn(x) − fm(x)| < �. Letting m tend to ∞shows that | fn(x) − f (x)| ≤ � for n ≥ N . Hence limn fn(x) = f (x) uniformlyfor x in S.

3. Uniform Convergence 19

In practice, uniform convergence often arises with infinite series of functions,and then the definition and results about uniform convergence are to be applied tothe sequence of partial sums. If the series is

∑∞k=1 ak(x), onewants

∣∣∑nk=m ak(x)

∣∣to be small for all m and n sufficiently large. Some of the standard tests forconvergence of series of numbers yield tests for uniform convergence of series offunctions just by introducing a parameter and ensuring that the estimates do notdepend on the parameter. We give two clear-cut examples. One is the uniformalternating series test or Leibniz test, given in Corollary 1.18. A generalizationis the handy test given in Corollary 1.19.

Corollary 1.18. If for each x in a nonempty set S, {an(x)}n≥1 is a mono-tone decreasing sequence of nonnegative real numbers such that limn an(x) = 0uniformly in x , then

∑∞n=1 (−1)nan(x) converges uniformly.

PROOF. The hypotheses are such that∣∣∑n

k=m (−1)kak(x)∣∣ ≤ supx |am(x)|

whenever n ≥ m, and the uniform convergence is immediate from the uniformCauchy criterion.

Corollary 1.19. If for each x in a nonempty set S, {an(x)}n≥1 is a monotonedecreasing sequence of nonnegative real numbers such that limn an(x) = 0uniformly in x and if {bn(x)}n≥1 is a sequence of real-valued functions on Swhose partial sums Bn(x) =

∑nk=1 bk(x) have |Bn(x)| ≤ M for some M and all

n and x , then∑∞

n=1 an(x)bn(x) converges uniformly.

PROOF. If n ≥ m, summation by parts givesn∑

k=mak(x)bk(x) =

n−1∑k=m

Bk(x)(ak(x) − ak+1(x)) + Bn(x)an(x) − Bm−1(x)am(x).

Let � > 0 be given, and choose N such that ak(x) ≤ � for all x whenever k ≥ N .If n ≥ m ≥ N , then

∣∣∣ n∑k=m

ak(x)bk(x)∣∣∣ ≤ n−1∑

k=m|Bk(x)|(ak(x) − ak+1(x)) + M� + M�

≤ Mn−1∑k=m

(ak(x) − ak+1(x)) + 2M�

≤ Mam(x) + 2M�≤ 3M�,

and the uniform convergence is immediate from the uniform Cauchy criterion.

20 I. Theory of Calculus in One Real Variable

A third consequence can be considered as a uniform version of the result thatabsolute convergence implies convergence. In practice it tends to be fairly easyto apply, but it applies only in the simplest situations.

Proposition 1.20 (Weierstrass M test). Let S be a nonempty set, and let { fn}be a sequence of real-valued functions on S such that | fn(x)| ≤ Mn for all x inS. Suppose that

∑n Mn < +∞. Then

∑∞n=1 fn(x) converges uniformly for x in

S.

PROOF. If n ≥ m ≥ N , then ∣∣∑nk=m+1 fk(x)∣∣ ≤ ∑nk=m | fk(x)| ≤ ∑nk=m Mk ,and the right side tends to 0 uniformly in x as N tends to infinity. Therefore theresult follows from the uniform Cauchy criterion.

EXAMPLES.

(1) The series∞∑n=1

1

n2xn

converges uniformly for−1 ≤ x ≤ 1 by theWeierstrass M test with Mn = 1/n2.(2) The series

∞∑n=1

(−1)n x2 + nn2

converges uniformly for −1 ≤ x ≤ 1, but the M test does not apply. To seethat the M test does not apply, we use the smallest possible Mn , which is Mn =supx

∣∣(−1)n x2+nn2 | = n+1n2 . The series ∑ n+1n2 diverges, and hence the M testcannot apply for any choice of the numbers Mn . To see the uniform convergenceof the given series, we observe that the terms strictly alternate in sign. Also,

x2 + nn2

≥ x2 + (n + 1)(n + 1)2 because

x2

n2≥ x

2

(n + 1)2 and1

n≥ 1n + 1 .

Finallyx2 + nn2

≤ n + 1n2

→ 0uniformly for−1 ≤ x ≤ 1. Hence the series converges uniformly by the uniformLeibniz test (Corollary 1.18).

Having developed some tools for proving uniform convergence, let us applythe notion of uniform convergence to interchanges of limits involving functionsof a real variable. For a point of reference, recall the diagrams of interchanges oflimits at the beginning of the section. We take the column index to be n and think

3. Uniform Convergence 21

of the row index as a variable t , which is tending to x . We make assumptionsthat correspond to (i) and (ii) in Proposition 1.16, namely that { fn(t)} convergesuniformly in t as n tends to infinity, say to f (t), and that fn(t) converges to somelimit fn(x) as t tends to x . With fn(x) defined as this limit, fn is continuousat x . In other words, the assumptions are that the sequence { fn} is uniformlyconvergent to f and each fn is continuous.

Theorem 1.21. If { fn} is a sequence of real-valued functions on [a, b] that arecontinuous at x and if { fn} converges to f uniformly, then f is continuous at x .REMARKS. This is really a consequence of Proposition 1.16 except that one of

the indices, namely t , is regarded as continuous and not discrete. Actually, there isa subtle simplification here, by comparison with Proposition 1.16, in that { fn(x)}at the limiting parameter x is being assumed to tend to f (x). This correspondsto assuming (b) in the proposition, as well as (i) and (ii). Consequently the proofof the theorem will be considerably simpler than the proof of Proposition 1.16.In fact, the proof will be our first example of a 3� proof. In many applicationsof Theorem 1.21, the given sequence { fn} is continuous at every x , and then theconclusion is that f is continuous at every x .

PROOF. We write

| f (t) − f (x)| ≤ | f (t) − fn(t)| + | fn(t) − fn(x)| + | fn(x) − f (x)|.Given � > 0, choose N large enough so that | fn(t)− f (t)| < � for all t whenevern ≥ N . With such an n fixed, choose some δ of continuity for the functionfn , the point x , and the number �. Each term above is then < �, and hence| f (t) − f (x)| < 3�. Since � is arbitrary, f is continuous at x .Theorem 1.21 in effect uses only conclusion (c) of Proposition 1.16, which

concerns the equality of the two iterated limits. Conclusion (d) gives a strongerresult, namely that the double limit exists and equals each iterated limit. Thestrengthened version of Theorem 1.21 is as follows.

Theorem 1.21′. If { fn} is a sequence of real-valued functions on [a, b] thatare continuous at x and if { fn} converges to f uniformly, then for each � > 0,there exist an integer N and a number δ > 0 such that

| fn(t) − f (x)| < �whenever n ≥ N and |t − x | < δ.PROOF. If � > 0 is given, choose N such that | fn(t) − f (t)| < �/2 for all

t whenever n ≥ N , and choose δ in the conclusion of Theorem 1.21 such that|t − x | < δ implies | f (t) − f (x)| < �/2. Then

| fn(t) − f (x)| ≤ | fn(t) − f (t)| + | f (t) − f (x)| < �2 + �2 = �whenever n ≥ N and |t − x | < δ. Theorem 1.21′ follows.

22 I. Theory of Calculus in One Real Variable

In interpreting our diagrams of interchanges of limits to get at the statement ofTheorem 1.21, we took the column index to be n and thought of the row index asa variable t , which was tending to x . It is instructive to see what happens whenthe roles of n and t are reversed, i.e., when the row index is n and the columnindex is the variable t , which is tending to x . Again we have fn(t) convergingto f (t) and limt→x fn(t) = fn(x), but the uniformity is different. This timewe want the uniformity to be in n as t tends to x . This means that the δ ofcontinuity that corresponds to � can be taken independent of n. This is the notionof “equicontinuity,” and there is a classical theorem about it. The theorem isactually stronger than Proposition 1.16 suggests, since the theorem assumes lessthan that fn(t) converges to f (t) for all t .Let F = { fα | α ∈ A} be a set of real-valued functions on a bounded interval

[a, b]. We say that F is equicontinuous at x ∈ [a, b] if for each � > 0, there issome δ > 0 such that |t−x | < δ implies | f (t)− f (x)| < � for all f ∈ F. The setF of functions is pointwise bounded if for each t ∈ [a, b], there exists a numberMt such that | f (t)| ≤ Mt for all f ∈ F. The set is uniformly equicontinuous on[a, b] if it is equicontinuous at each point x and if the δ can be taken independentof x . The set is uniformly bounded on [a, b] if it is pointwise bounded at eacht ∈ [a, b] and the bound Mt can be taken independent of t .

Theorem 1.22 (Ascoli’s Theorem). If { fn} is a sequence of real-valued func-tions on a closed bounded interval [a, b] that is equicontinuous at each point of[a, b] and pointwise bounded on [a, b], then

(a) { fn} is uniformly equicontinuous and uniformly bounded on [a, b],(b) { fn} has a uniformly convergent subsequence.PROOF. Since each fn is continuous at each point, we know from Theorems

1.10 and 1.11 that each fn is uniformly continuous and bounded. The proof of(a) amounts to an argument that the estimates in those theorems can be arrangedto apply simultaneously for all n.First consider the question of uniformboundedness. Choose, byTheorem1.11,

some xn in [a, b] with | fn(xn)| equal to Kn = supx∈[a,b] | fn(x)|. Then choose asubsequence on which the numbers Kn tend to supn Kn in R

∗. There will be noloss of generality in assuming that this subsequence is our whole sequence. Applythe Bolzano–Weierstrass Theorem to find a convergent subsequence {xnk } of {xn},say with limit x0. By pointwise boundedness, find Mx0 with | fn(x0)| ≤ Mx0 forall n. Then choose some δ of equicontinuity at x0 for � = 1. As soon as k is largeenough so that |xnk − x0| < δ, we have

Knk = | fnk (xnk )| ≤ | fnk (xnk ) − fnk (x0)| + | fnk (x0)| < 1+ Mx0 .Thus 1+ Mx0 is a uniform bound for the functions fn .

3. Uniform Convergence 23

The proof of uniform equicontinuity proceeds in the same spirit but takeslonger to write out. Fix � > 0. The uniform continuity of fn for each n meansthat it makes sense to define

δn(�) = min{1, sup

{δ′ > 0

∣∣∣∣ | f (x)− f (y)| < � whenever |x− y| < δ′and x and y are in the domain of f}}

.

If |x − y| < δn(�), then | fn(x) − fn(y)| < �. Put δ(�) = infn δn(�). Let us seethat it is enough to prove that δ(�) > 0: If x and y are in [a, b] with |x−y| < δ(�),then |x − y| < δ(�) ≤ δn(�). Hence | fn(x) − fn(y)| < � as required.Thus we are to prove that δ(�) > 0. If δ(�) = 0, then we first choose an

increasing sequence {nk} of positive integers such that δnk (�) < 1k , and we nextchoose xk and yk in [a, b] with |xk − yk | < δnk (�) and | fk(xk) − fk(yk)| ≥ �.Applying the Bolzano–Weierstrass Theorem, we obtain a subsequence {xkl } of{xk} such that {xkl } converges, say to x0. Then

lim supl

|ykl − x0| ≤ lim supl

|ykl − xkl | + lim supl

|xkl − x0| = 0+ 0 = 0,

so that {ykl } converges to x0. Now choose, by equicontinuity at x0, a numberδ′ > 0 such that | fn(x) − fn(x0)| < �2 for all n whenever |x − x0| < δ′. Theconvergence of {xkl } and {ykl } to x0 implies that for large enough l, we have|xkl − x0| < δ′/2 and |ykl − x0| < δ′/2. Therefore | fkl (xkl ) − fkl (x0)| < �2 and| fkl (ykl ) − fkl (x0)| < �2 , from which we conclude that | fkl (xkl ) − fkl (ykl )| < �.But we saw that | fk(xk) − fk(yk)| ≥ � for all k, and thus we have arrived at acontradiction. This proves the uniform equicontinuity and completes the proofof (a).To prove (b), we first construct a subsequence of { fn} that is convergent at

every rational point in [a, b]. We enumerate the rationals, say as x1, x2, . . . . Bythe Bolzano–Weierstrass Theorem and the pointwise boundedness, we can finda subsequence of { fn} that is convergent at x1, a subsequence of the result thatis convergent at x2, a subsequence of the result that is convergent at x3, and soon. The trouble with this process is that each term of our original sequence maydisappear at some stage, and then we are left with no terms that address all therationals. The trick is to form the subsequence { fnk } of the given { fn} whosek th term is the k th term of the k th subsequence we constructed. Then the k th,(k + 1)st, (k + 2)nd, . . . terms of { fnk } all lie in our k th constructed subsequence,and hence { fnk } converges at the first k points x1, . . . , xk . Since k is arbitrary,{ fnk } converges at every rational point.Let us