Online Notes for Advanced Calculus II …klassen/AdvancedCalcII/Online...Eric Klassen, Florida State...

Online Notes for Advanced Calculus II(MAA4227/MAA5307)

Eric Klassen, Florida State University

Spring, 2020

Eric Klassen, Florida State University Online Notes for Advanced Calculus II (MAA4227/MAA5307)


These lecture notes are based on Rudin’sPrinciples of Mathematical Analysis, 2nd Edition (McGraw Hill)


Can two different power series define the same function?

(Starting on page 177 of Rudin...)

Suppose∑

anxn and

∑bnx

n both converge to the same functionf (x) on the interval (−R,R).

Question: Is it possible that an 6= bn for some n?

Answer: No!

Why? Because by Theorem 8.1, (∀n ≥ 0)

an =f (n)(0)

n!= bn,

So, if two power series are equal on some interval, they must haveidentical coefficients!


The following theorem shows that we can conclude that two powerseries must have identical coefficients under a much weakerhypothesis than agreeing on a whole interval!

Theorem (8.5)

Suppose the series∑

anxn and

∑bnx

n converge for allx ∈ (−R,R). Let

E =x ∈ (−R,R) :

∑anx

n =∑

bnxn.

If E has a limit point in (−R,R), then (∀n ≥ 0), an = bn;i.e. the two power series are identical.


Proof of Theorem 8.5

For all n ≥ 0, let cn = an − bn, and set

f (x) =∑

cnxn =

∑anx

n −∑

bnxn.

Note that E = f −1(0) ∩ (−R,R). Since f is continuous, E is arelatively closed subset of (−R,R). Let

A = x ∈ (−R,R) : x is a limit point of EB = (−R,R)− A

Note that A ⊆ E because E is closed; also, A is a closed subset of(−R,R), since the set of limit points of a set is always closed byExercise 6 in Ch. 2. Therefore, B is open.

We will now show that A is also open!


Claim: A is open

Proof of Claim:Let x0 ∈ A. By Theorem 8.4, we can expand f as a power seriesabout x0:

f (x) =∞∑n=0

dn(x − x0)n.

We will now prove that dn = 0 for all n = 0, 1, 2, 3, . . . : Supposethis is not true; let k be the smallest non-negative integer suchthat dk 6= 0. Then factoring out (x − x0)k gives

f (x) = (x − x0)kg(x)

where

g(x) =∞∑

m=0

dk+m(x − x0)m.


Since g(x0) = dk 6= 0 and g is continuous, it follows that ∃δ > 0such that g(x) 6= 0 for all x ∈ Nδ(x0).

Hence, f (x) = (x − x0)kg(x) 6= 0 for all x ∈ Nδ(x0)− x0. Hencex0 is not a limit point of E . But this contradicts the fact thatx0 ∈ A! This contradiction proves that dn = 0 for alln = 0, 1, 2, 3, . . .

If follows that f (x) = 0 for all x such that |x − x0| < R − |x0|,proving that A contains a neighborhood of x0, which implies thatA is open since x0 ∈ A was arbitrary.

Thus(−R,R) = A ∪ B

expresses (−R,R) as the disjoint union of two open sets!


Since (−R,R) is connected, we conclude that A or B must beempty. But A 6= ∅, so we conclude that B = ∅, implying that(−R,R) = A. We showed above that

A ⊆ E ⊆ (−R,R),

so we now know that

(−R,R) ⊆ E ⊆ (−R,R).

Of course it follows that E = (−R,R). But we observed abovethat if two power series are equal on a whole interval, theircoefficients must all be equal, proving Theorem 8.5.


Remark (illustrated on next slide)

For Theorem 8.5 to apply, the limit point of E must be in(−R,R). If the limit point is R or −R, the theorem is notnecessarily true. For example, let

f (x) = sin(x/(1− x2))

g(x) = 0

These functions can both be expressed as convergent power serieson (−1, 1). The set of points at which they are equal has both 1and -1 as limit points. However, the two functions are obviouslynot equal to each other on (−1, 1).


f (x) = sin(x/(1− x2)) and g(x) = 0 for x ∈ (−1, 1)


Exponential and Logarithmic Functions

Define

E (z) =∞∑0

zn

n!(0! = 1)

Ratio Test =⇒ E (z) converges for all z ∈ C. Hence we have awell defined function

E : C→ C.

Theorem

For all z ,w ∈ C, E (z)E (w) = E (z + w).


Proof

E (z)E (w) =∞∑n=0

zn

n!

∞∑m=0

wm

m!=∞∑n=0

∞∑m=0

zn

n!

wm

m!

=∞∑n=0

n∑k=0

zkwn−k

k!(n − k)!=∞∑n=0

n∑k=0

1

n!

n!zkwn−k

k!(n − k)!

=∞∑n=0

1

n!

n∑k=0

(n

k

)zkwn−k =

∞∑n=0

1

n!(z + w)n = E (z + w).

The third equal sign is obtained by grouping together sets ofmonomials with constant total degree. The change in order ofsummation is justified by Theorem 3.50, since the series areabsolutely convergent.


Since, for all z ∈ C,

E (z)E (−z) = E (0) = 1,

we conclude that for all z ∈ C,

E (z) 6= 0 and E (−z) = 1/E (z).

x ≥ 0 =⇒ E (x) = 1 + x1! + x2

2! + · · · > 0.Since E (−x) = 1/E (x), it follows that

(∀x ∈ R),E (x) > 0.

Since for all x > 0, E (x) > 1 + x , we conclude that

limx→∞

E (x) =∞.

Also,

limx→−∞

E (x) = limx→∞

E (−x) = limx→∞

1

E (x)= 0.


Monotonicity of E (x) on R

By definition of E (z), 0 ≤ x < y =⇒ E (x) < E (y). UsingE (x) = 1/E (−x), one easily deduces:

E is strictly increasing on the entire real axis.


Computing the derivative E ′(x)

Lemma

limh→0

E (h)− 1

h= 1.

Proof.

limh→0

E (h)− 1

h= lim

h→0

(1 + h1! + h2

2! + . . . )− 1

h

= limh→0

(1

1!+

h

2!+

h2

3!+ . . .

)= 1.


Theorem

For all x ∈ R, E ′(x) = E (x)

Proof.

E ′(x) = limh→0

E (x + h)− E (x)

h= lim

h→0

E (x)E (h)− E (x)

h

= limh→0

E (x)

(E (h)− 1

h

)= E (x).


Verifying that E (n) = en for n ∈ J

From E (z + w) = E (z)E (w), it is easy to deduce usingmathematical induction that for all z1, . . . , zn ∈ C,

E (z1 + z2 + · · ·+ zn) = E (z1)E (z2) . . .E (zn).

In Chapter 3, we defined e = 1 + 11! + 2

2! + . . . so obviouslyE (1) = e. Hence, for all n ∈ J,

E (n) = E (1 + · · ·+ 1) = E (1) . . .E (1) = en.


Verifying the E (p) = ep for rational numbers p

Suppose p = n/m, (n,m ∈ J). Then

E (p)m = E (p) . . .E (p) = E (mp) = E (n) = en.

Since E (p) is a positive real number whose m-th power = en, itfollows that

E (p) = en/m = ep

by the definition of rational exponents given in Ex. 6 of Chapter 1.If p ∈ Q is negative, then

E (p) =1

E (−p)=

1

e−p= ep.

So we have shown that for all p ∈ Q,

E (p) = ep.


Since Q is dense in R, E (x) is the unique continuous functionR→ R that agrees with ex on Q. So for now on, we use thenotation ez or exp(z) for the function E (z).

The following theorem gathers together several importantproperties of the function ex : R→ R.


Theorem on real exponentials

Theorem (8.6)

1 ex is continuous and differentiable on R2 (ex)′ = ex

3 ex is strictly increasing and ex > 0 for all x ∈ R4 ex+y = exey for all x , y ∈ R5 lim

x→∞ex =∞; lim

x→−∞ex = 0

6 limx→∞

xne−x = 0 for all n ∈ J.



We have already proved (1)-(5). For a quick proof of (6), notethat the definition of ex implies that for x > 0,

ex >xn+1

(n + 1)!

Inverting both sides, reversing the inequality, and multiplying by xn

yields

0 < xne−x <(n + 1)!

x.

Letting x →∞, (6) then follows by the squeeze theorem.


Logarithm Function

Let R+ = x ∈ R : x > 0.

Because ex is continuous and limx→∞

ex =∞ and limx→−∞

ex = 0, it

follows that exp : R→ R+ is surjective.

Because ex is strictly increasing, exp : R→ R+ is injective; henceexp : R→ R+ is bijective. Therefore exp has an inverse

L : R+ → R.


Some facts about Logarithms

Lemma

L is strictly increasing.

Proof.

We need to prove that for all x , y ∈ R+, x < y =⇒ L(x) < L(y).Proceeding by contrapositive, we will instead prove thatL(x) ≥ L(y) =⇒ x ≥ y . So, assume that L(x) ≥ L(y). There aretwo cases.

Case 1. Suppose L(x) > L(y). Because exp is increasing, we knowexp(L(x)) > exp(L(y)), so x > y .

Case 2. Suppose L(x) = L(y). Then exp(L(x)) = exp(L(y)), sox = y , establishing the contrapositive.


Continuity of L

Lemma

L is continuous.

Proof.

Fix x ∈ R. Note that

exp : [x − 1, x + 1]→ [exp(x − 1), exp(x + 1)]

is continuous and bijective.Since [x − 1, x + 1] is compact, Theorem 4.17 implies that

L = exp−1 : [exp(x − 1), exp(x + 1)]→ [x − 1, x + 1]

is also continous. Since x was arbitrary, L is continous on R+.


Derivative of L

Lemma

L : R+ → R is differentiable and L′(x) = 1/x .


Proof.

Let x ∈ R. Compute

L′(x) = limy→x

L(y)− L(x)

y − x=

1

limy→x

(y−x

L(y)−L(x)

)=

1

limy→x

(E(L(y))−E(L(x))

L(y)−L(x)

) =1

limz→L(x)

(E(z)−E(L(x))

z−L(x)

)=

1

E ′(L(x))=

1

E (L(x))=

1

x


Properties of L

Since E (0) = 1, L(1) = 0. It follows that for all x ∈ R+,

L(x) =

∫ x

1

1

tdt.

In some treatments of this subject, this formula is taken as thedefinition of log, and exp is defined to be log−1!

Claim:

For all u, v ∈ R+, L(uv) = L(u) + L(v).


Proof.

L(uv) = L(E (L(u))E (L(v))) = L(E (L(u) + L(v))) = L(u) + L(v).

Properties we’ve already proved of E easily imply that

limx→∞

L(x) =∞ and limx→0+

L(x) = −∞.


More properties of L

It’s easy to show that for all p ∈ Q, and for all x ∈ R+,

pL(x) = L(xp)

and, henceE (pL(x)) = xp.

Motivated by this, the expression “xp” is often defined for allx ∈ R+ and for all p ∈ R by the formula

xp = E (pL(x))(= ep log x)

Recall that a different definition was given in Ex. 6 of Chapter 1,but this one is actually easier to use.


Properties of real exponents

Using this definition of xp, it is easy to derive laws of exponents forreal exponents, for example:

xα+β = xαxβ and (xα)β = xαβ

(xy)α = xαyα

d

dx(xα) = αxα−1 for all x ∈ R+, α ∈ R.

Henceforth, we write

ex = E (x) and log(x) = L(x)


Claim:

limx→∞

x−α log x = 0 for all α > 0.

Proof.

L’Hopital’s rule:

limx→∞

x−α log x = limx→∞

log x

xα= lim

x→∞

1/x

αxα−1

= limx→∞

1

αxα= 0


Definition of cos and sin

Definition

For all z ∈ C, define

C (z) =1

2(e iz + e−iz)

S(z) =1

2i(e iz − e−iz)


By definition of ez , ez = ez . Hence if x ∈ R,

C (x) =1

2(e ix + e−ix)

=1

2(e−ix + e ix) = C (x).

Similarly for S(x). This shows

x ∈ R =⇒ C (x) ∈ R and S(x) ∈ R.

From the definitions, for all z ∈ C,

e iz = C (z) + iS(z).


It follows that if x ∈ R, then

C (x) = Re(e ix) and S(x) = Im(e ix).

Also, if x ∈ R,

|e ix |2 = e ixe ix = e ixe−ix = 1.

Therefore, for all x ∈ R,|e ix | = 1

Hence for all x ∈ R,

C (x)2 + S(x)2 = 1.


The definitions of these functions imply that

C (0) = 1 and S(0) = 0.

Also, differentiating both sides of e ix = C (x) + iS(x) shows easilythat for all x ∈ R,

C ′(x) = −S(x) and S ′(x) = C (x).

Because C (x)2 + S(x)2 = 1, it follows that the parametrized curve

γ(t) = (C (t),S(t))

lies entirely on the unit circle. Note that

|γ′(t)| = |(−S(t),C (t))| =√

C (x)2 + S(x)2 = 1.


Hence the arclength of γ from γ(0) to γ(t) equals∫ t

0|γ′(s)|ds =

∫ t

01 ds = t.

Thus if we start at (1, 0) and move counterclockwise a distance oft along the unit circle,

x-coordinate = C (t)

y -coordinate = S(t)

This proves that according to the usual definitions of the trigfunctions,

C (t) = cos t and S(t) = sin t.


Definition of π

Rudin proves that there exists x ∈ R+ such that C (x) = 0. Let x0be the smallest such x .

Definition

π = 2x0.

C (π/2) = 0 =⇒ S(π/2) = ±1. Since, on [0, π/2],S ′(x) = C (x) ≥ 0, S(x) is increasing on [0, π/2], so S(π/2) = 1.It follows that

eπi/2 = C (π/2) + iS(π/2) = i .

Henceeπi = i2 = −1

ande2πi = (−1)2 = 1

.Eric Klassen, Florida State University Online Notes for Advanced Calculus II (MAA4227/MAA5307)

Periodicity of exponentials and trig functions

For all z ∈ C, n ∈ Z, ez+2πin = ez(e2πi )n = ez .

Theorem (8.7)

(a) For all z ∈ C, ez+2πi = ez

It then follows directly from the definitions that(b) For all x ∈ R,

S(x + 2π) = S(x)

C (x + 2π) = C (x)

(c) If 0 < t < 2π, then e it 6= 1(d) If z is a complex number with |z | = 1, there is a uniquet ∈ [0, 2π) such that e it = z .


Therefore, on the domain 0 ≤ t ≤ 2π,

γ(t) = e it

is a simple closed curve whose range is precisely the unit circle inC centered at 0.


Algebraic Completeness of C

Suppose P(z) is a polynomial with complex coefficients, andsuppose P(r) = 0 for some r ∈ C.

If we divide (z − r) into P(z) by polynomial long division, we get aquotient polynomial Q(z) and a remainder c ∈ C. Hence

P(z) = (z − r)Q(z) + c .

Substituting z = r in both sides⇒ c = 0. Hence, if P(r) = 0, then

P(z) = (z − r)Q(z)

for some polynomial Q(z).


Thus if we can prove that every polynomial P(z) of degree n has aroot, we get

P(z) = (z − r1)Q1(z) = (z − r1)(z − r2)Q2(z) = . . .

P(z) = c(z − r1)(z − r2) . . . (z − rn)

Thus it would follow that every polynomial P(z) over C factorscompletely into linear factors, and hence has at most n roots.


Theorem (8.8; Fundamental Theorem of Algebra)

Suppose a0, . . . , an ∈ C, n ≥ 1, and an 6= 0. Let

P(z) =n∑

k=0

akzk

Then there exists z ∈ C such that P(z) = 0.



Without loss of generality, assume an = 1.

Let µ = inf|P(z)| : z ∈ C. The strategy of our proof is asfollows:

1 We prove this inf is actually attained, i.e., that there existsz0 ∈ C such that |P(z0)| = µ.

2 We prove that µ = 0.

It then follows that P(z0) = 0, proving the theorem!


Proof that inf |P(z)| is attained

Suppose z ∈ C, |z | = R, where R > 0.

Then zn = P(z)− an−1zn−1 − · · · − a0

∴ |zn| ≤ |P(z)|+ |an−1||z |n−1 + · · ·+ |a0|Rn ≤ |P(z)|+ |an−1|Rn−1 + · · ·+ |a0|

∴ |P(z)| ≥ Rn − |an−1|Rn−1 − · · · − |a0||P(z)| ≥ Rn(1− |an−1|R−1 − · · · − |a0|R−n)

Clearly the RHS →∞ as R →∞

∴ ∃R0 ∈ R+ such that ∀R > R0, RHS > µ+ 1


Therefore, |z | > R0 =⇒ |P(z)| > µ+ 1

∀n ∈ J, choose zn ∈ C such that |P(zn)| < µ+ 1n .

Clearly, zn is a sequence in NR0(0).

Since NR0(0) is compact, zn has asubsequence yn that converges to apoint z0 ∈ NR0(0).


Since |P(z)| is continuous on NR0(0), it follows by Theorem 4.16that

|P(z0)| = |P(

limn→∞

yn)| = lim

n→∞|P(yn)| = µ;

the last equality follows by the squeeze theorem since for all n ∈ Jµ ≤ |P(yn)| ≤ µ+ 1

n .

Thus we have found z0 ∈ C such that |P(z0)| = µ.


Proof that µ = 0

Suppose µ = |P(z0)| 6= 0. Let Q(z) = P(z+z0)P(z0)

Q(z) is a non-constant polynomial, Q(0) = 1, and |Q(z)| ≥ 1(∀z ∈ C). Write

Q(z) = 1 + bkzk + · · ·+ bnz

n, where bk 6= 0

By Theorem 8.7 (4), ∃θ ∈ R such that

e ikθ = −|bk |bk

,

since − |bk |bkhas magnitude = 1, i.e.,

e ikθbk = −|bk |.


If r > 0, it follows that

|1 + bk rke ikθ| = |1− rk |bk ||

If we also assume rk < 1|bk | , then |1 + bk r

ke ikθ| = 1− rk |bk |By definition of Q,

|Q(re iθ)| ≤ |1 + bk rke ikθ|+ |bk+1|rk+1 + · · ·+ |bn|rn

∴ |Q(re iθ)| ≤ 1− rk |bk |+ |bk+1|rk+1 + · · ·+ |bn|rn

∴ |Q(re iθ)| ≤ 1− rk(|bk | − r |bk+1| − · · · − rn−k |bn|)

For sufficiently small r > 0, the last parenthesis is > 0.

Hence, for sufficient small r , |Q(re iθ)| < 1. Contradiction!Therefore µ = 0, so P(z0) = 0


“V is a vector space over C”

Completely analogous to vector spaces over R, except scalars ∈ C.

E.g. if α, β ∈ C, u, v ,w ∈ V

v + w = w + v

(u + v) + w = u + (v + w)

α(v + w) = αv + αw

(α + β)v = αv + βv

1 · v = v

0 · v = 0 ∈ V

Definitions of span, basis, linear independence, dimension all thesame as over R.


The most familiar example: Cn is a vector space over C ofdimension n.

Definition

A Hermitian inner product (HIP) on a vector space V over C is afunction 〈 , 〉 : V × V → C such that, assuming u, v ∈ V , λ ∈ C

1© 〈u, v〉 = 〈v , u〉 (conjugate − symmetric)

2© 〈λu, v〉 = λ〈u, v〉3© 〈u1 + u2, v〉 = 〈u1, v〉+ 〈u2, v〉

Notes : 1©+ 3© =⇒ 〈u, v1 + v2〉 = 〈u, v1〉+ 〈u, v2〉1©+ 2© =⇒ 〈u, λv〉 = λ〈u, v〉


1©⇒ (∀u ∈ V ), 〈u, u〉 ∈ R

since〈u, u〉 = 〈u, u〉

To be a HIP, we also require

4© 〈u, u〉 ≥ 0,∀u ∈ V , and

〈u, u〉 = 0 ⇐⇒ u = 0

The norm (or magnitude) is determined by

|u| =√〈u, u〉.


Standard example: V = Cn; if

v =

v1...vn

and w =

w1...wn

Define

〈v ,w〉 =n∑

i=1

viwi

Just like ordinary dot-product in Rn, except for taking complexconjugates of second factors!


Definitions

If v ,w ∈ V , v is orthogonal to w if 〈v ,w〉 = 0.(Note: 〈v ,w〉 = 0 ⇐⇒ 〈w , v〉 = 0)

A set v1, . . . , vk ⊆ V is orthonormal ⇐⇒ 〈vi , vj〉 =

0, i 6= j

1, i = j

Suppose v1, . . . , vn is an orthonormal set andu = α1v1 + · · ·+ αnvn. Then

〈u, u〉 = α1α1 + · · ·+ αnαn = |α1|2 + · · ·+ |αn|2

So |u| =√∑

|αi |2


Still assuming that v1, . . . , vn is orthonormal andu = α1v1 + · · ·+ αnvn,

〈u, vi 〉 = αi (∀i)

Hence, if we know u is a linear combination of the vi, we canrecover the coefficients by this formula.

Important exampleLet V = continuous functions [a, b]→ CThis is a complex vector space.

Define a Hermitian inner product on V by

〈f , g〉 =

∫ b

af (x)g(x)dx


Lemma

If V is a Hermitian C-vector space, v ,w ∈ V , and v⊥w , then

‖~v − ~w‖2 = ‖~v‖2 + ‖~w‖2, also ‖~v + ~w‖2 = ‖~v‖2 + ‖~w‖2

Proof.

‖~v − ~w‖2 = 〈v − w , v − w〉= 〈v , v〉 − 〈w , v〉 − 〈v ,w〉+ 〈w ,w〉= ‖v‖2 + ‖w‖2

[Note: if 〈v ,w〉 = 0, then 〈w , v〉 = 0]


Lemma

Suppose e1, . . . en ⊆ V are orthonormal and v ∈ V ; let

ci = 〈v , ei 〉 ∀i = 1, . . . , n.

Then (v −

n∑i=1

ciei

)⊥ ej ∀j = 1, . . . , n

Proof.

〈v −n∑

i=1

ciei , ej〉 = 〈v , ej〉 − cj〈ej , ej〉 = 0


Lemma

Let e1, . . . en ⊆ V be orthonormal and v ∈ V . Let ci = 〈v , ei 〉for all i = 1, . . . , n.

The element of spane1, . . . , en that is closest to v is∑

ciei , andno other element of spane1, . . . , en is equally close to v .

Proof.

Let∑

diei ∈ span e1, . . . , en.

|v −∑

diei |2 = |(v −∑

ciei ) +∑

(ci − di )ei |2

= |v −∑

ciei |2 + |∑

(ci − di )ei |2

= |v −∑

ciei |2 +∑|ci − di |2

Clearly, this is minimized when di = ci (∀i).



Definition

Suppose V is an HIP, e1, . . . , en is an orthonormal set, andv ∈ V . For each i = 1, . . . , n, let ci = 〈v , ei 〉. Then we call

n∑i=1

ciei

the projection of v into spane1, . . . , en. It has two importantproperties:

1 Of all vectors in spane1, . . . , en, it is closest to v .

2 The difference v −∑n

i=1 ciei is perpendicular to ei for alli = 1, . . . , n.


Fourier Series

Definition

A trigonometric polynomial is a finite sum of the form

f (x) = a0 +N∑

n=1

(an cos (nx) + bn sin (nx)) (?)

where x ∈ R and ai , bi ∈ C

Note: f : R→ C

By the definitions of cos x and sin x in terms of e ix , (?) can also bewritten

f (x) =N∑

n=−Ncne

inx (?’)


where

cn =1

2(an − ibn) and c−n =

1

2(an + ibn) for n = 1, . . . ,N

and c0 = a0. Clearly every trigonometric polynomial is periodicwith period 2π.

By the Fundamental Theorem of Calculus,

1

2π

∫ π

−πe inxdx =

1, n = 0

0, n 6= 0, n ∈ Z


Lemma1√2πe inx∞n=−∞

is an orthonormal set in the Hermitian vector

space of continuous functions [−π, π]→ C

Proof.

⟨1√2π

e inx ,1√2π

e imx

⟩=

∫ π

−π

1

2πe inxe−imxdx

=1

2π

∫ π

−πe i(n−m)xdx =

1, n = m

0, n 6= m


Lemma

If f (x) =∑N

n=−N cneinx then for all n = −N, . . . ,N,

cn =1

2π

∫ π

−πf (x)e−inxdx .

Proof. ∫ π

−πf (x)e−imxdx =

N∑n=−N

cn

∫ π

−πe i(n−m)xdx = 2πcm

∴ ∀m = −N, . . . ,N cm =1

2π

∫ π

−πf (x)e−imxdx


Lemma

If f (x) =∑N

n=−N cneinx , then

f (x) ∈ R (∀x) ⇐⇒ c−n = cn (∀n = 0, . . . ,N)

Proof.( =⇒ ) If f (x) is real, then

c−n =1

2π

∫ π

−πf (x)e inxdx =

1

2π

∫ π

−πf (x)e−inxdx = cn


(⇐=) If each c−n = cn, then

f (x) = c0 +N∑

n=1

cneinx + cne

−inx

It follows immediately that

f (x) = f (x) (∀x), so f (x) ∈ R

Continuing to assume that f : R→ C is defined by

f (x) =N∑

n=−Ncne

inx , (?)

clearly f is periodic on R, with period 2π.


Definition

A trigonometric series is a series of the form

∞∑n=−∞

cneinx x ∈ R, cn ∈ C

.

Its N-th partial sum is defined to be

N∑n=−N

cneinx


If f ∈ R (i.e., if f is Riemann integrable) on [−π, π], then for alln ∈ Z, we define

cn =1

2π

∫ π

−πf (x)e−inxdx

and then define∞∑

n=−∞cne

inx

to be the Fourier series of f (x).

(The numbers cn are called the Fourier coefficients of f .)


Main Question:

Under what circumstances (and in what sense) does the Fourierseries of f converge to f ?

We have already shown that if f is a finite trigonometricpolynomial, then it is equal to its Fourier series (since its Fourierseries will be the same trigonometric polynomial).

Definition

An orthogonal system of functions on [a, b] is a sequence φn offunctions [a, b]→ C such that,

∀n 6= m,

∫ b

aφn(x)φm(x)dx = 0


Definition

If, in addition,

(∀n)

∫ b

a|φn(x)|2dx = 1

then φn is said to be orthonormal.

We’ve verified that the functions

e inx√2π

n∈Z

form an orthonormal

system on [−π, π].

It’s easy to verify that another orthonormal system on [−π, π] is1√2π, cos x√

π, sin x√

π, cos 2x√

π, sin 2x√

π, . . .


Two metrics on function spaces

We now have two different ways of making functions [a, b]→ Cinto a metric space.

1 dist(f , g) = sup|f (x)− g(x)| : x ∈ [a, b].

2 dist(f , g) =(∫ b

a (f (x)− g(x))(f (x)− g(x))dx)1/2

.

The first is often referred to as the “sup-metric” while the secondis the “L2-metric”. Note that the L2-metric is simply themagnitude |f − g | with respect to the HIP we have defined on thecomplex vector space of functions.


Given any f ∈ R on [a, b], and any orthonormal system φn on[a, b], we define the n-th Fourier coefficient of f relative to φn by

cn =

∫ b

af (t)φn(t)dt

We writef (x) ∼

∑n

cnφn(x) (4)

and call this the Fourier series of f relative to φn.

Note: We are saying nothing about the convergence of theseries (4); just naming it the Fourier series of f .


Theorem (8.11)

Let φnn∈J be an orthonormal system on [a, b].

Let

sn(x) =n∑

m=1

cmφm(x)

be the n-th partial sum of the Fourier series of f, and let

tn(x) =n∑

m=1

γmφm(x)

be an arbitrary function in spanφ1, . . . , φn, where each γm ∈ C.


Theorem (8.11)

Then ∫ b

a|f − sn|2dx ≤

∫ b

a|f − tn|2dx

and equality holds ⇐⇒ γm = cm, ∀m = 1, . . . , n.

ProofMimics earlier proof for general HIP spaces.

First, note that ∫ b

a|tn|2dx =

n∑m=1

|γm|2.


∫|f − tn|2 =

∫(f − tn)(f − tn)

=

∫f f −

∫f tn −

∫f tn +

∫tntn

=

∫|f |2 −

∑cmγm −

∑cmγm +

∑γmγm

=

∫|f |2 −

∑cmcm +

∑cmcm

−∑

cmγm −∑

cmγm +∑

γmγm

=

∫|f |2 −

∑|cm|2 +

∑(cm − γm)(cm − γm)

=

∫|f |2 −

∑|cm|2 +

∑|cm − γm|2

Clearly, this achieves its minimum ⇐⇒ γm = cm for all m.


Setting γm = cm, tn = sn in last equation, shows∫|f − sn|2 =

∫|f |2 −

n∑m=1

|cm|2∫|f − sn|2 =

∫|f |2 −

∫|sn|2

∴∫|sn|2 ≤

∫|f |2 (?)

(Projection onto subspace spanned by φ1, . . . , φn is smaller thanoriginal vector).


Theorem (8.12)

If φn is an orthonormal system on [a, b],and if

f (x) ∼∞∑n=1

cnφn(x)

then∞∑n=1

|cn|2 ≤∫ b

a|f (x)|2dx

In particular,limn→∞

cn = 0


Proof.

Equation (?) in the last proof shows

n∑m=1

|cm|2 ≤∫|f |2 for all n.

Since n∑

m=1

|cm|2∞

n=1

is a monotone increasing sequence that is bounded above by∫|f |2, it converges to a number ≤

∫|f |2.

The fact that∑|cn|2 converges =⇒ limn→∞ cn = 0.


We now assume that f : R→ C has period 2π and f ∈ R on[−π, π] (hence on every bounded interval). Recall

Definition (Fourier series of f )

f (x) ∼∞∑

n=−∞cne

inx

where each

cn =1

2π

∫ π

−πf (x)e−inxdx

Let

sN(x) = sN(f ; x) =N∑−N

cneinx

be the N-th partial sum of the Fourier series.


We have proved that (8.12)

1

2π

∫ π

−π|sN(x)|2 dx =

N∑−N|cn|2 ≤

1

2π

∫ π

−π|f (x)|2 dx

Define the Dirichlet kernel by

DN(x) =N∑

n=−Ne inx

Note: (e ix − 1)DN(x) = e i(N+1)x − e−iNx .

Multiplying both sides by e−12x

(e12ix − e−

12ix)DN(x) = e i(N+ 1

2)x − e−i(N+ 1

2)x


=⇒ DN(x) =sin((N + 1

2

)x)

sin(12x)

Now, using t = x − u, u = x − t, du = −dt,

sN(x) =N∑

n=−N

1

2π

(∫ π

−πf (t)e−intdt

)e inx

=1

2π

∫ π

−πf (t)

(N∑

n=−Ne in(x−t)

)dt

=1

2π

∫ π

−πf (x)DN(x − t) dt

Now change variables from t to u:


sN(x) = − 1

2π

∫ x−π

x+πf (x − u)DN(u) du

=1

2π

∫ x+π

x−πf (x − t)DN(t) dt

∴ sN(x) =1

2π

∫ π

−πf (x − t)DN(t) dt

(Periodicity =⇒ the interval of integration doesn’t matter, aslong as its length is 2π.)


Finally, a theorem about pointwise convergence!

Theorem (8.14)

Assume f has period 2π and f ∈ R on [−π, π], and supposex ∈ R. If ∃ δ > 0 and M <∞ such that

|f (x + t)− f (x)| ≤ M|t| (∀t ∈ (−δ, δ))

thenlim

N→∞sN(x) = f (x)

In other words, the Fourier series of f converges to f at the pointx .


Clearly, by Mean Value Theorem, the hypothesis holds if f isdifferentiable on (x − δ, x + δ) and f ′ is bounded on (x − δ, x + δ).Can just take

M = supt∈(−δ,δ)

|f ′(x + t)|.

Thus Theorem 8.14 implies that if f is differentiable and f ′ isbounded on R,

sN(x)→ f (x) ∀x ∈ R

Proof of 8.14Fixing x ∈ R, define:

g(t) =

f (x−t)−f (x)

sin(t/2) t ∈ [−π, π]− 00 t = 0


An easy computation shows that 12π

∫ π−π DN(t) dt = 1.

Therefore

sN(x)− f (x) =1

2π

∫ π

−πf (x − t)DN(t) dt − 1

2πf (x)

∫ π

−πDN(t) dt

=1

2π

∫ π

−π(f (x − t)− f (x))DN(t) dt

=1

2π

∫ π

−π(f (x − t)− f (x))

sin((N + 1

2)t)

sin(12 t)dt

=1

2π

∫ π

−πg(t) sin

((N +

1

2)t

)dt

=1

2π

∫ π

−πg(t) cos

( t2

)sinNt dt +

1

2π

∫ π

−πg(t) sin

( t2

)cosNt dt


It is easy to show g(t) cos(t2

)and g(t) sin

(t2

)are bounded and

Riemann integrable (bounded follows from the hypothesis on f nearx ; integrable follows from 6.10, which says that if f is boundedand has finitely many discontinuities, then it is integrable).

By Theorem 8.12, both these terms → 0 as N →∞ because theyrepresent Fourier coefficients with respect to a certain orthonormalbasis (sines and cosines).

This proves sN(x)→ f (x) as N →∞.

Corollary : If f (x) = 0 for all x ∈ (a, b), then

sN(x)→ 0 ∀x ∈ (a, b).


Example:

f and g are both pointwise limits of Fourier series; they agree on(0, π/2), but not everywhere!


Important difference between power series and Fourierseries

Contrast:If two functions are analytic (i.e. sums of power series) and theyagree on an interval, they agree everywhere by Theorem 8.5.

If two functions are sums of Fourier series, they can agree on aninterval, but disagree elsewhere!

“By changing the Fourier coefficients, can change behavior of f inone interval, while not changing it in another interval.”


Theorem (8.15)

If f is continuous with period 2π and ε > 0, then ∃ atrigonometric polynomial P such that

|P(x)− f (x)| < ε ∀x ∈ R

Proof.

Think of periodic functions as functions S1 → C(S1 = unit circle in C) by

f (e ix) = f (x)

Then use Theorem 7.33.


Theorem (8.16 - Parseval’s Theorem)

Suppose f , g ∈ R on [−π, π], both have period 2π, and haveFourier series

f (x) ∼∞∑−∞

cneinx g(x) ∼

∞∑−∞

γneinx

Then:

1© limN→∞

1

2π

∫ π

−π|f (x)− sN(f ; x)|2 dx = 0

2© 1

2π

∫ π

−πf (x)g(x) dx =

∞∑−∞

cnγn

3© 1

2π

∫ π

−π|f (x)|2 dx =

∞∑−∞|cn|2


Comments on Parseval’s Theorem

In Theorem 8.14, we showed that under certain hypotheses, theFourier series of f converges to f pointwise. Part 1© of Parseval’sTheorem shows that, under weaker hypotheses, the Fourier seriesof f converges to f with respect to the L2 metric. (In fact thesehypotheses can further be weakened using Lebesgue integration.)

Parts 2© and 3© of Parseval’s Theorem show that we can calculateHermitian inner products and L2 norms of functions in terms oftheir Fourier coefficients, just as we calculate inner products andnorms of vectors in Cn using their coefficients w.r.t. the standardorthonormal basis.


Proof of Theorem 8.16For any h ∈ R on [−π, π], define

‖h‖2 =

√1

2π

∫ π

−π|h(x)|2 dx

Let ε > 0. Though f might not be continuous, it is shown inExercise 12 of Chapter 6 that by linearly interpolating values of ffrom a sufficiently fine partition of [−π, π], we obtain a continuousfunction h with period 2π such that

‖f − h‖2 < ε


Then Theorem 8.15 =⇒ ∃ a trigonometric polynomial P suchthat |h(x)− P(x)| < ε for all x . Hence ‖h − P‖2 < ε.

Assuming P has degree N0, Theorem 8.11 =⇒

‖h − sN(h)‖2 ≤ ‖h − P‖2 < ε ∀N ≥ N0

Using the inequality right before Theorem 8.12, we showed

‖sN(h)− sN(f )‖2 = ‖sN(h − f )‖2 ≤ ‖h − f ‖2 < ε

[In fact, we showed∫|sN |2 ≤

∫|f |2, ∀f ∈ R]


Now, the triangle inequality for ‖ ‖2 (from Ex. 11, Chap. 6) =⇒

‖f − sN(f )‖2 ≤ ‖f − h‖2 + ‖h − sN(h)‖2 + ‖sN(h)− sN(f )‖< 3ε

for all N ≥ N0. This proves 1©.

Next

1

2π

∫ π

−πsN(f )g dx =

N∑n=−N

cn1

2π

∫ π

−πe inxg(x) dx

=N∑

n=−N

cnγn (?)


∣∣∣∣∫ f g −∫

sN(f )g

∣∣∣∣ ≤ ∫ |f − sN(f )||g |

(Schwarz inequality) ≤(∫|f − sN |2

∫|g |2

) 12

By 1©, the RHS → 0 as N →∞.

Substituting (?) in the LHS , implies 2©.

3© follows from 2© by substituting f for g !


Reminder of Holder’s Inequality

In Exercise 10 on page 139 of Rudin, we worked out a proof ofHolder’s Inequality. It states that if p and q are positive realnumbers that satisfy

1

p+

1

q= 1

and f and g are Riemann-integrable functions [a, b]→ C, then∣∣∣∣∫ b

afg dx

∣∣∣∣ ≤ (∫ b

a|f |pdx

)1/p (∫ b

a|g |qpdx

)1/q

.

The special case of p = q = 2 is just the Schwarz inequality.


Gamma function

Definition

For 0 < x <∞, define

Γ (x) =

∫ ∞0

tx−1e−t dt

Note: Need to look separately at

∫ 1

0and

∫ ∞1

- good Calculus II

problem to show it converges!

(For

∫ ∞1

, first do for integers x = n by math induction.

Then a non-integer is dominated by an integer.)


Theorem (8.18)

1 ∀ 0 < x <∞, Γ (x + 1) = xΓ (x) “functional equation”

2 Γ (n + 1) = n! ∀n = 0, 1, 2, 3, . . .

3 log Γ is convex (concave up) on (0,∞)

[g is “convex” (Exercise 23, page 101) if and only if

g(λx + (1− λ)y) ≤ λg(x) + (1− λ)g(y)]


Proof.

1 Integration by parts

2 First note Γ (1) = 1; then use mathematical induction andpart 1©.

3 Assume 1p + 1

q = 1.

Holder’s inequality =⇒ Γ(xp + y

q

)≤ Γ (x)

1p Γ (y)

1q

3© follows.

Recall that Holder’s inequality was proved in Exercise 10, page 139of Rudin.


Theorem (8.19)

If f : (0,∞)→ (0,∞) and, for all x > 0, it satisfies:

1 f (x + 1) = xf (x) ∀x ∈ (0,∞)

2 f (1) = 1

3 log f is convex

thenf (x) = Γ (x)

ProofSince Γ satisfies these conditions, it is enough to show that f isuniquely determined by the conditions.

Because of 1©, it is enough to do this for 0 < x ≤ 1.


Let ϕ(x) = log(f (x)).

Then

ϕ(x + 1) = log(f (x + 1)) = log(xf (x)) = log(x) + log(f (x))

= log(x) + ϕ(x) (0 < x <∞)

Also, ϕ(1) = 0 and ϕ is convex.

By 1© and 2©, f (n + 1) = n! (∀n ∈ J).

So,ϕ(n + 1) = log(n!)

∴ ϕ(n + 1)− ϕ(n) = log(n!)− log((n − 1)!) = log(n)

ϕ(n + 1)− ϕ(n) = log(n) (for n ∈ J)


By convexity,

ϕ

(x

1 + xn+

1

1 + x(n + 1 + x)

)≤ x

1 + xϕ(n) +

1

1 + xϕ(n + 1 + x)

ϕ(n + 1) ≤ xϕ(n) + ϕ(n + 1 + x)

1 + x

xϕ(n + 1) + ϕ(n + 1) ≤ xϕ(n) + ϕ(n + 1 + x)

ϕ(n + 1)− ϕ(n) ≤ ϕ(n + 1 + x)− ϕ(n + 1)

x

log(n) ≤ ϕ(n + 1 + x)− ϕ(n + 1)

x


Similarly,

ϕ((1− x)(n + 1) + x(n + 2)) ≤ (1− x)ϕ(n + 1) + xϕ(n + 2)

ϕ(n + 1 + x)− ϕ(n + 1) ≤ x(ϕ(n + 2)− ϕ(n + 1))

ϕ(n + 1 + x)− ϕ(n + 1)

x≤ log(n + 1)

So,

2© log(n) ≤ ϕ(n + 1 + x)− ϕ(n + 1)

x≤ log(n + 1)


ϕ(x + 1) = ϕ(x) + log(x)

ϕ(x + 2) = ϕ(x + 1) + log(x + 1)

= ϕ(x) + log(x) + log(x + 1)

ϕ(x + 3) = ϕ(x + 2) + log(x + 2)

= ϕ(x) + log(x) + log(x + 1) + log(x + 2)

...

ϕ(x + n + 1) = ϕ(x) + log(x) + log(x + 1) + · · ·+ log(x + n)

1©ϕ(x + n + 1) = ϕ(x) + log[x(x + 1) . . . (x + n)]


Substituting 1© into 2©,

log(n) ≤ ϕ(x) + log[x(x + 1) . . . (x + n)]− ϕ(n + 1)

x≤ log(n + 1)

x log(n) ≤ ϕ(x) + log[x(x + 1) . . . (x + n)]− ϕ(n + 1)

≤ x log(n + 1)

0 ≤ ϕ(x) + log[x(x + 1) . . . (x + n)]− log(n!)

− x log(n) ≤ x log

(n + 1

n

)

0 ≤ ϕ(x)− log

[nxn!

x(x + 1) . . . (x + n)

]≤ x log

(1 +

1

n

)


By squeeze theorem,

ϕ(x) = limn→∞

log

[nxn!

x(x + 1) . . . (x + n)

]

∴ f (x) = limn→∞

nxn!

x(x + 1) . . . (x + n)

Thus f (x) is determined by 1©, 2© and 3©,

∴ f (x) = Γ (x)

Note: We have established:

Γ (x) = limn→∞

nxn!

x(x + 1) . . . (x + n)


Theorem (8.20)

If x , y > 0, then∫ 1

0tx−1(1− t)y−1 dt =

Γ (x)Γ (y)

Γ (x + y)

Define the beta function by

B(x , y) =

∫ 1

0tx−1(1− t)y−1 dt

Proof∀y > 0

B(1, y) =

∫ 1

0(1− t)y−1 dt = −(1− t)y

y

∣∣∣∣10

=1

y


Next, prove that as a function of x , log(B(x , y)) is convex for eachfixed y .

Given p, q such that 1p + 1

q = 1:

B

(x

p+

z

q, y

)=

∫ 1

0txp+ z

q− 1

p− 1

q (1− t)yp+ y

q− 1

p− 1

q dt

=

∫ 1

0tx−1p (1− t)

y−1p t

z−1q (1− t)

y−1q dt

by Holder’s inequality

≤(∫ 1

0tx−1(1− t)y−1dt

) 1p(∫ 1

0tz−1(1− t)y−1dt

) 1q

≤ B(x , y)1pB(z , y)

1q


∴ log

(B

(x

p+

z

q, y

))≤ 1

plog(B(x , y)) +

1

qlog(B(z , y))

proving log(B(x , y)) is convex as a function of x .

Next, prove

B(x + 1, y) =x

x + yB(x , y) (∀x , y > 0)

B(x + 1, y) =

∫ 1

0tx(1− t)y−1 dt =

∫ 1

0

(t

1− t

)x

(1− t)x+y−1 dt


t

1− t= −1 +

t

1− t

Integrating by parts,

u =

(t

1− t

)x

dv = (1− t)x+y−1 dt

du = x

(t

1− t

)x−1 [ 1

(1− t)2

]dt v = −(1− t)x+y

x + y


B(x + 1, y)

= − tx(1− t)y

x + y

∣∣∣∣10

+

∫ 1

0

x

x + ytx−1(1− t)y+1

[1

(1− t)2

]dt

=x

x + yB(x , y)

Now, define f (x) =Γ (x + y)

Γ (y)B(x , y)


Check:

(b) f (1) =Γ (1 + y)

Γ (y)B(1, y)

= y · 1

y= 1 X

(a) f (x + 1) =Γ (x + 1 + y)

Γ (y)B(x + 1, y)

= (x + y)Γ (x + y)

Γ (y)

x

x + yB(x , y)

= xf (x) X


(c) Since

log(f (x)) = log(Γ (x + y)) + log(B(x , y))− log(Γ (y))

log(f (x)) is convex.

Therefore, by Theorem 8.19, f (x) = Γ (x).

Γ (x) =Γ (x + y)

Γ (y)B(x , y)

∴ B(x , y) =Γ (x)Γ (y)

Γ (x + y)X


Consequences

Making the substitution t = sin2 θ, dt = 2 sin θ cos θ dθ gives

B(x , y) = 2

∫ π2

0(sin θ)2x−1(cos θ)2y−1 dθ

So, this

=Γ (x)Γ (y)

Γ (x + y)

Plugging in x = y = 12 gives

2

∫ π2

0dθ =

Γ (12)2

Γ (1)


or

π = Γ

(1

2

)2

=⇒ Γ

(1

2

)=√π

Substituting t = s2 in the definition of Γ , gives

Γ (x) = 2

∫ ∞0

s2x−1e−s2ds

Plug x = 12 =⇒

Γ

(1

2

)= 2

∫ ∞0

e−s2ds =

∫ ∞−∞

e−s2ds

∴∫ ∞−∞

e−s2ds =

√π


Claim ∀x ∈ (0,∞)

Γ (x) =2x−1√πΓ(x

2

)Γ

(x + 1

2

)Proof Define

f (x) =2x−1√πΓ(x

2

)Γ

(x + 1

2

)Using Theorem 8.19, check:

(a) f (1) =1√πΓ

(1

2

)Γ (1) = 1 X

(b) log(f (x)) = (x − 1) log 2− log(√π)

+ log(Γ(x

2

))+ log

(Γ

(x + 1

2

))convex X


(c) f (x + 1) =2x√πΓ

(x + 1

2

)Γ(x

2+ 1)

=2x√πΓ

(x + 1

2

)x

2Γ(x

2

)

= x2x−1√πΓ(x

2

)Γ

(x + 1

2

)

= xf (x) X

∴ By Theorem 8.19, f (x) = Γ (x)


Stirling’s formula

limx→∞

Γ (x + 1)

(x/e)x√

2πx= 1

Sometimes written

Γ (x + 1) ∼(xe

)x √2πx

I won’t go through the proof of this formula here... you can read itin Rudin!


Chapter 9: Functions of Several Variables

Please read the first two pages of Chapter 9 on your own, up to9.4 Definitions. It should be familiar to you from Linear Algebra.Note: in this chapter all vector spaces are assumed to be subspacesof Rn.

Definition (9.4)

Let X,Y be vector spaces.A function A : X → Y is called a linear transformation if∀x, y ∈ X, c ∈ R

1 A(x + y) = Ax +Ay

2 A(cx) = cA(x)


Note: Often write Ax instead A(x) if A is linear.

Note that if A is linear, the action of A on X is determined by itsaction on a basis x1, . . . , xn of X, because if x ∈ X is arbitrary,then we can write x = c1x1 + · · ·+ cnxn, and hence

A(x) = c1A(x1) + · · ·+ cnA(xn)

A linear transformation A : X → X is called a linear operator on X.

If A : X → X is bijective (i.e. 1− 1 and onto), we say A isinvertible.

In that case, its inverse A−1 : X → X is defined, is another linearoperator on X and A A−1 = A−1 A = IdX


Theorem (9.5)

Assume X is a finite dimensional vector space. Then a linearoperator A : X → X is 1− 1 if and only if it is onto.

ProofSee book. You learned this in Linear Algebra!

Definitions (9.6)

a If X,Y are vector spaces, letL(X,Y ) = linear transformations X → Y .Write L(X) for L(X,X).

Note:If c1, c2 ∈ R and A1, A2 ∈ L(X,Y ), then c1A1 + c2A2 ∈ L(X,Y )


b If X,Y and Z are vector spaces, A ∈ L(X,Y ) andB ∈ L(Y,Z), then B A ∈ L(X,Z).

We often write BA for B A.

Note:Even if X = Y = Z, in general AB 6= BA

c If A ∈ L(Rn,Rm), define the norm of A, ‖A‖, by

‖A‖ = sup |Ax| : x ∈ Rn and |x| ≤ 1

It follows easily that (∀x ∈ Rn)

|Ax| ≤ ‖A‖ |x|.

Also, if λ ∈ R and |Ax| ≤ λ|x| for all x ∈ Rn, then ‖A‖ ≤ λ.


Theorem (9.7)

1 If A ∈ L(Rn,Rm), then ‖A‖ ≤ ∞ and A : Rn → Rm isuniformly continuous.

2 If A,B ∈ L(Rn,Rm) and c ∈ R, then ‖A+B‖ ≤ ‖A‖+ ‖B‖and ‖cA‖ = |c | ‖A‖.If you define d(A,B) = ‖A−B‖, this makes L(Rn,Rm) intoa metric space.

3 If A ∈ L(Rn,Rm) and B ∈ L(Rm,Rk), then‖BA‖ ≤ ‖B‖ ‖A‖


Proof

1©: Let e1, . . . , en be the standard basis for Rn and supposex = c1e1 + · · ·+ cnen and |x| ≤ 1.It follows that |ci | ≤ 1 (∀i = 1, . . . , n).

Then

|Ax| =∣∣∣∑ ciAei

∣∣∣ ≤∑ |ci | |Aei| ≤∑|Aei|

Hence

‖A‖ ≤n∑

i=1

|Aei| <∞

To prove A uniformly continuous, let ε > 0 and let δ =ε

‖A‖.

Then if |x− y| < δ, |Ax− Ay| ≤ ‖A‖ |x− y| < ε.

(Case where ‖A‖ = 0 is trivial, since in that case A = 0.)


2©: The inequality in 2© follows from:

|(A + B)x| = |Ax + Bx| ≤ |Ax|+ |Bx| ≤ (‖A‖+ ‖B‖) |x|

Proof of ‖cA‖ = |c | ‖A‖ is similar.

Skip proof that L(Rn,Rm) is a metric space - it’s easy!

3© follows from:

|(BA)x| = |B(Ax)| ≤ ‖B‖ |Ax| ≤ ‖B‖ ‖A‖ |x|

We use the metric space structure on L(Rn,Rm) in thefollowing.


Theorem (9.8)

Let Ω = A ∈ L(X) such that A is invertible.

1 If A ∈ Ω, B ∈ L(Rn) and

‖B −A‖ ‖A−1‖ < 1

then B ∈ Ω

2 Ω is an open subset of L(Rn) and the function Ω → Ω givenby A→ A−1 is continuous on Ω.


Proof

1©: Let α =1

‖A−1‖, so ‖A−1‖ =

1

α,

and let β = ‖B − A‖. Since β · 1

α< 1, β < α.

For every x ∈ Rn,

α|x| = α|A−1Ax| ≤ α‖A−1‖ |Ax|= |Ax| ≤ |(A−B)x|+ |Bx| ≤ β|x|+ |Bx|

Hence,(α− β)|x| ≤ |Bx| (∀x ∈ Rn) (?)


Since α− β > 0, this =⇒ Bx 6= 0 if x 6= 0.

Therefore B is injective and, by Theorem 9.5, bijective.

HenceB ∈ Ω

2© This also proves

N 1‖A−1‖

(A) ⊆ Ω

so Ω is open in L(Rn).

Just need to prove the function A→ A−1 is continuous.In (?) replace x by B−1y, giving

(∀y ∈ Rn) (α− β)|B−1y| ≤ |BB−1y| = |y|


This proves

‖B−1‖ ≤ 1

α− β

Now, write B−1 −A−1 = B−1(A−B)A−1 and apply 9.7(3):

‖B−1 −A−1‖ ≤ ‖B−1‖ ‖A−B‖ ‖A−1‖ ≤ β

(α− β)α

Since β = ‖B −A‖, we can make B−1 as closed as desired to A−1

by making B sufficiently close to A (i.e. β close to 0).

This establishes continuity.


Matrices

Suppose X and Y are vector spaces andx1, . . . , xn is a basis of X and y1, . . . , ym is a basis of Y .Let A ∈ L(X ,Y ). For each j = 1 . . . , n we can write

Axj =m∑i=1

aijyi

where ∀ 1 ≤ j ≤ n, ∀ 1 ≤ i ≤ m, aij ∈ R.

These aij form an mx n matrix

[A] =

a11 a12 . . . a1na21 a22 . . . a2n

...... . . .

...am1 am2 . . . amn


[A] is called the matrix of A with respect to the given bases xjand yi.

Note: The j-th column of [A] gives the coefficients of Axj withrespect to the basis yi.

Suppose x ∈ X is arbitrary; write

x =n∑

j=1

cjxj


Then

Ax =n∑

j=1

cjAxj =n∑

j=1

cj

m∑i=1

aijyi =m∑i=1

n∑j=1

aijcj

yi

Thus the coordinates of Ax (with respect to yi) are

n∑j=1

aijcj


To sum up: If we express x ∈ X as a column vector

c1...cn

with

respect to the basis xj, then matrix multiplication

[A]

c1...cn

gives us Ax, expressed as a column vector with respect to the basisyi of Y .

This correspondence A←→ [A] gives a 1− 1 correspondencebetween

L(X ,Y )←→ mx n matrices over R


Don’t forget: the matrix [A] depends not only on A ∈ L(X ,Y ), butalso on our choice of bases for X and Y .

Now, suppose we have a third vector space Z with basisz1, . . . , zp and linear transformations

A : X → Y B : Y → Z

Then B A (written BA) has matrix

[BA] = [B] [A]

( matrix multiplication)with respect to the given bases.


An upper bound for ‖A‖

Suppose x1, . . . , xn and y1, . . . , ym are the standard bases ofRn and Rm.

Assume A ∈ L(Rn,Rm) has matrix [A] = (aij) with respect tothese bases.

Suppose x =

c1...cn

∈ Rn


Then

|Ax|2 =∑i

∑j

aijcj

2

(Schwarz inequality) ≤∑i

∑j

a2ij

·∑

j

c2j

=

∑i , j

a2ij

|x|2Hence

‖A‖ ≤

∑i , j

a2ij

12

(?)


Theorem

Suppose S is a metric space and ∀ i = 1, . . . ,m, ∀ j = 1, . . . , n,aij : S → R is continuous. For each p ∈ S , define Ap ∈ L(Rn,Rm)to be the linear transformation whose matrix is (aij(p)).Then the function

S → L(Rn,Rm)

defined byp 7→ Ap

is continuous.(Using the metric space structure on L(Rn,Rm), recentlyintroduced.)

Proof.

This follows easily by applying (?) to the difference B − A.Omit details.


Differentiation

Goal: Define the derivative of a function

f : Rn → Rm.

What kind of object should it be?

Recall case f : R→ R. We defined

f ′(x) = limh→0

f (x + h)− f (x)

h

Rewrite as

limh→0

f (x + h)− f (x)− f ′(x)h

h= 0


Thus, if we define r(h) = f (x + h)− f (x)− f ′(x)h, we see

limh→0

r(h)

h= 0

So r(0) = 0 and r(h)→ 0 as h→ 0 “very quickly”.

Sum up: The linear transformation

R→ R

given byh 7→ f ′(x)h

is a very good approximation of the non-linear maph 7→ f (x + h)− f (x) in the sense that the error termr(h) = f (x + h)− f (x)− f ′(x)h has the property that

limh→0

r(h)

h= 0.


Note: This is equivalent to

limh→0

|r(h)||h|

= 0

Moral of the Story: Instead of thinking of f ′(x) simply as anumber, we think of it as a linear transformation

R→ R

given byh 7→ f ′(x)h

Generalize:


Definition

Let E ⊆ Rn be open, x ∈ E , and assume f : E → Rm is a function.If ∃ a linear transformation A ∈ L(Rn,Rm) such that

limh→0

|f (x + h)− f (x)− Ah||h|

= 0 (? 1)

then we say f is differentiable at x and

f ′(x) = A

(Note: in the above limit, h ∈ Rn)

If f is differentiable at each x ∈ E , we say f is differentiable on E .

Repeat: For each given x ∈ E , f ′(x) is an element of L(Rn,Rm).This element changes as x changes.


Theorem

f has at most one derivative at x

I.e., if A1,A2 ∈ L(Rn,Rm) and both A1 and A2 satisfy (? 1), thenA1 = A2.

ProofLet B = A1 − A2. Then, ∀h ∈ Rn,

|Bh| = |(f (x + h)− f (x)− A2h)− (f (x + h)− f (x)− A1h)|≤ |f (x + h)− f (x)− A2h|+ |f (x + h)− f (x)− A1h|

∴|Bh||h|→ 0 as h→ 0


For fixed h ∈ Rn, this =⇒

limt→0

|B(th)||th|

= 0

But|B(th)||th|

=|tB(h)||th|

=|t||B(h)||t||h|

=|B(h)||h|

, which doesn’t

depend on t!

∴ B(h) = 0 ∀h ∈ Rn


Recall definition of derivative

Let E ⊆ Rn be open, and f : E → Rm. Fix x ∈ E . If there is alinear transformation A ∈ L(Rn,Rm) such that

limh→0

|f (x + h)− f (x)− Ah||h|

= 0 (?)

we write f ′(x) = A, and say “f is differentiable at x”.

We proved last time that, given f and x, there is at most onesuch A. Hence, f ′(x) is well-defined.


Remarks

1 (?) can be written as

f (x + h)− f (x) = f ′(x)h + r(h)

where limh→0

|r(h)||h|

= 0

I.e. “f (x + h)− f (x) is very well approximated by f ′(x)h”.

2 If f is differentiable on E , then f ′ : E → L(Rn,Rm) is afunction.

3 The formula in 1© shows that if f is differentiable at x,then f is continuous at x.

4 “f ′(x)” is sometimes called the differential of f at x orthe total derivative of f at x.


Ex.

Suppose A : Rn → Rm is a linear transformation.

Then

limh→0

|A(x + h)− A(x)− A(h)||h|

= limh→0

|Ax + Ah− Ax− Ah||h|

= 0

∴ For all x ∈ Rn, A′(x) = A


Theorem (9.15 - Chain Rule)

Suppose E ⊆ Rn is open, f : E → Rm, and g maps aneighborhood of f (E ) (in Rm) to Rk .Also, assume f is differentiable at x0 ∈ E and g is differentiable atf (x0).Define F : E → Rk by F = g f .Then F is differentiable at x0 and

F ′(x0) = g ′(f (x0))f ′(x0)

Note g ′(f (x0)) ∈ L(Rm,Rk), and f ′(x0) ∈ L(Rn,Rm).

Proof.

Omit. Very similar to proof of ordinary Chain Rule.


Partial Derivatives

Recall from Calculus III:Suppose f : Rn → R. The partial derivatives of f are defined as

∂f

∂xj(a1, . . . , an) = lim

h→0

f (a1, . . . , aj + h, . . . , an)− f (a1, . . . , an)

h

“The derivative of f (x1, . . . , xn) with respect to xj , holding theother variables constant”.

If a = (a1, . . . , an) and

ej = (0, . . . , 0, 1, 0, 0)

j-th placeWe can also write

∂f

∂xj(a) = lim

h→0

f (a + hej)− f (a)

h


Suppose E ⊆ Rn is open, and f : E → Rm. Let e1, . . . , en andu1, . . . ,um be standard bases of Rn and Rm.

The components of f are functions f1, . . . , fm (each fi : Rn → R)defined by

f (x) = (f1(x), f2(x), . . . , fm(x))

Equivalently, fi (x) = f (x) · ui

If x ∈ E , 1 ≤ i ≤ m, 1 ≤ j ≤ n, we can take partial derivative

∂fi∂xj

(x) = limt→0

fi (x + tej)− fi (x)

t

Other notation:∂fi∂xj

(x) = (Dj fi )(x)


Theorem (9.17)

Suppose E ⊆ Rn is open, and f : E → Rm is differentiable atx ∈ E . Then the partial derivatives (Dj fi ) all exist at x and

f ′(x)ej =m∑i=1

(Dj fi )(x)ui

1 ≤ j ≤ n

In other words, with respect to the standard bases

[f ′(x)] =

(D1f1)(x) . . . (Dnf1)(x)...

. . ....

(D1fm)(x) . . . (Dnfm)(x)


Thus, the partial derivatives of the components of f are the entriesof the matrix for f ′(x) with respect to the standard bases.

ProofBy definition of f ′(x),

limh→0

|f (x + h)− f (x)− f ′(x)h||h|

= 0

∴ f (x + tej)− f (x) = f ′(x)(tej) + r(tej)

where|r(tej)|

t→ 0 as t → 0 (since |tej| = |t|).


∴f (x + tej)− f (x)

t−

f ′(x)(tej)

t→ 0 as t → 0

∴f (x + tej)− f (x)

t→ f ′(x)(ej) as t → 0

∴ limt→0

f (x + tej)− f (x)

t= f ′(x)(ej) (?)

The i-th entry of the vector on the RHS of (?) is the ij-th entry ofthe matrix [f ′(x)].

The i-th entry of the vector on the LHS of (?) is

limt→0

fi (x + tej)− fi (x)

t= (Dj fi )(x)


Directional Derivative and Gradient

x0 ∈ Rn; e1, . . . , en standard basis for Rn (column vectors);f : Rn → R differentiable; u ∈ Rn, with |u| = 1.

Define the directional derivative of f at x0 in the direction u by

Duf (x0) =d

dt

∣∣∣∣t=0

f (x0 + tu)

Interpretation: Define γ : R→ Rn by γ(t) = x0 + tu.

γ is a parametrized straight line in Rn with a constant velocityγ ′(t) = u passing through x0 at t = 0.

R γ−−→ Rn f−−→ R


Duf (x0) = (f γ)′(0) = (by Chain Rule) f ′(γ(0))γ ′(0) = f ′(x0)u

Note: u =

u1...un

and f ′(x0) = [D1f (x0), . . . ,Dnf (x0)]

∴ Duf (x0) = D1f (x0)u1 + · · ·+ Dnf (x0)un


If we define the gradient of f at x0 ∇f (x0) ∈ Rn by

∇f (x0) = D1f (x0)e1 + · · ·+ Dnf (x0)en

then we haveDuf (x0) = ∇f (x0) · u

By Schwarz inequality, the value of u giving largest Duf (x0) is

u =∇f (x0)

|∇f (x0)|

So we say “∇f (x0) points in the direction of most rapid increaseof f .”


For this value of u,

Duf (x0) = ∇f (x0) · ∇f (x0)

|∇f (x0)|= |∇f (x0)|

“∇f (x0) points in the direction of most rapid increase of f startingat x0, and |∇f (x0)| gives the magnitude of this most rapidincrease.”


The Inverse Function Theorem

Suppose E ⊆ Rn is open, f : E → Rm, and f is differentiable onE . Then

f ′ : E → L(Rn,Rm).

If f ′ is continuous, we say f is continuously differentiable or write

f ∈ C′(E ).

(More modern notation: f ∈ C 1(E ))

Theorem (9.21)

f ∈ C′(E ) ⇐⇒ all the partial derivatives Dj fi are defined andcontinuous on E .

Proof.

Omit


Theorem (9.24 - Inverse Function Theorem)

Suppose E ⊆ Rn is open, f : E → Rn, and f ∈ C′(E ).Suppose a ∈ E , f ′(a) : Rn → Rn is invertible, and b = f (a).Then

1 ∃ open sets U,V ∈ Rn such that a ∈ U, b ∈ V , f is injectiveon U, and f (U) = V

2 If g : V → U is the inverse of f (which exists by 1©), theng ∈ C ′(V ) and g ′(b) = f ′(a)−1

“For a C′ function, invertibility of f ′(a) at a single point a =⇒invertibility of f on a neighborhood of a.”


Thank you!

Questions?


Date post:	20-Aug-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Online Notes for Advanced Calculus II …klassen/AdvancedCalcII/Online...Eric Klassen, Florida State...

Documents