MATH 353 LECTURE NOTESWEEK 9

INTRODUCTION TO FOURIER SERIES

J. WONG (FALL 2017)

Topics covered

• Introductory notes◦ Motivating example: Heat conduction in a metal bar◦ Connection to linear algebra

• Orthogonality, inner products◦ Inner products (dot product)◦ Orthogonal bases, eigenvectors/eigenvalues◦ Extending to functions: L2 (the infinite version of the dot product)◦ Orthogonality in L2 and the norm (again, infinite version)

• Fourier Series◦ Definition of Fourier series◦ Basic structure

− complete set− Orthogonality, mean-square convergence− As an approximation for periodic functions

◦ Calculating Fourier series for periodic functions− Two examples

1. Motivation

1.1. Linear algebra analogy. Suppose we have an n× n matrix A and wish to solve

Ax = b.

Assume A is invertible and symmetric. You know many ways of solving this problem, butlet’s go through the details of one approach that will be a useful analogy. Rather than solvea linear system, we would like to choose the right basis so that the n equations to solve areindependent.

To do so, recall that an n× n real symmetric matrix has distinct eigenvalues

λ1, λ2, · · ·λnand eigenvectors

v1,v2, · · ·vnthat form a basis for Rn. Not only that, but they are orthogonal:

vi · vj = 0 if i 6= j.1

2 J. WONG (FALL 2017)

Now both x and b can be written in terms of this basis:

x =n∑i=1

civi, b =n∑i=1

divi.

Now since Avi − λivi, plugging into Ax = b gives

n∑i=1

λicivi =n∑i=1

divi.

Since the vi’s form a basis, it must be true that the coefficients of each vi are equal, so

ci = di/λi.

This leaves only the problem of writing b in terms of the vi’s. Here is where orthogonalityis crucial. Take the dot product of the equation for b,

b =n∑i=1

divi,

with vj to get

b · vj =n∑i=1

di(vi · vj).

Now all the terms in the sum are zero except for i = j:

b · vj = divj · vj =⇒ di] =b · vjvj · vj

.

The solution to Ax = b is then

x =n∑i=1

civi, ci =1

λi

b · vivi · vi

.

Assuming we have the eigenvectors and eigenvalues, notice that at no point did we need tosolve any coupled linear systems. All the equations (for di and then ci) only had one termdivided by another term. The orthogonality of the basis allowed us to decompose b in termsof the basis by solving for the di’s one by one, and the eigenvalue property Av = λv let ussolve for ci in terms of di. In the coming weeks, we will see how to generalize these ideas todifferential equations.

1.2. Heat conduction in a metal bar. A metal bar with length L = π is initially heatedto a temperature of u0(x). The temperature distribution in the bar is u(x, t). Over time, weexpect the heat to diffuse or be lost to the environment until the bar is evenly heated.

MATH 353 LECTURE NOTES WEEK 9 INTRODUCTION TO FOURIER SERIES 3

Physicist Joseph Fourier, around 1800, studied this problem and in doing so drew attentionto a novel technique that has since become one of the cornerstones of applied mathematics.The approach outlined below hints at some of the deep structure we will uncover in theremainder of the course.

We will show later that the temperature can be modeled by the heat equation

∂u

∂t=∂2u

∂x2, t > 0 and x ∈ (0, π).

Assume that the temperature is held fixed at both ends. This condition is imposed in themodel through boundary conditions

u(0, t) = 0, u(π, t) = 0 for all t.

Notice that unlike initial conditions, there are boundary conditions at two different valuesof x. Thus we do not have an ‘initial value problem’ where we start at one point.

Lastly, the initial heat distribution is t = 0 is

u(x, 0) = u0(x)

which is the initial condition. The temperature should decrease as heat leaks out of thebar through the ends; eventually it all dissipates. The solution u(x, t) should predict this.

In summary, our goal is to find a function u(x, t) defined on [0, π] satisfying

∂u

∂t=∂2u

∂x2t > 0 and x ∈ (0, π),(1a)

u(0, t) = u(π, t) = 0 for t ≥ 0(1b)

u(x, 0) = u0(x).(1c)

We call (1) (the three equations together) an initial boundary value problem for u(x, t).

To solve the equation, we guess at an exponentially decaying solution

(2) u(x, t) = e−λtφ(x).

Our objective here is just to find a solution to the first two parts, (1a) and (1b) and worryabout the initial condition later.

Substituting into the PDE (1a), we find that

−λφ(x) = φ′′(x).

4 J. WONG (FALL 2017)

Now substitute into the boundary conditions (1b) (note that e−λt cancels out here) to get

φ(0) = 0, φ(π) = 0.

For convenience set λ = µ2. It follows that (2) satisfies the PDE (1a) and the boundaryconditions (1b) if the function g(x) solves the boundary value problem

(3) φ′′(x) + µ2φ(x) = 0, φ(0) = 0, φ(π) = 0.

This problem is not an initial value problem, but it is a constant-coefficient ODE, so we canstill solve it explicitly. The general solution is

φ = c1 sin(µx) + c2 cos(µx).

Imposing the condition φ(0) = 0 we find that

φ = c1 sin(µx).

The second condition, φ(1) = 0, requires that

sin(µπ) = 0.

Non-trivial solutions exist whenever µ is a non-zero integer. We have now found an infinitesequence of solutions to (3):

φn(x) = sin(nx), n = 1, 2, 3, · · ·Observe that (3) is a linear, homogeneous problem. In particular,

(4) φ1, φ2 are solutions to (3) =⇒ c1φ+ c2φ2 is a solution.

This means that for any constant an, the function

(5) ane−n2tφn(x)

is a solution to the heat conduction problem with initial data

u0(x) = an sin(nx).

Now the crucial question: what happens when the initial data is not a sine? No singlesolution of the form (5) will work. Fourier’s breakthrough was the realization that, using thesuperposition principle (4), the solution could be written as an infinite linear combinationof all the solutions of the form (5):

u(x, t) =∞∑n=1

ane−n2tφn(x).

Then u(x, t) solves the original problem (1) if the coefficients an satisfy

(6) u0(x) =∞∑n=1

anφn(x).

This idea is a generalization of what you know from linear algebra (representing vectors interms of a basis) but with basis functions {sin(nx) : n = 1, 2, 3, · · · }.

In fact, this set of functions has the rather remarkable orthogonality property

(7)

∫ π

0

φm(x)φn(x) dx =

∫ π

0

sin(mx) sin(nx) dx = 0, m 6= n.

MATH 353 LECTURE NOTES WEEK 9 INTRODUCTION TO FOURIER SERIES 5

To solve for the coefficient am, we can multiply (6) by sin(mx) and integrate:∫ π

0

u0(x) sin(mx) dx =

∫ π

0

∞∑n=1

an sin(mx) sin(nx) dx.

Now move the integral inside the sum (it is not trivial to show this is allowed!). By theproperty (7), only one of the terms in the sum will be non-zero:∫ π

0

u0(x) sin(mx) dx =

∫ π

0

∞∑n=1

an sin(mx) sin(nx) dx

=∞∑n=1

an

∫ π

0

sin(mx) sin(nx) dx

=

(∞∑

n=1,n6=m

an · 0

)+ am

∫ π

0

sin(mx) sin(mx) dx

= am

∫ π

0

sin2(mx) dx.

Magically, the infinite sum has been reduced to a simple equation for am:

(8) am =

∫ π0u0(x) sin(mx) dx∫ π0

sin2(mx) dx.

This process works for allm, so the solution to the heat conduction problem (5) with arbitraryinitial condition u0(x) is

u(x, t) =∞∑n=1

ane−n2t sin(nx)

with the coefficients given by the formula (8). Of course, all of the manipulations here areformal and unjustified - it is far from clear whether the series converges, or if it is valid toswap integrals and sums, and so on (Fourier did not know this either when first applyingthe method; it took several decades to settle the issue).

1.3. Observations and goals. The method in the example may seem rather mysterious,but it hints at some remarkable structure. We have identified eigenfunctions φn that satisfy

φ′′n = λφn, φn(0) = φn(π) = 0

and found that they have a special orthogonality property. Then, we exploited superpositionto build an infinite series, which has enough coefficients to match any initial condition. Thefunctions φn must span all possible initial conditions - they are a basis in some sense. Whilebuilding up the theory, we need to address some fundamental questions first:

• What does it mean for functions to be ’eigenfunctions’ and ’orthogonal’? What doesit mean for a collection of functions to be a basis, and for what? Think of this asgeneralizing eigenvalues and ortthogonal bases for Rn. The fact that the dimensionis infinite leads to serious complications.• What are the properties of the linear operator

L[φ] = φ′′

that arises in the ’eigenvalue’ problem L[φ] = λφ?

6 J. WONG (FALL 2017)

• How can these tools be used to solve PDEs (and what are the limitations)?• What are the implications of this theory for real problems?

1.4. Outline.

• L2 and orthogonality◦ functions in L2 (vectors in Rn)◦ inner product, orthogonality (dot product, orthogonal vectors)◦ orthogonal basis for L2 (orthogonal basis for Rn with eigenvectors)

• Fourier series, eigenfunctions◦ Appearance as solution to boundary value problem◦ operator L and eigenfunctions (A and eigenvectors of A)◦ Fourier series

• Heat equation, etc.◦ Introduction to PDEs◦ Solution using eigenfunctions

2. Inner products, orthogonality of functions

To study eigenfunctions and PDEs and so on, we need to identify the correct space wherethe functions of interest reside, and extend notions from linear algebra in Rn (see section 4)to this space of functions.

2.1. Square-integrable functions (L2[a, b]). For this section, we will consider real-valuedfunctions f defined on an interval [a, b]. Such a function is called square-integrable1 if∫ b

a

|f(x)|2 dx <∞.

We now define L2[a, b] to be the set of real-valued functions defined on [a, b] that are squareintegrable, i.e.

L2[a, b] = {f : [a, b]→ R such that

∫ b

a

|f(x)|2 dx <∞}.

This space will turn out to be the right one for studying Fourier series. We will not proveit, but if f and g are square integrable then f + g is also square integrable, so linear combi-nations of functions in L2[a, b] stay in L2[a, b].

Norm: We define a norm on the space (the ‘L2 norm’) as follows:

‖f‖ =

√∫ b

a

|f(x)|2 dx.

When applied to the difference of two functions f and g, i.e.

‖f − g‖2 =

∫ b

a

|f(x)− g(x)|2 dx

the norm is a way of measuring the distance between the two functions, analogous to theEuclidean distance for vectors:

‖x− y‖2 = (x1 − y1)2 + · · ·+ (xn − yn)2.

1Mathematicians will usually refer to a square integrable function as an ‘L2 function’.

MATH 353 LECTURE NOTES WEEK 9 INTRODUCTION TO FOURIER SERIES 7

Notice that this ‘distance’ ‖f − g‖ between f and g is a sort of weighted average of the areabetween the curves in the interval [a, b]. The quantity (??) is sometimes called the mean-square distance or mean-square error if g is some approximation to f .

Norm examples: Consider the norm for L2[−1, 1].

If f(x) = x2 then

‖f(x)‖2 =

∫ 1

−1x2 dx =

1

3x3∣∣∣1−1

=2

3.

If f(x) = |x| and g(x) = x then

‖f(x)− g(x)‖2 =

∫ 1

−1(x− |x|)2 dx =

∫ 0

−1(2x)2 dx =

4

3.

Note that f = g when x > 0 but they differ when x < 0.

Notation: For clarity, one sometimes writes ‖f‖2, ‖f‖L2 or ‖f‖L2[a,b]. Only this normwill be used, so we will just write ‖f‖. Be careful to keep track of the interval, because thenorm depends on it; e.g. if f(x) = 1 then ‖f‖ = 2 in L2[−1, 1] but ‖f‖ = 4 in L2[−2, 2].

Inner product: The inner product on L2[a, b] is

〈f, g〉 =

∫ b

a

f(x)g(x) dx.

Two functions are called orthogonal ‘with respect to the inner product’ if 〈f, g〉 = 0, anda set of functions {fn} is orthogonal if distinct pairs are all orthogonal (just as with Rn!).Unfortunately, there is no obvious intuition for orthogonal functions (compared to Rn, whereit means the two vectors are perpendicular).

Notation: Again, because this inner product is the only one we will work with, the ‘withrespect to...’ part will be omitted and the inner product used will be left implied.

Inner product examples: Consider the norm for L2[−1, 1]. Then

〈1, x〉 =

∫ 1

−1x dx = 0.

So the constant function 1 and the function x are orthogonal (in the interval [−1, 1]).

However,

〈1, x2〉 =

∫ 1

−1x2 dx =

2

3,

8 J. WONG (FALL 2017)

so 1 and x2 are not orthogonal. On the other hand, for g(x) = x2 − 1/3,

〈1, g〉 =

∫ 1

−1(x2 − 1/3) dx =

2

3− 2

3= 0.

This means that the set{1, x, x2 − 1/3}

is an orthogonal set in L2[−1, 1], whereas

{1, x, x2}is not.

2.2. L2[−`, `] as a space of 2`-periodic functions. A function f(x) is T -periodic (or‘has a period T ’) if

(9) f(x) = f(x+ T ) for all x.

That is, after a length T , the function repeats. A periodic function is therefore defined byits values on any interval of length T . For example, sinx has period 2π and cos(4πx) hasperiod 1/2.

Note that the period T as defined above is not unique (e.g. sin x is also periodic withperiod 4π). The fundamental period refers to the smallest possible T . The distinction isnot particularly important for the discussion here, however.

The point: We will often be interested in periodic functions defined for all x ∈ R. Ifthe period is 2` then we can identify such a periodic function with its restriction to [−`, `].Analysis of periodic functions is then done on that interval.

Similarly, any f ∈ L2[−`, `] corresponds to a 2`-periodic function defined by (9). Thatis, we construct a periodic function out of f by copying it on [`, 3`], then again on [3`, 5`]and so on. Thus we can view L2[−`, `] also as the space of 2`-periodic functions such thatthe integral of f 2 over one period is finite.

For example, consider the function f(x) = |x| in L2[−1, 1]. This corresponds to theperiodic function drawn below:

The periodic function sin(x), for instance, as a member of L2[−π, π] is continuous. If inL2[−π/2, π/2] however, sinx is not continuous (but is still in the space):

MATH 353 LECTURE NOTES WEEK 9 INTRODUCTION TO FOURIER SERIES 9

The values at the discontinuities −π/2, π/2 etc. are ambiguous.

3. Fourier series: fundamentals

In this section we consider the space L2[−`, `] as the space of 2`-periodic functions inthe sense of the previous section. In terms of the theory/computations, it will not matterwhether we think of f ∈ L2[−`, `] as really periodic on all of R or just defined on [−`, `] butit does matter conceptually for applications.

3.1. Definition. Following Fourier’s example, we observe that the set of functions

(10) constant, cos(πx

`), cos(

2πx

`), cos(

3πx

`), · · · sin(

πx

`), sin(

2πx

`), · · ·

forms an orthogonal set in L2[−`, `] with respect to the inner product

〈f, g〉 =

∫ `

−`f(x)g(x) dx.

Explicitly, the following identities hold (and are not too hard to prove; see homework):∫ `

−`cos

mπx

`sin

nπx

`dx = 0, for all m,n,(11) ∫ `

−`cos

mπx

`cos

nπx

`dx =

{0 m 6= n

` m = n and m 6= 0,(12)

∫ `

−`sin

mπx

`sin

nπx

`dx =

{0 m 6= n

` m = n.(13)

Note that the orthogonality of the constant with all the sines and cosines is contained in theabove (since cosmπx

`= const. when m = 0).

The Fourier series for a function f ∈ L2[−`, `] is given by

(14) f =a02

+∞∑n=1

(an cos

nπx

`+ bn sin

nπx

`

).

Note: we will study the sense in which f and the series are equal later. For now,we can use the orthogonality relations (11)-(13), to formally solve for the coefficients (i.e.ignoring any possible technical issues).

10 J. WONG (FALL 2017)

For am, take the inner product of both sides with cos mπx`

and use the linearity propertyto put the inner product inside the sum:

〈f, cosmπx

`〉 = 〈a0

2, cos

mπx

`〉+

∞∑n=1

(an〈cos

nπx

`, cos

mπx

`〉+ bn〈sin

nπx

`, cos

mπx

`〉)

Now by (11)-(13), all the terms on the right vanish except the n = m term in the cosinesum. If n ≥ 1 then

〈f, cosmπx

`〉 = an〈cos

mπx

`, cos

mπx

`〉.

Now employ (12) to evaluate the right side. We end up with the formula

(15) an =1

`

∫ `

−`f(x) cos

mπx

`dx, n ≥ 0.

Note that the a0 case has to be checked separately. Keeping the formula valid for n = 0 isthe reason for the choice of 1/2 has the constant basis function. Similarly, we get

(16) bn =1

`

∫ `

−`f(x) sin

mπx

`dx, n ≥ 1.

3.2. Periodic functions and the Fourier series. Note that the basis functions sin(nπx/`)etc. for L2[−`, `] are periodic with period 2`. Thus the Fourier series for f(x) defined on[−`, `] is always 2`-periodic as well! This fact makes a Fourier series a natural approximationfor a periodic function.

3.3. The main result. The fundamental result (which has profound implications in mathand physics) is that any function f in L2[−`, `] has a representation as a Fourier series.

In terms of a basis: Put another way, the set (10) of sines and cosines,

constant, cos(πx

`), cos(

2πx

`), cos(

3πx

`), · · · sin(

πx

`), sin(

2πx

`), · · ·

forms an orthogonal basis2 for L2[−`, `], which means that

(17) Every f ∈ L2[−`, `] can be written as f =a02

+∞∑n=1

(an cos

nπx

`+ bn sin

nπx

`

)with an, bn given by (15) and (16). The equality here is not in the sense that at each pointx, the series converges to f(x); we will resolve the ambiguity later.

In terms of periodic functions: The basis functions sin(nπx/`) etc. for L2[−`, `] areperiodic with period 2`. Thus the Fourier series for f(x) defined on [−`, `] is always 2`-periodic as well! This fact makes a Fourier series a natural approximation for a periodicfunction. Put yet another way, every reasonably nice periodic function is a superposition ofsines and cosines.

2Technical note just for completeness: the term ‘basis’ here is not exactly correct because it allows forinfinite linear combinations; there’s a slightly different term for this.

MATH 353 LECTURE NOTES WEEK 9 INTRODUCTION TO FOURIER SERIES 11

3.4. Partial sums; approximation and convergence (briefly). In general, the N -thpartial sum of the Fourier series for a periodic function f(x) is

SN(x) =

(a02

+N∑n=1

(an cos

nπx

`+ bn sin

nπx

`

))This partial sum is an approximation to f(x). In fact, it is often a very good approximation.The main result can therefore be rephrased yet again to say that every reasonably nice pe-riodic function can be approximated by a sum of sines and cosines.

Notation: ‘N -th partial sum’ might instead be defined to be the first N non-zero termsof the series rather than terms up to cosNπx/` and sinNπx/`; it depends on the series inquestion.

This notion of approximation lets us describe the equality in precisely; it means that

‖f − SN‖ → 0 as N →∞.That is, the ‘mean square error’ (i.e. the distance between f and SN as measured in the L2

norm)

‖f − SN‖2 =

∫ `

−`|f(x)− SN(x)|2 dx

goes to zero as we increase the number of terms in the approximation.

Because the error takes into account the difference |f(x) − SN(x)| at every point in theinterval, we would like to say that the convergence implies that the error goes to zero uni-formly, i.e.

maxx∈[−`,`]

|f(x)− SN(x)| → 0.

One might care about the above if trying to use a Fourier series to approximate a function(it should be a good estimate everywhere!). If f(x) is nice enough this is true, but it is nottrue in general. The distinction between the various notions of convergence is subtle; wewill consider it in more detail later after becoming better acquainted with Fourier series.

3.5. Examples. Some examples help to illustrate the defintions of the previous section.

3.5.1. Triangular wave. Let

f(x) =

{−x −1 ≤ x < 0

x 0 < x ≤ 1

and f(x) = f(x+ 2) when x /∈ [−1, 1]. This is the same as before:

12 J. WONG (FALL 2017)

To compute the Fourier series (note that ` = 1), we use (11)-(13). Some integration by partsis involved; in particular the formula∫

x cos ax dx =1

ax sin ax+

1

a2cos ax.

When n ≥ 1,

an =1

`

∫ `

−`f(x) cos(nπx) dx

= −∫ 0

−1x cos(nπx) dx+

∫ 1

0

x cos(nπx) dx

= 2

∫ 1

0

x cos(nπx) dx(change x→ −x in first term)

=[ 2

nπx sin(nπx) +

2

n2π2cos(nπx)

]∣∣∣∣∣1

0

=2

n2π2cos(nπx)

∣∣∣10

(since sin(nπ) = 0 for all n)

=2

n2π2((−1)n − 1).

Thus for n ≥ 1,

an =

{− 4n2π2 for odd n

0 for even n.

For a0 just compute

a0 =

∫ 1

−1f(x) dx = 2

∫ 1

0

x dx = 1.

For the sine terms, there is cancellation:

bn =1

`

∫ `

−`f(x) sin(nπx) dx

= −∫ 0

−1x sin(nπx) dx+

∫ 1

0

x sin(nπx) dx

= 0.

The two terms exactly cancel. The Fourier series representation for f is therefore

f(x) =1

2− 4

π2

∞∑n=1

1

(2n− 1)2cos((2n− 1)πx).

Now let’s define the partial sum containing N non-zero terms as follows (not the same asthe general definition earlier to avoid terms that are zero):

SN(x) =1

2− 4

π2

N∑n=1

1

(2n− 1)2cos((2n− 1)πx)

so S1 = 12− 4

π2 cosπx, S2 = 12− 4

π2 (cosπx+ 19

cos 3πx) and so on.

MATH 353 LECTURE NOTES WEEK 9 INTRODUCTION TO FOURIER SERIES 13

A plot of the approximations shows that the agreement is quite good, even with only afew terms:

-3 -2 -1 0 1 2 3

x

0

0.2

0.4

0.6

0.8

11 term2 terms3 terms

A plot of the mean-square error

‖f − SN‖2 =

∫ 1

−1|f(x)− SN(x)|2 dx

shows that the mean-square error does indeed go to zero as N →∞. In fact, it goes to zerorather quickly (exponentially fast!). The maximum error

maxx∈[−`,`]

|f(x)− SN(x)|

also goes to zero as N → ∞, but much slower; the agreement is not so good near the peakof the triangle where there is a sharp corner.

0 10 20 30 40 50

N

10 -7

10 -6

10 -5

10 -4

10 -3

10 -2

10 -1

M.S. errormax. error

14 J. WONG (FALL 2017)

3.5.2. Square wave. Let

f(x) =

{−1 −1 ≤ x < 0

1 0 < x ≤ 1

and f(x) = f(x+ 2) when x /∈ [−1, 1]. Then a0 = 0 and

an =

∫ 1

−1f(x) cos(nπx) dx = −

∫ 0

−1cos(nπx) dx+

∫ 1

0

cos(nπx) dx = 0,

bn =

∫ 1

−1f(x) sin(nπx) dx

=

∫ 0

−1sin(nπx) dx+

∫ 1

0

sin(nπx) dx

= 2

∫ 1

0

sin(nπx) dx

= − 2

nπcos(nπx)

∣∣∣10

= − 2

nπ((−1)n − 1)

so bn = 4/nπ when n is odd and bn = 0 when n is even. Thus the Fourier series for f(x) is

f(x) =4

π

∞∑n=1

1

2n− 1sin((2n− 1)πx).

The Nth partial sum SN Is

SN(x) =4

π

N∑n=1

1

2n− 1sin((2n− 1)πx).

Error: A plot of the approximation shows that the partial sums converge nicely wheref is continuous, but do not perform well at all near the discontinuity:

-3 -2 -1 0 1 2 3

x

-1

-0.5

0

0.5

1 2 terms4 terms12 terms

MATH 353 LECTURE NOTES WEEK 9 INTRODUCTION TO FOURIER SERIES 15

The partial sums tend to oscillate and overshoot the discontinuity by a significant amount.Again, define Sm(x) to be the first m terms of the series above, e.g. S2 = 4

π(sin πx+ 1

3sin 3πx).

Then as in the previous example,

‖f − SN‖ → 0 as N →∞.Zooming in on the top part of the discontinuity at x = 1:

0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1

x

0.8

0.85

0.9

0.95

1

1.05

1.1

1.15

1.2

2 terms4 terms12 terms24 terms100 terms

As the number of terms increases, the overshoot does not decrease in magnitude. The am-plitude of the bad oscillations stays about the same; the width, however, shrinks. Thus ateach point x 6= 1, the series will converge to f(x), but the maximum error is

maxx∈[−1,1]

|f(x)− SN(x)| ≈ 0.18 as N →∞.

16 J. WONG (FALL 2017)

4. Review: Inner products and orthogonality in Rn

Reviewing the relevant ideas in Rn is helpful. Recall that the 2-norm (or Euclidean norm)of a vector x is

‖x‖ =√x21 + x22 + · · ·+ x2n.

The quantity ‖x − y‖ measures the distance between two vectors in x,y ∈ Rn (which isliterally the distance between the two points x and y), The inner product (or dot product)of two vectors is

〈x,y〉 = x1y1 + · · ·+ xnynalso denoted x · y. The inner product on Rn has several important properties:

i) Symmetry:〈x,y〉 = 〈y,x〉.

ii) Linearity in each argument: If x, y, z are vectors then

〈x + y, z〉 = 〈x, z〉+ 〈y, z〉 and 〈x,y + z〉 = 〈x,y〉+ 〈x, z〉.If c is a scalar then

〈cx,y〉 = c〈x,y〉 and 〈x, cy〉 = c〈x,y〉.iii) Positive definiteness:

〈x,x〉 ≥ 0 and 〈x,x〉 = 0 =⇒ x = 0.

Note that the Euclidean norm is given by

‖x‖ =√〈x,x〉.

Property (iii) ensures that ‖x‖ = 0 if and only if x = 0.

Two vectors x,y ∈ Rn are orthogonal if their inner product is zero:

x,y orthogonal ⇐⇒ 〈x,y〉 = 0.

A basis v1, · · · ,vn for Rn is called orthogonal if every pair of distinct basis elements isorthogonal, i.e.

{vj} is an orthogonal basis if 〈vj,vk〉 = 0 for j 6= k.

MATH 353 LECTURE NOTES WEEK 9 INTRODUCTION TO FOURIER SERIES 17

5. A preview of future topics

In the coming weeks, we will develop a method inspired by the above to solve

ut = −L[u] + h, x ∈ (a, b) and t > 0

with some initial condition u(x, 0) = u0(x) and boundary conditions at x = a and x = b.

The differential operator L contains only x-derivatives, e.g L = − d2

dx2. It will be the same

sort of operator we encountered in studying second-order linear equations.

We can identify some key step from Fourier’s heat conduction problem:

A) Find ‘eigenfunctions’ that solve a boundary value problem of the form

L[φn] = λnφn, with some conditions on φ at x = a and x = b.

B) Represent the solution as a sum of coefficients (depending on time) times eigenfunc-tions:

u =∞∑n=0

an(t)φn(x).

C) Find (simple) equations for the coefficients and solve them to get the solution.

To make (C) work, we will need the functions φn to have specific properties, namely, some-thing like (7) that will make sure the equation for cn(t) decouples from the others. InFourier’s example, the equation is

a′n = −λnan, an(0) = coeff. of φn in the initial condition

and the ‘eigenvalue’ is λn = n2.

In a sense, we are converting (complicated) linear PDEs into a linear algebra problem plusan ODE problem. Doing so will require developing substantial theory in several areas:

• Vector spaces of functions, orthogonality: How do we make sense of series like

∞∑n=0

cnφn(x)

and extend the notion of a basis to functions? Where does the condition (7) comefrom? Essentially, we will extend familiar results from linear algebra in Rn to spacesof functions.• Boundary value problems / eigenvalue problems: What is the structure of

ODEs when the boundary conditions are at different points (not IVPs)? Theseproblems are quite different from the IVPs of previous weeks! We will also developtheory for eigenvalue problems (what properties do the eigenvalues and eigenfunctionshave?).• Important PDEs (what important equations can be solved this way?):

There are three essential PDEs (the heat equation, wave equation, and Laplace’sequation) that can be studied using the framework we develop here. These threeequations describe a wide range of phenomena in physics, engineering, and more.

18 J. WONG (FALL 2017)

• Fourier series (an extremely important special case): The case where theeigenfunctions are sines/cosines is of critical importance - with implications that gofar beyond their use as a tool for solving PDEs. We will look at such series (Fourierseries) in detail.