Midterm II Review Session Slides

transcript

Review for Midterm II

Math 20

December 4, 2007

Announcements

I Midterm I 12/6, Hall A 7–8:30pm

I ML Office Hours Wednesday 1–3 (SC 323)

I Old exams and solutions on website

Outline

Rank and other Linear AlgebraLinear dependenceRank

EigenbusinessEigenvector and EigenvalueDiagonalizationThe Spectral Theorem

Functions of several variables

Graphing/Contour PlotsPartial Derivatives

DifferentiationThe Chain RuleImplicit Differentiation

OptimizationUnconstrained OptimizationConstrained Optimization

Rank and other Linear AlgebraLearning Objectives

I Determine whether a set of vectors is linearly independent

I Find the rank of a matrix

Linear Independence

DefinitionLet {a1, a2, . . . , an} be a set of vectors in Rm. We say they arelinearly dependent if there exist constants c1, c2, . . . , ck ∈ R, notall zero, such that

c1a1 + c2a2 + · · ·+ ckan = 0.

If the equation only holds when all c1 = c2 = · · · = cn = 0, thenthe vectors are said to be linearly independent.

Deciding linear dependence

We showed

a1, . . . , an LD ⇐⇒ c1a1 + · · ·+ cnan = 0 has a nonzero sol’n

⇐⇒(a1 . . . an

)︸︷︷︸A

︸︷︷︸

= 0 has a nonzero sol’n

⇐⇒ system has some free variables

⇐⇒ rref(A) has a column with no leading entry to it

Example

Determine if the vectors101

3−22

are linearly dependent.

Solution

0 − 2 2

←−

1 3 00 −2 20 −1 1

0 1 − 1

←−−3

0 1 − 1

So the vectors are linearly dependent.

Example

3−22

Solution

0 − 2 2

←−

1 3 00 −2 20 −1 1

0 1 − 1

←−−3

0 1 − 1

Example

3−22

Solution

0 − 2 2

←−

1 3 00 −2 20 −1 1

0 1 − 1

←−−3

0 1 − 1

Deciding linear independence

a1, . . . , an LI ⇐⇒ every column of rref(A) has a leading entry to it

⇐⇒ A ∼(

Example

3−220

021−1

Solution

0 − 2 2

− 1 0 1

0 1 −1

− 1 0 1

←−

0 1 −1

0 0 − 2

So the vectors are linearly independent.

DefinitionThe rank of a matrix A, written r(A) is the maximum number oflinearly independent column vectors in A.

If A is a zero matrix, wesay r(A) = 0.

DefinitionThe rank of a matrix A, written r(A) is the maximum number oflinearly independent column vectors in A. If A is a zero matrix, wesay r(A) = 0.

Example

1 3 00 −2 21 2 1

1 0 30 1 −10 0 0

this matrix has rank 2.

Example

1 3 00 −2 21 2 1−1 0 1

1 0 00 1 00 0 10 0 0

this matrix has rank 3.

Another way to compute rank

TheoremBook Theorem 14.1 The rank of A is the size of the largestnonvanishing minor of A.

Rank and consistency

FactLet A be an m× n matrix, b an n× 1 vector, and Ab the matrix Aaugmented by b.Then the system of linear equations Ax = b has a solution (isconsistent) if and only if r(A) = r(Ab).

Rank and redundancy

FactLet A be an m× n matrix, b an n× 1 vector, and Ab the matrix Aaugmented by b. Suppose that r(A) = r(Ab) = k < m (m is thenumber of equations in the system Ax = b).Then m − k of the equations are redundant; they can be removedand the system has the same solutions.

Rank and freedom

FactLet A be an m× n matrix, b an n× 1 vector, and Ab the matrix Aaugmented by b. Suppose that r(A) = r(Ab) = k < n (n is thenumber of variables in the system Ax = b).Then n − k of the variables are free; they can be chosen at willand the rest of the variables depend on them, getting infinitelymany solutions.

Outline

EigenbusinessLearning Objectives

I Determine if a vector is an eigenvalue of a matrix

I Determine if a scalar is an eigenvalue of a matrix

I Find all the eigenvalues of a matrix

I Find all the eigenvectors of a matrix for a given eigenvalue

I Diagonalize a matrix

I Know when a matrix is diagonalizable

Eigenbusiness

DefinitionLet A be an n × n matrix. The number λ is called an eigenvalueof A if there exists a nonzero vector x ∈ Rn such that

Ax = λx. (1)

Every nonzero vector satisfying (1) is called an eigenvector of Aassociated with the eigenvalue λ.

ExampleMidterm II, Fall 2006, Problem 4

Let A =

(4 −21 1

)Problem

)an eigenvector for A?

SolutionUse the definition of eigenvector:(

4 −21 1

)So the vector is an eigenvector corresponding to the eigenvalue 3.

ExampleMidterm II, Fall 2006, Problem 4

Let A =

(4 −21 1

)Problem

)an eigenvector for A?

SolutionUse the definition of eigenvector:(

4 −21 1

)So the vector is an eigenvector corresponding to the eigenvalue 3.

Let A =

(4 −21 1

)ProblemIs 0 an eigenvalue for A?

SolutionThe number 0 is an eigenvalue for A if and only if the determinantof A− 0I = A is zero. But

det A = 4 · 1− 1 · (−2) = 6.

So it’s not.

Let A =

(4 −21 1

)ProblemIs 0 an eigenvalue for A?

SolutionThe number 0 is an eigenvalue for A if and only if the determinantof A− 0I = A is zero. But

det A = 4 · 1− 1 · (−2) = 6.

So it’s not.

Methods

I To find the eigenvalues of a matrix A, find the determinant ofA− λI. This will be a polynomial in λ (called thecharacteristic polynomial of A, and its roots are theeigenvalues.

I To find the eigenvector(s) of a matrix corresponding to aneigenvalue λ, do Gaussian Elimination on A− λI.

Diagonalization Procedure

I Find the eigenvalues and eigenvectors.

I Arrange the eigenvectors in a matrix P and the correspondingeigenvalues in a diagonal matrix D.

I If you have “enough” eigenvectors so that the matrix P issquare and invertible, the original matrix is diagonalizable andequal to PDP−1.

Example

Problem

Let A =

(2 32 1

). Diagonalize.

SolutionTo find the eigenvalues, find the characteristic polynomial and itsroots:

|A− λI| =

∣∣∣∣2− λ 32 1− λ

∣∣∣∣ = (2− λ)(1− λ)− 6

= λ2 − 3λ− 4 = (λ+ 1)(λ− 4)

So the eigenvalues are −1 and 4.

Example

Problem

Let A =

(2 32 1

). Diagonalize.

SolutionTo find the eigenvalues, find the characteristic polynomial and itsroots:

|A− λI| =

∣∣∣∣2− λ 32 1− λ

∣∣∣∣ = (2− λ)(1− λ)− 6

= λ2 − 3λ− 4 = (λ+ 1)(λ− 4)

So the eigenvalues are −1 and 4.

To find an eigenvector corresponding to the eigenvalue −1,

A + I =

(3 32 2

(1 10 0

(1−1

)is an eigenvector.

To find an eigenvector corresponding to the eigenvalue 4,

A− 4I =

(−2 32 −3

(1 −3/20 0

)is an eigenvector.

(1 3−1 2

P−1 =1

(2 −31 1

(1 3−1 2

)(−1 00 4

(2 −31 1

The Spectral Theorem

Theorem (Baby Spectral Theorem)

Suppose An×n has n distinct real eigenvalues. Then A isdiagonalizable.

Theorem (Spectral Theorem for Symmetric Matrices)

Suppose An×n is symmetric, that is, A′ = A. Then A isdiagonalizable. In fact, the eigenvectors can be chosen to bepairwise orthogonal with length one, which means that P−1 = P′.Thus a symmetric matrix can be diagonalized as

A = PDP′,

The Spectral Theorem

Theorem (Baby Spectral Theorem)

Suppose An×n has n distinct real eigenvalues. Then A isdiagonalizable.

Theorem (Spectral Theorem for Symmetric Matrices)

Suppose An×n is symmetric, that is, A′ = A. Then A isdiagonalizable. In fact, the eigenvectors can be chosen to bepairwise orthogonal with length one, which means that P−1 = P′.Thus a symmetric matrix can be diagonalized as

A = PDP′,

Outline

Functions of several variablesLearning Objectives

I identify functions, graphs, and contour plots

I find partial derivatives of functions of several variables

Types of functions

I linear

I polynomial

I rational

I Cobb-Douglas

I etc.

Examples

ProblemIn each of the following, find the domain and range of the function.Is it linear? polynomial? rational? algebraic? Cobb-Douglas?

(a) f (x , y) = y − x

(b) f (x , y) =√

y − x

(c) f (x , y) = 4x2 + 9y2

(d) f (x , y) = x2 − y2

(e) f (x , y) = xy

(f) f (x , y) = y/x2

(g) f (x , y) =1√

16− x2 − y2

(h) f (x , y) =√

9− x2 − y2

(i) f (x , y) = ln(x2 + y2)

(j) f (x , y) = e−(x2+y2)

(k) f (x , y) = arcsin(y − x)

(l) f (x , y) = arctan( y

Graphing/Contour Plots

A function of two variables can be visualized by

I its graph: the surface (x , y , f (x , y) in R3

I a contour plot: a collection of level curves

Example

Graph and contour plot of f (x , y) = y − x

-2 -1 0 1 2

Example

Graph and contour plot of f (x , y) = y − x

-2 -1 0 1 2

Example

Graph and contour plot of f (x , y) =√

y − x

-2 -1 0 1 2

Example

y − x

-2 -1 0 1 2

Example

Graph and contour plot of f (x , y) = 4x2 + 9y2

-2 -1 0 1 2

Example

Graph and contour plot of f (x , y) = 4x2 + 9y2

-2 -1 0 1 2

Example

Graph and contour plot of f (x , y) = x2 − y2

-2 -1 0 1 2

Example

Graph and contour plot of f (x , y) = x2 − y2

-2 -1 0 1 2

Example

Graph and contour plot of f (x , y) = xy

-2 -1 0 1 2

Example

Graph and contour plot of f (x , y) = xy

-2 -1 0 1 2

Example

Graph and contour plot of f (x , y) = yx2

-2 -1 0 1 2

Example

Graph and contour plot of f (x , y) = yx2

-2 -1 0 1 2

Example

Graph and contour plot of f (x , y) =1√

16− x2 − y2

-4 -2 0 2 4

Example

Graph and contour plot of f (x , y) =1√

16− x2 − y2

-4 -2 0 2 4

Example

9− x2 − y2

-3 -2 -1 0 1 2 3

Example

9− x2 − y2

-3 -2 -1 0 1 2 3

Example

Graph and contour plot of f (x , y) = ln(x2 + y2)

-3 -2 -1 0 1 2 3

Example

Graph and contour plot of f (x , y) = ln(x2 + y2)

-3 -2 -1 0 1 2 3

Example

Graph and contour plot of f (x , y) = e−(x2+y2)

-3 -2 -1 0 1 2 3

Example

Graph and contour plot of f (x , y) = e−(x2+y2)

-3 -2 -1 0 1 2 3

Example

Graph and contour plot of f (x , y) = arcsin(y − x)

-2 -1 0 1 2

Example

Graph and contour plot of f (x , y) = arcsin(y − x)

-2 -1 0 1 2

Example

Graph and contour plot of f (x , y) = arctan( y

-2 -1 0 1 2

Example

Graph and contour plot of f (x , y) = arctan( y

-2 -1 0 1 2

The process

To differentiate a function of several variables with respect to oneof the variables, pretend that the others are constant.

Examples

Example

Let f (x , y) = 3x + 2xy2 − 2y4. Find both the partial derivatives off .

SolutionWe have

∂x= 3 + 2y2 ∂f

∂y= 4xy − 8y3

Examples

Example

Let f (x , y) = 3x + 2xy2 − 2y4. Find both the partial derivatives off .

SolutionWe have

∂x= 3 + 2y2 ∂f

∂y= 4xy − 8y3

Example

Let w = sinα cosβ. Find both the partial derivatives of w .

SolutionWe have

∂α= cosα cosβ

∂β= − sinα sinβ

Example

Let w = sinα cosβ. Find both the partial derivatives of w .

SolutionWe have

∂α= cosα cosβ

∂β= − sinα sinβ

Example

Let f (u, v) = arctan(u/v). Find both the partial derivatives of f .

SolutionFor this it’s important to remember the chain rule!

1 + (u/v)2

Another way to write this is

u2 + v2

Example

Let f (u, v) = arctan(u/v). Find both the partial derivatives of f .

SolutionFor this it’s important to remember the chain rule!

1 + (u/v)2

Another way to write this is

u2 + v2

Example

Let u =√

x21 + x2

2 + · · ·+ x2n . Find all the derivatives of u.

SolutionWe have a partial derivative for each index i , but luckily they’resymmetric. So each derivative is represented by:

∂xi=

x21 + x2

2 + · · ·+ x2n

∂xi(x2

1 + x22 + · · ·+ x2

=xi√

x21 + x2

2 + · · ·+ x2n

Example

Let u =√

x21 + x2

2 + · · ·+ x2n . Find all the derivatives of u.

SolutionWe have a partial derivative for each index i , but luckily they’resymmetric. So each derivative is represented by:

∂xi=

x21 + x2

2 + · · ·+ x2n

∂xi(x2

1 + x22 + · · ·+ x2

=xi√

x21 + x2

2 + · · ·+ x2n

Example

Let f (x , y) = 3x + 2xy2 − 2y4. Find all the second derivatives.

Solution

∂x2= 0

∂x ∂y= 4y

∂y ∂x= 4y

∂y2= −24y2

Example

Let f (x , y) = 3x + 2xy2 − 2y4. Find all the second derivatives.

Solution

∂x2= 0

∂x ∂y= 4y

∂y ∂x= 4y

∂y2= −24y2

Tangent Planes

FactThe tangent plane to z = f (x , y) through (x0, y0, z0 = f (x0, y0))has normal vector (f ′1(x0, y0), f ′2(x0, y0),−1) and equation

f ′1(x0, y0)(x − x0) + f ′2(x0, y0)(y − y0)− (z − z0) = 0

orz = f (x0, y0) + f ′1(x0, y0)(x − x0) + f ′2(x0, y0)(y − y0)

I This is the best linear approximation to f near (x0, y0).

I is is the first-degree Taylor polynomial (in two variables) for f .

Tangent Planes

f ′1(x0, y0)(x − x0) + f ′2(x0, y0)(y − y0)− (z − z0) = 0

Tangent Planes

f ′1(x0, y0)(x − x0) + f ′2(x0, y0)(y − y0)− (z − z0) = 0

Outline

Fact (The Chain Rule, version I)

When z = F (x , y) with x = f (t) and y = g(t), then

z ′(t) = F ′1(f (t), g(t))f ′(t) + F ′2(f (t), g(t))g ′(t)

dt=∂F

dt+∂F

We can generalize to more variables, too. If F is a function ofx1, x2, . . . , xn, and each xi is a function of t, then

dt=∂F

dt+∂F

dt+ · · ·+ ∂F

Fact (The Chain Rule, version I)

When z = F (x , y) with x = f (t) and y = g(t), then

z ′(t) = F ′1(f (t), g(t))f ′(t) + F ′2(f (t), g(t))g ′(t)

dt=∂F

dt+∂F

We can generalize to more variables, too. If F is a function ofx1, x2, . . . , xn, and each xi is a function of t, then

dt=∂F

dt+∂F

dt+ · · ·+ ∂F

Tree Diagrams for the Chain Rule

∂F∂x

∂F∂y

To differentiate with respect to t, find all “leaves” marked t.Going down each branch, chain (multiply) all the derivativestogether. Then add up the result from each branch.

dt=∂F

dt+∂F

Fact (The Chain Rule, Version II)

When z = F (x , y) with x = f (t, s) and y = g(t, s), then

∂t=∂F

∂t+∂F

∂t∂z

∂s=∂F

∂s+∂F

Example

Suppose z = xy2, x = t + s and y = t − s. Find ∂z∂t and ∂z

∂s at(t, z) = (1/2, 1) in two ways:

(i) By expressing z directly in terms of t and s beforedifferentiating.

(ii) By using the chain rule.

Solution (i)

We have

z = (t + s)(t − s)2 = s3 − ts2 − t2s + t3

∂t= −s2 − 2ts + 3t2

∂s= 3s2 − 2ts − t2

Example

Suppose z = xy2, x = t + s and y = t − s. Find ∂z∂t and ∂z

∂s at(t, z) = (1/2, 1) in two ways:

(i) By expressing z directly in terms of t and s beforedifferentiating.

(ii) By using the chain rule.

Solution (i)

We have

z = (t + s)(t − s)2 = s3 − ts2 − t2s + t3

∂t= −s2 − 2ts + 3t2

∂s= 3s2 − 2ts − t2

Solution (ii)

We havez = xy2 x = t + s y = t − s

∂t=∂z

∂t+∂z

= y2 · 1 + 2xy · 1 = (t − s)2 + 2(t + s)(t − s)

∂s=∂z

∂s+∂z

= y2 · 1 + 2xy(−1) = (t − s)2 − 2(t + s)(t − s)

These should be the same as in the previous calculation.

Theorem (The Chain Rule, General Version)

Suppose that u is a differentiable function of the n variablesx1, x2, . . . , xn, and each xi is a differentiable function of the mvariables t1, t2, . . . , tm. Then u is a function of t1, t2, . . . , tm and

∂ti=

∂ti+∂u

∂ti+ · · ·+ ∂u

In summation notation

∂ti=

n∑j=1

Implicit DifferentiationThe Big Idea

FactAlong the level curve F (x , y) = c, the slope of the tangent line isgiven by

= −∂F/∂x

∂F/∂x= −F ′1(x , y)

F ′2(x , y)

Tree diagram

∂x+∂F

More than two variables

The basic idea is to close your eyes and use the chain rule:

Example

Suppose a surface is given by F (x , y , z) = c . If this defines z as afunction of x and y , find z ′x and z ′y .

SolutionSetting F (x , y , z) = c and remembering z is implicitly a functionof x and y, we get

∂x+∂F

= 0 =⇒(∂z

= −F ′xF ′z

∂y+∂F

= 0 =⇒(∂z

= −F ′yF ′z

More than two variables

The basic idea is to close your eyes and use the chain rule:

Example

Suppose a surface is given by F (x , y , z) = c . If this defines z as afunction of x and y , find z ′x and z ′y .

SolutionSetting F (x , y , z) = c and remembering z is implicitly a functionof x and y, we get

∂x+∂F

= 0 =⇒(∂z

= −F ′xF ′z

∂y+∂F

= 0 =⇒(∂z

= −F ′yF ′z

Tree diagram

∂x+∂F

= 0 =⇒(∂z

= −F ′xF ′z

ExampleProblem 16.8.4

ProblemLet D = f (r ,P) denote the deman for an agricultural commoditywhen the price is P and r is the producers’ total advertisingexpenditure. Let supply be given by S = g(w ,P), where w is anindex for how favorable the weather has been. Assumeg ′w (w ,P) > 0. Equilibrium now requires f (r ,P) = g(w ,P).Assume that this equation defines P implicitly as a differentiablefunction of r and w. Compute P ′w and comment on its sign.

Solution

We havef (r ,P)− g(w ,P) ≡ 0

f − g

− ∂g

∂w− ∂g

Answer

So (∂P

∂w∂f

∂P− ∂g

∂P< 0 and

∂P> 0. We assumed that

∂w> 0. So in this case,(

meaning the price decreasing with improving weather.

Outline

OptimizationLearning Objectives

I Find the critical points of a function defined on an open set(so unconstrained)

I Classify the critical points of a function

I Find the critical points of a function restricted to a surface(constrained)

Theorem (Fermat’s Theorem)

Let f (x , y) be a function of two variables. If f has a localmaximum or minimum at (a, b), and is differentiable at (a,b), then

∂x(a, b) = 0

∂y(a, b) = 0

As in one variable, we’ll call these points critical points.

Theorem (The Second Derivative Test)

Let f (x , y) be a function of two variables, and let (a, b) be acritical point of f . Then

I If ∂2f∂x2

∂2f∂y2 −

(∂2f

∂x∂y

)2> 0 and ∂2f

∂x2 > 0, the critical point is a

local minimum.

I If ∂2f∂x2

∂2f∂y2 −

(∂2f

∂x∂y

)2> 0 and ∂2f

∂x2 < 0, the critical point is a

local maximum.

I If ∂2f∂x2

∂2f∂y2 −

(∂2f

∂x∂y

)2< 0, the critical point is a saddle point.

All derivatives are evaluated at the critical point (a, b).

Example

ProblemFind and classify the critical points of

f (x , y) = 4xy − x4 − y4

SolutionWe have ∂f

∂x = 4y − 4x3 and ∂f∂y = 4x − 4y3. Both of these are

zero when y = x3 and x = y3 So x9 = x. Since

x9 − x = x(x8 − 1) = x(x4 + 1)(x2 + 1)(x + 1)(x − 1)

the real solutions are x = 0, x = 1, and x = −1. Thecorresponding y values are 0, 1, and −1. So the critical points are

(0, 0), (1, 1), (−1,−1)

Example

ProblemFind and classify the critical points of

f (x , y) = 4xy − x4 − y4

SolutionWe have ∂f

∂x = 4y − 4x3 and ∂f∂y = 4x − 4y3. Both of these are

zero when y = x3 and x = y3 So x9 = x. Since

x9 − x = x(x8 − 1) = x(x4 + 1)(x2 + 1)(x + 1)(x − 1)

the real solutions are x = 0, x = 1, and x = −1. Thecorresponding y values are 0, 1, and −1. So the critical points are

(0, 0), (1, 1), (−1,−1)

The second derivatives are

∂x2= −12x2 ∂2f

∂y ∂x= 4

∂x ∂y= 4

∂y ∂x= −12y2

H(x , y) = 4

(−3x2 1

1 −3y2

)At (0, 0), the matrix is

(0 11 0

), which has determinant < 0. So

it’s a saddle point. At the other two points, the matrix is(−3 11 −3

), which has positive determinant. So those points are

local maxima.

Graph and contour plot of f (x , y) = 4xy − x4 − y4

-2 -1 0 1 2

Theorem (The Method of Lagrange Multipliers)

Let f (x1, x2, . . . , xn) and g(x1, x2, . . . , xn) be functions of severalvariables. The critical points of the function f restricted to the setg = 0 are solutions to the equations:

∂xi(x1, x2, . . . , xn) = λ

∂xi(x1, x2, . . . , xn) for each i = 1, . . . , n

g(x1, x2, . . . , xn) = 0.

Note that this is n + 1 equations in the n + 1 variables.x1, . . . , xn, λ.

ProblemFind the critical points and values of

f (x , y) = ax2 + 2bxy + cy2

subject to the constraint that x2 + y2 = 1.

SolutionWe have

f ′x = λg ′x =⇒ 2ax + 2by = λ(2x)

f ′y = λg ′y =⇒ 2bx + 2cy = λ(2y)

So the critical points happen when(a bb c

ProblemFind the critical points and values of

f (x , y) = ax2 + 2bxy + cy2

subject to the constraint that x2 + y2 = 1.

SolutionWe have

f ′x = λg ′x =⇒ 2ax + 2by = λ(2x)

f ′y = λg ′y =⇒ 2bx + 2cy = λ(2y)

So the critical points happen when(a bb c

The critical values are

f (x , y) =(x y

)(a bb c

)=(x y

)= λ(x2 + y2) = λ

I The critical points are eigenvectors!

I The critical values are eigenvalues!

The critical values are

f (x , y) =(x y

)(a bb c

)=(x y

)= λ(x2 + y2) = λ

I The critical points are eigenvectors!

I The critical values are eigenvalues!

Midterm II Review Session Slides

Technology