Day One: Derivatives, Optimization, and...

transcript

Day One: Derivatives, Optimization, and Integrals

SOC Methods Camp

September 3rd, 2019

SOC Methods Camp Day One: Derivatives, Optimization, and Integrals September 3rd, 2019 1 / 1

Outline

Review and application of derivatives

I Chain ruleI Derivatives of logs/exponentsI Two tools important for optimization

F Higher-order derivativesF Partial derivatives

I Basic optimizationReview and application of integrals

Outline

Review and application of derivativesI Chain rule

I Derivatives of logs/exponentsI Two tools important for optimization

Outline

Review and application of derivativesI Chain ruleI Derivatives of logs/exponents

I Two tools important for optimization

Outline

Review and application of derivativesI Chain ruleI Derivatives of logs/exponentsI Two tools important for optimization

Outline

F Higher-order derivatives

F Partial derivativesI Basic optimization

Review and application of integrals

Outline

I Basic optimization

Outline

Review of topics covered in summer assignment

Interval notation, domain, and range

An interval is a connected subset of the real number line. There areinfnite real numbers that exist between two points on the number line(say, between 0 and 15) so really, what’s relevant for our descriptionare these two end points.

If we use intervals to describe the range of a variable in a dataset, it isokay if the interval encompasses values that don’t actually exist in thedataset.The domain and range of functions are often expressed in intervalnotation. These important intervals represent the range of possiblevalues that x or y can take on.Some common sets for domain/range in interval notation:

I all real numbers (the entire number line): (−∞,∞)I all real numbers except 0: (−∞, 0) ∪ (0,∞)

An interval is a connected subset of the real number line. There areinfnite real numbers that exist between two points on the number line(say, between 0 and 15) so really, what’s relevant for our descriptionare these two end points.If we use intervals to describe the range of a variable in a dataset, it isokay if the interval encompasses values that don’t actually exist in thedataset.

The domain and range of functions are often expressed in intervalnotation. These important intervals represent the range of possiblevalues that x or y can take on.Some common sets for domain/range in interval notation:

An interval is a connected subset of the real number line. There areinfnite real numbers that exist between two points on the number line(say, between 0 and 15) so really, what’s relevant for our descriptionare these two end points.If we use intervals to describe the range of a variable in a dataset, it isokay if the interval encompasses values that don’t actually exist in thedataset.The domain and range of functions are often expressed in intervalnotation. These important intervals represent the range of possiblevalues that x or y can take on.

Some common sets for domain/range in interval notation:

An interval is a connected subset of the real number line. There areinfnite real numbers that exist between two points on the number line(say, between 0 and 15) so really, what’s relevant for our descriptionare these two end points.If we use intervals to describe the range of a variable in a dataset, it isokay if the interval encompasses values that don’t actually exist in thedataset.The domain and range of functions are often expressed in intervalnotation. These important intervals represent the range of possiblevalues that x or y can take on.Some common sets for domain/range in interval notation:

I all real numbers (the entire number line): (−∞,∞)

I all real numbers except 0: (−∞, 0) ∪ (0,∞)

Why derivatives?

Derivatives might seem like a relic from our high school days withunclear relevance for statistics

But consider the following very applied (stylized) cases

Why derivatives?

Derivatives might seem like a relic from our high school days withunclear relevance for statisticsBut consider the following very applied (stylized) cases

Application one: investigating rates of change underdifferent conditions

Scenario: You want to construct a formal model, similar to the blackversus white population change model we reviewed in the summerassignment, where:

the x axis represents the number of an individual’s friends who havejoined a protestthe y axis represents an individual’s probability of joining the protest.

You want to see how the rate at which the individual’s probability of joiningthe protest changes (the slope of the curve) at varying number of friendsunder different assumptions about the strength of influence friends have onprotesting

Application one: investigating rates of change underdifferent conditions

Weaker social influence

Stronger social influence

0 50 100 150 200Number of friends at protest

Application two: inferring unobserved weights thatgenerate observed patternsScenario: You’re fitting a model to assign weights to different variables(e.g., education; rural versus urban residence) in predicting whether or not aperson will develop an opioid addiction (1 = yes; 0 = no). You have a setof data on who developed addiction and their demographic characteristics,and want to find weights for each characteristic in a way that maximizes theprobability of observing the correct pattern of 1’s and 0’s in the addictionvariable.

This is basically what you’re trying to do when you run a regression:assign weights to variables to maximize the likelihood of predicting(using the regression model) your observed outcomes given your knowninputs.Derivatives help us with maximum likelihood estimation byallowing us to find critical points.

Why focus on chain rule and logs/exponents?

Chain rule: real-world processes and functions that describe them arecomplex; chain rule provides some tools for breaking down functionsinto composite parts to more easily find the derivative

Logs/exponents: ex and log(x) (which unless otherwise specified,refers to the natural log) each have simple derivatives; so a frequentapproach is to:

1 Take a function that has a lot of products/things that make derivativedifficult

2 Take the log of that function, remembering rules of logs/exponents andthat the log of a product is the sum of its logs (which all of you ablyshowed in the summer assignment!)

3 Find the derivative more easily!

For steps two and three, practice with derivatives of logs/exponents isimportant

Chain rule: real-world processes and functions that describe them arecomplex; chain rule provides some tools for breaking down functionsinto composite parts to more easily find the derivativeLogs/exponents: ex and log(x) (which unless otherwise specified,refers to the natural log) each have simple derivatives; so a frequentapproach is to:

3 Find the derivative more easily!For steps two and three, practice with derivatives of logs/exponents isimportant

3 Find the derivative more easily!

For steps two and three, practice with derivatives of logs/exponents isimportant

Chain rule: mechanics

When do we use the chain rule? When we have what’s called acomposite function- e.g., a function nested inside another function

Notation:

I (f ◦ g) = f (g(x))I Chain rule = ∂

∂x f (g(x)) = f ′(g(x)) ∗ g ′(x)

More colloquial: think about...

1 Take the derivative of the "outer layer" of the function (f (x)) whilekeeping the inner layer (g(x)) constant while you take the derivative

2 Then take the derivative of the "inner layer" of the function (g(x))3 Then multiply the results from step 1 and step 2 and simplify as needed

When do we use the chain rule? When we have what’s called acomposite function- e.g., a function nested inside another functionNotation:

∂x f (g(x)) = f ′(g(x)) ∗ g ′(x)

I (f ◦ g) = f (g(x))

I Chain rule = ∂∂x f (g(x)) = f ′(g(x)) ∗ g ′(x)

∂x f (g(x)) = f ′(g(x)) ∗ g ′(x)

More colloquial: think about...1 Take the derivative of the "outer layer" of the function (f (x)) while

keeping the inner layer (g(x)) constant while you take the derivative

∂x f (g(x)) = f ′(g(x)) ∗ g ′(x)

keeping the inner layer (g(x)) constant while you take the derivative2 Then take the derivative of the "inner layer" of the function (g(x))

3 Then multiply the results from step 1 and step 2 and simplify as needed

∂x f (g(x)) = f ′(g(x)) ∗ g ′(x)

keeping the inner layer (g(x)) constant while you take the derivative2 Then take the derivative of the "inner layer" of the function (g(x))3 Then multiply the results from step 1 and step 2 and simplify as needed

Chain rule practice

1 f (g(x)) =√3x4

2 f (g(x)) = x ∗√1− x2

Try these on your own and then we’ll review as a group...

Chain rule practice: solution one1 Step one: take the derivative of the "outer layer" of the function (f (x))

while keeping the inner layer constant while you take the derivative

I Outer layer: f (x) =√u = u 1

I Derivative of outer layer while keeping inner layer constant:

f ′(g(x)) = 12 (3x4)− 1

I Rearrange derivative of outer layer to make more readable:

f ′(g(x)) = 12 ∗

1√3x4

2 Step two: take the derivative of the "inner layer" of the function

I Inner layer: g(x) = 3x4

I Derivative of inner layer: g ′(x) = 12x3

3 Step three: multiply results from steps one and two: 12 ∗

12x3√

3x4 = 6x3√

f ′(g(x)) = 12 (3x4)− 1

f ′(g(x)) = 12 ∗

1√3x4

12x3√

3x4 = 6x3√

f ′(g(x)) = 12 (3x4)− 1

f ′(g(x)) = 12 ∗

1√3x4

12x3√

3x4 = 6x3√

f ′(g(x)) = 12 (3x4)− 1

f ′(g(x)) = 12 ∗

1√3x4

12x3√

3x4 = 6x3√

f ′(g(x)) = 12 (3x4)− 1

f ′(g(x)) = 12 ∗

1√3x4

12x3√

3x4 = 6x3√

f ′(g(x)) = 12 (3x4)− 1

f ′(g(x)) = 12 ∗

1√3x4

12x3√

3x4 = 6x3√

f ′(g(x)) = 12 (3x4)− 1

f ′(g(x)) = 12 ∗

1√3x4

12x3√

3x4 = 6x3√

f ′(g(x)) = 12 (3x4)− 1

f ′(g(x)) = 12 ∗

1√3x4

12x3√

3x4 = 6x3√

Chain rule practice: solution one4 Step four: Simplify (could have also simplified at an earlier step by

taking the constant out of the original equation)

Break out the x4 from the square root in the denominator and divide itinto the numerator:

6x3√3x4

= 6x√3

Multiply both numerator and denominator by√3:

6x√3∗√3√3

= 6x√3

Simplify:6x√3

3 = 2x√3

6x3√3x4

= 6x√3

6x√3∗√3√3

= 6x√3

Simplify:6x√3

3 = 2x√3

6x3√3x4

= 6x√3

6x√3∗√3√3

= 6x√3

Simplify:6x√3

3 = 2x√3

Chain rule practice: solution two

f (g(x)) = x ∗√1− x2

Use the product rule ∂∂x f (x) ∗ g(x) = f ′(x) ∗ g(x) + f (x) ∗ g ′(x):

x ∗ (√1− x2)′ + 1 ∗

√1− x2

We’re going to be using the chain rule on (√1− x2)′

Take the derivative of the "outer layer" of the function (f (x)) whilekeeping the inner layer constant while you take the derivative

I Outer layer:√u

I Derivative of outer layer while keeping inner layer constant: 12 ∗

1√1−x2

I Take the derivative of the "inner layer" of the function: g(x) = 1− x2;g ′(x) = −2x

f (g(x)) = x ∗√1− x2

x ∗ (√1− x2)′ + 1 ∗

√1− x2

I Outer layer:√u

1√1−x2

f (g(x)) = x ∗√1− x2

x ∗ (√1− x2)′ + 1 ∗

√1− x2

I Outer layer:√u

1√1−x2

f (g(x)) = x ∗√1− x2

x ∗ (√1− x2)′ + 1 ∗

√1− x2

I Outer layer:√u

1√1−x2

f (g(x)) = x ∗√1− x2

x ∗ (√1− x2)′ + 1 ∗

√1− x2

I Outer layer:√u

1√1−x2

f (g(x)) = x ∗√1− x2

x ∗ (√1− x2)′ + 1 ∗

√1− x2

I Outer layer:√u

1√1−x2

Multiply inner and outer layer from previous step and simplify:

12 ∗

−2x√1− x2

= −x√1− x2

Plug back into original product rule equation and simplify:

x ∗−x

√1− x2

1− x2 =−x2√

1− x2+

√1− x2

=−x2√

1− x2+√

1− x2√

1− x2

=−x2 +

√1− x2

=1− 2x2√

1− x2

Multiply inner and outer layer from previous step and simplify:

12 ∗

−2x√1− x2

= −x√1− x2

Plug back into original product rule equation and simplify:

x ∗−x

√1− x2

1− x2 =−x2√

1− x2+

√1− x2

=−x2√

1− x2+√

1− x2√

1− x2

=−x2 +

√1− x2

=1− 2x2√

1− x2

Why start with chain rule?

The chain rule is often used with derivatives of logs and exponents, whichare important tools for making functions easier to work with in maximum

likelihood estimation.

Derivatives of logs and exponents

Derivatives:

∂∂x ln(x) = 1

∂∂x e

x = ex

Derivatives:

∂∂x ln(x) = 1

∂x ex = ex

Derivatives:

∂∂x ln(x) = 1

∂x ex = ex

Some examples to practice on your worksheet

1 ∂∂x ln(x5)

2 ∂∂x ln(x2 + 3)

3 ∂∂x e

Derivatives of logs and exponents: solutions

1 ∂∂x ln(x5) = ∂

∂x 5 ∗ ln(x) = 5x

2 ∂∂x ln(x2 + 3)

I Outer layer derivative: 1x2+3

I Inner layer derivative: 2xI Multiply: 2x

3 ∂∂x e

I Outer layer derivative: e5x+2

I Inner layer derivative: 5I Multiply: 5e5x+2

1 ∂∂x ln(x5) = ∂

∂x 5 ∗ ln(x) = 5x

2 ∂∂x ln(x2 + 3)

3 ∂∂x e

1 ∂∂x ln(x5) = ∂

∂x 5 ∗ ln(x) = 5x

2 ∂∂x ln(x2 + 3)

3 ∂∂x e

1 ∂∂x ln(x5) = ∂

∂x 5 ∗ ln(x) = 5x

2 ∂∂x ln(x2 + 3)

I Inner layer derivative: 2x

I Multiply: 2xx2+3

3 ∂∂x e

1 ∂∂x ln(x5) = ∂

∂x 5 ∗ ln(x) = 5x

2 ∂∂x ln(x2 + 3)

3 ∂∂x e

1 ∂∂x ln(x5) = ∂

∂x 5 ∗ ln(x) = 5x

2 ∂∂x ln(x2 + 3)

3 ∂∂x e

1 ∂∂x ln(x5) = ∂

∂x 5 ∗ ln(x) = 5x

2 ∂∂x ln(x2 + 3)

3 ∂∂x e

1 ∂∂x ln(x5) = ∂

∂x 5 ∗ ln(x) = 5x

2 ∂∂x ln(x2 + 3)

3 ∂∂x e

I Inner layer derivative: 5

I Multiply: 5e5x+2

1 ∂∂x ln(x5) = ∂

∂x 5 ∗ ln(x) = 5x

2 ∂∂x ln(x2 + 3)

3 ∂∂x e

Two useful tools for optimization

Why tools for optimization?We’ve practiced finding the derivative of various functions

Optimization involves finding the set of points at which the derivativeis equal to zero (critical points), which since the derivative is the slopeof the function, is the point at which this slope equals zero:

−2000

0 5000 10000 15000 20000healthnav

Why tools for optimization?We’ve practiced finding the derivative of various functionsOptimization involves finding the set of points at which the derivativeis equal to zero (critical points), which since the derivative is the slopeof the function, is the point at which this slope equals zero:

−2000

0 5000 10000 15000 20000healthnav

Why tools for optimization?

Sometimes we can find these points by actually finding the derivativeand setting it equal to zero (analytic optimization: our focus today)

Other times, an analytic solution is impossible so we instead rely onalgorithms that help us "climb the hill" of a curve or jump around thespace a function covers to find these points (numerical optimization)For both forms of optimization, we need additional tools, which is thefocus of the next section

Sometimes we can find these points by actually finding the derivativeand setting it equal to zero (analytic optimization: our focus today)Other times, an analytic solution is impossible so we instead rely onalgorithms that help us "climb the hill" of a curve or jump around thespace a function covers to find these points (numerical optimization)

For both forms of optimization, we need additional tools, which is thefocus of the next section

Sometimes we can find these points by actually finding the derivativeand setting it equal to zero (analytic optimization: our focus today)Other times, an analytic solution is impossible so we instead rely onalgorithms that help us "climb the hill" of a curve or jump around thespace a function covers to find these points (numerical optimization)For both forms of optimization, we need additional tools, which is thefocus of the next section

Two useful tools

1 Tool one: higher-order derivatives (derivatives of derivatives...adinfinitum)

2 Tool two: partial derivatives (derivatives for functions with more thanone variable)

Two useful tools

1 Tool one: higher-order derivatives (derivatives of derivatives...adinfinitum)

2 Tool two: partial derivatives (derivatives for functions with more thanone variable)

Tool one: higher order derivatives

−2000

0 5000 10000 15000 20000healthnav

On the function to the left,which we’ll review in greaterdetail later, we can use the firstderivative to find where theslope of the curve equals zero

From visualizing the function,we can clearly see that it’s amaximum

But in order to confirm that, weuse higher-order derivatives tohelp us check how the slopechanges on either side of thatflat point on the curve to assesswhether it’s a maximum,minimum, or neither

−2000

0 5000 10000 15000 20000healthnav

−2000

0 5000 10000 15000 20000healthnav

Not much to learn...but in addition to the first derivatives we’ve beenfinding, you should be familiar with:

I Second derivative: f ′′(x) = ∂2f (x)dx2 = ∂

∂x ( ∂∂x f (x))

I Third derivative: f ′′′(x) = ∂3f (x)dx3

I Etc. until there remains nothing left in the function of which to take thederivative (that is, all constants)

Example:

I f (x) = 5x3 + 3x2

I ∂f (x)∂x = 15x2 + 6x

I ∂2f (x)dx2 = 30x + 6

I ∂3f (x)dx3 = 30

∂x ( ∂∂x f (x))

Example:

I f (x) = 5x3 + 3x2

I ∂f (x)∂x = 15x2 + 6x

I ∂2f (x)dx2 = 30x + 6

I ∂3f (x)dx3 = 30

∂x ( ∂∂x f (x))

Example:

I f (x) = 5x3 + 3x2

I ∂f (x)∂x = 15x2 + 6x

I ∂2f (x)dx2 = 30x + 6

I ∂3f (x)dx3 = 30

∂x ( ∂∂x f (x))

Example:

I f (x) = 5x3 + 3x2

I ∂f (x)∂x = 15x2 + 6x

I ∂2f (x)dx2 = 30x + 6

I ∂3f (x)dx3 = 30

∂x ( ∂∂x f (x))

Example:

I f (x) = 5x3 + 3x2

I ∂f (x)∂x = 15x2 + 6x

I ∂2f (x)dx2 = 30x + 6

I ∂3f (x)dx3 = 30

∂x ( ∂∂x f (x))

Example:I f (x) = 5x3 + 3x2

I ∂f (x)∂x = 15x2 + 6x

I ∂2f (x)dx2 = 30x + 6

I ∂3f (x)dx3 = 30

∂x ( ∂∂x f (x))

I ∂f (x)∂x = 15x2 + 6x

I ∂2f (x)dx2 = 30x + 6

I ∂3f (x)dx3 = 30

∂x ( ∂∂x f (x))

I ∂f (x)∂x = 15x2 + 6x

I ∂2f (x)dx2 = 30x + 6

I ∂3f (x)dx3 = 30

∂x ( ∂∂x f (x))

I ∂f (x)∂x = 15x2 + 6x

I ∂2f (x)dx2 = 30x + 6

I ∂3f (x)dx3 = 30

Tool two: partial derivatives

What is a partial derivative? Derivative of a function that has twoor more variables with respect to one of those variables

I What we’ve been doing : f (x) = x2...f ′(x) = 2xI When we’d use partial derivative: f (x , y) = x2 + 5y4

Notation:

I ∂f (x ,y)∂x : partial derivative of f (x , y) with respect to (w.r.t.) x

I ∂f (x ,y)∂y : partial derivative of f (x , y) with respect to (w.r.t.) y

How do we find?

I ∂f (x ,y)∂x : take derivative of function but treat y as a constant

I ∂f (x ,y)∂y : take derivative of function but treat x as a constant

I What we’ve been doing : f (x) = x2...f ′(x) = 2x

I When we’d use partial derivative: f (x , y) = x2 + 5y4

Notation:

How do we find?

Notation:

How do we find?

Notation:

How do we find?

Notation:I ∂f (x ,y)

∂x : partial derivative of f (x , y) with respect to (w.r.t.) x

How do we find?

∂x : partial derivative of f (x , y) with respect to (w.r.t.) xI ∂f (x ,y)

∂y : partial derivative of f (x , y) with respect to (w.r.t.) y

How do we find?

∂y : partial derivative of f (x , y) with respect to (w.r.t.) yHow do we find?

f (x , y) = x2y5 + ey + ln(x)

Find the following on your own:

1 ∂f (x ,y)∂x

2 ∂f (x ,y)∂y

3 ∂2f (x ,y)∂x∂y

4 ∂2f (x ,y)∂x2

Tool two: partial derivatives solutions

f (x , y) = x2y5 + ey + ln(x)

Find (green = take derivative; gray = treat as constant):

1 ∂f (x ,y)∂x : x2y5 + ey + ln(x) = 2xy5 + 1

2 ∂f (x ,y)∂y : x2y5 + ey + ln(x) = 5x2y4 + ey

3 ∂2f (x ,y)∂x∂y : can do in two steps

1∂f (x ,y)

∂x = 2xy5 + 1x

2 ∂∂y (2xy5 + 1

x ) = 10xy4

4 ∂2f (x ,y)∂x2 : again, two steps

1∂f (x ,y)

∂x = 2xy5 + 1x

2 ∂∂x (2xy5 + 1

x ) = 2y5 − 1x2

f (x , y) = x2y5 + ey + ln(x)

1 ∂f (x ,y)∂x : x2y5 + ey + ln(x) = 2xy5 + 1

x2 ∂f (x ,y)

∂y : x2y5 + ey + ln(x) = 5x2y4 + ey

1∂f (x ,y)

∂x = 2xy5 + 1x

2 ∂∂y (2xy5 + 1

x ) = 10xy4

1∂f (x ,y)

∂x = 2xy5 + 1x

2 ∂∂x (2xy5 + 1

x ) = 2y5 − 1x2

f (x , y) = x2y5 + ey + ln(x)

1 ∂f (x ,y)∂x : x2y5 + ey + ln(x) = 2xy5 + 1

x2 ∂f (x ,y)

∂y : x2y5 + ey + ln(x) = 5x2y4 + ey

1∂f (x ,y)

∂x = 2xy5 + 1x

2 ∂∂y (2xy5 + 1

x ) = 10xy4

1∂f (x ,y)

∂x = 2xy5 + 1x

2 ∂∂x (2xy5 + 1

x ) = 2y5 − 1x2

f (x , y) = x2y5 + ey + ln(x)

1 ∂f (x ,y)∂x : x2y5 + ey + ln(x) = 2xy5 + 1

x2 ∂f (x ,y)

∂y : x2y5 + ey + ln(x) = 5x2y4 + ey

3 ∂2f (x ,y)∂x∂y : can do in two steps1

∂f (x ,y)∂x = 2xy5 + 1

2 ∂∂y (2xy5 + 1

x ) = 10xy4

1∂f (x ,y)

∂x = 2xy5 + 1x

2 ∂∂x (2xy5 + 1

x ) = 2y5 − 1x2

f (x , y) = x2y5 + ey + ln(x)

1 ∂f (x ,y)∂x : x2y5 + ey + ln(x) = 2xy5 + 1

x2 ∂f (x ,y)

∂y : x2y5 + ey + ln(x) = 5x2y4 + ey

∂f (x ,y)∂x = 2xy5 + 1

x2 ∂

∂y (2xy5 + 1x ) = 10xy4

1∂f (x ,y)

∂x = 2xy5 + 1x

2 ∂∂x (2xy5 + 1

x ) = 2y5 − 1x2

f (x , y) = x2y5 + ey + ln(x)

1 ∂f (x ,y)∂x : x2y5 + ey + ln(x) = 2xy5 + 1

x2 ∂f (x ,y)

∂y : x2y5 + ey + ln(x) = 5x2y4 + ey

∂f (x ,y)∂x = 2xy5 + 1

x2 ∂

∂y (2xy5 + 1x ) = 10xy4

1∂f (x ,y)

∂x = 2xy5 + 1x

2 ∂∂x (2xy5 + 1

x ) = 2y5 − 1x2

f (x , y) = x2y5 + ey + ln(x)

1 ∂f (x ,y)∂x : x2y5 + ey + ln(x) = 2xy5 + 1

x2 ∂f (x ,y)

∂y : x2y5 + ey + ln(x) = 5x2y4 + ey

∂f (x ,y)∂x = 2xy5 + 1

x2 ∂

∂y (2xy5 + 1x ) = 10xy4

4 ∂2f (x ,y)∂x2 : again, two steps1

∂f (x ,y)∂x = 2xy5 + 1

2 ∂∂x (2xy5 + 1

x ) = 2y5 − 1x2

f (x , y) = x2y5 + ey + ln(x)

1 ∂f (x ,y)∂x : x2y5 + ey + ln(x) = 2xy5 + 1

x2 ∂f (x ,y)

∂y : x2y5 + ey + ln(x) = 5x2y4 + ey

∂f (x ,y)∂x = 2xy5 + 1

x2 ∂

∂y (2xy5 + 1x ) = 10xy4

4 ∂2f (x ,y)∂x2 : again, two steps1

∂f (x ,y)∂x = 2xy5 + 1

x2 ∂

∂x (2xy5 + 1x ) = 2y5 − 1

Summing up

What we’ve done:

I Reviewed how to find the derivatives of various functionsI Reviewed two tools useful for optimization: higher-order derivatives and

partial derivativesWhere we’re going next:

I Simple case: optimizing a function with one variable and with astraightforward solution

F Stylized example: finding the number of health navigators to maximizeACA enrollment

Summing up

What we’ve done:I Reviewed how to find the derivatives of various functions

I Reviewed two tools useful for optimization: higher-order derivatives andpartial derivatives

Where we’re going next:

Summing up

What we’ve done:I Reviewed how to find the derivatives of various functionsI Reviewed two tools useful for optimization: higher-order derivatives and

partial derivatives

Where we’re going next:

Summing up

Simple case of univariate optimization

Example 1: Health navigatorsSuppose a community health center is interested in hiring health navigators to helpenroll patients into the ACA. They want to maximize the number of patients theyenroll, and are aware that the following relationships hold, letting h = healthnavigator.

Productivity of each health navigator = 50h 23

Cost that detracts from other efforts at ACA enrollment = 2hPutting them together, and letting A = patients enrolled in ACA:

A(h) = 50h23 − 2h

Cost that detracts from other efforts at ACA enrollment = 2h

Putting them together, and letting A = patients enrolled in ACA:

A(h) = 50h23 − 2h

Cost that detracts from other efforts at ACA enrollment = 2hPutting them together, and letting A = patients enrolled in ACA:

A(h) = 50h23 − 2h

Visualizing the function

−2000

0 5000 10000 15000 20000healthnav

Example 1: why use optimization?

We can see that. . .

The curve flattens out a little before 5000 navigators

That flat point appears to be a maximumHow do we formally (i.e. mathematically) demonstrate theseobservations?

The curve flattens out a little before 5000 navigatorsThat flat point appears to be a maximum

How do we formally (i.e. mathematically) demonstrate theseobservations?

The curve flattens out a little before 5000 navigatorsThat flat point appears to be a maximumHow do we formally (i.e. mathematically) demonstrate theseobservations?

Example 1: finding the critical point(s)

1 Define the domain of interest: in this case, it is where h ≥ 0 andA(h) ≥ 0 (we don’t want there to be fewer ACA enrollees as a result ofthe health navigators)

I For this particular equation, 0 = 50h 23 − 2h =⇒ h = 15625

I Domain of interest: h = [0, 15625]2 Find the critical point(s) of the function: we find the critical

point(s) by setting the derivative of the function equal to zero andsolving for the number of health navigators:

Solve for critical point(s) on your worksheet

I Domain of interest: h = [0, 15625]

2 Find the critical point(s) of the function: we find the criticalpoint(s) by setting the derivative of the function equal to zero andsolving for the number of health navigators:

Example 1: finding the critical point(s)1 Define the domain of interest: in this case, it is where h ≥ 0 and

A(h) ≥ 0 (we don’t want there to be fewer ACA enrollees as a result ofthe health navigators)

A(h) = 50h23 − 2h

A′(h) =100

3h13− 2 = 0

2 =100

(h13 )3 =

33 =125000

27= 4629.23

Example 1: why do we need a second derivative?

−2000

0 5000 10000 15000 20000healthnav

Now we’ve been able toformalize the intuition thatthe slope flattens a littlebefore 5000 navigators, bysolving for h ≈ 4629

How do we formalize (i.e.mathematically demonstrate)that this is indeed a localmax and not a local min?

Example 1: why do we need a second derivative?

−2000

0 5000 10000 15000 20000healthnav

Now we’ve been able toformalize the intuition thatthe slope flattens a littlebefore 5000 navigators, bysolving for h ≈ 4629How do we formalize (i.e.mathematically demonstrate)that this is indeed a localmax and not a local min?

Example 1: using the second derivative1 Second derivative test: We found the critical point(s), but how to

make sure that it’s a maximum rather than a minimum (if we didn’thave the useful graph...)

1 Take second derivative of original function:

A′(h) = 1003h 1

3− 2

A′′(h) = − 1009h 4

2 Evaluate the second derivative at each critical point (in this case, onlyone) to check its sign

A′′(h) = − 1009 ∗ (4629) 4

3= −0.00014

3 Signs of second derivativef ′(x) f ′′(x) Classification

0 > 0 local min0 < 0 local max0 0 local min, local max, or inflection

point (where f(x) changes concavity)

A′(h) = 1003h 1

3− 2

A′′(h) = − 1009h 4

A′′(h) = − 1009 ∗ (4629) 4

3= −0.00014

A′(h) = 1003h 1

3− 2

A′′(h) = − 1009h 4

A′′(h) = − 1009 ∗ (4629) 4

3= −0.00014

A′(h) = 1003h 1

3− 2

A′′(h) = − 1009h 4

A′′(h) = − 1009 ∗ (4629) 4

3= −0.00014

point (where f(x) changes concavity)SOC Methods Camp Day One: Derivatives, Optimization, and Integrals September 3rd, 2019 37 / 1

Graphical represenation

Basic idea: as we approach the maximum, the slope of the curve (firstderivative) becomes less and less positive. Because the slope is going, forinstance, from 0.937 around health navigators = 1000 to 0.463 aroundhealth navigators = 2000 to 0.199 around health navigators = 3000, theslope of that slope (the second derivative) is negative

To illustrate graphically. . .

−2000

0 5000 10000 15000 20000healthnav

−2000

0 5000 10000 15000 20000healthnav

Graphical representation

−2000

0 5000 10000 15000 20000healthnav

What is an integral?

We use integrals to find the area under a curve.

Integrals are sometimes called called antiderivates, because integrationand derivation are opposite procedures.

I For example, if f (x) = 13x

3, the derivative is x2. The antiderivative ofx2 is f (x) = 1

The integral of a function f(x) is written as:

F (x) =∫f (x) dx

The integral of a function over a set interval [a,b] is called a definiteintegral and is denoted as follows:

F (x) =∫ b

af (x) dx = F (b)− F (a)

We use integrals to find the area under a curve.Integrals are sometimes called called antiderivates, because integrationand derivation are opposite procedures.

F (x) =∫f (x) dx

F (x) =∫ b

af (x) dx = F (b)− F (a)

F (x) =∫f (x) dx

F (x) =∫ b

af (x) dx = F (b)− F (a)

F (x) =∫f (x) dx

F (x) =∫ b

af (x) dx = F (b)− F (a)

F (x) =∫f (x) dx

F (x) =∫ b

af (x) dx = F (b)− F (a)

Properties of definite integrals

Antiderivatives of common functions

Integration by Parts

Integration by parts allows us to solve integrals in which two functionsare multiplied together.

The rule for integration by parts is:∫uv dx = u

∫v dx −

∫u′ (

∫v dx)dx

Or, another way of looking at it:∫uv ′ = uv −

∫vu′

Integration by parts allows us to solve integrals in which two functionsare multiplied together.The rule for integration by parts is:

∫uv dx = u

∫v dx −

∫u′ (

∫v dx)dx

∫vu′

Integration by parts allows us to solve integrals in which two functionsare multiplied together.The rule for integration by parts is:

∫uv dx = u

∫v dx −

∫u′ (

∫v dx)dx

∫vu′

Let’s do some practice. Try to solve these examples on scrap paper:∫exx dx∫ex sin(x) dx

Integration by Parts: SolutionsFor the first problem, let u = x and v ′ = ex . We know that u’ = 1 and∫ex dx = v = ex . So using our rule for integration by parts, we get:

xex − ex

ex (x + 1) + C

For the second problem, we will let u = sin(x) and v ′ = ex . We then canfind that:

u′ = cos(x)∫ex dx = ex

Putting that together, we get:∫ex sin(x) dx = sin(x)ex −

∫cos(x)ex dx

While this may look more complicated, we can actually use integration byparts again to get to the solution we need.

Solution continuedSo, let u = cos(x) and v ′ = ex . Then:

u′ = −sin(x)∫ex dx = ex

Using our intergration by parts rule, we get:

∫ex sin(x) dx = sin(x)ex − (cos(x)ex −

∫−sin(x)ex dx)∫

ex sin(x) dx = ex sin(x)− excos(x)−∫ex sin(x) dx

2∫ex sin(x) dx = ex sin(x)− excos(x)∫ex sin(x) dx = ex sin(x)− excos(x)

Why do we care about integrals in statistics?

Integrals help us find the probability of continuous events. You will learn alot more about this in the stats sequence next year, but for now just takeour word on this:

There are things called probability density functions.

The areas under these functions help us figure out the probability ofgiven events.

And, as we said at the start, integrals can be used to find the areas under acurve!

Why do we care about integrals in statistics?

Integrals help us find the probability of continuous events. You will learn alot more about this in the stats sequence next year, but for now just takeour word on this:

There are things called probability density functions.The areas under these functions help us figure out the probability ofgiven events.

And, as we said at the start, integrals can be used to find the areas under acurve!

Day One: Derivatives, Optimization, and...

Documents