Last Time

Last Time

• Histograms– Binomial Probability Distributions– Lists of Numbers– Real Data– Excel Computation

• Notions of Center– Average of list of numbers– Weighted Average

Administrative Matters

Midterm I, coming Tuesday, Feb. 24

• Excel notation to avoid actual calculation– So no computers or calculators

• Bring sheet of formulas, etc.

• No blue books needed

(will just write on my printed version)



• Material Covered:

HW 1 – HW 5

– Note: due Thursday, Feb. 19– Will ask grader to return Mon. Feb. 23– Can pickup in my office (Hanes 352)– So this weeks HW not included



• Extra Office Hours:– Monday, Feb. 23 8:00 – 9:00– Monday, Feb. 23 9:00 – 10:00– Monday, Feb. 23 10:00 – 11:00– Tuesday, Feb. 24 8:00 – 9:00– Tuesday, Feb. 24 9:00 – 10:00– Tuesday, Feb. 24 1:00 – 2:00



• How to study:– Rework HW problems

• Since problems come from there• Actually do, not “just look over”• In random order (as on exam)• Print HW sheets, use as a checklist

– Work Practice Exam• Posted in Blackboard “Course Information” Area

Reading In Textbook

Approximate Reading for Today’s Material:

Pages 277-282, 34-43

Approximate Reading for Next Class:

Pages 55-68, 319-326

Big Picture

• Margin of Error

• Choose Sample Size

Need better prob tools

Start with visualizing probability distributions

Big Picture

• Margin of Error

• Choose Sample Size

Need better prob tools

Start with visualizing probability distributions,

Next exploit constant shape property of Bi

Big Picture


Next exploit constant shape property of Binom’l

Big Picture



Centerpoint feels p

Big Picture



Centerpoint feels p Spread feels n

Big Picture



Centerpoint feels p Spread feels n

Now quantify these ideas, to put them to work

Notions of Center

Will later study “notions of spread”

Notions of Center

Textbook: Sections 4.4 and 1.2

Recall parallel development:

(a) Probability Distributions

(b) Lists of Numbers

Study 1st, since easier

Notions of Center


“Average” or “Mean” of x1, x2, …, xn

Mean = =

common

notation

xn

xn

ii

1

Notions of Center

Generalization of Mean:

“Weighted Average”

Intuition: Corresponds to finding balance

point of weights on number line

1x 2x 3x

Notions of Center

Generalization of Mean:

“Weighted Average”

Intuition: Corresponds to finding balance

point of weights on number line

1x 2x 3x

Notions of Center

Textbook: Sections 4.4 and 1.2

Recall parallel development:

(a) Probability Distributions


Notions of Center

(a) Probability distributions, f(x)

Approach: use connection to lists of numbers

Notions of Center



Recall: think about many repeated

draws

Notions of Center



Draw X1, X2, …, Xn from f(x)

Notions of Center



Draw X1, X2, …, Xn from f(x)

Compute and express in terms of f(x)X

Notions of Center

n

XXX n

1

Notions of Center

n

XXX n1

n

XX ii 22#11#

Rearrange list, depending on values

Notions of Center

n

XXX n1

n

XX ii 22#11#

Number of Xis that are 1

Notions of Center

n

XXX n1

n

XX ii 22#11#

Apply Distributive Law of Arithmetic

2

2#1

1#

n

X

n

X ii

Notions of Center

n

XXX n1

n

XX ii 22#11#

Recall “Empirical Probability Function”

2

2#1

1#

n

X

n

X ii

22ˆ11ˆ ff

Notions of Center

n

XXX n1

22ˆ11ˆ ff

Notions of Center

n

XXX n1

22ˆ11ˆ ff

2211 ii XPXP

Frequentist approximation

Notions of Center

n

XXX n1

22ˆ11ˆ ff

2211 ii XPXP

2211 ff

Notions of Center

n

XXX n1

22ˆ11ˆ ff

2211 ii XPXP

A weighted average of values that X takes on

2211 ff

Notions of Center

n

XXX n1

22ˆ11ˆ ff

2211 ii XPXP

A weighted average of values that X takes on, where weights are probabilities

2211 ff

Notions of Center

n

XXX n1

A weighted average of values that X takes on, where weights are probabilities

2211 ff

This concept deserves its own name:Expected Value

Expected Value

Define Expected Value of a random variable X:

Expected Value


xfxEXx

Expected Value


Useful shorthand notation

xfxEXx

Expected Value


Recall f(x) = 0, for most x, so sum

only operates for values X takes

on

xfxEXx

Expected Value

E.g. Roll a die, bet (as before):

Expected Value


Win $9 if 5 or 6, Pay $4, if 1, 2 or 3, otherwise

(4) break even

Expected Value



(4) break even

Let X = “net winnings”

Expected Value



(4) break even


otherwise

x

x

x

xf

0

0

4

9

)(61

21

31

Expected Value



(4) break even


Are you keen to play?

otherwise

x

x

x

xf

0

0

4

9

)(61

21

31

Expected Value


otherwise

x

x

x

xf

0

0

4

9

)(61

21

31

Expected Value


Weighted average, wts & values

otherwise

x

x

x

xf

0

0

4

9

)(61

21

31

x

xfxEX )(

Expected Value


Weighted average, wts & values

otherwise

x

x

x

xf

0

0

4

9

)(61

21

31

6121

31 0)4(9)( x

xfxEX

Expected Value


i.e. weight average of values 9, -4 & 0, with

weights of “how often expect”, thus “expected”

otherwise

x

x

x

xf

0

0

4

9

)(61

21

31

10)4(9)( 61

21

31 x

xfxEX

Expected Value


Conclusion: on average in many plays, expect

to win $1 per play.

otherwise

x

x

x

xf

0

0

4

9

)(61

21

31

10)4(9)( 61

21

31 x

xfxEX

Expected Value

Caution: “Expected value” is not what is

expected on one play

(which is either 9, -4 or 0)

But instead on average, over many plays

HW: 4.73, 4.74 (1.9, 1)

Expected Value

Real life applications of expected value:

• Decision Theory

– Operations Research

– Rational basis for making business decisions

– In presence of uncertainty

– Common Goal: maximize expected profits

– Gives good average results over long run

Expected Value


• Decision Theory

• Casino Gambling

– Casino offers games with + expected value

(+ from their perspective)

– Their goal: good overall average performance

– Expected Value is a useful tool for this

Expected Value


• Decision Theory

• Casino Gambling

• Insurance

– Companies make profit

– By writing policies with + expected value

– Their goal is long run average performance

Expected Value


• Decision Theory

• Casino Gambling

• Insurance

• State Lotteries

– State’s view: games with + expected value

– Raise money for state in long run overall

Flip Side of Expected Value

Decisions made against expected value:



• Casino Gambling

– Why do people play?



• Casino Gambling


– Odds are against them

– For sure will lose “over long run”



• Casino Gambling


– Odds are against them

– For sure will lose “over long run”

– But love of short run successes, can make

eventual long term loss worthwhile

– Are buying entertainment



• Casino Gambling

• Insurance

– Why should you buy it?



• Casino Gambling

• Insurance


– You lose in expected value sense



• Casino Gambling

• Insurance


– You lose in expected value sense

– But E not applicable since you only play once

– Avoids chance of catastrophic loss

– Allows low cost sharing of risk



• Casino Gambling

• Insurance

• State Lotteries




• Casino Gambling

• Insurance

• State Lotteries


– Clear loss in expected value



• Casino Gambling

• Insurance

• State Lotteries


– Clear loss in expected value

– But only play once (hopefull), so not applicable

– Worth feeling of hope from buying ticket?


Interesting issues about State Lotteries:

A very different type of tax

Big Plus:

• Only totally voluntary tax

Big Minus:

• Tax paid mostly by poor



Key Lesson: Expected Value tells what

happens on average over long run,

not in one play



Key Lesson: Expected Value tells what

happens on average over long run,

not in one play

Conclude Expected Value not good for

everything, but very good for many things

Another Inverse View of Expected Value

Suppose you have $5000, and need $10,000



e.g. you owe mafia $5000 (gambling debt?),

clean out safe at work for $5000.





If give to mafia, you go to jail





If give to mafia, you go to jail, so decide to

raise another $5000 by gambling.



and can make even bets, with P[win] = 0.48




Can really do this, e.g. bet on Red in game of

Roulette at a casino




Important question:

Make one large bet? Or many small bets?

Something in between?





E[Gain] = 0 x P[loss] + $2 x P[win]

= 0 x (0.52) + $2 x (0.48) =

$0.96





E[Gain] = $0.96

Interpretation: “expect” to lose $0.04 every

time you play





E[Gain] = $0.96


time you play

Why games are so profitable for casinos





E[Gain] = $0.96


time you play

Many plays Expected Value dictates result





E[Gain] = $0.96


time you play

So best to make just one large bet!



time you play




time you play


After many plays, will surely lose!

(lesson of expected value)


Another View:

Strategy P[win $1,0000]

one $5000 bet 0.48 ≈ ½


Another View:


one $5000 bet 0.48 ≈ ½

two $2500 bets ≈ (0.48)2 ≈ ¼


Another View:


one $5000 bet 0.48 ≈ ½

two $2500 bets ≈ (0.48)2 ≈ ¼

four $1250 bets ≈ 1/16


Another View:


one $5000 bet 0.48 ≈ ½

two $2500 bets ≈ (0.48)2 ≈ ¼

four $1250 bets ≈ 1/16

many bets no chance



time you play


Casino Folklore: Sometimes people really

make such bets

Expected Value

Binomial Expected Value:

For X ~ Bi(n,p),

Expected Value


For X ~ Bi(n,p), Expected Value is

probability weighted average of values

x

xfxEX )(

Expected Value



Use Binomial Probability Distribution

x

xnx

xpp

x

nxxfxEX 1)(

Expected Value



After a long and tricky calculation

(details beyond scope of this course)

x

xnx

xpp

x

nxxfxEX 1)(

pn

Expected Value

For X ~ Bi(n,p),

• Makes sense:

“Expect” to win proportion p, of n trials

pnEX

Expected Value

For X ~ Bi(n,p),

• Makes sense:


• Just use this formula from here on out

pnEX

Expected Value

For X ~ Bi(n,p),

• Makes sense:


• Just use this formula from here on out

• E.g. to capture “shifting mean”

pnEX

Expected Value

For X ~ Bi(n,p),

HW:

5.28a, mean part only (900)

5.29a, mean part only

pnEX

Properties of Expected Value

Linearity:

For X ~ f(x) and Y ~ g(y)

E(aX + bY) =


Linearity:


E(aX + bY) = Σx Σy (ax + by) f(x) g(y)

Weighted average, where

weights

are probabilities


Linearity:


E(aX + bY) = Σx Σy (ax + by) f(x) g(y) =

= Σx Σy (ax f(x) g(y) + by f(x)

g(y))

(Distributive Rule)


Linearity:



= Σx Σy (ax f(x) g(y) + by f(x) g(y)) =

= Σx Σy axf(x)g(y) + Σx Σy axf(x)g(y)

(Associative Property of Addition)


Linearity:



= Σx Σy (ax f(x) g(y) + by f(x) g(y)) =

= Σx Σy axf(x)g(y) + Σx Σy axf(x)g(y) =

= (Σxaxf(x)) Σyg(y) + (Σxf(x)) Σybyg(y)

(Distributive Rule)


Linearity: For X ~ f(x) and Y ~ g(y)

E(aX + bY) =

= (Σxaxf(x)) Σyg(y) + (Σxf(x)) Σybyg(y)



E(aX + bY) =

= (Σxaxf(x)) Σyg(y) + (Σxf(x)) Σybyg(y) =

= Σx a x f(x) + Σy b y g(y)

(Since Σx f(x) = Σy y g(y) = 1)



E(aX + bY) =


= Σx a x f(x) + Σy b y g(y) =

= a Σx x f(x) + b Σy y g(y)

(Distributive Rule)



E(aX + bY) =


= Σx a x f(x) + Σy b y g(y) =

= a Σx x f(x) + b Σy y g(y) =

= a E(X) + b E(Y)



E(aX + bY) = a E(X) + b E(Y)

i.e. E(linear combo) = linear combo (E)


HW:

4.81 (mean part only)

4.84 (mean part only)


HW: C18 An insurance company sells 1378

policies to cover bicycles against theft for 1

year. It costs $300 to replace a stolen

bicycle and the probability of theft is

estimated at 0.08. Suppose there is no

chance of more than one theft per

individual.


HW: C18 (cont.)

a. Calculate the expected payout for each

policy, to give a break even price for each

policy. ($24)

b. If 2 times the break even price is actually

charged, what is the company’s expected

profit per policy, if the theft rate is actually

0.10? ($18)

Research Corner

Recall Hidalgo Recall Hidalgo

StampData &StampData &

Movie over binwidthMovie over binwidth

Research Corner

Recall Hidalgo Recall Hidalgo

StampData &StampData &

Movie over binwidthMovie over binwidth

Main point:Main point:

Binwidth drives Binwidth drives

histogram performancehistogram performance

Research Corner

Less known fact:Less known fact:

Bin Bin locationlocation also has also has

Serious effectSerious effect

(even for fixed width)(even for fixed width)

Research Corner

How many bumps?How many bumps?

~2?~2?

Research Corner


~3?~3?

Research Corner


~7?~7?

Research Corner


~7?~7?

Research Corner

Explanation?Explanation?

Compare with “smoothedCompare with “smoothed

version” called version” called

““Kernel Density Estimate”Kernel Density Estimate”

Research Corner




Peaks appear:Peaks appear:

when when entirely in entirely in a bina bin

Research Corner




Peaks disappear:Peaks disappear:

when split betweenwhen split between

two bins bintwo bins bin

Research Corner

Question: If understand problem with Question: If understand problem with histogram, using histogram, using Kernel Density EstimateKernel Density Estimate

Then why not use Then why not use KDEKDE for data analysis? for data analysis?

Will explore Will explore KDEKDE later. later.

Notions of Center

Caution about mean:

Works well for ~symmetric distributions

Notions of Center

Caution about mean:


E.g. Buffalo Snowfalls

Notions of Center

Caution about mean:



Analyzed in:http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg6.xls

http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg6.xls

Notions of Center

Caution about mean:



Mean = 80.3

(from Excel)

Notions of Center

Caution about mean:



Mean = 80.3

(from Excel)

Notions of Center

Caution about mean:



Mean = 80.3

Visually sensible

Notion of “Center”

Notions of Center

Caution about mean:

But poorly for asymmetric distributions

Notions of Center

Caution about mean:


E.g. British Suicides Data

Notions of Center

Caution about mean:



• Time (in days) to suicide attempt

• Of Suicide Patients

• After Initial Treatment

Notions of Center

Caution about mean:



Analyzed in:http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg6.xls


Notions of Center

Caution about mean:



Notions of Center

Caution about mean:



Clearly not mound shaped

Notions of Center

Caution about mean:




Very asymmetric

Notions of Center

Caution about mean:




Very asymmetric

Called “right skewed”

Notions of Center

Caution about mean:



Mean = 122.3

Notions of Center

Caution about mean:



Mean = 122.3

Notions of Center

Caution about mean:



Mean = 122.3

Sensible as “center”??

Notions of Center

Caution about mean:



Mean = 122.3


%(data ≥) = 30.2%

Notions of Center

Caution about mean:



Mean = 122.3


%(data ≥) = 30.2%

Too Small…

Notions of Center

Perhaps better notion of “center”:

• Take center to be point in middle

• I.e. have 50% of data smaller

• And 50% of data larger

This is called the “median”

Notions of Center

Median: = Value in middle (of sorted list)

Notions of CenterMedian: = Value in middle (of sorted list)

Unsorted E.g: Sorted E.g:

3 0

1 1

27 2

2 3

0 27



3 0

1 1

27 2

2 3

0 27

One in middle???



3 0

1 1

27 2

2 3

0 27

One in middle??? NO, must sort



3 0

1 1

27 2

2 3

0 27

Sensible version of “middle”

Notions of CenterWhat about ties?

Sorted E.g:

0

1

2

3


Sorted E.g:

0

Tie for point in 1

middle 2

3


Sorted E.g:

0

Tie for point in 1

middle 2

3

Break by taking average (of two tied values):


Sorted E.g:

0

Tie for point in 1

middle 2

3

Break by taking average (of two tied values):

e.g. Median = 1.5



3 0

1 1

27 2

2 3

0 27

EXCEL: use function “MEDIAN”

Notions of CenterEXCEL: use function “MEDIAN”

Very similar to other functions

E.g. see:http://www.stat-or.unc.edu/webspace/courses/marron/UNCstor155-2009/ClassNotes/Stor155Eg6.xls


Notions of Center


Mean = 80.3

Notions of Center


Mean = 80.3

Median = 79.6

(from Excel)

Notions of Center


Mean = 80.3

Median = 79.6

Very similar

Notions of Center


Mean = 80.3

Median = 79.6

Very similar

(expected from

symmetry)

Notions of Center


Mean = 122.3

Median = 77.5

Substantially different

Notions of Center


Mean = 122.3

Median = 77.5


Notions of Center


Mean = 122.3

Median = 77.5


But which is better?

Notions of Center


Mean = 122.3

Median = 77.5



Goal 1: ½ - ½ middle

Notions of Center


Mean = 122.3

Median = 77.5



Goal 1: ½ - ½ middle

Goal 2: long run average

Date post:	30-Jan-2016
Category:	Documents
Upload:	linus
View:	225 times
Download:	0 times

Last Time

Documents