AMS 5AMS 5

CHANCE VARIABILITYCHANCE VARIABILITY

The Law of AveragesThe Law of Averages

When tossing a fair coin the chances of tails and heads are the same: 50% and 50%. So if the coin is tossed a large number of times, the number of heads and the number of tails should be approximately equal. This is the law of averages.The number of heads will be off half the number of tosses by some amount. That amount is called chance error. The chance error increases with the number of tosses in absolute terms, butit decreases in relative terms.

The Law of AveragesThe Law of AveragesQ: A coin is tossed and you win a dollar if there are more than

60% heads. Which is better: 10 tosses or 100?A: 100 tosses is better. As the number of tosses increase you are

more likely to be close to 50%, according to the law of averages.

Q: Same as before, but you win one dollar if there are exactly 50% heads.

A: 10 tosses is better, since in absolute terms you are more likely to be off the expected value when the number of tosses is large.

Q: 100 tickets are drawn with replacement from one of two boxes: one contains two tickets with -1 and two with 1. The other contains one ticket with -1 and one with 1. One hundred tickets will be drawn at random with replacement from one of them and the amount on the ticket will be paid to you, which box do you prefer?

A: In this case the expected payoff is the same for both boxes, since they both have 50% 1 and 50% -1.

Random VariablesRandom Variables

�� Chance ProcessesChance Processes

�� Random VariablesRandom Variables

�� Example 1: Toss a coin 1000 times and report Example 1: Toss a coin 1000 times and report the number of heads. If you repeat the the number of heads. If you repeat the experiment the number of heads will turn out experiment the number of heads will turn out differently.differently.

�� Example 2: The amount of money won or lost at Example 2: The amount of money won or lost at roulette. roulette.

�� Example 3: The percentage of Democrats in a Example 3: The percentage of Democrats in a random sample of voters.random sample of voters.

Variables whose results depend on the outcome of a random experiment

Box ModelBox Model• Find an analogy between the process being studied (e.g. sampling voters) and drawing numbers at random from a box.• Connect the variability you want to know about (e.g. estimate for the Democratic vote) with the chance variability in the sum of the numbers drawn from the box. The analogy between a chance process and drawing from a box is called box model.Consider a box model for a roulette. A roulette wheel has 38 pockets. 1 through 36 are alternatively colored red and black, plus 0 and 00 which are colored green. So, there are 18 red pockets and 18 black ones. Suppose you win $1 if red comes out and loose $1 if either a black number or 0 or 00 come out. Your chance of winning is 18 to 38 and your chance of loosing is 20 to 38. A box representation is

Box ModelBox Model

Suppose now that you bet a dollar on a single number and that, if you win, you get your $1 plus $35, but you loose your $1 if any other number comes up. A box model of this bet is given by

What is your net gain after 100 plays? This correspond to the sum of 100 draws made at random with replacement from the above box. To calculate this amount we need the concept of Expected value.

Expected Value and Standard ErrorExpected Value and Standard Error

A chance process is running. It delivers a number. Then we re-run it and it delivers another number, and so on. The numbers delivered by the process vary around the expected value, the amounts off being similar in size to the standard error. Example: Count the number of heads in 100 tosses of a coin. The expected value of the number of heads is 50. You actually toss the coin and the results are:• 57 heads, you are off by 7• 46 heads, you are off by -4• 47 heads, you are off by -3The amounts off are similar in size to the standard error.The expected value and the standard error depend on the random process that generates the numbers.

Expected Value and Standard ErrorExpected Value and Standard Error

Consider the box and draw a ticket at random with

replacement 100 times. What is the expected value of the sum ofthe tickets?The chance of a 1 is 75% and the chance of a 5 is 25%. So weexpect to see 25 x 5 + 75 x 1 = 200.

Notice that this number is equal to 100 x 2 = 200 which is the number of draws times the average number in the box. As a general rule we have that the expected value is given by:

Expected Value and Standard ErrorExpected Value and Standard ErrorQ: Suppose you play Keno, a game where you win $2 and pay $1 to play. You have 1 chance in 4 to win. How much should you expect to win if you play 100 times?A: A box representation of the game is given by

so the average in the box is: ($2- 3 x $1)/4=-$0.25. So you are expected to `win' -$25. Of course, if you keep playingyou are expected to loose more money!Q: Consider the box

and suppose 25 draws with replacement are made from the box. What is the expected sum?A: Each number should appear 1/5 of the time, that is 5 onaverage. So the expected value of the sum is5 x 0 + 5 x 2 + 5 x 3 + 5 x 4 + 5 x 6 = 75 = 25 x 3.

Expected Value and Standard ErrorExpected Value and Standard Error

In the last example we will not see each ticket appearing exactly 5 times. The actual sum we observe will be off by a chance error

sum = observed value = expected value + chance errorThe standard error gives a measure of how large the chance erroris likely to be. When drawing at random with replacement from a box we can calculate the standard error for the sum of the drawsas:

where SD of box stands for the standard deviation of the list of numbers in the box. Notice that the SE increases as the number of draws increases, but only by a factor equal to the square root of the number of draws.

Expected Value and Standard ErrorExpected Value and Standard Error

Expected Value and Standard ErrorExpected Value and Standard Error

Can we make the previous statements more precise?Consider again the box with the five numbers. We know that in 25draws the expected value is 75 and the SE is 10. Also, we observethat in 25 draws the sum ranges from 0 to 150 (all 0 through all6).What are the chances that the sum will be between 50 and 100?To answer this question we observe that:

50 - 75 = -25 = -2.5 x 10 and 100 – 75 = 25 = 2.5 x 10

So that 50 and 100 are 2.5 times SEs away from the expected value. We say that 25 is 2.5 standard units.

Expected Value and Standard ErrorExpected Value and Standard Error

We can apply the following approximation that will be betterjustified by the use of the Central Limit Theorem:

• 68% of the draws will be within one standard unit of the expected value.• 95% of the draws will be within two standard units of the expected value.• 99% of the draws will be within 2.5 standard units of the expected values.

For our example the ranges we get for one, two and 2.5 standard units are 75 ± 10, 75 ± 20 and 75 ± 25.

Expected Value and Standard ErrorExpected Value and Standard Error

�� ExampleExample: : Suppose there are 10,000 independent plays on a roulette wheel in a casino. Suppose all plays are of $1 on red at each play. What are the chances that the casino will win more than $250 from these plays?

The box model for this problem is

The expected value of the casino's net gain is the average:

times the number of plays 10000 x $0.05 = $500. So the casino expects to win $500 a month.

$20 $18$0.05

38

−≈

Expected Value and Standard ErrorExpected Value and Standard Error

We now need to calculate the SD. We have 20 deviations of 0.95 and 18 deviations of -1.05 from the average. So

and being conservative we can approximate this number to 1. So the SE is approximately

$250 is 2.5 SE units from the expected gain of $500, so we see that there is 99% chance that the net gain for the casino will be between $250 and $750.

10000 100.=

Probability HistogramsProbability Histograms

Consider the box: Consider the box:

Then, the chances of obtaining a ticket with a 1are 4/7, the chances of a3 are 1/7 and the chancesof a 4 are 2/7. We candisplay that informationgraphically in a probability histogram.

Each box is centered at a number and its area corresponds to theprobability of that number. The sum of the areas of the boxes isequal to one. This histograms are used to represent chance and not the frequency of data.

Probability HistogramsProbability HistogramsEmpirical histograms based on the frequencies of observed outcomes of an experiment converge to the corresponding probability histograms, as can be seen by the example of rolling two dice.

Probability HistogramsProbability HistogramsConsider now taking the product of the two dice. The convergence is also true then, although the probability histogram is much more irregular than the one obtained for the sum. The regularity is a general feature related to the sum.

Probability HistogramsProbability HistogramsConsider the problem of tossing a fair coin a certain number oftimes n. We can obtain the probability histogram for each n. We observe thatthe probability histogram of the number of tails con-verges to a very regular curve (the NORMAL) as thenumber of tosses isincreased.

Normal ApproximationNormal ApproximationWe can approximate the probability histogram of the sum of heads in a large number of coin tosses using the normal curve.

Example: A coin is tossed 100 times, what is the probability of getting:a) exactly 50 heads?b) between 45 and 55 heads inclusive?c) between 45 and 55 heads exclusive?

Solution:a) We can look at the probability histogram for this case. Weobserve that the chances corresponding to 50 are equal to the area of the box that has a base from 49.5 to 50.5. The area of this box is 7.96%.

Normal ApproximationNormal ApproximationWhat about an approximation using the normal curve?First step is to calculate the mean and standard deviation.Consider a box model where there is a zero for the tail and 1 forthe head, We draw a ticket from this box 100 times

The expected number of heads is 100 × ½ = 50 and the standard error is given by the square root law Now we have to convert to standard units:

So the normal approximation consists of the area under the normal curve for the interval (-0.1,0.1). According to the normal table, this is equal to 7.97%.

100 1/ 2 5.× =

49.5 50 50.5 500.01 and 0.01

5 5

− −= − =

Normal ApproximationNormal Approximationb) The probability of getting between 45 and 55 heads is equal to the areas of the rectangles between 45 and 55 in the probability histogram. This is approximated by the area under the normal curve for the interval (44.5,55.5). In standard units this corresponds to the interval (-1.1,1.1), which has a probability of 72.87% according to the table.c) This time the probability is given by the areas of the rectangles between 46 and 54, which is approximately the area under the curve corresponding to the interval (45.5,54.5), this is the interval (-0.9,0.9) in standard units, which has a probability of 63.19%.Very often it is not specified if the end points are included or not.In that case we consider the approximation using the given interval. So, for the previous example, we would have (45,55) that is converted to (-1,1) in standard units and yields 68.27% probability.

When can we use the normal When can we use the normal approximationapproximation

Consider the box . Then the probability histogram

for the tickets in the box is far from being normal.

Nevertheless, if we consider the experiment of drawing tickets from the box and sum the results over and over again, then the probability histogram of the sum will be approximated by the normal curve.

What if we consider the product of the tickets? In that case theprobability histogram will not be approximated by a normal curve, no matter how many draws from the box we take.

The Central Limit TheoremThe Central Limit TheoremIn general it is true that the probability histogram of the sum of draws from a box of tickets will be approximated by the normal curve. This is a mathematical fact that can be expressed and proved as a theorem.

The reason why the CLT is used as an approximation fordistributions of lists of numbers is that it often happens that theuncertainty in the data can be thought of as the sum of severalsources of randomness.