Tobias Econ 472

Law of Large Numbers (LLN)and

Central Limit Theorem (CLT)

Tobias Econ 472

Consistency and LLN

• As shown in class, a law of large numbers is a powerful theorem that can be used to establish the consistency of an estimator.

• We illustrate what we mean by consistency by showing what happens to the sampling distributions of sample averages as the sample size tends toward infinity.

Tobias Econ 472

Consistency and LLN

• We again consider the case of random (iid) sampling from a uniform distribution.

• We obtain 5,000 random samples of sizes n = 1,2,5,50 and 1,000.

• For each experiment, we calculate the sample average of the drawn values. Doing this 5,000 different times (for each sample size n) enables us to characterize the sampling distributions of the estimators.

Tobias Econ 472

Consistency and LLN

• The following 5 slides present those sampling distributions for n = 1,2,5,50 and 1,000.

Tobias Econ 472

Tobias Econ 472

Tobias Econ 472

Tobias Econ 472

Tobias Econ 472

Tobias Econ 472

Results

• As we can see from the progression of these slides, the sampling distribution collapses around the population average, (i.e., .5), as n approaches infinity. This is what we mean by the consistency of the sample average under iid sampling.

• We also see that the Normal approximation to the sample average appears to work well for moderate to large n, but not so well for very small n.

Tobias Econ 472

Central Limit Theorem

Tobias Econ 472

CLT

• As suggested by the last point, the central limit theorem is a powerful statistical tool that can be used to establish that the sampling distribution of the standardized sample average converges to a standard Normal distribution as the sample size n ! 1

Tobias Econ 472

CLT, continued

• By standardized sample average, we mean taking the sample average, subtracting off its mean, and then dividing through by its standard deviation.

• Since

Tobias Econ 472

CLT, continued• the CLT can be used to establish that:

• Where “!” means “converges to as n approaches 1” and N(0,1) denotes a standard Normal distribution.

Tobias Econ 472

CLT, continued

• To demonstrate this convergence, we again illustrate with random sampling from a uniform distribution.

• We obtain random samples of sizes n=1,2,5 and 1,000 from the uniform distribution, and calculate the sample average and the standardized sample average.

• We do this 5,000 times for each sample size.

Tobias Econ 472

CLT, continued

• We then characterize the sampling distributions of the standardized sample averages and compare them to the standard Normal distribution.

• The results of this exercise are found on the following 4 slides:

Tobias Econ 472

Tobias Econ 472

Tobias Econ 472

Tobias Econ 472

Tobias Econ 472

CLT results

• As you can see, for very small n, the Normal approximation is not very accurate.

• For this exercise, the normal approximation is reasonable even for n=5.