Review of Probability (Topics of PAT)MATH 448: Mathematical Statistics - Section 01

Sanjeena DangOffice: WH 128

Fall 2017, Binghamton University

1 / 23

Welcome!!

Instructor: Sanjeena Dang.

Assistant Professor at the Department of Mathematical Sciences.

PhD (University of Guelph, Canada).

My primary area of research is clustering and classification of highdimensional data with applications in bioinformatics.

Class Meetings:MWF 8:00 am – 9:30 am at SW-206.

Office hours:MW 12:00 pm –1:00 pm at WH128OR By Appointment.

2 / 23

Basic Information

Textbook: Mathematical Statistics with Applications (7th ed.) byWackerly, Mendenhall, and Scheaffer.

The course will cover:

Review of Chapters 1–7 (very briefly).

Note: you will need to have the basic knowledge from MATH 447 forprobability aptitude test (PAT).

Mainly, Chapters 8–10.

myCourses will be used for:

Announcements

Homework and grades.

Lecture notes.

3 / 23

Course Assessment

4 / 23

Record keeping

Lecture notes, Quiz and test grades as well as homeworkassignments will be posted on the myCourses.

Announcements will be made via emails (VERY IMPORTANT!!Check your emails!) and also posted on the myCourses’sannouncement!

Homework are assigned regularly and will be collected on the dateposted. It is VERY IMPORTANT that you do the homework in atimely manner as some homework problems may become the quizproblems.

5 / 23

How to succeed? (Let the data speak!)Don’t miss the class!

Miss 0 Qs (15) Miss 4 Qs (10)

200

300

400

500

600

700

800

Absence V.S. Final grade

Fin

al g

rade

This is the final grades from MATH 448, section 01, Spring 2015.

6 / 23

How to succeed? (Let the data speak!)Do well on your quizzes.

This means DO YOUR HOMEWORK.Read and understand the lecture notes and book thoroughly.

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

0 50 100 150

020

040

060

080

010

00

Tests score V.S. Quiz score

Quiz score

Test

s sc

ore

This is the final grades from MATH 448, section 01, Spring 2015.

Quiz scores are highly correlated with your final grades.

7 / 23

Review: Some key concepts

An experiment is the process by which an observation is made.

A sample space is a set of all possible outcomes.

An event is a possible outcome (a subset of the sample space).

A probability measure P(:) is function defined over the sample/eventspace mapping to [0,1] and it describes some attributes of events.

Independence of two events:If A and B are independent, P(AB) = P(A)P(B).

8 / 23

Conditional Probability

By definition: P(A j B) � P(AB)=P(B), called conditionalprobability of event A given event B (occurs).

Independence: three rules for checking.

Multiplicative law: P(A j B)P(B) = P(AB)

Additive law: P(A [B) = P(A) + P(B)� P(AB)

Event-decomposition (use diagram), law of total probability andBayes’ Rule.

9 / 23

Random variables and their distributions

Random variable (r.v.): A function that maps from an outcome(sample space) to a real value.

For discrete r.v., we have probability mass function (pmf).

For continuous r.v., we have probablity density function (pdf).

Cumulative distribution function (cdf):

For discrete random variables, it is a step function.

For continuous random variables, it is the area under the PDF curve tothe left of y .

10 / 23

Discrete Random Variables

Expectation, variance and definition of all below.

Bernoulli random variable.

Binomial random variable.

Geometric random variable.

Poisson random variable.

Moment generating function:

definition and how to calculate;

how to retrieve moments of a distribution from the given momentgenerating function.

11 / 23

Examples: Binomial distribution

X � Binomial (10; 0:4); x = 0; 1; : : : ; 10:

f (x = 3)

F (x = 3)

E(X )

Var(X )

Moment generating function and first moment.

12 / 23

Practice for PAT

Repeat what we did for Binomial random variable for:

Bernoulli random variable.

Geometric random variable.

Poisson random variable.

Negative binomial random variable.

13 / 23

Continuous Random Variable

Expectation, variance and MGF of all below:

Uniform distribution.

Normal (important): standardization; algebra involving normal r.v.’s.

Gamma distribution.

�2 distribution.

Exponential distribution.

Beta distribution (MGF does not exist in closed form).

Know how to get CDF from PDF or from PDF to CDF.

Probability inequalities (Chebyshev’s).

Algebra involving expectations and variance.E(aX ) = aE(X ), and Var(aX ) = a2 Var(X ).

14 / 23

Examples: Binomial distribution

X � Exp(�); x � 0:

f (x )

F (x )

E(X )

Var(X )

Moment generating function and first moment.

15 / 23

Practice for PAT

Repeat what we did for exponential random variable for:

Uniform random variable.

Normal variable.

Gamma random variable.

�2 random variable.

Exponential random variable.

Beta random variable (MGF does not exist in closed form).

16 / 23

Multivariate

For discrete case, we have joint probability mass function.

Know how to get marginal and conditional distributions from the jointprobability mass function.

For continuous case, we have joint probability density function.

Know how to get marginal and conditional distributions from the jointdensity (by integrating out unwanted variable).

Be careful about the possible region (domain) - must declare.

17 / 23

Example

Let Y1 and Y2 have the joint density function

f (y1; y2) = e�(y1+y2) for y1 > 0 and y2 > 0, and 0 elsewhere:

What is P(Y1 < 1;Y2 > 5)?

What is P(Y1 +Y2 < 3)?

Marginal density of Y1 and Y2.

Coditional density of Y1 and Y2.

18 / 23

You need to know!

Expectation of a function of random variable(s).

Use proper integration. Be careful about the integration region.

Covariance: definition; its relation to variance.

Algebra involving expectation, variance and covariance when thereare multiple random variables.

E.g., E(X1 +X2) =?, Var(X1 +X2) =?, when X1 and X2 areindependent, or when they are not.

E.g., Cov(X1 +X2;Y1 +Y2) =?

19 / 23

Transformation of a random variable

Find the distribution of a function of random variable(s). Note: theoutput of this function is also a random variable as its inputs are allrandom.

CDF method: That is to try to find P(Y � y) by manipulating theinequality inside the parenthesis and convert to a question about findthe probability of an event regarding U (section 6.3).

Let’s look at an example.

Order statistic, esp. the max and the min., expectation and density.

20 / 23

Review of the max of Y1 : : : Yn

Notation: Y(n) = max fY1 : : :Yng.CDF of Y(n): use the CDF method

FY(n)(y) � P(Y(n) � y)

= P(Yi � y ; for all i)

=Y

i

P(Yi � y)

= [F (y)]n

PDF of Y(n): derivative of the CDF

fY(n)(y) = n [F (y)]n�1f (y)

Expectation: integration of y � fY(n)(y) over a reasonable range.

21 / 23

Review of the min of Y1 : : : Yn

Notation: Y(1) = min fY1 : : :Yng.CDF of Y(1): use the CDF method

FY(1)(y) � P(Y(1) � y)

= 1� P(Y(1) > y)= 1� P(all Yi ’s are > y)= 1� [1� F (y)]n

PDF of Y(1): derivative of the CDF

fY(1)(y) = n [1� F (y)]n�1f (y)

Expectation: integration of y � fY(1)(y) over a reasonable range.

22 / 23

Sampling distribution and CLT

A statistic is a function of observable random variables in a sampleand known constants.

The distribution of a statistic (when viewed as a random variable) iscalled the sampling distribution.

The most famous statistic is the sample mean (average of thesample), X .

E(X ) = E(X1):

Var(X ) = Var(X1)=n . Know why.

CLT: the distribution of X will be normal distributed (approximately),when n is large.

23 / 23