COMP 170 L2

L18: Random Variables: Independence and VariancePage 1

COMP 170 L2

Outline

Independence of RVs

Distribution Functions of RVs

Independence between RVs

Expectation of product of independent RVs

Variance of RV

Definition and Examples

Additivity

Standard deviation

Central limit theorem

Page 2

COMP 170 L2

Distribution Functions of RVs

P(X=k) viewed as a function of k: The distribution function of X.

In general

Page 3

COMP 170 L2

Probability Weight and Distribution FunctionPage 4

COMP 170 L2

Distribution Functions of RVsPage 5

COMP 170 L2

Distribution Functions of RVsPage 6

For coin example

D(1, 9) = P(1 <= X <=9 ) ~= 1

COMP 170 L2

Outline

Independence of RVs

Distribution Functions of RVs

Independence between RVs

Expectation of product of independent RVs

Variance of RV

Definition and Examples

Additivity

Standard deviation

Central limit theorem

Page 7

COMP 170 L2

Independence Between Events

E is independent of F

If P(E|F) = P(E)

The information that “Event F occurred” does not change the

probability of E

COMP 170 L2

Random Variable and Event

Given a rand variable, we can define many difference events

Page 9

COMP 170 L2

Independence between RVsPage 10

COMP 170 L2

Independence between RVsPage 11

COMP 170 L2

Independence between RVsPage 12

No need to further check

P(X=0, Y=1) ?= P(X=0)P(Y=1)

P(X=1, Y=0) ? = P(X=1)P(Y=0)

P(X=1, Y=1) ?= P(X=1)P(Y=1)

COMP 170 L2

Independence between RVs

Draw two cards from a deck of 52 cards

X: number on first card

Y: number on second card

X and Y are independent when drawing with replacement

X and Y are not independent when drawing without replacement

Page 13

COMP 170 L2

Outline

Independence of RVs

Distribution Functions of RVs

Independence between RVs

Expectation of product of independent RVs

Variance of RV

Definition and Examples

Additivity

Standard deviation

Central limit theorem

Page 14

COMP 170 L2

Independence and ExpectationPage 15

COMP 170 L2

Independence and ExpectationPage 16

COMP 170 L2Page 17

COMP 170 L2Page 18

COMP 170 L2

Illustrating Proof via ExamplePage 19

COMP 170 L2

Illustrating Proof via ExamplePage 20

COMP 170 L2

Outline

Independence of RVs

Distribution Functions of RVs

Independence between RVs

Expectation of product of independent RVs

Variance of RV

Definition and Examples

Additivity

Standard deviation

Sum of independent RVs

The Trend

Central limit theorem

Page 21

COMP 170 L2

Variance of RVs

Probability starts with a process whose outcome is uncertain

Sample space: the set of all possible outcomes

A random variable (RV) is a function defined on the sample space

Different runs of the process might yield different outcomes

The RV might take different values in different runs

In other words, the value of RV varies across different runs

Some RVs vary more and some vary less

Number of heads in 1 coin flip

Number of heads in 10 coin flips

Number of heads in 100 coin flips

Page 22

COMP 170 L2

Variance of RVs

Variance of an RV X

Measures how much it varies (across different runs of process)

Relative to the mean value

Page 23

COMP 170 L2Page 24

COMP 170 L2Page 25

COMP 170 L2

Which RV vary the most?

Flip fair coin

Number of heads in 1 flip

Number of heads in 10 flips

Number of heads in 100 flips

Page 26

Variance

1/4

10/4

100/4

COMP 170 L2

Outline

Independence of RVs

Distribution Functions of RVs

Independence between RVs

Expectation of product of independent RVs

Variance of RV

Definition and Examples

Additivity

Standard deviation

Central limit theorem

Page 27

COMP 170 L2

Calculating VariancePage 28

COMP 170 L2

Example 1: We have already seen

Number of heads in n flips

n=1

n=10

(x1+X2+…+X10)

n=100

(X1+X2+…+X100)

Page 29

Variance

1/4

10/4 = 10 * 1/4

100/4 = 100 * 1/4

Additivity is true here.

COMP 170 L2Page 30

Additivity is true here.

COMP 170 L2

Example 3 (Counter Example) Page 31

COMP 170 L2

Example 3 (Counter Example) Page 32

COMP 170 L2Page 33

COMP 170 L2

Two LemmasPage 34

COMP 170 L2Page 35

COMP 170 L2Page 36

COMP 170 L2Page 37

An Application of Theorem 5.29

COMP 170 L2

A CorollaryPage 38

COMP 170 L2

Outline

Independence of RVs

Distribution Functions of RVs

Independence between RVs

Expectation of product of independent RVs

Variance of RV

Definition and Examples

Additivity

Standard deviation

Central limit theorem

Page 39

COMP 170 L2

Standard Deviation

Standard deviation is another measure of how much a rv varies or

how much a distribution spread out

It is the square root of variance.

Page 40

Next page

Shows several distributions, variances and standard deviations

Highlights the differences between variance and standard

deviation

COMP 170 L2

Distributions, Variances, and Standard Deviations Page 41

COMP 170 L2

The examples on the previous page show

Standard deviation is a natural measure of “spread” of distribution

Variance is easier to manipulate mathematically. Will see this in

further study of probability theory and statistics.

Page 42

Distributions, Variances, and Standard Deviations

COMP 170 L2

Outline

Independence of RVs

Distribution Functions of RVs

Independence between RVs

Expectation of product of independent RVs

Variance of RV

Definition and Examples

Additivity

Standard deviation

Central limit theorem

Page 43

COMP 170 L2

A Pattern

If we flip of a coin a large number of times,

“The number of heads” has bell-shaped distribution.

This phenomenon is not unique to coin flipping

Page 44

COMP 170 L2Page 45

Test with .8 probability of getting correct answer

COMP 170 L2Page 46

COMP 170 L2Page 47

COMP 170 L2

Normal DistributionPage 48

COMP 170 L2Page 49

COMP 170 L2Page 50

COMP 170 L2

Recap: 13-05-2010

COMP 170 L2

Recap: 13-05-2010

E is independent of F

If P(E|F) = P(E)

The information that “Event F occurred” does not change the

probability of E

COMP 170 L2

Recap: 13-05-2010

Flip fair coin

Number of heads in 1 flip: Variance = 1/4

Number of heads in 10 flips: Variance = 10/4

Number of heads in 100 flips: Variance = 100/4

COMP 170 L2

Number of times until first success

Throw a fair die

How many times, on average, do you need to throw the die until

you see a 1?

P(getting 1 at each throw) = 1/6

Answer: 1/(1/6) = 6

Page 54

COMP 170 L2

Number of times until first success

Throw a fair die

How many times, on average, do you need to throw the die until

you see a 2 and 3 in that order?

Expected number of throws to see 2: 6

After that, expected number of throws to see 3: 6

Answer: 6+6 = 12

Page 55

COMP 170 L2

Number of times until first success

Throw a fair die

How many times, on average, do you need to throw the die until

you see a 2 and 3 , where the order does not matter?

P( get 2 or 3 in one throw ) = 1/3

Expected number throws until you see one of 2 or 3: 3

After that, P( get the other number in each throw) = 1/6

Expected number of throws until you see the other number: 6

Answer: 3+6 = 9

Page 56

COMP 170 L2

Old Exam Question 1Page 57

P( at least one copy of T1 in 20 weeks)

= 1 – P( no copy of T1 in 20 weeks)

= 1 – P ( no T1 in week1 AND no T1 in week 2 AND …)

= 1- P(no T1 in week1) P(no T1 in week 2) …

=

COMP 170 L2

COMP 170 L2

Can we do this?

P(all 10 types of toys in 20 weeks)

= P(copy of T1 in 20 weeks AND copy of T2 in 20 weeks AND …)

= P(copy of T1 in 20 weeks) P(copy of T2 in 20 weeks)…

NO, events not independent

Correct way

Page 59

P(all 10 types of toys in 20 weeks) = 1 – P(A)

Use inclusion-exclusion to calculate P(A)

COMP 170 L2

COMP 170 L2Page 61

Old Exam Question 2

COMP 170 L2

COMP 170 L2