axdif

8/12/2019 axdif

1/38

Random Variables Monotonic Transformations Expectations

PSCI 356:

STATISTICS FOR POLITICAL RESEARCH I

Class 5: Probability Theory: Random Variables andDistribution Functions II

8/12/2019 axdif

2/38


Outline

1 Random Variables

Discrete Random Variables

Continuous Random Variables

2 Monotonic Transformations

3

ExpectationsProperties of Expectations

Measures of Dispersion

8/12/2019 axdif

3/38



Random Variables

Random Variable: numerical outcome of a random experiment.

1 Definition: Adiscrete random variableis a random variable that can

take on only a finite (or countably infinite) number of values.

2 Definition: Acontinuous random variable is a random variable thatcan take on a continuum of values (uncountable number of values).

LetF(x)be the CDF for a continuous random variableX.Properties ofF(same as for discrete random variables):

limxF(x) =0, limx+F(x) =1

F(x)is increasing sox1

8/12/2019 axdif

4/38




In dealing with continuous random variables, we talk about the

distribution of the random variable over the sample space in terms of the

probability density function [PDF]not probability mass functions.

Definition: TheCDF of a continuous random variable is

F(x) =P(X x) = x

f(x)x. Iffis continuous atx, then thePDF isf(x) =F(x)/y.

In other words, the PDF is the derivative of the CDF.

A PDFf(x)is a function such that:

f(x) 0 for allx,

8/12/2019 axdif

5/38




EXAMPLE:The Uniform distribution on the interval[0, 1]chooses any number between 0 and 1. The PDF of the Uniform

is given by:

f(x) =

1 if 0 x10 else

f(x)

8/12/2019 axdif

6/38

R d V i bl M t i T f ti E t ti

8/12/2019 axdif

7/38



Continuous Random VariablesEXAMPLE: Uniform Distribution

The CDF of the Uniform Distribution described above is (note the pdf is just

the derivative ofF(x): given by:

F(x) =

0 ifx1

f(x) =

1 if 0 x 10 else

x

Pr(X0 and ,Yis a normal randomvariable if the probability distribution is defined as:

p(X=x) =p(x) = 1

2e(x)

2/(22), x

Standard Normal: = 0 and =1


8/12/2019 axdif

9/38



Continuous Random VariablesNormal Probability Distributions

How might Gauss have guessed this?


8/12/2019 axdif

10/38



Continuous Random VariablesStandard Normal Probability Distributions


8/12/2019 axdif

11/38



Continuous Random VariablesOther Probability Distributions

Gamma Binomial Probability Distribution

Beta Probability Distribution

8/12/2019 axdif

12/38


8/12/2019 axdif

13/38

p

Monotonic Transformation

EXAMPLE:Say pdffX(x) =exp(x), x>0 and 0

elsewhereFind the pdf for Y = 1/X

fY(y) =e(1/y)

1y2

Find the pdf for Y = ln(X)

fY(y) =e

ey

|ey

|


8/12/2019 axdif

14/38

p

What have we learned?

Events in a sample space can be used to model

experiments.

Probability measure on the events enables us to evaluate

the frequency of events.(Discrete and Continuous) random variables can be

defined on the sample space to quantify events.

Random variables can be described/characterized using

pmf/pdf and cdf.

However, is there a summary measure of a quantity of interest

that can be used to characterize the outcome of the

experiment? Of course there is. We refer to such quantities as

moments.


8/12/2019 axdif

15/38

Measures of Central TendencyDiscrete Random Variable: Expected Value E[X]

One characteristic that we might desire is the expected realization of

the random variable X. For a discrete random variable X with values

xi, 1=1...N, we calculate theexpected value(or expectationormeanor the mean of the probability distribution) as follows:

E[X] =

Ni=1

xif(xi) = = x

EXAMPLE 1Let X be the number of heads in 2 fair coin tosses. Then

E[X] =0 1/4+1 1/2+2 1/4= 1.

Pr(X=x)

0 1 2

0

.25

.50


8/12/2019 axdif

16/38

Measures of Central TendencyDiscrete Random Variable: Median(X)

An alternative measure of central tendency is given by the medianof a

random variable. The medianxMof random variable X is defined as the

choicexMsuch thatP(X xM) 1/2.

x

Pr(X

1. Calculate the expected value,median and mode.

The median of X is solved for as follows.

12

=

m

1 2x3x

1

2 = 2 12 x

2m1

12

= 1m2 1

11

m2 = 1

2

m=

2

Inspection of the PDF reveals that the mode of X is obviously at 1.

Note that as the above example indicates, there is no reason to expectthat all measures of central tendency will be identical. Consequently,

depending on the measure you choose, your characterization of the

random variable may differ.

Furthermore, there is no necessary reason for the median and mode to

be unique although it is the case that the mean (E[X]) is unique.


8/12/2019 axdif

23/38

Measures of Central Tendency

EXAMPLEConsider the random variable X with the following

PMF.

x 0 1 2 3 4

f(x) .2 .3 .1 .3 .1

E[X] =0(.2) +1(.3) +2(.1) +3(.3) +4(.1) =1.8.4

mode(X) ={1, 3}

median(X)[1, 2]sinceP(X xm) 12 .5

4

Note that there is no reason why E[X] has to be an actual value of X.5Sometimes expresses as 1.5.


8/12/2019 axdif

24/38

Properties of Expectations


For any function of the random variable g(x) we can stillcalculate the expected value of the transformed" randomvariable using the following result:

E[g(x)] =

x

g(xi)f(xi)if X discrete

E[g(x)] = g(x)f(x)xif X continuousNote that it isnotthe case thatE[X2] = (E[X])2.


8/12/2019 axdif

25/38



EXAMPLEConsider the random variable X with the following PMF.

x 0 1 x 2 0 1

f(x) .5 .5 f (x2) .5 .5

E[X] =0(.5) +1(.5) =.5implying that(E[X])2 =.52 =.25.

E[X2] =0(.5) + 1(.5) =.5.EXAMPLELet g(x) = a + b(x).

E[a+bX] =

(a+bx)f(x)x

=

af(x)x+

bxf(x)x

= a

f(x)x+b

xf(x)x

= a 1+b E[X]6= a+bE[X]

As the above example demonstrates, for a constant c,E[c] =c, andE[cX] =cE[X]

6By definition of the PDF and E[X] respectively.


8/12/2019 axdif

26/38



Just as there are several measures of central tendency, so too are there

several measures of dispersion. The two most commonly used are the

rangeand thevariance.

The range is defined as: range(X) = max(X) - min(X). Note that it is

usually very uninformative.

A more useful description of a random variables dispersion is given by

the variance. Intuitively, the variance of a random variable measures

how much the random variable X typically" deviates from its typical"

value (or distance from the population mean). Mathematically:

Var[X] =E[(X E[X])2

].

2 =E[(X E[X])2] =

x(x E[X])2f(x)if X is discretex

(x E[X])2f(x)xif X is continuous


8/12/2019 axdif

27/38



Why is the difference squared? Well, suppose not.

i(xi E[X])f(x) =

ixif(x)

iE[X]f(x)

= E[X] E[X]if(x)7= E[X]

E[X]8

= 0.

In other words, the average deviation of X from its average is always0;

values above the mean cancel out values below the mean by definition

of the mean.

Consequently, we need to square the difference. Note that we couldalso take the absolute difference but that is harder" to work with

mathematically.

7By definition of E[X] and since E[X] is a constant respectively.8Since

if(x) =1 by definition of probability.


8/12/2019 axdif

28/38



EXAMPLE

x 1 2 3 4 5

f(x) .2 .2 .2 .2 .2

E[X] =1(.2) +2(.2) +3(.2) +4(.2) +5(.2) =.2 +.4 +.6 +.8 +1= 3Var[X] =E[(XE[X])2] = (1 3)2(.2) + (2

3)2(.2) + (3 3)2(.2) + (4 3)2(.2) + (5 3)2(.2) =2.5


8/12/2019 axdif

29/38



We can also derive another expression for the variance.

E[(XE[X])2

] = E[X2

2XE[X] + (E[X])2

]= E[X2]2E[XE[X]] +E[(E[X])2]= E[X2]2E[X]E[X] +E[(E[X])2]= E[X2]2E[X]2 +E[X]2

= E[X2]E[X]2


8/12/2019 axdif

30/38



EXAMPLEConsider the continuous random variable with the PDF.Calculate Var[X].

f(x) =

2(1 x) if0 < x

8/12/2019 axdif

31/38



E[X2] =1

0 x22(1 x)x

= 21

0 x2 x3x

= 2

10

x2x

1

0 x3x

= 2

13

x3 ]10 14 x4 ]10

= 2

13 1

4

= 2

112

= 1

6

2 =Var[X] =E[X2]

E[X]2

= 16 19= 1

18


8/12/2019 axdif

32/38



The interpretation of the variance may not be that intuitive, as it tells us

the average squared deviation of a random variable X. Consequently, if

X is measured in dollars, Var[X] is measures insquareddollars.

To make interpretation easier, we often times refer to the standard

deviation of a random variable. The standard deviation is simply thesquare root of the variance.

More generally, we can characterize a random variable using a

moment generating function. The moment generating function allows

us to define a sequence of moments which can completely characterize

the probability distribution.

The kth moment around zero is defined as E[0 E[X]]k orE[X]k.Note that the first moment about zero is the mean: E[X].

The kth moment around the mean is defined as: E[(X E[X])k].The second moment about the mean is the variance.


M f Di i

8/12/2019 axdif

33/38



Why is this terminology useful? Because it provides a common

framework for talking about our measures of central tendency and

dispersion.

Higher moments about the mean also have special terms associated

with them.

The third moment around the mean E[(X E[X])3] is calledtheskewof the distribution. The skew tells us whether the

dispersion about the mean is symmetric (if skew= 0 ), or if it is

negatively skewed (if skew< 0; implying that E[X] < median(X))or positively skewed (if skew> 0; implying that E[X]> median(X)).


M f Di i

8/12/2019 axdif

34/38


Measures of DispersionSkewness

50 100 150

0.0

0

0.0

2

0.0

4

Positive Skew

x

Density

EX

med[X]

50 100 150

0.0

00

0.0

05

0.0

10

0.0

15

Symmetric

x

Density EX

med[X]

50 100 150

0.0

0

0.0

2

0.0

4

Negative Skew

x

Density

EX

med[X]



8/12/2019 axdif

35/38


Measures of DispersionKurtosis

The fourth moment about the mean E[(XE[X])4

]isknown as thekurtosisand it measures how thick thetails" of the distribution are.9

9When measure using higher powers we often normalize and use E[(XE[X])k]

(

Var[X])k

becauseE[(X E[X])k]increases dramatically as k increases.



8/12/2019 axdif

36/38


Measures of DispersionKurtosis

0 50 150

0.0

00

0.0

05

0.0

10

0.0

15

0.0

20

x

Density

0 50 150

0.0

00

0.0

05

0.0

10

0.0

15

0.0

20

x

Density

8/12/2019 axdif

37/38



8/12/2019 axdif

38/38



IfXis a random variable with finite variance, then for any

constants a and b,

Var[aX+b] = E[(aX+b)E[(aX+b)]2

= E[aX+baE[X]b]2

= E[aXaE[X]]2

= a2E[XE[X]]2

= a

2

Var[X]

Date post:	03-Jun-2018
Category:	Documents
Upload:	peter-newman
View:	223 times
Download:	0 times

axdif

Documents