Chapter 03 Random Variables Part A

8/3/2019 Chapter 03 Random Variables Part A

http://slidepdf.com/reader/full/chapter-03-random-variables-part-a 1/55

Definition of a Random Variable

Random Variable [m-w.org]

: a variable that is itself a function of the result of a statistical

experiment in which each outcome has a definite

probability of occurrence

Copyright © Syed Ali Khayam 20093




A random variable is a mapping from an outcome s of a

random experiment to a real number

: X X S S → ⊂

domainrange





:X S S →

head

an om

ExperimentSample Space Random

VariableX(s)

tail0 1 R

S x





:X S S → ⊂

X (s)

1 2 R3 4 5 6

Random

Sample Space, S

Random Variable

S x


Image courtesy of www.buzzle.com/

6




More than one outcomes can be mapped to the same real

number:X S S →

X(s)

Random

0 1

S x

xper menSample Space

Random Variable


Image courtesy of www.buzzle.com/

7



Types of Random Variables

Discrete random variables: have a countable (finite or infinite)image S x = {0, 1}

S x = {…, -3, -2, -1, 0, 1, 2, 3, …}

Continuous random variables: have an uncountable image S x = (0, 1]

S x = R

Mixed random variables: have an image which contains

continuous and discrete parts S x = {0} U (0, 1]

We will mostly focus on discrete and continuous random

var a es




screte an om ar a es




Probability Mass Function

The Probability Mass Function (pmf ) or the discrete probability

density function provides the probability of a particular point in

For a countable S X ={a0, a2, …, an}, the pmf is the set of

probabilities

( ) { }Pr , 1,2, ,X k k p a X a k n = = = …

p X (ak )

S X ={a0=0, a1=1, …, a5 =5},


X

10



Properties of a PMF

P1: ( )0 1X k p a ≤ ≤

P2: ( ) 1k X

X k a S p a ∈ =∑

X k

S X ={a0=0, a1=1, …, a5 =5},




Cumulative Distribution Function (CDF) of Discrete

Random Variable

The Cumulative Distribution Function (CDF) for a discrete rv is

defined as:

( ) { } ( )PrX X

x t F t X t p x

≤= ≤ = ∑

p X ( x )

mf

F X ( x )

CDF

X 1 2 3 4 X 1 2 3 4




Cumulative Distribution Function (CDF) of Discrete

Random Variable

CDF can be used to find the probability of a range of values in a

rv’s image:

{ } { } { }( ( )

Pr Pr Pr

X X

a X b X b X a F b F a

< ≤ = ≤ − ≤= −

p X ( x ) F X ( x )

pmf CDF


X 1 2 3 4 X 1 2 3 4

13



Properties of a CDF

F1: ( )0 1X F x x ≤ ≤ ∀ − ∞ < < ∞

F2: ( )X X a a ⇒

p X ( x )

pmf

F X ( x )

X 1 2 3 4 X 1 2 3 4




Properties of a CDF

F3:( )lim 0x X F x →−∞ =

F4:

mx X x →∞ =

( ) ( ) ( )1 1X i X i X i F x F x p x + += +

p X ( x ) F X ( x )

pmf CDF


X 1 2 3 4 X 1 2 3 4

15



Expected Value of a Discrete Random

Variable The expected value, expectation or mean of a discrete rv is the

“average” value of the random variable

What is the average value of a random variable whose image is

S = 1 6 7 9 13 ?





Variable What is the average value of the following random variable

whose image is S X ={1, 6, 7, 9, 13}?

If your answer is 7.2 then you assumed that all of the values in the

rv’s ima e have e ual wei hts

( )

1 1 1 1 1 1

1 6 7 9 13 1 6 7 9 13 7.25 5 5 5 5 5× + × + × + × + × = + + + + =





Variable Mathematically, the expected value of a discrete random

variable is:

{ } { }Prk X

X k k a S

X a X a µ∈Ε = = =∑

,

In such cases, we say that the expected value does not exist




Variance of a Random Variable

Variance of a rv is a measure of “the amount of variation of a rv

around its mean”

Intuitively, which of the following discrete rvs has a higher

variance?

X

0.5E { X }=3.87

X

0.4E { X }=5.2

0.033

.

0.2

0.1


1 6 7 9 13 X 1 6 7 9 13 X

20




Intuitively, which of the following discrete rvs has a higher

variance?

q X ( x ) has a higher variance because it varies more around its mean

than p X ( x )

p X ( x )

E X =3.87

q X ( x )

E { X }=5.20.5

0.4

0.2

0.4

1 6 7 9 13

0.033

X 1 6 7 9 13

0.1

X





Mathematically, the variance of a discrete rv is defined as:

2var r

k X

X k k

a S

a a σ

∈

= = − =





x x

0.5

0.4

E { X }=3.87 0.4 E { X }=5.2

var { X }=15.36

0.033

0.2

0.1

var { X }=9.91

1 6 7 9 13 X 1 6 7 9 13 X

{ } ( )2var 1 3.87 0.5X = − × + { } ( )2

var 1 5.2 0.4X = − × +

( ) ( )

( ) ( )

2 2

2 2

6 3.87 0.4 7 3.87 0.033

9 3.87 0.033 13 3.87 0.033

− × + − × +

− × + − ×

( ) ( )

( ) ( )

2 2

2 2

6 5.2 0.2 7 5.2 0.2

9 5.2 0.1 13 5.2 0.1

− × + − × +

− × + − ×


.= .=

23



Standard Deviation of a Random Variable

In many scenarios, we use the square root of the variance

called its standard deviation

{ }varX X σ =




Standard Deviation of a Random Variable

p X ( x )

0.5E { X }=3.87

q X ( x )

0.4E { X }=5.2

0.4

0.2σ

X =3.14

σ X =3.92

1 6 7 9 13

0.033

X 1 6 7 9 13

.

X

{ }var 15.36 3.92X X σ = = ={ }var 9.91 3.14X X σ = = =




Discrete Uniform Random Variable

A discrete uniform rv, D, has a finite image and all the elements

of the image have equal probabilities

Pr{D=k }

1/n

What do you think are the expected value and standard

k x1 x2 x3 xn

deviation of this random variable?




Bernoulli Random Variable

A Bernoulli Random Variable is defined on a single event A

This rv is based on an experiment called a Bernoulli trial

The experiment is performed and the event A either happens or does not

happen

Thus the sam le s ace of a Bernoulli rv is binar

Sample Space

Bernoulli Random

VariableX sB

A=head

er n o ul l i T c

o

Ac

= Not head= tail

0 1 Ri al : T o s s

i n


S x

27





This rv is based on a the experiment called a Bernoulli trial


happen

Thus the sample space of a Bernoulli rv is binary

Bernoulli Random

VariableX sP

A=Pak wins

R

k i s t an c r i

pl a y s A u

Ac =Pak losses 0 1

Sample Space

k e t t e am

s t r al i a


Image Courtesies of http://www.tribuneindia.com and

images.google.com

28





This rv is based on a the experiment called a Bernoulli trial


happen

Thus the sample space of a Bernoulli rv is binary

Bernoulli Random

Of course the Pr{A} = 0 for this

ex eriment

A=Pak wins

VariableX(s)

P ak i s t a

pl a y

Ac =Pak losses0 1

R c r i c k e t t

A u s t r al i


Sample Space

Image Courtesies of http://www.tribuneindia.com and

images.google.com

am

29




Typical examples of Bernoulli rvs in communication:

Transmit a bit over a wireless channel

0−

> bit is received error-free 1 −> bit received is not received error-free => bit is received with errors

Outcomes:

0 −> packet is received

1 −> acket is not received => acket is lost en-route due to con estion





Sample space of a Bernoulli rv, I, is binary

Both outcomes are mapped to real numbers,

Traditionally: I( A) = 1 and I( Ac) = 0 are used to represent a Bernoulli rv’s

outcomes

The pmf of I is:

Pr{I = 1} = p

Pr{I = 0} = 1 − p

Pr{I =k }

p

I

1- p


0 1

31




Example:

Consider the experiment of a fair coin toss. What is the expected

value and the variance of this Bernoulli rv?

Pr{I =k }

0.5

I 0 1




Discrete Random Variables

Example:

Consider the experiment of a fair coin toss. What is the expected

va ue an e var ance o s ernou rv

Since the coin toss is fair: Pr{I = 1} = 0.5 and Pr{I = 0} = 0.5

. . .

var{I} = (1−0.5)2x(0.5) + (0−0.5)2x(0.5) = 0.25

Pr{I =k }

=

σ I =0.5

0.5

.


0 1

33




Example:

Consider a binary symmetric channel with probability of bit-error

0.1. What is the expected value and the variance of this Bernoulli

rv?

Pr{I =k }

0.9

I 0 1





Example:

Consider a binary symmetric channel with probability of bit-error 0.1. What is the

ex ected value and the variance of this Bernoulli rv?

The event of interest here is a bit-error:

=> Pr{I = 1} = 0.9 and Pr{I = 0} = 0.1

= . . = .

var{I} = (1−0.9)2x(0.9) + (0−0.9)2x(0.1) = 0.09

Note that the variance of this pmf is smaller than the variance of the coin tosspm

Pr{I =k }

0.9σ I =0.3

0.1

µ I =0.9


I 0 1

35



Binomial Random Variable

Consider a collection of n independent Bernoulli trials

A Binomial Random Variable is the total number of occurrences

of an event A in this independent Bernoulli collection

Send n bits count the number of bits that are received with errors

Send n packets, count the number of packets that are not lost





If I j ( A)=1 and I j ( Ac)=0, j =1,2,…,n, are used to represent the

outcomes of the Bernoulli trials then the Binomial Random

ar a e, , s

n

X I = ∑

So what is the ima e of ?

1 j =





If I j ( A)=1 and I j ( Ac)=0, j =1,2,…,n, are used to represent the

outcomes of the Bernoulli trials then the Binomial Random

, ,

n

j X I = ∑

So what is the image of X ?

1 j =

S X = {0, 1, 2, 3, …, n}





Pr{I j (A)=1} = p and Pr{I j (A)=0} = 1- p

Then a Binomial rv X is defined as:

n

X I =1 j =

An t e pm o a nom a rv s

n k k

n −

r p pk = = −





The pmf of a Binomial rv X is:

n −

; , rn p p p

k

= = = −

s pm g ves e pro a y a exac y ou o a o a o n

Bernoulli trials were successes

Be very careful about the definition of a success





Pr{ X =k }

k


Image courtesy of Wikipedia article on Binomial Distribution

41




Example:

Consider a binary symmetric channel with probability of bit-error p.

n e pro a y a a pac e o n s s rece ve w one

or more errors?





Example:



or more errors?

Pr{I j = 1} = 0.1 and Pr{I j = 0} = 0.9, j =1,2,…,n

Then the probability that a packet is received with errors is

( ) ( ) ( ) ( ){ }Pr 1 2 3X X X X n = ∪ = ∪ = ∪ ∪ =…





Example:



or more errors?

{ } ( ) ( ) ( ) ( ){ }

{ } { } { } { }

Pr pkt with errs Pr 1 2 3

Pr 1 Pr 2 Pr 3 Pr

X X X X n

X X X X n

= = ∪ = ∪ = ∪ ∪ =

= = + = + = + + =

…

…

( ) ( ) ( ) ( )1 2 31 2 3

1 1 1 11 2 3

n n n n nn n n n n

p p p p p p p pn

− − − − = − + − + − + + −

…

( )1

1

n

n i i

i

p pi −

= = − ∑





Example:



or more errors?n

n i n

−

There is an easier way to compute the same probability by noting

1

r p w errsi

p pi

=

= −

that:

{ } { } { } { }Pr pkt with errs Pr 0 1 Pr 0 1 Pr 0X X X = > = − < = − =

( )00

1 10

n n p p− = − −


p= − −

45

Connection Between Bernoulli and Binomial



Connection Between Bernoulli and Binomial

RVs

I ( A)=1, I ( A)=0n Bernoulli trials

A

Pr{ A}

I

Pr{I =1}= p

X

{ } ( )Pr 1

n k k

n

X k p pk −

= = −

Pr{I =0}=1- p

Bernoulli Trial Bernoulli RV Binomial RV


d bl



Geometric Random Variable

A Geometric Random Variable, M, is the number of Bernoulli

trials until the first occurrence of an event A

The experiment is stopped as soon as event A is observed

The image of a Geometric random variable is infinite but

countable

M = , , , …


G i R d V i bl




The pmf of a Geometric Random Variable, Z , is

1k −= = = −

Al s o c al

g e o

OR

e d

t h em

m e

t r i c p

Depending on whether the success trial is included in the total

rZ p p p= = = − o d i f i e d

f

count or not


G t i R d V i bl





Image courtesy of Wikipedia article on Geometric Distribution

Connection between Geometric and Binomial




RVs Can we find the probability of a Geometric random variable

using the Binomial random variable?

{ } ( )Pr 1k

Z k p p= = − Geometric

{ } ( )Pr 1n k k

n Z k p p

k

− = = −

Binomial





RVsCan we find the probability of a Geometric random variable using

the Binomial random variable?

11 k k k

− −

{ } ( ) { }1

1 1

Pr 1 Prk

X k p p Z k −

= = − = =





RVs In k Bernoulli trials, there are k ways in which you can have 1

success and k -1 failures

Since the Binomial random variable counts successes andfailures it sums and considers all the k outcomes to ether





RVsFor k =4, a Binomial random variable X jointly considers the

outcomes 0001, 0010, 0100, 1000

Pr{ X = 1} = Pr{0001} + Pr{0010} + Pr{0100} + Pr{1000}Since the underlying Bernoulli trials are independent:

Pr{ X = 1} = (k )Pr{one success in k trials}

,one of these outcomes, 0001

=> Pr{ Z = 1} = Pr{ X = 1} / k


Memoryless Property



Memoryless Property

It can be shown that the Geometric rv satisfies the memoryless

property

The memoryless property is satisfied when:

{ } { }Pr PrZ j k Z k Z j = + ≥ = =

This property is also called the Markov Property


Memoryless Property



Memoryless Property

The memoryless property is satisfied when:

{ } { }Pr PrZ j k Z k Z j = + ≥ = =

For the Geometric rv, RHS of the above equation is:

r p p= = −

property, we need to show that

Pr 1j

Z k Z k = = −


Memoryless Property



Memoryless Property

To prove that the Geometric rv satisfies the memoryless

property, we need to show that

Let’s expand the LHS using the definition of conditional

{ } ( )Pr 1j

Z j k Z k p p= + ≥ = −

probability

{ }Pr Z j k Z k

= + ≥∩{ }

{ }

Pr

Pr

Z k

Z j k

≥

= +=

( )1 2

1

1 1 1

j k

k k k

p p+

+ +

−=

− − − …


Memoryless Property



Memoryless Property

Continued from last page

j k +−

( ) ( ) ( )( )

1 2Pr

1 1 1

1

k k k

j k

Z j k Z j p p p p p p

p p

+ +

+

= + ≥ =

− + − + − +−

…

( ) ( ) ( ) ( )( )

( )

0 1 21 1 1 1

1

k

j k

p p p p p p p

p p

+

=− − + − + − +

−

…

( )

( )

1

1

k

j

p

p p

−

= −

Summation over all possible values

of the Geometric pmf


Memoryless Property



Memoryless Property

It can also be shown that the Geometric rv is the only discrete

random variable that satisfies the memoryless property

Because of the memoryless property, the Geometric rv can bethought of as the number of failures between two successes

OR the inter-arrival time between successes


Date post:	07-Apr-2018
Category:	Documents
Upload:	yasir-ateeq
View:	222 times
Download:	0 times

Chapter 03 Random Variables Part A

Documents