Date post: | 29-Dec-2015 |
Category: |
Documents |
Upload: | lindsay-curtis |
View: | 214 times |
Download: | 0 times |
tch-prob 1
Chap 3. Random VariablesThe outcome of a random experiment need not be a number.
However, we are usually interested in some measurement or numeric attribute of the outcome.
For example, toss a coin n times, total number of heads = ?
What is the prob. of the resulting numerical values ?
A random variable X is a function that assigns a real number, , to each outcome .
X
X
real line
s domain
Sx Range
( )X x
tch-prob 2
Ex.3.1 : HHH HHT HTH THH HTT THT TTH TTT
: 3 2 2 2 1 1 1 0
Let B be some subset of Sx
Event B in Sx occurs whenever
event A in S occurs.
Events A and B are equivalent events.
X
:A X in B A
B
S
real line
:P B P A P X in B
0,1,2,3XS
tch-prob 4
3.2 Cumulative Distribution Function
Cumulative distribution function (cdf) of a random variable X is defined as
a convenient way of specifying the probability of all semi-infinite intervals of the real line of the form .
:
XF x P X x for x
P X x
( , ]x
tch-prob 5
Cdf has the following properties:
i. cdf is a prob., axiom 1 & corollary 2.
ii. is the entire sample space & Axiom II.
iii. is an empty set. Corollary 3.
iv. nondecreasing function, corollary 7.
v. continuous from the right. h>0
0 1X
F x
lim 1X
F xx x
lim 0X
F xx
,X X
if a b then F a F b
lim0X X X
F b F b h F bh
x
tch-prob 6
vi.
vii.
if cdf is continuous at b.
viii. Corollary 1.
X X
P a X b F b F a
X a a X b X b
vi, 0...
0
X X
X X
P X b F b F b
Let a b in P b X b F b F b as
P X b
1X
P X x F x
( ) ( ) ( ) ( )X X X X X XP a X b F a F a F b F a F b F a
a X b X a a X b
tch-prob 7
Ex.3.4. Fig 3.3 toss a coin 3 times. Count Heads.
X
7/8
1/2
1
1/8
Fx(x)
1 3 3 11 2 38 8 8 80 0
1 0
F x u x u x u x u xXx
u xx
X
1/8
fx(x)
0 1 32
1/8
3/83/8 probability mass function (pmf)
For discrete random variable, cdf is
right-continuous, staircase function of x.
1
2
3
4
1/ 8 0
3/ 8 1( )
3 / 8 2
1/ 8 3
X k
x
xp x
x
x
tch-prob 8
Ex.3.5. The transmission time X of messages obeys the exponential probability law with parameter .
is continuous for all x, its derivative exists everywhere except at x=0.
pdf.
0 0
1 0
0
find 1X
x
x
e x
xP X x e x
cdf F x P X x P X x
Fx(x) 1
x
'0 0
0X
xF x
xe x
F'x(x)
x
1 xe xe
( )XF x
tch-prob 9
Continuous random variable
cdf is continuous everywhere, and smooth enough.
can be written as an integral of some
nonnegative function f(x)
property (vii)
( )xX
f t dtF x
0 for allP X x x
tch-prob 10
Random variable of mixed type
Ex. 3.6 The waiting time X of a customer in a queueing system is zero if he finds the system idle (p), and an exponentially distributed random length of time if he finds the system busy (prob. 1-p).
0 0
(1 )(1 ) 0X x
x
p p e xF x
[ ] [ | ] [ | ](1 )
XF x P X x
P X x idle p P X x busy p
tch-prob 11
3.3 Probability Density Function (pdf)
The pdf of X, if it exists, is defined as
( ) ( )
( ) ( )
0
XX
X X
X X
X
dF xf x
dx
P x X x h F x h F x
F x h F xh
h
as h P x X x h f x h
density
tch-prob 12
i.
ii.
iii.
iv.
0X
f x
XbP a X b f x dxa
X XxF x f t dt
1X
f t dt
since is nondecreasing( )XF x
pdf completely specifies the behavior of continuous random variables.
tch-prob 14
The derivative of the cdf does not exist at points where the cdf is not
continuous.
To generalize pdf for discrete random variable.
Define delta function
xu x t dt
0 x
1
0 x
0 0
1 0u x
x
x
( )x
tch-prob 15
Ex
X k k
X k k
p u
p
F x x x xxk
f x x x xxk
X k k
p x P X x
X
1/8
fx(x)
0 1 32
1/8
3/83/8
X
7/8
1/2
1
1/8
Fx(x)
tch-prob 16
Conditional cdf’s and pdf’s
The conditional cdf of X given A is
satisfies all the properties of a cdf.
The conditional pdf. of X given A is
0
X
X
P X x AF x A if P A
P A
F x A
X Xdf x A F x Adx
tch-prob 17
Ex.3.10 The lifetime X of a machine has a continuous cdf .
Find the conditional cdf and pdf given A={X>t}.
0
1
1
X
X XX
X
XX
X
F x X t P X x X t
P X x X t
P X t
x t
F x F tF x X t
x tF t
f xx X t x t
F tf
( )XF x
tch-prob 18
3.4 Some important random variables
- Discrete Random Variables
1. Bernoulli r.v.
2. Binomial Random variable
X: number of times a certain event occurs in n independent trials.
0( )
1A
notif in AI
if in A
0 1
1I
I
p p
p p
1, 2, ,
1 0, ,
A A n A
n kk
X I I In
P X k p p for k nk
Indicator function for A
is a r.v. with pmfAI
tch-prob 19
3. Geometric r.v.
M indep. Bernoulli trials until the first success
or M’=M-1 , number of failures before a success
the only discrete r.v. that satisfies the memoryless property:
11 1,2,
kP M k p p k
' 1 1 0,1,2,kP M k P M k p p k
1
for all , 1
1: 1 1
[ , ] [ ] (1 )
[ ] [ ]k
P M k j M j P M k j k
j k jPf P M j p P M k j p
P M k j M jP M k j M j P M k j p
P M j P M j
tch-prob 20
4. Poisson r.v.
counting the number of occurrences of an event in a time period.
average number of event occurrences in a time interval t.
0,1,2,...!
:
kP N k e k
k
1!0!0
ke e e
kk
ke
kk
t
Figure 3.10.
tch-prob 21
Binomial prob. Poisson prob. As
, 0,n p np
11
1
1
. . 1!
1! !1
1 ! 1 ! 11
1
1 1 1 1
1
0,1,2,...1
kn kk
k
n kk
k
n kkk
k k
ni e p p p e
k k
np p
k n k pkpnp k n k p
p pk
kn k p nk p k n
as nk
p p for kk
tch-prob 23
- Continuous r.v.
1. uniform r.v.
2. Exponential r.v.
model the time between event occurrences.
1
0 and
0
1
a x bb a
x a x b
x a
x aa x b
b ax b
f xx
F xx
a b X
1---b-a
a b X
1
0 0
0
0 0
1 0
X
X
xf x
xe x
xF x
xe X
: rate at which events occur
fx(x)
x
tch-prob 24
Exponential r.v. is limiting form of the geometric r.v.
- An interval of duration T is divided into subintervals of length
- Perform a Bernoulli trial on each subinterval with prob. of success
- The number of subintervals until the occurrence of a successful event is a geometric r.v. M.
- Thus, the time until the occurrence of the first successful event is X=M (T/n)
0 T 2 3T T T
n n n
Tn
pn
1
1
tT
nP X t P M tT
ntp T
tn Tn
tTe e as n
tch-prob 25
For a Poisson r.v., the time between events is an exponentially
distributed r.v. with parameter events/sec.
Exponential r.v. also has the memoryless propertyT
0
( )
P X t h X tP X t h X t for h
P X t
t hP X t h etP X t e
he P X h
The probability of having to wait at least h additional seconds given that one has already been waiting t seconds = The probability of waiting at least h seconds when one first begin to wait.
tch-prob 26
3. Gaussian (Normal) r.v.
Sum of a large number of small r.v.s
p.d.f
cdf.
change of variable
where
2' 22
21 22
2
1 '2
X
x mx
xx
f x e
P X x e dx
m
2
21
2
tx m
X e dt
x m
F x
'x mt
X2 2m m m m m
x
2
212
tx
e dtx
tch-prob 27
Ex. 3.14. Show that Gaussian pdf integrates to one.
2 2
2
2 22 21 12 2 2
22
12
x y
yx xe dx e dx e dy
e dxdy
cos , sin ,let x r y r 2
2
2
22
0 0
2
0
2
12
10
r
r
r
e rdrd
re dr
e
tch-prob 28
Ex.3.15.
4.753, 9.506 /
6[ 0] 102 2
2v
v m v vP Y P N v Q
v
-6
Output voltage , where is input voltage and is
Gaussian noise with m=0, =2. Find such that P[ <0]=10 .
Y V N V N
V Y
tail of the pdf
21 2Q 12
Q 1 Q
tx x e dtx
x x x
Q-function Table 3.3
It is sometimes convenient to work with Q(x).
tch-prob 29
Q(x)
x Q(x)
0 0.500
1.0 0.159
2.0 2.28E-2
3.0 1.35E-3
4.0 3.17E-5
5.0 2.87E-7
6.0 9.87E-10
k
1 1.2815
2 2.3263
3 3.0902
4 3.7190
5 4.2649
6 4.7535
7 5.1993
)10(1 kQx
tch-prob 30
4. Gamma r.v.
where is the gamma function
10 , 0, 0X
xx ef x x
z
0
1 0
12
1 0
1 !
z
z
z xx e dx z
z z z
m m
m non negative integer
11
,1 !X
m xx em f x
m
m-Erlang r.v.
exponential r.v.
Figure 3.14
tch-prob 31
※ : the time until the occurrence of the mth event
Assume the times between events are exponential r.v., (Poisson r.v. limiting case)
Let N(t) be the Poisson r.v. for the number of events in t seconds.
※ iff
m th event occurs before t m or more events occur in t second.
, , ,1 2X X Xm
1 2S X X Xm m
( )S t N t mm
1
0
11
1
1
1!
1 ! !
1 !
m m
kmt
k
k kmt t t
mk
m
t
Fs t P S t P N t m
te
k
t tfs t e e e
k k
te
k
m-Erlang
mS
tch-prob 32
3.5 Functions of a Random Variable
X: r. v. g(x): real-valued function
Y=g(X) is also a r.v.
Event C in Y <=> equivalent event B in X
[ in ] [ ( ) in ] [ in ]P Y C P g X C P X B
( )x g x
{ } { ( ) }Y y g X y
[{ }] [{ ( ) }]P Y y P g X y
tch-prob 33
Ex 3.21. X: # of active speakers in a group of N indep. speakers
p: Prob. that a speaker is active
A Voice transmission system can transmit up to M voice signals at
a time
Y: # of Signals discarded
in {0,1,2,...,M}
0
0 10
1 0
N jj
N M kM k
X
x MY X M
X M M x NM N
P Y P pjj
NP Y k P X M k p p k N M
M k
p
M N
N-M
Y
X
tch-prob 34
Y=aX+b , where a is nonzero.
Suppose X is continuous and has cdf , Find .
0
1 0
X
X
y b y bP X F if aa a
y b y bP X F if aa a
Y
x
y
Y=ax+b
a>0 YF y P Y y P aX b y
pdf.
1 0
1 0
1
X
X
X
y bf adF y a aYf yY dy y bf a
a a
y bfa a
( )YF y( )XF x
tch-prob 35
Ex.3.24 X: Gaussian
2
2
0 0
0
02 2
2 2
YX X
X XY
X X
Y X
P Y y P X y P y X y
yF y
F y F y y
f y f yf y y
y y
f y f yy y
If has n solutions, , then will have n terms.1 2, , ,
nx x x
X
y
0( )
Yf y0 ( )y g x
tch-prob 36
Consider a nonlinear function Y=g(X)
event
Its equivalent event
yC y Y y dy
1 1 1
2 2 2
3 3 3
1 1 2 2 3 3
y
Y
y X X X
B x X x dx
x dx X x
x X x dx
P C f y dyy
P B f x dx f x dx f x dx
yy+dy
x1 x2 x3x3+dx3
dx2 is negative
XY
k
X
k
f xf y
dykx xdx
dxf x dy x xk
Ex.3.27.
tch-prob 37
Ex. 3.28 Y=cos(X) , X: uniformly distributed in
for –1< y <1 , y has two solutions
(0,2 ]
10
1 0
cos
2
x y
x x
10
0
0
2 2
20
2
cos
sin( ) sin cos
si
1
1
n
1,
d
y x
z
yx y
x
y z y
d
y
z
xx
y
0 1cos y
2
tch-prob 39
3.6 The Expected Value of R.V.s
c.d.f. or p.d.f provides complete description of a r.v.
Sometimes interested in a few parameters that summarize the information
Expected value of X or mean of X is defined by
for discrete r.v.
X
k x kk
E X t f t dt
or E X x p x
X
k x kk
E X t f t dt
or E X x p x
The expected value is defined if
tch-prob 40
When the pdf is symmetric about a point m, i.e.,
If
Ex. .Gasussian
uniform
2
E X m
a bE X
( ) ( ), X Xf m x f m x then E X m
0 X
x
t m f t dt
tf t dt m
tch-prob 41
00 0
0
0 0
1
X X X
X
Xtf t dt tF t F t dt t F t d
F dt
t
t
0
0
1 x
k
E X F t dt
or E X P X k
X continuous
X integer-valued
When X is a non-negative r.v.
1
tch-prob 42
Ex. Exponential r.v.
0 0
0
( ) ( ) 1
11
t tx x
tx
t
f t e F t e
E X F t dt e dt
or t e dt
Expected value of Y=g(X)
Y
X
E Y yf y dy
g x f x dx
tch-prob 43
Ex.3.33 constant
uniform r.v. in
cos ,Y a t
(0,2 )
2
0
2 2 2
cos
1cos
221
sin 002
cos
E Y E a t
a t d
a t
E Y E a t
2 2
2 2 22
0
cos(2 2 )2 2
1cos(2 2 )
2 2 2 2
a aE t
a a at d
, ,a t
tch-prob 44
If c is a constant
If
1
1
20 1 2
20 1 2
20 1 2
( )
: ( )
[ ] [ ] [ ] [ ] [ ]
[ ] [ ] [ ]
n
kk
n
kk
nn
nn
nn
Y g X
E Y E g X
Ex Y g X a a X a X a X
E Y E a E a X E a X E a X
a a E X a E X a E X
E X c E X c
[ ] ( ) ( )
[ ] ( ) ( ) [ ]
E c cf x dx c f x dx c
E cX cxf x dx c xf x dx cE X
tch-prob 45
2
22
22
22
2
2
Var X E X E X
E X E X X E X
E X E X E X E X
E X E X
2
2
2
2
2
2
2
2
2 2
1
2
1
2
1
2
x m
x m
x m
e
x m e dx
e
Variance of X
Expected value provides limited info., also want to know variation magnitude
Ex. 3.38. Var. of Gaussian
(x-m)
dx
tch-prob 46
Var [c]=0
Var [X+c]= Var [X]
Var [cX]= 2 [ ]c Var X
2 2
2 2
22 2
0E c E c E c c
E X c E X c E X E X
E cX E cX E c X E X
nth moment of the random variable X
( )n nXE X x f x dx
tch-prob 47
3.7 Markov and Chebyshev Inequalities
Suppose X is a non-negative r.v.,
0[ ] ( ) ( )
( )
( )
a
x xa
xa
xa
E X tf t dt tf t dt
tf t dt
af t dt aP X a
E XP X a
a
Markov inequality
1
a
tch-prob 48
Suppose are known.
Let
and use Markov inequality, we obtain
2 2( )D X m
2[ ] , [ ]E X m Var X
22
2 22 2
22 2
2
2
2
E X mP D a
a a
P X m aa
P X m aa
Chebyshev inequality
tch-prob 49
Ex. 3.42 Suppose
Then the Chebeshev inequality for gives
2[ ] , [ ]E X m Var X
2
2 2 2
1 (=0.25 for 2)P X m k k
k k
Now suppose that we know that X is a Gaussian r.v., then for k=2
a k
2 0.0456P X m
tch-prob 50
3.9 Transform Methods
useful computational aids in the solution of equations that involves derivatives and integrals of functions.
A. Characteristic Function
Expected value of
Fourier Transform of
1.
2
j XX
j
X
xX X
j x
f x
E e
f x e d
e d inve
x
rse FT
j X
X
e
f x
Ex.3.47. Exponential r.v.
check p.101.
0 0
j xx j xX e e dx e dx
j
tch-prob 51
If X is a discrete r.v. .
kj xX X k
k
p x e If X is a discrete integer-valued r.v.,
j kX X
k
p k e
A periodic function of with period 2
2
0
10, 1, 2,...
2j k
X X
X
p k e d k
Fourier series coefficients of
tch-prob 52
If f(x) is a periodic function of period , then f(x) can be represented as
, Fourier series of ( )
1( ) , 0, 1, 2,
2
jkxk
k
jkxk
f x c e f x
c f x e dx k
2
tch-prob 53
Moments of X can be obtained from by X
0
2
2 2
0
0
1
( ). 1 ...
2!
1 ... ...2! !
nn
Xn n
X X
n n
X
nn n
Xn
dE X
j d
j xpf f x j x dx
j E X j E Xj E X
nd
jE Xd
dj E X
d
tch-prob 54
Ex.3.49 exponential
2
3
22 2
222 2 2
'
' 0 1
2''
'' 0 2
2 1 1
X
X
X
X
X
wj
jw
j
E Xj
j
E Xj
Var X E X E X
Check p.101
tch-prob 55
B. Probability Generating Function
For nonnegative r.v.
a. if N is nonnegative integer-valued r.v.
prob. Generating Function of N
0
0
1|
!
NN
kN
k
k
jN
N N
N
zk
G z E z
p k z
dp
G e
k G zk dz
tch-prob 56
11 1
0 0
22
1 120 0
2
2" ' '
|
| 1 1
1
1 1 1
kN z N z N
k k
kN z N z N
k k
N N N
dG z p k kz kp k E N
dz
dG z p k k z k k p k
dz
E N N E N E N
Var N G G G
tch-prob 57
b. if X is a non-negative continuous r.v.
Laplace transform of the pdf
0
01
sx sXX
nnn
sn
X s f x e dx E e
dE X X s
ds
Ex. 3.51. Laplace transform of the gamma pdf
1
0
1
0
1
0
1
xsx
s x
y
x eX s e dx
x e dx
y e dys
s