1 Chapter 1 Random Process 1.1 Introduction (Physical phenomenon) Deterministic model : No uncertainty about its time- dependent behavior at any instant of time . Random model :The future value is subject to “chance”(probability) Example: Thermal noise , Random data stream 1.2 Mathematical Definition of a Random Process (RP) The properties of RP a. Function of time. b. Random in the sense that before conducting an experiment, not possible to define the waveform. Sample space S function of time, X(t,s) mapping
Transcript
1
Chapter 1 Random Process1.1 Introduction (Physical phenomenon) Deterministic model : No uncertainty about its time- dependent behavior at any instant of time . Random model :The future value is subject to “chance”(probability)
Example: Thermal noise , Random data stream 1.2 Mathematical Definition of a Random Process (RP) The properties of RP a. Function of time. b. Random in the sense that before conducting an experiment, not possible to define the waveform.Sample space S function of time, X(t,s) mapping
2
(1.1)2T:The total observation interval (1.2) = sample function
At t = tk, xj (tk) is a random variable (RV).To simplify the notation , let X(t,s) = X(t)X(t):Random process, an ensemble of time function together with a probability rule.
Difference between RV and RPRV: The outcome is mapped into a numberRP: The outcome is mapped into a function of time
Tt-Tt,sX )( S
)(),( txstXs jjj )(tx j
3
Figure 1.1 An ensemble of sample functions:},,2,1|)({ njtx j
4
1.3 Stationary ProcessStationary Process :
The statistical characterization of a process is independent of the time at which observation of the process is initiated.
Nonstationary Process:
Not a stationary process (unstable phenomenon )
Consider X(t) which is initiated at t = ,
X(t1),X(t2)…,X(tk) denote the RV obtained at t1,t2…,tk
For the RP to be stationary in the strict sense (strictly stationary)
The joint distribution function
(1.3)
For all time shift , all k, and all possible choice of t1,t2…,tk
)...()..( ,)()(,,)(),...,( 1111
kk xxFxxFkk t,...,XtXτtXτtX
5
X(t) and Y(t) are jointly strictly stationary if the joint
finite-dimensional distribution of and
are invariant w.r.t. the origin t = 0.
Special cases of Eq.(1.3)
1. for all t and (1.4)
2. k = 2 , = -t1
(1.5)
which only depends on t2-t1 (time difference)
)()( 1 ktXtX )()( 1 'tY'tY j
)(F)(F)(F )()( xxx XτtXtX
),(F),( 212 )((0),)(),( 121
21xxxxF ttXXtXtX
6Figure 1.2 Illustrating the probability of a joint event.
7
Figure 1.3 Illustrating the concept of stationarity in Example 1.1.
8
1.4 Mean, Correlation,and Covariance FunctionLet X(t) be a strictly stationary RP
The mean of X(t) is
(1.6)
for all t (1.7)
fX(t)(x) : the first order pdf.
The autocorrelation function of X(t) is
for all t1 and t2 (1.8)
)()( tXEtX xxxf tX d )(
)(
)(
),(
),(
)()( )(R
12
-
- 2121)((0)21
-
- 2121)()(21
2121
12
21
ttR
dxdxxxfxx
dxdxxxfxx
tXtXE,tt
X
ttXX
tXtX
X
X
9
The autocovariance function
(1.10)
Which is of function of time difference (t2-t1).
We can determine CX(t1,t2) if X and RX(t2-t1) are known.Note that:
1. X and RX(t2-t1) only provide a partial description.
2. If X(t) = X and RX(t1,t2)=RX(t2-t1), then X(t) is wide-sense stationary (stationary process).3. The class of strictly stationary processes with finite second-order moments is a subclass of the class of all stationary processes.4. The first- and second-order moments may not exist.
)(
)()( )(C2
12
2121
XX
XXX
ttR
tXtXE,tt
10
Properties of the autocorrelation functionFor convenience of notation , we redefine
1. The mean-square value
2.
3.
(1.11) allfor , )()()( ttXτtXERX
(1.12) 0 τ , )((0) 2 tXERX
(1.13) τ)()( RRX
(1.14) (0))( XX RR
11
0)]([)]()([2)]([ 22 tXEtXtXEtXE
0)(2)0(2 XX RR
)0()()0( XXX RRR
0)(2)]([2 2 XRtXE
)0(|)(| XX RR
Proof of property 3:
0]))()([( Consider 2 tXτtXE
12
The RX() provides the interdependence information
of two random variables obtained from X(t) at times
seconds apart
13
Example 1.2 (1.15)
(1.16)
(1.17)
Θ)2cos()( tπfAtX c
elsewhere,0
,2
1)( πθπ
πf
f
θπ π
)2cos(2
)()()(2
tπfA
tXτtXER cX
14
Appendix 2.1 Fourier Transform
15
We refer to |G(f)| as the magnitude spectrum of the signal g(t), and refer to arg {G(f)} as its phase spectrum.
16
DIRAC DELTA FUNCTION
Strictly speaking, the theory of the Fourier transform is applicable only to time functions that satisfy the Dirichlet conditions. Such functions include energy signals. However, it would be highly desirable to extend this theory in two ways:
1. To combine the Fourier series and Fourier transform into a unified theory, so that the Fourier series may be treated as a special case of the Fourier transform.
2. To include power signals (i.e., signals for which the average power is finite) in the list of signals to which we may apply the Fourier transform.
17
The Dirac delta function or just delta function, denoted by , is defined as having zero amplitude everywhere except at , where it is infinitely large in such a way that it contains unit area under its curve; that is
and
)(t
0t
,0)( t 0t
1)( dtt
)()()( 00 tgdttttg
)()()( tgdtg
(A2.3)
(A2.4)
(A2.5)
(A2.6)
18
19
Example 1.3 Random Binary Wave / Pulse
1. The pulses are represented by ±A volts (mean=0).
2. The first complete pulse starts at td.
3. During , the presence of +A
or –A is random.
4.When , Tk and Ti are not in the same pulse
interval, hence, X(tk) and X(ti) are independent.
elsewhere,0
0,1
)( TtTtf d
dTd
nTttTn d )1(
Ttt ik
0)()()()( ikik tXEtXEtXtXE
20
Figure 1.6Sample function of random binary wave.
21
0)()()()( ikik tXEtXEtXtXE
4. When , Tk and Ti are not in the same pulse
interval, hence, X(tk) and X(ti) are independent.
Ttt ik
22
5. For
X(tk) and X(ti) occur in the same pulse interval
kkik tttTtti , 0 ,
T-tt
-ttT- t
id
ikd
i.e.,
iff
TttT
ttA
dtT
A
dttftXtXE
ttTtAttXtXE
ikik
-ttT-
d
dd
-ttT-
Tik
ikddik
ik
ik
d
)(1
)(A)()(
elsewhere0,
--, )()(
2
0
2
0
2
2
23
6. Similar reason for any other value of tk
What is the Fourier Transform of ?Reference : A.Papoulis, Probability, Random Variables and Stochastic Processes, Mc Graw-Hill Inc.
)(XR
where, 0,
),(1)(2
ikX -ttτTτ
TτT
τAR
24
Cross-correlation Function
and
Note and are not general even
functions.
The correlation matrix is
If X(t) and Y(t) are jointly stationary
(1.20) )()()(
(1.19) )()()(
uXtYEt,uR
uYtXEt,uR
YX
XY
),( utRYX),( utRXY
),(),(
),(),(),(
utRutR
utRutRut
YYX
XYXR
utτwhere
RR
RR
YYX
XYX
(1.21) )( )(
)( )()(
R
25
(1.22) )(
)](()([
)]()([
)]()([ )(
,Let
τR
tXtYE
XYE
YXEτR
μτt
YX
XY
Proof of :
)]()( E[)( τtYtXτRXY
)()( τRR YXXY
26
Example 1.4 Quadrature-Modulated Process
where X(t) is a stationary process and is uniformly distributed over [0, 2].
),2(sin)()(
)2(cos)()(
2
1
tπftXtX
tπftXtX
c
c
)2sin()(2
1
)231()2sin()222sin()(2
1
)22sin()2cos()()(
)()((ττ) 2112
τπfτR
. τπfEτπftπfEτR
τπftπftπfEτtXtXE
τtXtXER
cX
cccX
ccc
.orthogonal are )( and )(
, 0)( 0,)2sin( ,0 At
21
12
tXtX
Rτπfc
= 0
27
1.5 Ergodic Processes
Ensemble averages of X(t) are averages “across the process”.
Long-term averages (time averages) are averages “along the
process ”
DC value of X(t) (random variable)
If X(t) is stationary,
(1.24) )(2
1)(
T
Tx dttxT
Tμ
(1.25) 2
1
)(2
1)(E
?
X
T
T X
T
Tx
μ
dt μT
dttxET
T μ
28
represents an unbiased estimate of
The process X(t) is ergodic in the mean, if
The time-averaged autocorrelation function
If the following conditions hold, X(t) is ergodic in the
autocorrelation functions
0)(varlim b.
)(lim a.
T μ
μTμ
xT
XxT
)(Tx X
variable.randoma s ) (
)261()()(2
1)(
iT,R
. dt txτtxπ
τ,TR
x
T
Tx
0)(varlim
)(),(lim
,TR
τRTR
xT
XxT
29
1.6 Transmission of a random Process Through a Linear Time-Invariant Filter (System)
where h(t) is the impulse response of the system
If E[X(t)] is finiteand system is stable
If X(t) is stationary,H(0) :System DC response.
111 )()()( dττtXτhtY
-
(1.28) )()(
)()(
(1.27) )()(
)()(
111
111
111
- X
-
-
Y
dττtμτh
dττtxEτh
dττtXτhE
tYE tμ
(1.29) ),0( )( 11 Hμdττhμμ X
-XY
30
Consider autocorrelation function of Y(t):
If is finite and the system is stable,
If (stationary)
Stationary input, Stationary output
)301( )()( )()(
)()()(
222111 . dττμXτhdττtXτhE
YtYEt,R
Y
)](E[ 2 tX
)311( )()( )( ) 212211 . τ,τtRτhdττhdτ(t,μR X
Y
)(),( 2121 ττμtRτμτtR XX
)321( )()()()( 212121 .dτdττττRτhτhτR X
Y
X
Y .dτdτττRτhτhtYER )331( )()()()()0( 2112212
31
1.7 Power Spectral Density (PSD) Consider the Fourier transform of g(t),
Let H(f ) denote the frequency response,
dfπftjfGtg
dtπftjtgfG
)2exp()()(
)2exp()()(
filter) theof response conjugate(complex )(
)361(
)351(
(1.34)
)2exp()()2exp()(
)2exp()()(
)()()2exp()()(
)2exp()()(
222
111222
2112212
11
f*
H
.
.
dfτjRfτj)h(τdτfdf H
dτ fτj)τ(τRτhdτfdf H
dτdτ ττRτh dffτjfHtYE
df πfτjfHτh
X
X
X
12 -
32
: the magnitude responseDefine: Power Spectral Density ( Fourier Transform of )
Recall Let be the magnitude response of an ideal narrowband filter
f : Filter Bandwidth
)371( )2exp())()(22 . dτfj(τRfHdftYE
- X-
)( fH
)τR(
)391( )()()(
)381( )2exp()()(
22 .dffSfHtYE
. dπfτRfS
- X
- XX
)331( )()()( 2 11212
.dτdτττRτhtYE X- -
W/Hz in )(Δ2)(
,continuous is )( andΔ If2
cX
Xc
ff StYE
fSff
)( fH
(1.40) ff,
ff,|f|H
f
f
c
c
2121
0
1)(
33
Properties of The PSD
Einstein-Wiener-Khintahine relations:
is more useful than !
)431( )2exp()()(
)421( )2exp()()(
. df fjτSτR
. dfjτRfS
XX
XX
)( fSX )(τRX
)()( τRfS XX
34
.df fS
fSf
e.
.fS
τ udufujuR
dτfjτRfd. S
. f fS
fSftYE
t Xc.
. df fStXb. E
. dτRa. S
X
XX
X
X
XX
X
X
X
XX
p )481( )(
)()(
:pdfa withassociated be can PSDThe
)471( )(
, )2exp()(
)2exp()()(
)461( allfor 0)(
0)()Δ2()(
,stationary is)(If
)451( )()(
)441( )()0(
2
2
35
Example 1.5 Sinusoidal Wave with Random Phase
)( )(2exp ,2Appendix
)()(4
)2exp()2exp()2exp(4
)2exp()()(
)2cos(2
)(
) ,(~ ),2cos()(
2
2
2
cc
cc
cc
XX
cX
c
ffdffj
ffffA
dfjfjdfjA
dfjRfS
fA
R
UtfAtX
36
Example 1.6 Random Binary Wave (Example 1.3)
Define the energy spectral density of a pulse as
(1.50) )(sincA
)2exp()1()(
0
)1()(
0if
1if ,
, )(
22
2
2
f TT
dfjT
AfS
T
TT
AR
m(t)
m(t)
A
AtX
T
TX
X
(1.52) )(
)(S
(1.51) )(sinc)( 222
T
ff
f TTAfε
g
X
g
37
Example 1.7 Mixing of a Random Process with a Sinusoidal Process
(1.55) )()(4
1
))(2exp())(2exp()(4
1
)2exp()( )(
(1.54) )2cos()(2
1
)224cos()2cos()(2
1
)2cos()22cos()()(
)()( )(
(1.53) )(0,2~ , )2cos()()(
cXcX
ccX
YY
cX
cccX
ccc
Y
c
ffSffS
dffjffjR
dfjRfS
fR
ftffER
tfftfEtXτtXE
tYτtYER
UtftXtY
We shift the to the right by , shift it to the left by ,
add them and divide by 4.
)( fSX cf cf
38
Relation Among The PSD of The Input and Output Random Processes
Recall (1.32)
(1.58) )()(
)(*)()(
)2exp()2exp()2exp()()()()(
or ,
)2exp()()()()(
(1.32) )()()()(
2
021020121
210 021
2 12121
2 12121
fSfH
fHfHfS
dτdτdτfjfjfjRhhfS
Let
dddfjRhhfS
ddRhhR
X
X
XY
XY
XY
h(t)X(t)
SX
(f)
Y(t)
SY
(f)
39
Relation Among The PSD and The Magnitude Spectrum of a Sample Function
Let x(t) be a sample function of a stationary and ergodic Process X(t).In general, the condition for Fourier transformable is
This condition can never be satisfied by any stationary x(t) with infinite duration.
We may write
If x(t) is a power signal (finite average power)
Time-averaged autocorrelation periodogram function
(1.61) )()(2
1lim)(
average timeTake Ergodic
(1.60) )2exp()(),(
dttxtxT
R
dtftjtxTfX
T
TTX
T
T
(1.59) )( dttx
(1.62) ),(2T
1 )()(
2
1 2TfXdttxtx
T
T
T
40
Take inverse Fourier Transform of right side of (1.62)
From (1.61),(1.63),we have
Note that for any given x(t) periodogram does not converge as
Since x(t) is ergodic
(1.67) is used to estimate the PSD of x(t)
(1.63) )2exp(),(2
1)()(
2
1 2df fTjTfX
Tdttxtx
T
T
(1.64) )2exp(),(2
1lim)(
2dffjTfX
TR
TX
T
(1.67) )2exp()(
2
1lim
),(2
1lim)(
)2exp()()( (1.43) Recall
(1.66) )2exp()(2
1lim)(
)2exp()(2
1lim)()(
2
2
2
2
T
TT
TX
XX
TX
TXX
dtftjtxET
TfXET
fS
dffjfSR
dffjTfXET
R
dffjTfXET
RRE
41
Cross-Spectral Densities
(1.72) )()()(
(1.22) )()(
)2exp()()(
)2exp()()(
real. benot may )( and )(
(1.69) )2exp()()(
(1.68) )2exp()()(
fSfSfS
τRτR
dfπfτjfSτR
dfπfτjfSτR
fSfS
dfjRfS
dfjRfS
YXYXXY
YXXY
YXYX
XYXY
YXXY
YXYX
XYXY
42
Example 1.8 X(t) and Y(t) are zero mean stationary processes.
Consider
Example 1.9 X(t) and Y(t) are jointly stationary.
(1.75) )()()(
)()()(
fSfSfS
tYtXtZ
YXZ
)()()()(
(1.77))()()( )(
Let
),()()(
)()()()(
)()( ),(
21
21212211
21212211
22221111
fSfHfHfS
ddRhhR
ut τ
ddutRhh
duYhdtXhE
uZtVEutR
XYVY
XYVZ
XY
VZ
F
43
1.8 Gaussian ProcessDefine : Y as a linear functional of X(t)
The process X(t) is a Gaussian process if every linear functional of X(t) is a Gaussian random variable
T
dttXtgY0
(1.79) )()(
(1.81) (0,1) as , )2
exp(2
1)( Normalized
(1.80) 2
)(exp
2
1)(
2
2
2
Ny
yf
yyf
Y
Y
Y
Y
Y
( g(t): some function)
( e.g g(t): (e) )
Fig. 1.13 Normalized Gaussian distribution
44
Central Limit Theorem
Let Xi , i =1,2,3,….N be (a) statistically independent R.V.
and (b) have mean and variance .Since they are independently and identically distributed (i.i.d.)Normalized Xi
The Central Limit TheoremThe probability distribution of VN approaches N(0,1)as N approaches infinity.
N
iiN
i
i
XiX
i
YN
Y
Y
NiXY
1
1V Define
.1Var
,0 EHence,
1,2,...., )(1
Xμ 2
Xσ
45
Properties of A Gaussian Process1.
Gaussian is 0 ,)()()(
(1.81) variablerandom Gaussiana is definitionBy
)()()( where
)()(
)()()(
)()()( Define
)()()(
0
0
0
0
0
0
0
0
t dXthtY
Z
dtthtgg
dτXg
dτ dt Xthtg
dt dτXthtgZ
dXthtY
T
Y
T
T
Y
T
Y
T
X(t)h(t)
Y(t)
Gaussian Gaussian
46
2. If X(t) is Gaussisan
Then X(t1) , X(t2) , X(t3) , …., X(tn) are jointly Gaussian.
Let
and the set of covariance functions be
,....,n,itXE itX i21 )()(
Σ
Σ
μ
μxΣμx
X
matrix covariance oft determinan
)},({matrix covariance
,....,, vector mean where
(1.85) ))()(2
1exp(
)2(
1)...,( Then
)()()( where
21 , )()(),(
1,
21
1
21
2,1)(),...,(
21
)()(
1
nikikX
Tn
TnntXtX
Tn
tXitXkikX
ttC
xxf
t,....,Xt,XtX
,...,n,k,itXtXEttC
n
ik
47
3. If a Gaussian process is stationary then it is strictly stationary.
(This follows from Property 2)
4. If X(t1),X(t2),…..,X(tn) are uncorrelated as
Then they are independent Proof : uncorrelated
is also a diagonal matrix(1.85)
.21, ]))(()([( where,
0
0
22
2
21
,n,itXEtXE iii
n
Σ
2
2
1
1
21
2,1)(,),(
2exp
2
1)( and )( where
)()(
))(2
1exp(
)2(
1)...,(
1
i
Xi
i
iXii
n
iiXX
T
nntXtX
i
i
i
n
xxftXX
xff
xxf
x
μxΣμx
0)])()()([( )()( ik tXitXk tXtXE
1Σ
48
1.9 Noise
· Shot noise
· Thermal noise
k: Boltzmann’s constant = 1.38 x 10-23 joules/K, T is the absolute temperature in degree Kelvin.
22
22
22
amps 41
41
volts 4
fkTGfR
kTVER
IE
fkTRVE
TNTN
TN
49
· White noise
(1.95) )(2
)(
receiver theof re temperatunoise equivalent:
(1.94)
(1.93) 2
)(
0
0
0
NR
T
kTN
NfS
W
e
e
W
50
Example 1.10 Ideal Low-Pass Filtered White Noise
)2sinc(
(1.97) )2exp(2
)(
(1.96) 02)(
0
0
0
BBN
df fjN
R
B f
B f-B NfS
B
BN
N
51
Example 1.11 Correlation of White Noise with a Sinusoidal Wave
White noise
(1.99) 2
)2(cos
)2cos()2cos()(2
2
(1.95) From
)2cos()2cos(),(2
)2cos()2cos()()(2
)2cos()()2cos()(2
is )( of varanceThe
(1.98) )2cos()(2
)('
0
020
210 0 212102
210 0 2121
210 0 2121
210 0 22112
0
T
c
T T
cc
T T
ccW
T T
cc
T T
cc
T
c
N dtt f
T
N
dt dtt ft fttN
T
dt dtt ft fttRT
dt dtt ft ftwtwET
dt dtt ftwt ftwT
E
tw'
dttftwT
tw
X T
dt0
integer is , , )2cos(2
kT
kftf
Tcc
)(tw )(tw'
52
1.10 Narrowband Noise (NBN)
Two representations
a. in-phase and quadrature components (cos(2 fct) ,sin(2 fct))
b.envelope and phase
1.11 In-phase and quadrature representation
signals pass-low are )( and )(
(1.100) )2sin()()2cos()()(
tntn
t ftnt ftntn
Q
Q
I
ccI
53
Important Properties1.nI(t) and nQ(t) have zero mean.
2.If n(t) is Gaussian then nI(t) and nQ(t) are jointly Gaussian.
3.If n(t) is stationary then nI(t) and nQ(t) are jointly stationary.
4.
5. nI(t) and nQ(t) have the same variance .
6.Cross-spectral density is purely imaginary.
7.If n(t) is Gaussian, its PSD is symmetric about fc, then nI(t) and nQ
(t) are statistically independent.
(1.101) otherwise
0
, )()()()(
Bf-BffSffSfSfS cNcN
NN QI
2
0N
(1.102)
otherwise
0
,
)( )(
Bf-BffSffSj
fSfS
cNcN
NNNN IQQI
54
Example 1.12 Ideal Band-Pass Filtered White Noise
).2sinc(2)()(
), offactor (a (1.97) withCompare
(1.103) )2cos()2sinc(2
)2exp()2exp()2sinc(
)2exp(2
)2exp(2
)(
0
0
0
00
BBNRR
fBBN
fj fjBBN
df fjN
df fjN
R
QI
c
c
c
c
NN
c
cc
Bf
Bf
Bf
BfN
55
1.12 Representation in Terms of Envelope and Phase Components
Let NI and NQ be R.V.s obtained (at some fixed time) from nI(t)
and nQ(t). NI and NQ are independent Gaussian with zero mean and variance .
(1.107) )(
)(tan)(
Phase
(1.106) )()()(
Envelope
(1.105) )(2cos)()(
1
21
22
tn
tnt
tntntr
ttftrtn
I
Q
QI
c
2
56
(1.108) )2
exp(2
1),(
2
22
2, QI
QINN
nnnnf
QI
(1.112)
(1.111) sin
(1.110) cosLet
(1.109) )2
exp(2
1),(
2
22
2 ,
r dr dψ dndn
ψrn
ψrn
dn dnnn
dndnnnf
QI
Q
I
QIQI
QIQINN QI
57
Substituting (1.110) - (1.112) into (1.109)
(1.118) elsewhere 0
0 , )2
exp()(
)()(. let , econveniencFor
on.distributi Rayleighis )(
(1.115) elsewhere 0
0 , )2
exp()(
(1.114) elsewhere
20
02
1)( ,20
(1.113) )2
exp(2
),(
)2
exp(2
),(),(
2
2
2
2
2
2
2,
2
2
2
, ,
νf
rfνfσ
rν
rf
rrr
rf
f
rrrf
rdrdrr
rdrdrfdndnnnf
V
RV
R
R
Ψ
ΨR
ΨRQIQINN QI
58
Figure 1.22 Normalized Rayleigh distribution.
59
1.13 Sine Wave Plus Narrowband Noise
If n(t) is Gaussian with zero mean and variance
1. and are Gaussian and statistically independent.
2.The mean of is A and that of is zero.
3.The variance of and is .
)()(
)2sin()()2cos()()(
(1.119) )()2cos()(
tnAtn
tftntftntx
tntfAtx
II
cQcI
c
dependent. are and
)2
cos2rexp(
2),(
have we, proceduresimilar a Follow
(1.124) )(
)(tan(t)
(1.123) )()( )( Let
2
)(exp
2
1),(
2
22
2,
1-
2
122
2
2
2,
2
R
ArArrf
tn
tn
tntntr
nAnnnf
ΨR
I
Q
QI
QIQINN QI
2
2
)(' tnI )(tnQ
)(' tnI )(tnQ
)(' tnI )(tnQ
60
The modified Bessel function of the first kind of zero
