Digital Signal Processing-Random processes

7/30/2019 Digital Signal Processing-Random processes

1/121

1

Random Processes, OptimalFiltering and Model-based Signal

Processing

Elena Punskayawww-sigproc.eng.cam.ac.uk/~op205


2/121

2

Overview of the course


3/121

3

A good text for the course:

Monson H. Hayes, Statistical Digital

Signal Processing and Modeling,

Willey, 1996


4/1214

Discrete-time Random Processes

A random process is a rule that maps every outcome of anexperiment to a function.

family of functions (a single function is

identified by the outcome k

and it is just

a function of, for example, time, where t =

nT, and T the sampling interval)


5/1215

Random Processes


6/1216

Ensemble representation of adiscrete-time random process

where t = nT, and T the sampling interval

1

2

.

.

.

N

From pdf

f()

samplespace

Xn1 Xn2 Xn3 Random Vector

n1

n2

n3

Random variable


7/121

7

Discrete-time and Continuous-timeRandom process


8/121

8

Example: the harmonic process


9/121

9

Example: the harmonic process

f()

1/2

0=0T, where T sampling periodo true frequency

0

normalised frequencyindependent of T

A f b f h d h


10/121

10

A few members of the random phasesine ensemble


11/121

11

Correlation functions- Autocorrelation


12/121

12

Cross-Correlation function


13/121

13

Stationarity


14/121

14

Stationarity

Joint pdf:Random vectors, see Fig.1

Xn1

Xn2

Xn3

Random Vector

n1

n2

n3


15/121

15

Stationarity


16/121

16

Strict-Sense Stationarity


17/121

17

Wide-sense stationarity


18/121

18

Wide-sense stationarity

WSS sometimes known as weakly

stationary

This only applies for finite variance processes: a

SSS process with infinite variance is not WSS


19/121

19

Example: random phase sine-wave


20/121

20

Take three WSS conditions in turn


E[sin()]=-

sin()f() d=

sin()(1/2)d=0odd function

E[cos()]=- cos()f() d= cos()(1/2)d=0sin() = 0, sin(-) = 0

Condition 1 Ok

constant

sum of angles

constants


21/121

21

nth mth from the same member of ensemble

sinAsinB=0.5[cos(A-B)-cos(A+B)]

fixed constants

cos(A+B)=cosAcosB-sinAsinB,where B = 2and A fixed constantsimplify and show as before

E[cos(2)] = E[sin(2)]

= 0

Condition 2 Ok



22/121

22

3. Check variance is finite

x2 = E[(Xn-)

2], = 0

x2 = E[Xn

2] (autocorrelation)

= rXX[0]= 0.5A2[cos(n-n)0]

= 0.5A2 <

Condition 3 Ok



23/121

23

Power Spectra

Autocorrelation function from power


24/121

24

Autocorrelation function from powerspectrum

P S


25/121

25

Power Spectra

contribution to the mean-

square in a frequency

interval

lT lTuTuT 2

E l P S


26/121

26

Example: Power Spectrum

E l P S t


27/121

27

Example: Power Spectrum

cos as half sum of complex exp

Recall from Part 1B: Fourier series representation of a function of,

C1m= exp(jm0) and C2m= exp(-jm0)

E.g., C2m corresponds to impulse

train of period 2

C2m

= (1/2)(-0

)exp(-jm)d =

= (1/2)exp(-jm0)Sum of two impulse

trains of period 2


28/121

Whit N i


29/121

29

White Noise

=cXX[0]

1

m

[m]

Whit N i


30/121

30

White Noise

rXX[m] = cXX[m] = X2[m]

[m]=1 only form=0

E l hit G i i (WGN)


31/121

31

Example: white Gaussian noise (WGN)

1

E ample: hite Ga ssian noise (WGN)


32/121

32

Xn1 Xn2 Xn3 Random Vector

n1

n2

n3

fXn(xn) = N (xn | = 0, X2)

Example: white Gaussian noise (WGN)

Example: white Gaussian noise


33/121

33


since allXnis independent



34/121

34


Statistical characteristics are the same irrespective of shifts along the time

axis.

An observer looking at the process from sampling time n1 would not be able

to tell the difference in the statistical characteristics of the process if hemoved to a different time n2

Linear systems and random processes


35/121

35


e.g. Digital Filter



36/121

36




37/121

37




38/121

38




39/121

39




40/121

40


Time reversed impulse response



41/121

41




42/121

42


Example: Filtering white noise


43/121

43




44/121

44




45/121

45




46/121

46


m = -1, m = 0, m =1



47/121

47

p g

PeriodicMinima

-

Maxima

H(z) has a zero at z = -b0/b1=-1/0.9

Hence |H(ej) | has a minimum at= +2n

maximum at=0 + 2n

=z-plane

Ergodic Random processes


48/121

48

g p



49/121

49

g p



50/121

50

g p



51/121

51

g p

Example


52/121

52

p

f f

Example


53/121

53

p

as we were supposed to obtain

Example


54/121

54

Ergodic processes


55/121

55

Comment: Complex-valued processes


56/121

56

Summary


57/121

57

Looked at discrete-time random processes (most results for continuous

time random processes follow through almost directly)

Defined Correlation functions (auto- and cross-)

Stationarity (strict sense and wide sense 3 conditions)

Power spectrum (and calculation of autocorrelation function using

the inverse DTFT)

White noise (in particular, white Gaussian noise)

Ergodic processes

Linear system and a wide-sense stationary process

Revision: Continuous time random processes see handouts, notcovered during lectures

Optimal Filtering


58/121

58

The general Wiener filtering problem


59/121

59

+

observed

unobserved

H(z)

Need to designthis filter

The general Wiener filtering problem


60/121

60

The Discrete-time Wiener Filter


61/121

61

H(z)

Need to design

this filter

Filtering observed noisy signal


62/121

62

Mean-squared error (MSE)


63/121

63

Mean-squared error (MSE)


64/121

64

Error signal

+_

error

H(z)

Need to design

this filter

+

The Discrete-time Wiener Filter


65/121

65

Derivation of Wiener filter


66/121

66



67/121

67



68/121

68

rdx

[-q] = rxd

[q]



69/121

69



70/121

70

Mean-squared error for the optimal filter


71/121

71

Mean-squared error for the optimal filter


72/121

72

Thus, minimum error is:

Important Special Case: UncorrelatedSignal and Noise Processes


73/121

73



74/121

74



75/121

75

0



76/121

76

leads to Wiener Filter

and are

real and positive

1



77/121

77

1 +

1

1 +

1

1

SNR()

=

Signal-to-noise ratio

Example: AR Process


78/121

78

A 1storder all-pole, also known as, autoregressive process

(AR), is generated by passing a zero-mean white

noise through a first-order all-pole IIR filter

H(z) =1

1 - z-1

2

Example: AR Process


79/121

79

This is our AR

process

Example: AR Process


80/121

80

=

Example: Deconvolution


81/121

81

Consider the following process

where vn is zero-mean unit variance white noise uncorrelated with dn

Assume dn is a WSS AR(1) process with rdd[k] = (0.5)|k|

Determine the optimal Wiener filter to estimate dn from xn

xn =dn+ 0.8 dn-1 + vn



82/121

82



83/121

83and take DTFT ofrxdand rxx to obtain the Wiener filter

The FIR Wiener filter


84/121

84



85/121

85



86/121

86

The correlation matrix


87/121

87



88/121

88

Positive definite



89/121

89

Example: AR process

C id 1st

d AR hi h h t l ti f ti


90/121

90

Consider 1 order AR process which has autocorrelation function

The process is observed in zero mean white noise with variance , which

is uncorrelated with :

Design the 1st order FIR Wiener Filter for estimation of from

Now

We need to find

rdd[k] = |k|, with -1 < < 1

= |k| +v2[k]

Example: AR process


91/121

91

Example: AR process


92/121

92

1+v2

1+v2

1

-1

dddd

-1

Example: Noise cancellation


93/121

93

Design a Wiener Filter to estimate dn

Signal

Source

Noise

Source

+dn

vn

xn=dn+vn



94/121

94

First, assume dn and vn are stationary and ergodic so that we

could estimate

Also assume that a long segment ofvn is available during asilent section of music/speech

large

v vvv


For the FIR Wiener Filter


95/121

95





96/121

96




97/121

97

In the above equation rxx and rvv were estimated asexplained before and the equation below can be solvedusing, for example, Matlab

In fact, audio signals are non-stationary

Thus, need to apply to short quasi-stationary batches ofdata, one by one

Can also successfully implement a frequency domainversion using FFT

Model-based Signal Processing


98/121

98

Autoregressive moving-average model


99/121

99

Model-based Signal Processing


100/121

100

ARMA modelling


101/121

101

ARMA modelling


102/121

102

ARMA modelling


103/121

103

ARMA modelling


104/121

104

ARMA modelling


105/121

105

Autoregressive models


106/121

106

AR model


107/121

107

Autocorrelation function of AR model


108/121

108



109/121

109



110/121

110

The inverse z-

transform of1/A(z)

Yule-Walker Equations


111/121

111



112/121

112

wn

= [n] and forn = 0



113/121

113



114/121

114

r = 0

r = 1

r = P

`



115/121

115



116/121

116

Solving for AR coefficients


117/121

117

Example: AR coefficients estimation

Takep = 2


118/121

118

We have measured

rXX[0] = 6.14rXX[1] = 3.08

rXX[2] = -2.55

In practice this would be measured using ergodicity of the

process

large

Yule-Walker Equations:



119/121

119

6.14 3.08

3.08 6.14

3.08

-2.55

-0.95

0.89



120/121

120

= 6.14 +3.08 x (-0.95) +(-2.55) x (0.89)


121/121

121

Date post:	04-Apr-2018
Category:	Documents
Upload:	gramoz-cubreli
View:	216 times
Download:	0 times

Digital Signal Processing-Random processes

Documents