Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 1 of 78 ECE 3800
Henry Stark and John W. Woods, Probability, Statistics, and Random Variables for Engineers, 4th ed.,
Pearson Education Inc., 2012. ISBN: 978-0-13-231123-6
Chapter 9 Random Processes Sections 9.1 Basic Definitions 544 9.2 Some Important Random Processes 548 9.3 Continuous-Time Linear Systems with Random Inputs 572 White Noise 577 9.4 Some Useful Classifications of Random Processes 578 Stationarity 579 9.5 Wide-Sense Stationary Processes and LSI Systems 581 Wide-Sense Stationary Case 582 Power Spectral Density 584 An Interpretation of the Power Spectral Density 586 More on White Noise 590 Stationary Processes and Differential Equations 596 9.6 Periodic and Cyclostationary Processes 600 9.7 Vector Processes and State Equations 606 State Equations 608 Summary 611 Problems 611 References 633
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 2 of 78 ECE 3800
9.1 Basic Concepts
A random process is a collection of time functions and an associated probability description.
When a continuous or discrete or mixed process in time/space can be describe mathematically as a function containing one or more random variables.
A sinusoidal waveform with a random amplitude. A sinusoidal waveform with a random phase. A sequence of digital symbols, each taking on a random value for a defined time period
(e.g. amplitude, phase, frequency). A random walk (2-D or 3-D movement of a particle)
The entire collection of possible time functions is an ensemble, designated as tx , where one particular member of the ensemble, designated as tx , is a sample function of the ensemble. In general only one sample function of a random process can be observed!
Think of: 20,sin twAtX
where A and w are known constants.
Note that once a sample has been observed … 111 sin twAtx
the function is known for all time, t.
Note that, 2tx is a second time sample of the same random process and does not provide any “new information” about the value of the random variable.
221 sin twAtx
There are many similar ensembles in engineering, where the sample function, once known, provides a continuing solution. In many cases, an entire system design approach is based on either assuming that randomness remains or is removed once actual measurements are taken!
For example, in communications there is a significant difference between coherent (phase and frequency) demodulation and non-coherent (i.e. unknown starting phase) demodulation.
On the other hand, another measurement in a different environment might measure 21212 sin twAtx
In this “space” the random variables could take on other values within the defined ranges. Thus an entire “ensemble” of possibilities may exist based on the random variables defined in the random process.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 3 of 78 ECE 3800
For example, assume that there is a known AM signal transmitted:
twtAbts sin1
at an undetermined distance the signal is received as
20,sin1 twtAbty
The received signal is mixed and low pass filtered …
20,cossin1cos twtwtAbthtwtythtx
20,sin2sin5.01cos twtAbthtwtythtx
If the filter removes the 2wt term, we have
20,sin2
1cos tAbtwtythtx
Notice that based on the value of the random variable, the output can change significantly! From producing no output signal, ( ,0 ), to having the output be positive or negative ( 20 toorto ). P.S. This is not how you perform non-coherent AM demodulation.
To perform coherent AM demodulation, all I need to do is measured the value of the random variable and use it to insure that the output is a maximum (i.e. mix with mtw cos , where.
1tm
Note: the phase is a function of frequency, time, and distance from the transmitter.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 4 of 78 ECE 3800
From our textbook
Random Stochastic Sequence
Definition 8.1-1. Let P,, be a probability space. Let . Let ,nX be a mapping of the sample space into a space of complex-valued sequences on some index set Z. If, for each fixed integer Zn , ,nX is a random variable, then ,nX is a ransom (stochastic) sequence. The index set Z is all integers, n , padded with zeros if necessary,
Definition 9.1-1. Let P,, be a probability space. then define a mapping of X from the sample space to a space of continuous time functions. The elements in this space will be called sample functions. This mapping is called a random process if at each fixed time the mapping is a random variable, that is, ,tX for each fixed t on the real line t .
Example sets of random sequence.
Figure 8.1-1 Illustration of the concept of random sequence X(n,ζ), where the ζ domain (i.e., the sample space Ω) consists of just ten values. (Samples connected only for plot.)
Example sets of random process.
Figure 9.1-1 A random process for a continuous sample space Ω = [0,10].
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 5 of 78 ECE 3800
Example 9.1-2
Separable random process may be constructed by combining a deterministic sequence with one or more random variables.
The classic example already shown is a sinusoid with random amplitude and phase: tfAtX 02sin,
Where the amplitude and phase are R.V. defined based on the probability space selected.
Example9.1‐3
A random process used to model a continuous sequence of random communication symbols. nTtpnAtX
n
In a communication class, Dr. Bazuin would typically use the following TntpAtX
nn , for tp non zero for Tkt 0
Here An is the amplitude and phase of a complex communication symbol and p(t) is the deterministic time function, the simplest of which is a rectangular pulse in time.
This can be used to describe a wide range of digital communication systems, including; Phase-Shift Keyed (PSK) or Quadrature Amplitude Modulation (QAM) communication signals.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 6 of 78 ECE 3800
TheapplicationoftheExpectedValueOperator
Moments play an important role and, for Ergodic Processes, they can be estimated from a single process in time of the infinite number that may be possible.
Therefore, tXEtX
and the correlation functions (auto- and cross-correlation) *2121, tXtXEttRXX *2121, tYtXEttRXY
and the covariance functions (auto- and cross-correlation) *221121, ttXttXEttK XXXX *221121, ttYttXEttK YXXY
with *212121 ,, ttttRttK XXXXXX
Note that the variance can be computed from the auto-covariance as tttXttXEttK XXXXX 2*,
and the “power” function can be computed from the auto-correlation
2*, tXEtXtXEttRXX For real X(t) tttXEttR XXXX 222,
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 7 of 78 ECE 3800
Example9.1‐5Auto‐correlationofasinusoidwithrandomphase
Think of: ,sin twAtX
where A and w are known constants. And theta is a uniform pdf covering the unit circle.
The mean is computed as
twAEtXEtX sin twEAtXEtX sin
dtwAtXEtX
sin2
1
twAtXEtX cos2
002
coscos2
AtwtwAtXEtX
( What would happen if 0 instead? )
The auto-correlation is computed as
*21*2121 sinsin, twAtwAEtXtXEttRXX 2cos21cos21, 21212*2121 ttwttwEAtXtXEttRXX
2cos2
cos2
, 212
21
2
21 ttwEAttwAttRXX
212
21
2
21 cos20cos
2, ttwAttwAttRXX
( This works if 0 instead. )
Note that if A was a random variable (independent of phase) we would have …
wAERttwAEttR XXXX cos2cos2,2
21
2
21
and we would still have
002
AEtXEtX
Note: this Random Process is Wide-Sense stationary (mean and variance not a function of time)
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 8 of 78 ECE 3800
Definition9.1‐3
All correlation and covariance functions are positive semidefinite.
All auto-correlation functions are diagonal dominate.
Using the Cauchy-Schwartz Inequality
221121 ,,, ttRttRttR XXXXXX
which for a WSS random process becomes
0XXXX RR
AdditionalPropertiesforreal,WSSrandomprocesses.
220 XXXXR
0max XXXX RR
XXXX RR
For signals that are the sum of independent random variable, the autocorrelation is the sum of the individual autocorrelation functions.
tYtXtW
YXYYXXWW RRR 2
If X is ergodic and zero mean and has no periodic component, then
0lim
XXR
InterpretationofWSSautocorrelation…
The statistical (or probabilistic) similarity of future (or past) samples of a random process to other samples of the process for an ergodic random process.
How similar is a time shifted version of a function to itself?
Nominal definition of ergodicity … the time base statistics are equivalent to the probabilistic based statistics of a stationary random sequence or process.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 9 of 78 ECE 3800
For the autocorrelation defined as:
2121212121 ,, xxfxxdxdxXXEttRXX
For WSS processes:
XXXX RtXtXEttR 21,
If the process is ergodic, the time average is equivalent to the probabilistic expectation, or
txtxdttxtx
T
T
TT
XX 21lim
and
XXXX R
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 10 of 78 ECE 3800
A strange autocorrelation
Arandomprocesshasasamplefunctionoftheform
else
tAtX
,010,
where A is a random variable that is uniformly distributed from 0 to 10.
Find the autocorrelation of the process.
100,101
aforaf
Using
2121, tXtXEttRXX
1,0,, 21221 ttforAEttRXX
1,0,101, 21
10
0
221 ttfordaattRXX
1,0,30
, 2110
0
3
21 ttforattRXX
1,0,3
10030
1000, 2121 ttforttRXX
221121 1,0,1,0,0, ttorttforttRXX
Not WSS as it is a function of time!
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 11 of 78 ECE 3800
Example: tfAtx 2sin for A a uniformly distributed random variable 2,2A
212121 2sin2sin, tfAtfAEtXtXEttRXX
2121
22121 2cos2cos2
1, ttfttfAEtXtXEttRXX
2121221 2cos2cos21, ttfttfAEttRXX
for 12 tt
212
21 2cos2cos1221, ttffttRXX
2121 2cos2cos2416, ttffttRXX
A non-stationary process! It is still a function of both time variables!
The time based formulation:
txtxdttxtx
T
T
TT
XX 21lim
T
TTXX
dttfAtfAT
2sin2sin21lim
T
TTXX
dttffT
A 22cos2cos21
21lim2
T
TTXX
dttfT
AfA 22cos21lim
22cos
2
22
fAfAXX 2cos22cos222
Acceptable, but the R.V. is still present?! To find a value not dependent upon a R.V
ffAEE XX 2cos24
162cos2
2
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 12 of 78 ECE 3800
Example: tfAtx 2sin for a uniformly distributed random variable 2,0
212121 2sin2sin, tfAtfAEtXtXEttRXX
22cos2cos21, 2121
22121 ttfttfEAtXtXEttRXX
22cos2
2cos2
, 212
21
2
2121 ttfEAttfAtXtXEttRXX
Of note is that the phase need only be uniformly distributed over 0 to π in the previous step!
212
2121 2cos2, ttfAtXtXEttRXX
for 12 tt
fARXX 2cos22
but
fARR XXXX 2cos22
Assuming a uniformly distributed random phase “simplifies the problem” !!!
Also of note, if the amplitude is an independent random variable, then
fAERXX 2cos22
The time based formulation:
txtxdttxtx
T
T
TT
XX 21lim
T
TTXX
dttfAtfAT
2sin2sin21lim
T
TTXX
dttffT
A 222cos2cos21lim
2
2
fAXX 2cos22
This appears to be stationary but not technically ergodic … due to the R.V. in the time AC.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 13 of 78 ECE 3800
Example:
TttrectBtx 0 for B =+/-A with probability p and (1-p) and t0 a uniformly
distributed random variable
2,
20TTt . Assume B and t0 are independent.
TttrectB
TttrectBEtXtXEttRXX 01012121 ,
Tttrect
TttrectBEtXtXEttRXX 0101
22121 ,
As the RV are independent
Tttrect
TttrectEBEtXtXEttRXX 0201
22121 ,
Tttrect
TttrectEpApAttRXX 0201
2221 1,
2
2
002012
211,
T
TXX dtTT
ttrectT
ttrectAttR
For 21 0 tandt
2
2
002 11,0
T
TXX dtTT
trectAR
The integral can be recognized as being a triangle, extending from –T to T and zero everywhere else.
TtriARXX
2
T
TTT
A
TTT
A
T
RXX
,0
0,1
0,1,0
2
2
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 14 of 78 ECE 3800
The time based formulation:
txtxdttxtx
T
T
TTXX 2
1lim
C
CCXX
dtT
ttrectBT
ttrectBC
00
21lim
A change in variable for the integral 0ttt . And only integrate over the finite interval T.
2
2
2 11T
TXX dtT
trectT
B
For 20T
T
BTTT
BdtT
BT
TXX
122111 22
2
2
2
For 02 T
T
BTTT
BdtT
BT
TXX
122111 22
2
2
2
And
TttriBXX
2
Not ergodic as taking the expected value of the time autocorrelation … however …
TttriBEE XX
2
TttripApAE XX
122
TttriAE XX
2
This is identical to the probabilistic autocorrelation previously computed!
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 15 of 78 ECE 3800
Some Important Random Processes
AsynchronousBinarySignaling
The pulse values are independent, identically distributed with probability p that amplitude is a and q=1-p that amplitude is –a. The start of the “zeroth” pulse is uniformly distributed from –T/2 to T/2
22
,1 TDTforD
Dpdf
Determine the autocorrelation of the bipolar binary sequence, assuming p=0.5.
kk T
TkDtrectXtX
Note: the rect function is defined as
else
TtT
Ttrect
,022
,1
Determine the Autocorrelation 2121, tXtXEttRXX
kk
nnXX T
TkDtrectXT
TnDtrectXEttR 2121,
n kknXX T
TkDtrectXT
TnDtrectXEttR 2121,
n kkknXX T
TkDtrectXT
TnDtrectXXEttR 2121,
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 16 of 78 ECE 3800
n kkknXX T
TkDtrectXT
TnDtrectEXXEttR 2121,
For samples more than one period apart, Ttt 21 , we must consider apapapapapapapapXXE jk 1111
222 112 ppppaXXE jk 144 22 ppaXXE jk
For p=0.5 0144 22 ppaXXE jk
For samples within one period, Ttt 21 ,
2222 1 aapapXEXXE kkk 0144 221 ppaXXE kk
For samples within one period, Ttt 21 , there are two regions to consider, the sample bit overlapping and the area of the next bit.
kXX T
TkDtrectT
TkDtrectEattR 21221,
But the overlapping area … should be triangular. Therefore
0,112
2
2
2
1
TfordtXXET
dtXXET
RT
Tkk
T
TkkXX
TfordtXXET
dtXXET
RT
Tkk
T
TkkXX
0,112
2
1
2
2
or
0,112
2
2
TfordtT
aRT
TXX
Tfordtt
aRT
TaXX
0,112
2
2
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 17 of 78 ECE 3800
Therefore
TforT
Ta
TforT
TaRXX
0,
0,
2
2
or recognizing the structure
TTforT
aRXX
,12
This is simply a triangular function with maximum of a2, extending for a full bit period in both time directions.
For unequal bit probability
Tforppa
TTforT
ppT
ta
Ra
XX
,144
,144
22
22
As there are more of one bit or the other, there is always a positive correlation between bits (the curve is a minimum for p=0.5), that peaks to a2 at = 0.
Note that if the amplitude is a random variable, the expected value of the bits must be further evaluated. Such as,
22 kk XXE
21 kk XXE
In general, the autocorrelation of communications signal waveforms is important, particularly when we discuss the power spectral density later in the textbook.
If the signal takes on two levels a and b vs. a and –a, the result would be
bpbpapbpbpapapapXXE jk 1111 For p = 1/2
2
22
241
21
41
babbaaXXE jk
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 18 of 78 ECE 3800
And 222 1 bpapXEXXE kkk
For p = 1/2
22
222
2221
bababaXEXXE kkk
Therefore,
Tforba
TTforT
baba
RXX
,2
,122
2
22
For a = 1, b = 0 and T=1, we have
Tfor
TTforTRXX
,41
,141
41
Figure 9.2-2 Autocorrelation function of ABS random process for a = 1, b = 0 and T = 1.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 19 of 78 ECE 3800
Examples of discrete waveforms used for communications, signal processing, controls, etc.
(a) Unipolar RZ & NRZ, (b) Polar RZ & NRZ , (c) Bipolar NRZ , (d) Split-phase Manchester, (e) Polar quaternary NRZ.
From Cahp. 11: A. Bruce Carlson, P.B. Crilly, Communication Systems, 5th ed., McGraw-Hill, 2010. ISBN: 978-0-07-338040-7
In general, a periodic bipolar “pulse” that is shorter in duration than the pulse period will have the autocorrelation function
wwwp
wXX ttfortt
tAR
,12
for a tw width pulse existing in a tp time period, assuming that positive and negative levels are equally likely.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 20 of 78 ECE 3800
Digital signal autocorrelation functions give rise to a range of Power Spectral Density results. The following shows some of the expected frequency responses for digital waveforms.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 21 of 78 ECE 3800
Exercise6‐3.1–CooperandMcGillem
a) An ergodic random process has an autocorrelation function of the form 1610cos164exp9 XXR
Find the mean-square value, the mean value, and the variance of the process.
The mean-square (2nd moment) is 222 41161690 XXRXE
The constant portion of the autocorrelation represents the square of the mean. Therefore 1622 XE and 4
Finally, the variance can be computed as, 2516410 2222 XXRXEXE
b) An ergodic random process has an autocorrelation function of the form
1
642
2
XXR
Find the mean-square value, the mean value, and the variance of the process.
The mean-square (2nd moment) is
222 6160 XXRXE
The constant portion of the autocorrelation represents the square of the mean. Therefore
414
164lim 2
222
t
XE and 2
Finally, the variance can be computed as, 2460 2222 XXRXEXE
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 22 of 78 ECE 3800
PoissonCountingProcess
Applications and properties
arrival times radioactive decay’ memoryless property mean arrival rates
Complicated analysis and derivation left for reading in the textbook.
RandomTelegraphSignal
The random telegraph signal was originally defined based on a telegraph operator or someone manually sending Morse code. The signal may also represent “zero crossings” in an FM modulated signal.
The signal is a binary signal with random transitions in time.
Figure 9.2-4 Sample function of the random telegraph signal.
Let X(0) = +/-a with equal probability. Use the Poisson arrival process from Chap. 8 as the time of transition to the opposite level. The arrival time is now a R.V.
The probability of signal level correlation at two seperate4 times, assuming different symbols
aPaaPaaPaaPaaaPaaPaaaPaaPa
aaPaaaPaaaaPaaaaPa
tXtXEttRXX
||
||,,,,
,
2
2
22
2121
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 23 of 78 ECE 3800
For P(a)=1/2
aaPaaPaaPaaPattRXX ||||21, 221
This becomes the probability of odd or even transitions after the average time interval of the mean of the Poisson arrival time. With this for “the positive time axis” it becomes
0,!
exp!
exp00
2
kodd
k
keven
k
XX kkaR
0,!
1exp0
2
k
kk
XX kaR
The summation can be determined as for tau>0
0,2exp2 aRXX
To include both positive and negative time (the property of positive and negative autocorrelation)
2exp2aRXX
Figure 9.2-5 The symmetric exponential correlation function of an RTS process (a = 2.0, λ = 0.25).
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 24 of 78 ECE 3800
BinaryPhaseShiftKeying
Figure 9.2-6 System for PSK modulation of Bernoulli random sequence B[n].
The communications symbols represent 180 degree phae shifting of a sinusoidal waveform.
ttfts ac 2cos
where
TktTkforkta 1,
and
0,2
1,2nbif
nbifn
Finally
k
c kTktftX 2cos
k
cc kTktfkTktftX sin2sincos2cos
k
c kTktftX sin2sin
Typically T is selected so that the symbols form “complete” cosine waveforms integerTfc
nc
kcXX nTntfkTktfEttR sin2sinsin2sin, 2121
nc
kcXX nTntfkTktfEttR sin2sinsin2sin, 2121
nc
kcXX nkTntfTktfEttR sinsin2sin2sin, 2121
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 25 of 78 ECE 3800
Notice that
nknkE sinsin
The following becomes similar to the previous “rectangular magnitude becoming a triangular autocorrelation. In addition, we typically define a filtering function so to remove frequencies at twice the frequency of interest.
TttTktfTktfttR ck
cXX 212121 ,0,2sin2sin,
This is where the text stops … using some further analysis and assumptions.
TttTkttfttfttR ck
cXX 21212121 ,0,22cos2cos21,
Setting 21 tt vary over two T. In addition, kT is an integer
TtandTTtfftR ck
cXX 222 0,22cos2cos21,
Averaging across all possible t2, (alternately, if the original equations had a random phase component …)
222cos2cos21, 22 tfEftR c
kcXX
The autocorrelation would become
TTT
triftR cXX
,2cos
21, 2
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 26 of 78 ECE 3800
There is an alternate derivation that focuses on “one” symbol cycle for each of the t1 and t2 sequences, particularly if symbols have zero cross-correlation
nknkE sinsin
only one symbol and in fact the same symbol time shifted is of interest. If one symbol remains fixed and the other varies in time.
tf
Ttrecttf
TtrectEttR ccXX 2cos2cos,
222cos2cos
21, tff
Ttrect
TtrectEttR ccXX
The expected value of the random phase, component goes to zero and
cXX fT
trectTtrectEttR 2cos
21,
If a time average in t is performed the result becomes.
cXX fT
triR 2cos21
If the “symbols” do not have equal probability, there will be a cross-correlation component. Then, the “envelope” of the autocorrelation has a triangular sections (as above) and the rest is a “DC magnitude” that will multiple the cosine “modulated waveform” element. Such as
ccXX fbfT
triaR 2cos2cos21
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 27 of 78 ECE 3800
Note: the following notes are from Cooper and McGillem ….
The Autocorrelation Function
The autocorrelation is defined as:
2121212121 ,, xxfxxdxdxXXEttRXX
The above function is valid for all processes, stationary and non-stationary. For WSS processes:
XXXX RtXtXEttR 21, If the process is ergodic, the time average is equivalent to the probabilistic expectation, or
txtxdttxtx
T
T
TT
XX 21lim
and XXXX R
Properties of Autocorrelation Functions 1) 220 XXERXX or 20 txXX 2) XXXX RR 3) 0XXXX RR 4) If X has a DC component, then Rxx has a constant factor.
tNXtX NNXX RXR 2
5) If X has a periodic component, then Rxx will also have a periodic component of the same period.
20,cos twAtX
wAtXtXERXX cos22
6) If X is ergodic and zero mean and has no periodic component, then 0lim
XXR
7) Autocorrelation functions can not have an arbitrary shape. One way of specifying shapes permissible is in terms of the Fourier transform of the autocorrelation function.
wallforRXX 0
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 28 of 78 ECE 3800
The Crosscorrelation Function The crosscorrelation is defined as:
2121212121 ,, yxfyxdydxYXEttRXY
2121212121 ,, xyfxydxdyXYEttRYX
The above function is valid for all processes, jointly stationary and non-stationary. For jointly WSS processes:
XYXY RtYtXEttR 21, YXYX RtXtYEttR 21,
Note: the order of the subscripts is important for cross-correlation! If the processes are jointly ergodic, the time average is equivalent to the probabilistic expectation, or
tytxdttytx
T
T
TT
XY 21lim
txtydttxty
T
T
TT
YX 21lim
and XYXY R YXYX R
Properties of Crosscorrelation Functions 1) The properties of the zoreth lag have no particular significance and do not represent mean-square values. It is true that the “ordered” crosscorrelations must be equal at 0. .
00 YXXY RR or 00 YXXY 2) Crosscorrelation functions are not generally even functions. However, there is an antisymmetry to the ordered crosscorrelations:
YXXY RR 3) The crosscorrelation does not necessarily have its maximum at the zeroth lag. It can be shown however that 00 YYXXXY RRR As a note, the crosscorrelation may not achieve this maximum anywhere … 4) If X and Y are statistically independent, then the ordering is not important
YXtYEtXEtYtXERXY and
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 29 of 78 ECE 3800
YXXY RYXR 5) If X is a stationary random process and is differentiable with respect to time, the crosscorrelation of the signal and it’s derivative is given by
d
dRR XXXX Similarly,
22
dRdR XXXX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 30 of 78 ECE 3800
Measurement of The Autocorrelation Function
We love to use time average for everything. For wide-sense stationary, ergodic random processes, time average are equivalent to statistical or probability based values.
txtxdttxtx
T
T
TT
XX 21lim
Using this fact, how can we use short-term time averages to generate auto- or cross-correlation functions?
An estimate of the autocorrelation is defined as:
T
XX dttxtxTR
0
1ˆ
Note that the time average is performed across as much of the signal that is available after the time shift by tau.
Digital Time Sample Correlation
In most practical cases, the operation is performed in terms of digital samples taken at specific time intervals,
t . For tau based on the available time step, k, with N equating to the available
time interval, we have:
kN
iXX ttktixtixtktN
tkR0
11ˆ
kN
iXXXX kixixkN
kRtkR0
11ˆˆ
In computing this autocorrelation, the initial weighting term approaches 1 when k=N. At this point the entire summation consists of one point and is therefore a poor estimate of the autocorrelation. For useful results, k
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 31 of 78 ECE 3800
It can be shown that the estimated autocorrelation is equivalent to the actual autocorrelation; therefore, this is an unbiased estimate.
kN
iXXXX kixixkN
EkREtkRE0
11ˆˆ
tkRkNkN
tkRkN
kixixEkN
tkRE
XX
kN
iXX
kN
iXX
111
11
11ˆ
0
0
tkRtkRE XXXX ˆ As noted, the validity of each of the summed autocorrelation lags can and should be brought into question as k approaches N.
Biased Estimate of Autocorrelation
As a result, a biased estimate of the autocorrelation is commonly used. The biased estimate is defined as:
kN
iXX kixixN
kR0
11~
Here, a constant weight instead of one based on the number of elements summed is used. This estimate has the property that the estimated autocorrelation should decrease as k approaches N.
The expected value of this estimate can be shown to be
tkRN
nkRE XXXX
11~
The variance of this estimate can be shown to be (math not done at this level)
M
MkXXXX tkRN
kRVar 22~
This equation can be used to estimate the number of time samples needed for a useful estimate.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 32 of 78 ECE 3800
Exercise 6-6.1
Find the cross-correlation of the two functions …
tftX 2cos2 and tftY 2sin10
Using the time average functions
tytxdttytx
T
T
TT
XY 21lim
T
TTXY
dttftfT
2sin102cos221lim
f
XY dttftff
1
0
2sin2cos1120
f
XY dtftff
1
0
2sin222sin2120
ff
XY dtffdttff
1
0
1
0
2sin10222sin10
f
fXY dtfftff
f1
0
1
02sin10222cos
410
fff
fXY 2sin1022cos222cos
410
fffXY 2sin1022cos224cos4
10
fffXY 2sin1022cos22cos4
10
fXY 2sin10
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 33 of 78 ECE 3800
Using the probabilistic functions
tytxERXY
tftfERXY 2sin102cos2
tftfERXY 2sin2cos20
fftfERXY 2sin2222sin10
2222sin102sin10 ftfEfRXY
From prior understanding of the uniform random phase ….
fRXY 2sin10
By the way, it is useful to have basic trig identities handy when dealing with this stuff …
bababa cos21cos
21sinsin
bababa cos21cos
21coscos
bababa sin21sin
21cossin
bababa sin21sin
21sincos
and aaa cossin22sin
1cos2sincos2cos 222 aaaa
as well as
aa 2cos121sin 2
aa 2cos121cos 2
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 34 of 78 ECE 3800
Functions of a random variable and time
Example: tafntX , where a is a wide-sense stationary ergodic process with a known pdf.
XXERRttR XXXXXX 0,0, 21
daapdftXtXRttRttR XXXXXX ,, 21
Exponential
tutAtX exp where A is a uniformly distributed random variable BA ,0 .
B
XX dABtutAtutAttR
0221121
1expexp,
B
XX dAABttuttttR
0
2212121
1,maxexp,
3
,maxexp,2
212121BttuttttRXX
For 21 0 tandt
0,exp3
,02BRXX
For 21 0 tandt
0,exp3
,02
BRXX
Therefore
exp3
2BRXX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 35 of 78 ECE 3800
MATLAB Signal Processing Examples
Fig_6_2: Cross Correlation Rxy and Ryx
Run the various y waveforms. Chirp and Sinc are popular and interesting.
Fig_6_3: Auto Correlation random Gaussian noise
Fig_6_4: Auto Correlation smoothers random Gaussian noise
Fig_6_9v2: Sin wave correlation in noise
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 36 of 78 ECE 3800
Section 9.4 Classifications of Random Processes
Definition9.4‐1.:LetXandYberandomprocesses.
(a) They are Uncorrelated if 21*21*2121 ,, tandtallfortttYtXEttR YXXY
(b) They are Orthogonal if 21*2121 ,0, tandtallfortYtXEttRXY
(c) They are Independent if for all positive integers n, the nth-order CDF of X and Y factors. That is
nnYnnX
nnnXY
tttyyyFtttxxxFtttyxyxyxF
,,,;,,,,,,;,,,,,,;,,,,,,
21212121
212211
Note that if two processes are uncorrelated and one of the means is zero, they are orthogonal as well!
Stationarity
A random process is stationary when its statistics do not change with the continuous time parameter.
TtTtTtxxxF
tttxxxF
nnX
nnX
,,,;,,,,,,;,,,
2121
2121
Overall, the CDF and pdf do not change with absolute time. They may have time characteristics, as long as the elements are based on time differences and not absolute time.
0,;,,;, 21212121 ttxxFttxxF XX
0,;,,;, 21212121 ttxxfttxxf XX
This implies that 0,0,, 21*2121 XXXXXX RttRtXtXEttR
Definition9.4‐3.:WideSenseStationary
A random process is wide-sense stationary (WSS) when its mean and variance statistics do not change with the continuous time parameter. We also include the autocorrelation being a function of one variable …
tofntindependedforRtXtXE XX ,,*
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 37 of 78 ECE 3800
Power Spectral Density
Definition9.1‐1:PSD
Let Rxx(t) be an autocorrelation function for a WSS random process. The power spectral density is defined as the Fourier transform of the autocorrealtion function.
diwRRwS XXXXXX exp
The inverse exists in the form of the inverse transform
dwiwtwStR XXXX exp21
Properties:
1. Sxx(w) is purely real as Rxx(t) is conjugate symmetric
2. If X(t) is a real-valued WSS process, then Sxx(w) is an even function, as Rxx(t) is real and even.
3. Sxx(w)>= 0 for all w.
Wiener–Khinchin Theorem For WSS random processes, the autocorrelation function is time based and has a spectral decomposition given by the power spectral density.
Also see:
http://en.wikipedia.org/wiki/Wiener%E2%80%93Khinchin_theorem
Why this is very important … the Fourier Transform of a “single instantiation” of a random process may be meaningless or even impossible to generate. But if the random process can be described in terms of the autocorrelation function (all ergodic, WSS processes), then the power spectral density can be defined.
I can then know what the expected frequency spectrum output looks like and I can design a system to keep the required frequencies and filters out the unneeded frequencies (e.g. noise and interference).
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 38 of 78 ECE 3800
Relation of Spectral Density to the Autocorrelation Function
For “the right” random processes, power spectral density is the Fourier Transform of the autocorrelation:
diwtXtXERwS XXXX exp
For an ergodic process, we can use time-based processing to arrive at an equivalent result …
txtxdttxtx
T
T
TT
XX 21lim
T
TT
XX dttxtxTtXtXE
21lim
diwdttxtx
TtXtXE
T
TT
XX exp21lim
dtdiwtxtxT
T
TT
XX
exp21lim
dtdiwttiwtxtxT
T
TT
XX
exp21lim
dtdtiwtxiwttxT
T
TT
XX
expexp21lim
dtdtiwtxiwttxT
T
TT
XX
expexp21lim
If there exists wXX
dtwXiwttxT
T
TTXX
exp
21lim
dttwitxT
wXT
TTXX
exp21lim
2wXwXwXXX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 39 of 78 ECE 3800
Properties of the Fourier Transform:
diwxxwX exp
For x(t) purely real
dwiwxxwX sincos
dwxidwxxwX sincos
dwxidwxwOiwEwX XX sincos
dwxwEX cos and
dwxwOX sin
Notice that:
wEdwxdwxwE XX
coscos
wOdwxdwxwO XX
sinsin
Therefore, the real part is symmetric and the imaginary part is anti-symmetric!
Note also, for real signals *wXwXconjwX
X(w) is conjugate symmetric about the zero axis.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 40 of 78 ECE 3800
Relatingthistoarealautocorrelationfunctionwhere XXXX RR wOiwER XXXX
dwiwRR XXXX sincos
dtwtiwttRR XXXX sincos
dtwttRidtwttRR XXXXXX sincos
wOiwER XXXX
Since Rxx is symmetric, we must have that
XXXX RR and wOiwEwOiwE XXXX
For this to be true, wOiwOi XX , which can only occur if the odd portion of the Fourier transform is zero! 0wOX .
This provides information about the power spectral density,
wERwS XXXXX
wEwS XXX
0 wS XX
The power spectral density necessarily contains no phase information!
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 41 of 78 ECE 3800
Example9.5‐3
Find the psd of the following autocorrelation function … of the random telegraph.
0,exp forRXX
Find a good Fourier Transform Table … otherwise
dwjRwS XXXX exp
dwjwS XX expexp
0
0
expexpexpexp dwjdwjwS XX
0
0
expexp dwjdwjwS XX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 42 of 78 ECE 3800
0
0
expexp
wjwj
wjwjwS XX
wjwj
wjwj
wjwj
wjwjwS XX
exp0exp
0expexp
wjwjwjwj
wjwjwS XX
11
222222
ww
wS XX
For a=3
Figure 9.5-2 Plot of psd for exponential autocorrelation function.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 43 of 78 ECE 3800
Example9.5‐4
Find the psd of the triangle autocorrelation function … autocorrelation of rect.
TtriRXX
or TT
RXX
,1
T
TXX dwjT
wS
exp1
T
TXX dwjT
dwjT
wS0
0
exp1exp1
TTTT
XX
dwjT
dwj
dwjT
dwjwS
00
00
expexp
expexp
TT
T
TXX
wjwj
wjwj
T
wjwj
wjwj
T
wjwj
wjwjwS
02
0
2
0
0
expexp1
expexp1
expexp
22
22
1expexp1
expexp11
1expexp1
wwTwj
wjTwjT
T
wTwj
wjTwjT
wT
wjwjTwj
wjTwj
wjwS XX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 44 of 78 ECE 3800
222expexp121
expexp1
expexp
wTwj
wTwj
TwT
wjTwjT
wjTwjT
T
wjTwj
wjTwjwS XX
22cos212sin2sin2
wTw
TwTwTw
wTwwS XX
TwwT
wS XX cos112
2
2
2
2
2
2
2sin
2sin212
Tw
Tw
TTwjwT
wS XX
Don’t you love the math ?!
Using a table is much faster and easier ….
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 45 of 78 ECE 3800
Deriving the Mean-Square Values from the Power Spectral Density
Using the Fourier transform relation between the Autocorrelation and PSD
diwRwS XXXX exp
dwiwtwStR XXXX exp21
The mean squared value of a random process is equal to the 0th lag of the autocorrelation
dwwSdwiwwSRXE XXXXXX 210exp
2102
dffSdwfifSRXE XXXXXX 02exp02
Therefore, to find the second moment, integrate the PSD over all frequencies.
As a note, since the PSD is real and symmetric, the integral can be performed as
0
22120 dwwSRXE XXXX
0
2 20 dffSRXE XXXX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 46 of 78 ECE 3800
Converting between Autocorrelation and Power Spectral Density
Using the properties of the functions we can actually different variations of Transforms!
The power spectral density as a function is always real, positive, and an even function in w/f.
You can convert between the domains using any of the following …
The Fourier Transform in w
diwRwS XXXX exp
dwiwtwStR XXXX exp21
The Fourier Transform in f
dfiRfS XXXX 2exp
dfftifStR XXXX 2exp
The 2-sided Laplace Transform (the jw axis of the s-plane)
dsRsS XXXX exp
j
jXXXX dsstsSj
tR exp21
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 47 of 78 ECE 3800
NotesonusingtheLaplaceTransform
ECE 3100 and ECE 3710 stuff …
(1) When converting from the s-domain to the frequency domain use:
jws or jsw
(2) As an even function, the PSD may be expected to have a polynomial form as: (Hint: no odd powers of w in the numerator or denominator!)
0
22
4242
2222
20
22
4242
2222
20
bwbwbwbwawawawaw
SwS mm
mm
m
nn
nn
nXX
This can be factored and expressed as:
sTsTsdsdscscwS XX
To compute the autocorrelation function for 0 use a partial fraction expansion such that
sd
sgsdsgwS XX
and solve for 0 using the LHP poles and zeros as
0,exp21
tfordsstsdsgtR
j
jXX
for determining 0 , use the RHP expansion, replace –s with s, perform the Laplace transform and replace t with –t.
Another hint, once you have 0 , make the image and skip the math tRtR XXXX .
Final Note … for sTsTsdsdscscwS XX
If you “define” T(s) as all the LHP poles and zeros and T(-s) as all the RHP poles and zeros, then T(s) will represent a (1) causal, (2) minimum phase, and (3) stable filter (if there are no poles on the jw axis).
You give me a power spectral density and I can design a filter that passes “signal energy” and filters out as much of the rest as possible!
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 48 of 78 ECE 3800
Example:InverseLaplaceTransform.
222
22
22
1
2
wA
w
AwS
X
XXX
Substitute s for w
ssA
sAsS XX
2
22
2 22
Partial fraction expansion
ss
Ass
sksks
ks
ksS XX
21010 2
20210
1010
222
0
AkAkk
kkskk
sA
sAsS XX
22
Taking the LHP Laplace Transform
Taking the RHP with –s and then –t.
0expexpexp222
2
tfortAtAtA
sAL
Combining we have
tARXX exp2
0exp2
tfortA
sAL
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 49 of 78 ECE 3800
7-6.3 A stationary random process has a spectral density of.
else
wwS XX ,0
2010,5
(a) Find the mean-square value of the process.
02
22
10 dwwSdwwSR XXXXXX
20
10
10
20
20
10
52
1252
152
10 dwdwdwRXX
501020
210
2100 20
10
wRXX
(b) Find the auto-correlation function the process.
dwtwjwStR XXXX exp21
10
20
20
10
expexp2
5 dwtwjdwtwjtRXX
10
20
20
10
expexp2
5tj
twjtj
twjtRXX
tj
tjtj
tjtj
tjtj
tjtRXX20exp10exp10exp20exp
25
tj
tjtj
tjtj
tjtj
tjtRXX10exp10exp20exp20exp
25
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 50 of 78 ECE 3800
ttttj
tjtj
tjtRXX
10sin20sin510sin220sin22
5
ttt
ttt
tRXX
15cos5sin10
21020cos
21020sin25
tttt
ttRXX
15cos5sinc5015cos5
5sin50
(c) Find the value of the auto-correlation function at t=0..
015cos05sinc50015cos05
05sin500
XX
R
115011500 XX
R
500 XXR
It must produce the same result!
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 51 of 78 ECE 3800
White Noise
Noise is inherently defined as a random process. You may be familiar with “thermal” noise, based on the energy of an atom and the mean-free path that it can travel.
As a random process, whenever “white noise” is measured, the values are uncorrelated with each other, not matter how close together the samples are taken in time.
Further, we envision “white noise” as containing all spectral content, with no explicit peaks or valleys in the power spectral density.
As a result, we define “White Noise” as tSRXX 0
20
0N
SwS XX
This is an approximation or simplification because the area of the power spectral density is infinite!
Nominally, noise is defined within a bandwidth to describe the power. For example,
Thermal noise at the input of a receiver is defined in terms of kT, Boltzmann’s constant times absolute temperature, in terms of Watts/Hz. Thus there is kT Watts of noise power in every Hz of bandwidth. For communications, this is equivalent to –174 dBm/Hz or –204 dBW/Hz.
For typical applications, we are interested in Band-Limited White Noise where
fW
WfN
SwS XX
020
0
The equivalent noise power is then:
WNNWSWdwSRXEW
WXX
00
002
2220
For communications, we use kTB where W=B and N0=kT.
How much noise power, in dBm, would I say that there is in a 1 MHz bandwidth?
dBmBdBkTdBkTBdB 11460174
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 52 of 78 ECE 3800
Receiver Sensitivity
What does it mean when you buy a radio receiver?
For a great receiver (spectrum analyzer grade), assume a 200 kHz FM radio bandwidth.
Noise Power kT -174. dBm/Hz
Equivalent Noise Bandwidth B 53. dB Hz
Receiver Noise Figure NF 10. dB
Signal Detection Threshold D 8. dB
Minimum Detectable Signal MDS -103. dBm
FM radio stations can transmit up to 1 Megawatt +90 dBm
Why doesn’t your receiver get blasted? Path loss, distance, higher noise figure, receiving antenna inefficiency, etc.
https://en.wikipedia.org/wiki/Path_loss
https://en.wikipedia.org/wiki/Friis_transmission_equation
But notice that your commercial; receiver is in microvolts, where 2.0 uV is very good. Power into 50 ohms is V^2/R or 8e-14 W = -130.97 dBW -101 dBm.
-103 dBm 1.6 uV
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 53 of 78 ECE 3800
FMRadioDesignDiagram–Somethingyoumayencounterinthefuture
Assume input to be digitized by a 12-bit ADC with 60 dB SNR
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 54 of 78 ECE 3800
Band Limited White Noise
fW
WfNSwS XX
02
00
The equivalent noise power is then:
002 20 SWdwSRXEW
WXX
Butwhatabouttheautocorrelation?
W
WXX dfftiStR 2exp0
tiWti
tiWtiS
tiftiStR
W
WXX
22exp
22exp
22exp
00
ti
WtiiStRXX
22sin2
0
For xtxtxt
sinc
WtSWtRXX 2sinc2 0
Using the concept of correlation, for what values will the autocorrelation be zero? (At these delays in time, sampled data would be uncorrelated with previous samples!)
,2,12
2
kforWkt
kWt
Sampling at 1/2W seems to be a good idea, but isn’t that the Nyquist rate!!
Also note, noise passed through a filter becomes band-limited, and the narrower the filter the smaller the noise power … but the wider is the sinc autocorrelation function.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 55 of 78 ECE 3800
Noise and Filtered Noise Matlab Simulation
Based on Cooper and McGillem HW Problem 6-4.6.
x=randn(N,1); % zero mean, unit power random signal [b,a] = butter(4,20/500); y=filter(b,a,x); % applying a digital filter y=y/std(y); % normalizing the output power Rxx=xcorr(x)/(N+1); Ryy=xcorr(y)/(N+1); DFTx = fftshift(fft(x))/N; DFTy = fftshift(fft(y))/N; DFTRxx = fftshift(fft(Rxx,2*N))/N; DFTRyy = fftshift(fft(Ryy,2*N))/N;
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 56 of 78 ECE 3800
Pink_Noise … If we call constant at all frequencies white noise, then noise in a limited low frequency band is sometimes called pink noise.
OK. I’m getting ahead …. we just did this.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 57 of 78 ECE 3800
The Cross-Spectral Density
Why not form the power spectral response of the cross-correlation function?
The Fourier Transform in w
diwRwS XYXY exp and
diwRwS YXYX exp
dwiwtwStR XYXY exp21
and
dwiwtwStR YXYX exp21
Properties of the functions
wSconjwS YXXY
Since the cross-correlation is real, the real portion of the spectrum is even the imaginary portion of the spectrum is odd
There are no other important (assumed) properties to describe
Note: the trick using the Laplace transform to form the positive and negative portions of the “time-based” cross-correlation is required to determine the correct “inverse transform” of the “Cross” Power Spectral Density.
OK … Time to talk about linear transfer functions … filters!.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 58 of 78 ECE 3800
Section 9.3 Continuous-Time Linear Systems with Random Inputs
Linear system requirements:
Definition 9.3-1 Let x1(t) and x2(t) be two deterministic time functions and let a1 and a2 be two scalar constants. Let the linear system be described by the operator equation
txLty then the system is linear if “linear super-position holds”
txLatxLatxatxaL 22112211 for all admissible functions x1 and x2 and all scalars a1 and a2.
For x(t), a random process, y(t) will also be a random process.
Linear transformation of signals: convolution in the time domain txthty
th ty tx
Linear transformation of signals: multiplication in the Laplace domain
sXsHsY
sX sH sY
The convolution Integrals (applying a causal filter)
0
dhtxty
or
t
dxthty
Where for physical realize-ability, causality, and stability constraints we require
00 tforth and
dtth
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 59 of 78 ECE 3800
Example: Applying a linear filter to a random process 03exp5 tfortth
tMtX 2cos4
where M and are independent random variables, uniformly distributed [0,2].
We can perform the filter function since an explicit formula for the random process is known.
t
dxthty
t
dMtty 2cos43exp5
tt
dtdtMty 2cos3exp203exp5
t
t
diiiit
tMty
2exp2exp3exp10
33exp5
t
iiit
iiitMty
232exp3exp
232exp3exp10
35
23
2exp23
2exp103
5i
itii
itiMty
49
2exp232exp23103
5 itiiitiiMty
ttMty 2sin22cos31320
35
Linear filtering will change the magnitude and phase of sinusoidal signals (DC too!).
tMtX 2cos4
69.33,2cos4135
35
tMty
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 60 of 78 ECE 3800
Expectedvalueoperatorwithlinearsystems
For a causal linear system we would have
0
dhtxty
and taking the expected value
0
dhtxEtyE
0
dhtxEtyE
0
dhttyE
For x(t) WSS
00
dhdhtyE
Notice the condition for a physically realizable system!
The coherent gain of a filter is defined as:
00
Hdtthhgain
Therefore, 0HXEhXEtYE gain
Note that:
dttfithfH 2exp
For a causal filter
0
2exp dttfithfH
At f=0
0
0 dtthH
And 0HtyE
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 61 of 78 ECE 3800
Whataboutacross‐correlation?(Convertinganauto‐correlationtocross‐correlation)
For a linear system we would have
dhtxty
And performing a cross-correlation (assuming real R.V. and processing)
dhtxtxEtytxE 2121
dhtxtxEtytxE 2121
dhtxtxEtytxE 2121
dhttRtytxE XX 2121 ,
For x(t) WSS
dhRRtytxE XXXY
hRRtytxE XXXY
What about the other way … YX instead of XY
And performing a cross-correlation (assuming real R.V. and processing)
2121 txdhtxEtxtyE
dhtxtxEtxtyE 2121
dhtxtxEtxtyE 2121
dhttRtxtyE XX 2121 ,
For x(t) WSS … see the next page
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 62 of 78 ECE 3800
For x(t) WSS
dhttRRtxtyE XXYX
dhRRtxtyE XXYX
Perform a change of variable for lamba to “-kappa” (assuming h(t) is real, see text for complex0
dhRRtxtyE XXYX
Therefore
dhRRtxtyE XXYX
hRRtxtyE XXYX
Whatabouttheauto‐correlationofy(t)?
And performing an auto-correlation (assuming real R.V. and processing)
222211112121 , dhtxdhtxEttRtytyE YY
112222112121 , dhdhtxtxEttRtytyE YY
112222112121 , dhdhtxtxEttRtytyE YY
112222112121 ,, dhdhttRttRtytyE XXYY
For x(t) WSS
122112 ddhhRRtytyE XXYY
112221 dhdhRRtytyE XXYY
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 63 of 78 ECE 3800
The output autocorrelation can also be defined in terms of the cross-correlation as
111 dhRRtytyE XYYY
hRRtytyE XYYY
The cross-correlation can be used to determine the output auto-correlation!
Continue in this concept, the cross correlation is also a convolution. Therefore,
hhRRtytyE XXYY
If h(t) is complex, the term in h(-t) must be a conjugate.
TheMeanSquareValueataSystemOutput
Based on the output autocorrelation formula
1221122 0 ddhhRRtyE XXYY
211122
2 0 ddhRhRtyE XXYY
dhRhRtyE XXYY 02
Based on the input to output cross-correlation formula
dhRRtytyE XYYY
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 64 of 78 ECE 3800
Example:WhiteNoiseInputstoacausalfilter
Let tNtRXX 20
0122
0211
2 0 ddhRhRtYE XXYY
0122
021
01
2
20 ddhNhRtYE YY
0
11102
20 dhhNRtYE YY
0
12
102
20 dhNRtYE YY
For a white noise process, the mean squared (or 2nd moment) is proportional to the filter power.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 65 of 78 ECE 3800
Example:RCfilter
The RC low-pass filter
CRs
CRCRs
CsR
CssH
1
1
11
1
1
Inverse Laplace Transform
tuCRt
CRth
exp1
Coherent Gain of the RC Filter
00
Hdtthhgain
0
exp1 dtCRt
CRhgain
CR
CRt
CRCR
CRt
CRhgain
1
exp1
1
exp1 0
10expexp1
CRCR
hgain
If driven by a white noise process, what is the output power?
0
202
2 dhNtYE
0
202 exp1
2 d
CRCRNtYE
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 66 of 78 ECE 3800
0
202 2exp1
2 d
CRCRNtYE
CR
CRCR
NtYE
2
2exp1
20
202
CR
NCR
NtYE
411
21
2 002
ComparingNoisePowerinthefilterbandwidth
Power in band-limited noise
B
B
W
WW df
NdwNNE 10102 1
21
21
2
BNWNWNNE W 0002 222
2 where W is in rad/sec and B in Hz
The noise power in an RC RC
NYE RC 41
02
For an equivalent band-limited noise process to have the same power (assume a brick wall filter)
2002 41
2 RCWYE
RCN
WNNE
RCNWN
41
2 00
Therefore RC
W2
or RC
BW4
12
where B is in Hz
Note that the nominal -3dB band (½ power) of an RC network is
RCW dB
13 or RC
B dB 2
13
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 67 of 78 ECE 3800
Comparing these two, the equivalent noise bandwidth is greater than the –3dB bandwidth by
dBWW 32
or dBBB 342
Note: B in Hz and W in rad/sec.
0 0.5 1 1.5 2 2.5 3 3.5 4-0.2
0
0.2
0.4
0.6
0.8
1
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 68 of 78 ECE 3800
The power spectral density output of linear systems
The first cross-spectral density
hRR XXXY
diwRwS XYXY exp
diwhRwS XXXY exp
Using convolution identities of the Fourier Transform (if you want the proof it isn’t bad, just tedious)
wHwSwS XXXY
The second cross-spectral density
hRR XXYX
diwRwS YXYX exp
diwhRwS XXYX exp*
Using convolution identities of the Fourier Transform (if you want the proof it isn’t bad, just tedious)
*wHwSwS XXYX
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 69 of 78 ECE 3800
The output power spectral density becomes
hhRR XXYY
diwRwS YYYY exp
diwhhRwS XXYY exp
Using convolution identities of the Fourier Transform
*wHwHwSwS XXYY
2wHwSwS XXYY
This is a very significant result that provides a similar advantage for the power spectral density computation as the Fourier transform does for the convolution.
This leads to the following table.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 70 of 78 ECE 3800
AdditionalTopics
System analysis with a noise input …
tx tn
th ty tr
Where the signal of interest is x(t), n(t) is a noise or interfering process. The signal plus noise is r(t) and the received system output is y(t) which has been filtered.
We have tntxtr
Assuming WSS with x and n independent and n zero mean
tntxtntxEtrtrERRR
tntntxtntntxtxtxERRR
NNXXRR RtxtnEtntxERR
NNNXXXRR RRR 2
NNXXRR RRR
And then
* hhRRR NNXXYY
hhRhhRR NNXXYY
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 71 of 78 ECE 3800
Signal‐to‐Noise‐RatioSNR(alwaysdoneforpowers)
The signal-to-noise ratio is the power ratio of the signal power to the noise power.
The input SNR is defined as
00
2
2
NN
XX
Noise
Signal
RR
tNEtXE
PP
The output SNR is defined as
0
02
2
hhRhhR
thtNEthtXE
PP
NN
XX
Noise
Signal
For a white noise process and assuming the “filter” does not change the input signal (unity gain), but strictly reduces the noise power by the equivalent noise bandwidth of the filter.
We have
dhN
dwwHwSR XXYY202
2210
With appropriate filtering with unity gain where the signal exists and bandwidth reduction for the noise
EQXXYY BNdwwSR
02
10
or EQXXYY BNRR 000
The output SNR is defined as EQ
XX
Noise
Signal
BNR
PP
0
0
The narrower the filter applied prior to signal processing, the greater the SNR of the signal! Therefore, always apply an analog filter prior to processing the signal of interest!
From the definition of band-limited noise power, the equation for the equivalent noise bandwidth (performed as a previous example)
12
10
12
12
20 dhNdhRthtNE NN
dffHdtthBEQ22
21
21
Under the unity gain condition
dtth1
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 72 of 78 ECE 3800
Otherwise, the equivalent noise bandwidth can be defined as
dffHfH
BEQ2
2max12
For a real, low pass filter this simplifies to
dffHH
BEQ2
2012
Using Parseval’s Theorem
dffHdwwHdtth 22221
2
2
2
2
02
dtth
dtth
H
dtthBEQ
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 73 of 78 ECE 3800
ExamplesofLinearSystemFrequency‐DomainAnalysis
Noise in a linear feedback system loop.
sX sY
1
1 ssA
sN
Linear superposition of X to Y and N to Y.
sNsYsXssAsY
1
sNsXssA
ssAsY
11
1
sNsXssA
ssAsssY
11
2
sNAss
sssXAss
AsY
2
2
2
There are effectively two filters, one applied to X and a second apply to N.
Ass
AsH X 2 and Ass
sssH N
22
sNsHsXsHsY NX
Generic definition of output Power Spectral Density:
wSwHwSwHwS NNNXXXYY 22
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 74 of 78 ECE 3800
Change in the input to output signal to noise ratio.
dwwS
dwwSSNR
NN
XX
In
dwwHN
wSH
dwwSwH
dwwSwHSNR
N
XXX
NNN
XXX
Out20
2
2
2
2
0
EQ
XX
X
N
XX
Out BN
wS
dwH
wHN
wSSNR
0
2
20
02
Where
dwH
wHB
X
NEQ 2
2
0
If the noise is added at the x signal input, the expected definition of noise equivalent bandwidth results.
Notes and figures are based on or taken from materials in the course textbook: Probability, Statistics and Random Processes for Engineers, 4th ed., Henry Stark and John W. Woods, Pearson Education, Inc., 2012.
B.J. Bazuin, Fall 2016 75 of 78 ECE 3800
Systems that Maximize Signal-to-Noise Ratio
SNR is defined as EQNoise
Sign