DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

STOCHASTIC SIGNALS AND PROCESSES

Lecture 1

WELCOME

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Introduction to probability and random variablesA deterministic signal can be derived by

mathematical expressions.A deterministic model (or system) will always

produce the same output from a given starting condition or initial state.

Stochastic (random) signals or processes• Counterpart to a deterministic process• Described in a probabilistic way• Given initial condition, many realizations of the

process

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Introduction to probability

Define the probability of an event A as:

where N is the number of possible outcomes of the random

experiment and NA is the number of outcomes favorable to the

event A.

For example: A 6-sided die has 6 outcomes. 3 of them are even, Thus P(even) = 3/6

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Axiomatic Definition of Probability• A probability law (measure or function)

that assigns probabilities to events such that:oP(A) ≥ 0oP(S) =1o If A and B are disjoint events (mutually exclusive), i.e. A ∩ B = ∅, then P(A ∪ B) = P(A) + P(B)

That is if A happens, B cannot occur.

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Some Useful Properties

: probability of impossible event

, the complement of A

If A and B are two events, then

If the sample space consits of n mutually exclusive events such that , then

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Joint and marginal probability

Joint probability:is the likelihood of two events occurring together.

Joint probability is the probability of event A occurring at the same time event B occurs. It is P(A ∩ B) or P(AB).

Marginal probability:is the probability of one event, ignoring any

information about the other event. Thus P(A) and P(B) are marginal probabilities of events A and B

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Conditional probability

Let A and B be two events. The probability of event B given that event A has occured is called the conditional probability

If the occurance of B has no effect on A, we say A and B are indenpendent events. In this case

Combining both, we get , when A and B are indenpendent

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Random variables

A random variable is a function, which maps events or outcomes (e.g., the possible results of rolling two dice: {1, 1}, {1, 2}, etc.) to real numbers (e.g., their sum).

A random variable can be thought of as a quantity whose value is not fixed, but which can take on different values.

A probability distribution is used to describe the probabilities of different values occurring.

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Random variablesNotations:Random variables with capital letters: X, Y, ..., ZReal value of the random variable by lowercase letters (x,

y, …, z)

Types:Continuous random variables: maps outcomes to values of

an uncountable set. the probability of any specific value is zero.

Discrete random variable: maps outcomes to values of a countable set. Each value has probability ≥ 0. P(xi) = P(X = xi)

Mixed random variables

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Continuous random variablesDistribution function:

Properties:1. is either increasing or remains constant.

2.

3.

4.

5.

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Distribution functions

Cumulative distribution function (CDF) for a normal distribution

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Probability density function (pdf)Definition:

Properties:

Thus integration of gives probability

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Expectation operator

Is defined only for random variables or a function of them.

Expected value: Is the weighted average of all possible values that this random variable can take on.

Let

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Definition of moments

A moment of order k of a random variable is defined as:

Order 1:

Order 2: Mean of X

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Central moments

Defined in a similar way as moment

Variance of X

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Two random variables

Let X and Y be two random variables, we define the joint distribution function

and the joint probability density function as

≥ 0

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Useful Properties

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Useful Properties

The probability that X lies between x1 and x2 and Y lies between y1 and y2 is

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Indenpendent and uncorrelated random variablesLet X and Y be two random variables, we say that

X and Y are indenpendent if

X and Y are uncorrelated if

Indenpendence => uncorrelation

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Expected value

If X and Y are indenpendent we get The correlation

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Uncorrelated

If we say X and Y are orthogonal

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Correlation coefficient Covariance

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Random processesA random process may be viewed as a collection

of random variables, with time t as a parameter running through all real numbers.

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Example

With Φ a random variable uniformly distributed between 0 and 2π.

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

First and second order statistics

Expected value

Autocorrelation function

Properties: Stationarity, Ergodicity, Power spectrum

DEPARTMENT OF HEALTH SCIENCE AND TECHNOLOGY

ww

w.h

st.a

au.

dk

Homework

Workshop: Get familiar with the following terms

Probability density function, independency Autocorrelation, Stationarity, Ergodicity, Power spectrum.

Exercises on the web (click here)