Distribution Function properties

1. ( ) 0XF

2. ( ) 1XF 3. 0 ( ) 1XF x

1 2 1 24. ( ) ( ) X XF x F x if x x 1 2 2 15. ( ) ( ) X XP x X x F x F x

6. ( ) ( )X XF x F x

( ) ( 0) ( 1) (( ) ( 1) ( 2)2) ( 3 ( 3) )step function step function step function step function

X u x u xF x P X P X P X P uXu x x

Density Function

– We define the derivative of the distribution function FX(x) as the probability density function fX(x).

( )( ) X

X

dF xf x

dx

1

( ) ( )N

i ii

P x x x

1

( ) ( )N

i ii

dP X x u x x

dx

2

11 2

Properties of Density Function

1. ( ) 0 for all x

2. ( ) 1

3. ( ) ( )

4. ( )

X

X

x

X X

x

Xx

f x

f x dx

F x f d

P x X x f x dx

0

( ( ) (1 ) )N

k

kX

N kp pf kN

kxx

is called the binomial density function.

N = 6 p = 0.25

BinomialLet 0 < p < 1, N = 1, 2,..., then the function

! is thebinomialcoefficient

!( )!

the no. of combinations of objects from

N N

k k N k

k N

( ) (1 )k N kN

kpP k pX

5 possible combinations for 1 Head

The probability of obtaining one head in 5 coin tosses is

Two heads in 5 tosses

5 5!10

2 2!(5 2)!

2 32 1

10 X X 0.1653 3

!

!( )!N

k

N NC

k k N k

( ) (1 )k N kNP X k

kp p

7 10 710( 7) 0.4 (1 0.4)

7P X

7 3(120)0.4 (0.6) 0.0425

( ) (1 )k N kNP X k

kp p

N Probabilty of

successes Number of successesk

Probability  of 7 successes out of 1  0 independent trials

0

( ) ( )!

bk

Xk

f x xek

bk

0

( ) ( )!

bk

Xk

F x u xk

e kb

 is the parameter of the distribution.

We say X follows a Poisson distribution

with parameter (average rate)

0

( !

)k

bk

x kk

eb

 is the parameter of the distribution

(average rate)

bIn our book

1.8 birth/houre

1.8 birth/houre

an infinite number of probabilities to calculate

What is the probability of observing no birth (X=0) births in a given hour at the hospital?

01.8 1.8

( 4) 0!

P X e

1.8 e 0.165

1.8 birth/houre

Then Y is Poisson with 3.6

The Gaussian Random Variable

22 2

2

( )1( )

2X xa

x

xXf ex

2( , )X XX N a

Which is tabulated

( ) X

Xx

x aF x F

for real constants and 0a b

2

/

( )

1,

Uniform distribution 0 ,

1,

Exponential distribution            0 ,

2( )

The Rayleigh distribution ( ) 0

X

x a b

X

x a b

X

a x bf x b a

elsewhere

e x af x b

x a

x a e x af x b

2( )

1 ( )

0

x a b

X

x a

e x aF x

x a

Let X be a random variable and define the event  A = X x

Conditional Distribution

X

{X x}

X

BA

we define the conditional distribution function  F (x|B)

P{X x B}F (x|B) = P{X x |B} =

P(B)

P(A B)P(A|B) =

P(B)

Conditional Distribution and Density Functions

X

X

X

X 1 X 2 1 2

1 2 X 2 X 1

Properties of Conditional Distribution

(1) F ( |B) = 0

(2) F ( |B) = 1

(3) 0 F (x|B) 1

(4) F (x |B) F (x |B) if x < x

(5) P x < X x |B = F (x |B) F (x |B)

(6)

+

X X F (x |B) = F (x|B)

Conditional Density Functions

( | )( | ) X

X

dF x Bf x B

dx

2

1

X

X

x

X X

x

1 2 Xx

Properties of Conditional Density

(1) f (x|B) 0

(2) f (x|B) dx = 1

(3) F (x|B) = f (ξ|B)dξ

(4) P x < X x |B = f (x|B)dx

X

XX

F (x)x < b

F (b)F (x|X b) =

1 b x

X XF (x|X b) F (x)

The conditional density function derives from the derivative

XX

dF (x|X b)f (x|X b) =

dx

X Xf (x|X b) f (x)

X Xb

X X

f (x) f (x) = x < b

F (b) f (x)dx=

0 x b

Similarly for the conditional density function

Example 8 Let X be a random variable with an exponential probability density function given as

0

0 0( )

X

xe x

xf x

2

( )

( ) Since 2 ( | )

2

0 2

X

XX

f x

f x dxf x X

x

x

21

2

0 2

xee

x

x

Find the probability P( X < 1 | X ≤ 2 )

( | )2X Xf x

( )Xf x

1

0

( 1| )2 2| ) (X

X XP f x dX x 1

02

1

xe dx

e

1

2 11

ee

0.7310

1

02

1

xe dxe

Ch3 Operations on one random variable-Expectation

i X

i ix S

X

E X = X = x P(x ) if   X is discrete values

E X = X = xf (x)dx if   X is continuous value with Probability density

Expected value of a random variable

N

i ii=1

X

E g(X) = g(x )P(x )

E g(x) = g(x)f (x)dx

Expected value of a Function of a random variable

Conditional Expectation

We define the conditional density function for a given event

B = { X b}X

b

XX

f (x)x < b

f (x)dxf (x|X b) =

0 x b

we now define the conditional expectation in similar manner

x <

b

X X X

b x

b

b

E X|B = xf (x|B)dx = xf (x|B)dx + xf (x|B)dx

XF (b)

0

constant =

bX

b bX

f (x)= x dx + x 0 dx

f (x)dx

X

b

X

b

X

F (constant = b)

xf (x)dx=

f (x)dx

Moments

X

The expected value defined previously as

E X = X = xf (x)dx

th

n

n nn X

we can define the n  moment m as

m = E X = x f (x)dx

(about the origin)

11 Xm = E X = xf (x)dx = The expectedX val Xue of

th

n nn X

we define the n  moment

μ = E (X X) = (x X) f (x)dx

Central(about the m Moea ) sn ment

X

E 1 =1

0 00 X X

f (x)dx = 1

The area of the functioμ = E (X X) = (X X) f (x)dx = 1 f ( x) n

constan

1

t

1μ = E (X X) = E[X] E[X] = X X = 0

were we have used the fact that  E[ a ] = a

n nn Xμ = E (X X) = (x X) f (x)dx

Moments

i

N

ni=

i1

m E X = P(X x x = )n n

Moments about the origin Moments about the mean called central moments

1m E X = X

X

nn

n

μ = E (X X)

= (x X) f (x)dx

nn X

nm = E X = ) xx f (x d

n

i

n

N

i=1

μ = E (X X)

P(X = x= x X )n

i

2 2 2x 2 X

Thus the variance is given by

σ = μ = E (X X) = (x X) f (x)dx

2 2 2 2 2x 2 2 1σ = μ = E (X X) =E X X = m m

2 2 2x 2 X

2 2 22 1

Thus the variance is given by

σ = μ = E (X X) = (x X) f (x)dx

= E X X = m m

2c

2 2x + c x

2 2 2cx x

(1) σ 0 c  is  a constant 

(2) σ σ The variance does not change by shifting

(3) σ c σ

Properties of the variance 

3.3 Function that Give moments

nn X

n n

ω = 0

Φ (ω)= ( )

ω

dm j

d

X  ( )( ) XX

je f x dx

X Fourier Transform( ) )( ()X Xf x f x

-X

Inverse FourierTransform

X1

= Φ (ω)e dω2π

( ) )( j X

Xf x

Example Let X be a random variable with an exponential probability density function given as

0

0 0( )

X

xe x

xf x

1 ( )[ ] Xf x xm E X x d

0

xdxxe =1

X  ( ) xj Xe dxe

= Fourier Transform{ }

xe

1=

1 j

1=

1 j

X10

( ) ( )dm jd

X1

1( )

j

d dd d

2(1 )

j

j

20

1 (1 )

( ) j

jm j

=1

Now let us find the 1st moment (expected value) using the characteristic function

2

20

(1 (0))

j

j

3.4 Transformations of A Random Variable

1

1( )

( )

( ) ( ) x T y

x T y

Y X

dxf y f x

dy

OR

Nonmonotonic Transformations of a Continuous Random Variable

n

X n

n

x = x

f (x )( )

dT(x)dx

Yf y

Ch4: Multiple Random Variables

Joint Distribution and its Properties

6 6

X,Y n m m mn=1 m=1

F (x,y) = P(X x, Y y) = P(x ,y ) u(x x ) u(y y )

6 6

X,Y n m m mn=1 m=1

f (x,y) = P(x ,y ) δ(x x ) δ(y y )

y x

X,Y X,Y 1 2 1 2F (x,y) = f (ξ ,ξ )dξ dξ

2X,Y

X,Y

F (x,y)f (x,y) =

x y

Properties of the joint distribution

X,Y X,Y X,Y(1) F ( , ) = 0 F ( , y) = 0 F (x, ) = 0

XY(2) F ( , ) = 1

XY(3) 0 F (x,y) 1

X(4) F (x,y) is a nondecreasing function of both x and y

1 2 1 2 X,Y 2 2 X,Y 1 2 X,Y 2 1 X,Y 1 1(5) P x < X x ,y < Y y = F (x ,y ) F (x ,y ) F (x ,y ) + F (x , y ) 0

X,Y X X,Y Y(6) F (x, ) = F (x) F ( , y) = F (y)

Marginal Distribution Functions

X,Y X X,Y YF (x, ) = F (x) F ( , y) = F (y)

2X,Y

X,Y

F (x,y)f (x,y) =

x y

y x

X,Y X,Y 1 2 1 2F (x,y) = f (ξ ,ξ )dξ dξ

Joint Density and its Properties

Properties of the Joint Density

X,Y

X,Y x y

(1) f (x,y) 0

(2) f (x,y)d d = 1

y x

X,Y X,Y 1 2 1 2(3) F (x,y) = f (ξ ,ξ )dξ dξ

x

X X,Y 1 2 2 1

y

Y X,Y 1 2 1 2

(4) F (x) = f (ξ ,ξ )dξ dξ

F (y) = f (ξ ,ξ )dξ dξ

2 2

1 1

y x

1 2 1 2 X,Y x yy x(5) P x < X x ,y < Y y = f (x,y) d d

X X,Y

Y X,Y

(6) f (x) = f (x,y) dy

f (y) = f (x,y) dx

Properties (1) and (2) may be used as sufficient test to determine if some function can be a valid density function

Marginal Densities

Marginal Distribution

Conditional Distribution and Density

The conditional distribution function of a random variable X given some event B was defined as

X

P X x BF x|B =P X x|B = were P B 0

P B

The corresponding conditional density function was defined through the derivative

XX

dF x|Bf x|B =

dx

N

Ni

i K ii = 1

K

Ki

i = 1 KX KF

P(xP(x ,y )u(

(x|Y= y ),y )

u(x x )P(y

x x ) =

)P(y )

Ni K

ii = 1 K

KX K

X

dF (x|Y= y ) =

df (x|Y= y

P(x ,y )δ(x x

)) )

x P(y

x

X,Y 1 1

YY

X

f ξ ,y dξ For every y such that f (y) 0

f yF (x Y = y)

X,X

YX

Y

f x,ydF (Y = y) f x|Y = y =

dx f y

(2) X and Y are Continuous

(1) X and Y are Discrete

STATICAL INDEPENDENCE

k k

i i

i i

1

j j

j j

2 21N N

X j X jX i X i

X X k X ki X i

X 1 X

X j X j

X 2 XX N X N21

F ( , ) F ( )F ( )

F ( , , ) F ( )F ( )F ( )

x x

F ( , , ) F ( )

x x

x

F (

x x

x x

x x )

x

x Fx (x )x

k k

i i

i i

1

j j

j j

2 21N N

X j X jX i X i

X X k X ki X i

X 1 X

X j X j

X 2 XX N X N21

f ( , ) f ( )f ( )

f ( , , ) f ( )f ( )f ( )

x x

f ( , , ) f ( )

x x

x

f (

x x

x x

x x )

x

x fx (x )x

WF w = P W w = P X+Y w

w y

W X,YxF w = f (x,y) xd d y

w y

W Y XxF w = f (y) f (x) dx dy

WW Y X

Y X

using Leibnizerule we get 

dF wf w = = = f (y) f (x)

dwf (y)f (w x)dy

Convolution Integral

We seek the distribution or density of W=X+Y

X,Y X YIf X and are independent f (x,y) f (x)f (y)

WW

dF wf w =

dw

Operations on Multiple Random Variables

X,Y

i k X,Y i ki k

Cong(x,y)f (x,y)dxdyg = E g(X,Y) =

g(x ,y )P (x ,

tin

y Disc

uou

r

s

t) e e

n k n knk X,Ym = E X Y = x y f (x,y)dxdy

Joint Moment about the Origin

n thn0 n

k th0k

m = E[X ] the  n  moment m  of the one random variable X

m = E[Y ] the  k  moment m  of the one random variable Yk

n= n=

X,Y

n m n m

11XY = m = E[XY] = xyf (x,y)dxdy Continuous

=

R

x y P(x ,y ) Discrete

correlation

uncorrelated

0 OrthogonalXY

E[X]E[Y] =R

2 2

2 2

2 2X 20

2 2Y 02

The variance

σ μ = E (X X) = E[X ] X

σ μ = E (Y Y) = E[Y ] Y

Y XY11X = μ = E (X X)(Y Y) = E[X]E[Y]RC

covariance

0 if X and Y are uncorrelated

E[X]E[Y] if X and Y are orthogonal

Independence UncorrelationThe converse is not true in general except for Gaussian

Y XY11X = μ = E (X X)(Y Y) = E[X]E[Y]RC

covariance

0 if X and Y are uncorrelated

E[X]E[Y] if X and Y are orthogonal

XY

X Y1   ρ 1

The correlation coefficient

Cρ = σ σ

1 2

n+kX,Y 1 2n+k

nk n k1 2 ω =0, ω 0

X,Y 1 2

The joint moments can be found from the joint characteristic function

Φ (ω ,ω )m = ( j)

ω ω

f (x, y) Φ (ω ,ω ) 2D Fourier Transform with reversal of   signXY

i i 1 2 NY = T (X ,X ,...,X ) i = 1,2, ..., N

1j j 1 2 NX = T (Y ,Y ,...,Y ) j = 1,2, ..., N

1 2 N 1 2 N

1 1Y ,Y ,...,Y 1 N X ,X ,...,X 1 1 N Nf (y ,...,y ) f (x = T ,...,x = T ) J

1 11 1

1 N

1 1N N

1 N

T T

Y Y

J =

T T

Y Y

Random Process and its Applications to linear systems

Distribution and Density of Random Processes

1 1For a particular time t , t is a random variableX

1 11 t t t R.V has a distribution (x) , and a density (x)X XX F f

1

1 1

1t1 1 1 1t t

1

( )( ) ( ) =

X

X X

dF xF x P X t x f x

dx

A process that satisfies the followings :

( ) = constant ( ) ( ) ( )XXE X t X E X t X t R

Wide - Sense Stationary WSS

T

TT

The time average of a quantity is defined as

1A = lim dt

2T

XXR (t, t + τ) = E X(t)X(t + τ)

Autocorrelation Function and Its Properties

None random Deterministic Function

X(t) Y(t)=X(t)*h(t)h(t)

Linear SystemXXR (τ)YYR (τ)

None random Deterministic Function

Random Function Random Function

X Y = XH(0)

XXR (τ) YY XXR (τ) = R (τ) h( τ) h(τ)

( )XXS f ( ) =YYS f ( )XXS f *( )H f ( )H f

2

( )H f

= ( )XXS f df

2[ ( )] = (0) = ( )YY YYE Y t R S f df

2

XX[ ( )]=R (0)E X t2

= ( ) ( )XXS f H f df

Total power of the Input

2= ( ) ( )XXS f H f

Total power of the Output