PROBABILITY AND STATISTICS FOR ENGINEERING

transcript

Hossein Sameti

Department of Computer EngineeringSharif University of Technology

Let X represent a Binomial r.v ,Then from

for large n. In this context, two approximations are extremely useful.

kkn qp

kPkXkP

) " and between is of sOccurrence ("2

kkkkkk qp

XPXXXP

The Normal Approximation (Demoivre-Laplace Theorem)

If hen for k in the neighborhood of np, we can approximate

(for example, when with p held fixed) And we have:

1 2/)( 2 npqnpkknk enpq

dyedxenpq

kXkP yx

npqnpxk

. , 22

11 npq

npkxnpq

As we know,

If and are within with approximation:

We can express this formula in terms of the normalized integral

that has been tabulated extensively.

kkn qp

kPkXkP

1k 2k , , npqnpnpqnp

dyedxenpq

kXkP yx

npqnpxk

)(21)(

xerfdyexerfx y

. , 22

11 npq

npkxnpq

x erf(x) x erf(x) x erf(x) x erf(x)

0.10 0.15 0.20 0.25

0.30 0.35 0.40 0.45 0.50

0.55 0.60 0.65 0.70 0.75

0.01994 0.03983 0.05962 0.07926 0.09871 0.11791 0.13683 0.15542 0.17364 0.19146 0.20884 0.22575 0.24215 0.25804 0.27337

0.80 0.85 0.90 0.95 1.00

1.05 1.10 1.15 1.20 1.25

1.30 1.35 1.40 1.45 1.50

0.28814 0.30234 0.31594 0.32894 0.34134 0.35314 0.36433 0.37493 0.38493 0.39435 0.40320 0.41149 0.41924 0.42647 0.43319

1.60 1.65 1.70 1.75

1.80 1.85 1.90 1.95 2.00

2.05 2.10 2.15 2.20 2.25

0.43943 0.44520 0.45053 0.45543 0.45994 0.46407 0.46784 0.47128 0.47441 0.47726 0.47982 0.48214 0.48422 0.48610 0.48778

2.35 2.40 2.45 2.50

2.55 2.60 2.65 2.70 2.75

2.80 2.85 2.90 2.95 3.00

0.48928 0.49061 0.49180 0.49286 0.49379 0.49461 0.49534 0.49597 0.49653 0.49702 0.49744 0.49781 0.49813 0.49841 0.49865

A fair coin is tossed 5,000 times. Find the probability that the number of heads is between 2,475 to 2,525.

We need

Since n is large we can use the normal approximation. so that and

and So the approximation is valid for and

Example

Solution

).525,2475,2( XP

p 500,2np .35npq

,465,2 npqnp475,21 k .525,22 k

,535,2 npqnp

Using the table,

Example - continued

.21 2/

y dyekXkP

npkxnpq

516.075erf2

|)(|erf)(erf )(erf)(erf525,2475,2

xxxxXP

. 258.0)7.0(erf

The Poisson Approximation

For large n, the Gaussian approximation of a binomial r.v is valid only if p is fixed, i.e., only if and

What if is small, or if it does not increase with n?- for example when , such that is a fixed

number.

1np .1npq

np0p ,n np

The Poisson Theorem

,0p ,n np

knkn !

)1(!)!(

!)( 𝑛→ ∞→

Consider random arrivals such as telephone calls over a line.

n : total number of calls in the interval as we have Suppose Δ : a small interval of duration Let be the probability of k calls in the interval Δ We have shown that

).,0( TT n

.Tn 0 T

𝑃𝑛 (𝑘 )=𝑃 (𝑘𝑖𝑛∆ )=(𝑛𝑘)𝑝𝑘𝑞𝑛−𝑘𝑤h𝑒𝑟𝑒𝑝= ∆𝑇

p: probability of a particular call occurring in Δ: as

Normal approximation is invalid here because is not valid.

: probability of obtaining k calls (in any order) in an interval of duration Δ ,

knkn pp

kknnkP

)1(!)!(

.)/1()/1(

!112111

)/1(!)()1()1()(

knnnkP

nnppn !)(lim

,0 ,the Poisson p.m.f

Suppose - two million lottery tickets are issued - with 100 winning tickets among them.

a) If a person purchases 100 tickets, what is the probability of winning?

Example: Winning a Lottery

Solution

.105102

100 ticketsof no. Total

tickets winningof no. Total 56

pThe probability of buying a winning ticket

X: number of winning tickets

n: number of purchased tickets ,

P: an approximate Poisson distribution with parameter

So, The Probability of winning is:

Winning a Lottery - continued

.005.0105100 5 np

.005.01)0(1)1( eXPXP

b) How many tickets should one buy to be 95% confident of having a winning ticket?

we need

But or

Thus one needs to buy about 60,000 tickets to be 95% confident of having a winning ticket!

Winning a Lottery - continued

Solution

.95.0)1( XP

.320ln implies 95.01)1( eXP

3105 5 nnp .000,60n

A space craft has 100,000 components The probability of any one component being defective is The mission will be in danger if five or more components become defective. Find the probability of such an event.

n is large and p is small Poisson Approximation with parameter

Example: Danger in Space Mission

Solution

).0(102 5 p

2102000,100 5 np

.052.032

342211

!1)4(1)5(

keXPXP

Conditional Probability Density Function

, )( )( xXPxFX

.0)( ,)(

BAPBAP

)( |)( )|(BP

BxXPBxXPBxFX

.0)()(

)( )( )|(

,1)()(

)( )( )|(

BPBXPBF

Conditional Probability Density Function

Further,

Since for

),|()|( )(

)( )|)((

BxFBxFBP

BxXxPBxXxP

,12 xx

. )()()( 2112 xXxxXxX

)|()|(dx

BxdFBxf XX

21 .)|(|)(x

x X dxBxfBxXxP

XX duBufBxF

.)|()|(

Toss a coin and X(T)=0, X(H)=1. Suppose Determine

has the following form. We need for all x. For so that and

Example

Solution

}.{HB).|( BxFX

)|( BxFX

,)( ,0 xXx ,)( BxX.0)|( BxFX

( | )XF x B

For so that

For and

Example - continued

, )( ,10 TxXx

HTBxX )( .0)|( and BxFX

,)( ,1 xXx

}{ )( BBBxX 1)()()|( and

BPBPBxFX

( | )XF x B

Given suppose Find

We will first determine

For so that

Example

Solution

),(xFX .)( aXB ).|( BxfX

).|( BxFX

aXPaXxXPBxFX

xXaXxXax ,

)()()|(

aXPxXPBxF

)( , aXaXxXax .1)|( BxFX

and hence

Example - continued

, ,)()(

axaFxF

otherwise. ,0

,,)()(

)|()|( axaFxf

BxFdxdBxf

)|( BxFX

)|( BxfX

Let B represent the event with For a given determine and

Example

Solution

)()()( )(

)( )()(

|)( )|(

aFbFbXaxXP

bXaPbXaxXP

BxXPBxF

bXa )( .ab),(xFX )|( BxFX ).|( BxfX

For we have and hence

For we have so that

Example - continued

,bxa })({ )( )( xXabXaxX

.)()()()(

)()()()|(

aFbFaFxF

aFbFxXaPBxF

,bx bXabXaxX )( )( )( .1)|( BxFX

,ax ,)( )( bXaxX.0)|( BxFX

otherwise.,0

,,)()(

)()|( bxa

aFbFxf

)|( BxfX

Conditional p.d.f & Bayes’ Theorem First, we extend the conditional probability results to random variables: We know that If is a partition of S and B is an arbitrary event,

By setting we obtain:

)()|()()|()( 11 nn APABPAPABPBP

],,,[ 21 nAAAU

Son partition a form ,

)()|()()|()(

APAxfAPAxfxf

APAxFAPAxFxF

APAxXPAPAxXPxXP

}{ xXB

Conditional p.d.f & Bayes’ Theorem

Using:

We obtain:

)()|()|(BP

APABPBAP

|)| AP

AxXPxXAP

21 )( xXxB

)()()|()|(

APdxxf

dxAxfAP

xFxFAxFAxF

APxXxP

AxXxPxXxAP

Conditional p.d.f & Bayes’ Theorem

Let so that in the limit as

we also get

,0 , , 21 xxxx ,0

)|(|)(|lim

xXAPxXxAPX

)()|()|(| APxfxXAPAxf X

,)()|()|()(1

dxxfxXAPdxAxfAP XX

dxxfxXAPAP X )()|()(

(Total Probability Theorem)

Bayes’ Theorem (continuous version)

using total probability theorem in

We get the desired result

.)()|(

)()|()|(|

dxxfxXAP

xfxXAPAxfX

probability of obtaining a head in a toss. For a given coin, a-priori p can possess any value in (0,1). : A uniform in the absence of any additional information After tossing the coin n times, k heads are observed. How can we update this is new information?

Let A= “k heads in n specific tosses”. Since these tosses result in a specific sequence,

and using Total Probability Theorem we get

Example: Coin Tossing Problem Revisited

Solution

)( pfP

,)|( knkqppPAP

)( pfP

!)!()1()()|()(1

nkkndpppdppfpPAPAP knk

The a-posteriori p.d.f represents the updated information given the event A,

This is a beta distribution. We can use this a-posteriori p.d.f to make further predictions. For example, in the light of the above experiment, what can we say about the

probability of a head occurring in the next (n+1)th toss?

Example - continued

| ( | )P Af p A

|( | ) ( )( | )

( )( 1)! , 0 1 ( , ).

( )! !

P A P p f pf p AP A

n p q p n kn k k

)|(| Apf AP

Let B= “head occurring in the (n+1)th toss, given that k heads have occurred in n previous tosses”.

Clearly From Total Probability Theorem,

Using (1) in (2), we get:

Thus, if n =10, and k = 6, then

which is more realistic compared to p = 0.5.

Example - continued

,)|( ppPBP

0 .)|()|()( dpApfpPBPBP P

,58.0127)( BP

( 1)! 1( ) .( )! ! 2

k n kn kP B p p q dpn k k n

PROBABILITY AND STATISTICS FOR ENGINEERING

Documents