Independence and Bernoulli Trials
Sharif University of Technology 2
Independence
A, B independent implies: are also independent.
Proof for independence of :
Events A and B are independent if
).()()( BPAPABP
BAABBAAB )(
, BAAB
),()()())(1()()()()( BPAPBPAPBPAPBPBAP
)()()()()()()( BAPBPAPBAPABPBAABPBP
Sharif University of Technology 3
Application
let :
Then
Also
Hence it follows that Ap and Aq are independent events!
" the prime divides the number "pA p N
" the prime divides the number ".qA q N
1 1{ } , { }p qP A P A
p q
1{ } {" divides "} { } { }p q p qP A A P pq N P A P A
pq
Example
Sharif University of Technology 4
Some Properties of Independence
Let P(A)=0, Then:
Thus: P(AB) = P(A) P(B) = 0
The event of zero probability is independent of every other event
Other independent events cannot be ME
Since: 0)( ,0)( BPAP
.0)( ABP
AAB
,0)(0)()( ABPAPABP
Sharif University of Technology 5
If and Ais are independent,
Application in solving number theory problems.
Application
A family of events {Ai} are said to be independent, if for every finite sub collection
Remark
Application
,,,,21 niii AAA
n
ki
n
ki kk
APAP11
).(
,321 nAAAAA
nAAAA 21
.))(1()()()(11
21
n
ii
n
i
in APAPAAAPAP
,))(1(1)(1)(1
n
iiAPAPAP
Sharif University of Technology 6
Example
Each switch remains closed with probability p.
(a) Find the probability of receiving an input signal at the output.
Three independent parallel switches
Let Ai = “Switch Si is closed” and R = “input signal is received at the output”. Then
,)( pAP i
).()()()( );()()( 321321 APAPAPAAAPAPAPAAP jiji .31 i
Solution
.321 AAAR .33)1(1)()( 323
321 ppppAAAPRP
Sharif University of Technology 7
Example – continued
Since any event and its complement form a trivial partition,
Thus:
Another Solution
).()|()()|()( 1111 APARPAPARPRP
,1)|( 1 ARP2
321 2)()|( ppAAPARP
,33)1)(2()( 322 pppppppRP
Note:The events A1, A2, A3 do not form a partition, since they are not ME.
Moreover, .1)()()( 321 APAPAP
Sharif University of Technology 8
Example – continued
From Bayes’ theorem
Also, because of the symmetry :
Solution
.33
32
33
)1)(2(
)(
)()|()|(
2
2
32
211
1pp
pp
ppp
ppp
RP
APARPRAP
).|()|()|( 321 RAPRAPRAP
(b) Find the probability that switch S1 is open given that an input signal is received at the output.
Sharif University of Technology 9
Repeated Trials
A joint performance of the two experiments with probability models (1, F1, P1) and (2, F2, P2).
Two models’ elementary events: 1, 2
How to define the combined trio (, F, P)?
= 1 2
Every elementary event in is of the form = (, ).
Events: Any subset A B of such that AF1 and B F2
F : all such subsets A B together with their unions and complements.
Sharif University of Technology 10
Repeated Trials
In this model, The events A 2 and 1 B are such that
Since
the events A 2 and 1 B are independent for any A F1 and B F2.
So, for all A F1 and B F2
Now, we’re done with definition of the combined model.
).()( ),()( 2112 BPBPAPAP
,)()( 12 BABA
)()()()()( 2112 BPAPBPAPBAP
Sharif University of Technology 11
Generalization
Experiments with
let
Elementary events
Events in are of the form
and their unions and intersections.
If s are independent, and is the probability of the event in then
,,,, 21 n ,1 , and niPF ii
,,,, 21 n .ii
nAAA 21
n 21
,ii FA
)( ii APiA iF
1 2 1 1 2 2( ) ( ) ( ) ( ).n n nP A A A P A P A P A
iA
Sharif University of Technology 12
Example
A has probability p of occurring in a single trial. Find the probability that A occurs exactly k times, k n in n
trials.
The probability model for a single trial : (, F, P)
Outcome of n experiments is :
where and
A occurs at trial # i , if Suppose A occurs exactly k times in .
,,,, 021 n i 0
Solution
.Ai
Sharif University of Technology 13
Example - continued
If belong to A,
Ignoring the order, this can happen in N disjoint equiprobable ways, so:
Where
1 2, , ,
ki i i
.)()()( )()()(
}) ({}) ({}) ({}) ({}) ,,,,, ({)(21210
knk
knk
iiiiiiii
qpAPAPAPAPAPAP
PPPPPPnknk
,)()(
) trials" in timesexactly occurs ("
01
0knk
N
ii qNpNPP
nkAP
k
n
kkn
n
k
knnnN
!)!(
!
!
)1()1(
Sharif University of Technology 14
Bernoulli Trials
Independent repeated experiments Outcome is either a “success” or a “failure”
The probability of k successes in n trials is given by the above formula,
where p represents the probability of “success” in any one trial.
,,,2,1,0 ,
) trials" in timesexactly occurs (")(
nkqpk
n
nkAPkP
knk
n
Sharif University of Technology 15
Example
Toss a coin n times. Obtain the probability of getting k heads in n trials.
“head” is “success” (A) and let Use the mentioned formula.
In rolling a fair dice for eight times, find the probability that either 3 or 4 shows up five times ?
Thus
).(HPp
. } 4or 3either { success"" 43 ffA
3
1
6
1
6
1)()()( 43 fPfPAP
Example
Example
5 38
8( ) (5) (1/ 3) (2 / 3) 0.068282
5k n k
n
nP k p q P
k
Sharif University of Technology 16
Bernoulli Trials
Consider a Bernoulli trial with success A, where Let:
As are mutually exclusive,
so,
.)( qAP ,)( pAP
. trials" in soccurrence exactly " nkX k
.1)( 10 nXXXP
ji XX ,
n
k
n
k
knkkn qp
k
nXPXXXP
0 010 .)()(
,)(0
knkn
k
n bak
nba
1)( nqp
Sharif University of Technology 17
Bernoulli Trials
For a given n and p what is the most likely value of k ?
Thus if or
Thus as a function of k increases until
The is
the most likely number of successes
in n trials..2/1 ,12 pn
Fig. 2.2
)(kPn
k
.1!
!)!(
)!1()!1(
!
)(
)1( 11
p
q
kn
k
qpn
kkn
kkn
qpn
kP
kPknk
knk
n
n
),1()( kPkP nn pknpk )1()1( .)1( pnk
)(kPn pnk )1(
],)1[(max pnk
Sharif University of Technology 18
Effects Of p On Binomial Distribution
Sharif University of Technology 19
Effects Of p On Binomial Distribution
Sharif University of Technology 20
Effects Of n On Binomial Distribution
Sharif University of Technology 21
Example
In a Bernoulli experiment with n trials, find the probability that the
number of occurrences of A is between k1 and k2.
.)()(
) " and between is of sOccurrence ("
2
1
2
1
211 1
21
k
kk
knkk
kkkkkk qp
k
nXPXXXP
kkAP
Solution
Sharif University of Technology 22
Example
Suppose 5,000 components are ordered. The probability that a part is defective equals 0.1. What is the probability that the total number of defective parts does not exceed 400 ?
".components 5,000 among defective are parts " kYk
.)9.0()1.0(5000
)()(
5000400
0
400
040010
kk
k
kk
k
YPYYYP
Solution
Sharif University of Technology 23
If kmax is the most likely number of successes in n trials,
or
So,
This connects the results of an actual experiment to the axiomatic definition of p.
pnkpn )1(1)1( max
,max
n
pp
n
k
n
qp
.lim pn
km
n
Sharif University of Technology 24
Bernoulli’s Theorem
A : an event whose probability of occurrence in a single trial is p.
k : the number of occurrences of A in n independent trials
Then ,
i.e. the frequency definition of probability of an event and its
axiomatic definition ( p) can be made compatible to any degree of accuracy.
.2
n
pqp
n
kP
n
k
Sharif University of Technology 25
Proof of Bernoulli’s Theorem
Direct computation gives:
Similarly,
.)(
!)!1(
)!1(
!)!1(
!
)!1()!(
!
!)!(
!)(
1
11
0
111
0
1
1
10
npqpnp
qpiin
nnpqp
iin
n
qpkkn
nqp
kkn
nkkPk
n
inin
i
inin
i
knkn
k
n
k
knkn
kn
.)!1()!(
!
)!2()!(
!
)!1()!(
!)(
22
1
210
2
npqpnqpkkn
n
qpkkn
nqp
kkn
nkkPk
knkn
k
knkn
k
n
k
knkn
kn
Sharif University of Technology 26
Proof of Bernoulli’s Theorem - continued
Equivalently,
Using equations derived in the last slide,
,)( toequivalent is 222 nnpkpn
k
.)()()( 22
0
22
0
2 nkPnkPnpk n
n
kn
n
k
.2
)(2)()()(
2222
22
00
2
0
2
npqpnnpnpnpqpn
pnkPknpkPkkPnpk n
n
kn
n
kn
n
k
.
)( )()(
)()()()()()(
22
222
22
0
2
nnpkPn
kPnkPnpk
kPnpkkPnpkkPnpk
nnnpk
nnnpk
nnnpk
nnnpk
n
n
k
Sharif University of Technology 27
Proof of Bernoulli’s Theorem - continued
Using last two equations,
For a given can be made arbitrarily small by letting n become large.
Relative frequency can be made arbitrarily close to the actual probability of the event A in a single trial by making the number of experiments large enough.
As the plots of tends to concentrate more and more around
.2
n
pqp
n
kP
2/ ,0 npq
maxk,n )(kPn
Conclusion
Sharif University of Technology 28
Example: Day-trading strategy
A box contains n randomly numbered balls (not necessarily 1 through n)
m = np (p<1) of them are drawn one by one by replacement The drawing continues until a ball is drawn with a number larger than the
first m numbers.
Determine the fraction p to be initially drawn, so as to maximize the
probability of drawing the largest among the n numbers using this strategy.
Sharif University of Technology 29
Day-trading strategy: Solution
Let drawn ball has the largest number among:
all n balls,
the first k balls is in the group of first m balls, k > m.
= where
A = “largest among the first k balls is in the group of first
m balls drawn”
B = (k+1)st ball has the largest number among all n balls”.
Since A and B are independent , So,
P (“selected ball has the largest number among all balls”)
stk kX )1(
kX ,A B
. 1
1
)()()(kp
knp
nkm
nBPAPXP k
1 1
1 1( ) ln
ln .
n n n n
k npnpk m k m
P X p p p kk k
p p
Sharif University of Technology 30
Day-trading strategy: Solution
To maximize with respect to p, let:
So, p = e-1 ≈ 0.3679
This strategy can be used to “play the stock market”. Suppose one gets into the market and decides to stay up to 100 days. When to get out?
According to the solution, when the stock value exceeds the maximum among the first 37 days.
In that case the probability of hitting the top value over 100 days for the stock is also about 37%.
0)ln1()ln( pppdpd
Sharif University of Technology 31
Example: Game of Craps
A pair of dice is rolled on every play,- Win: sum of the first throw is 7 or 11
- Lose: sum of the first throw is 2, 3, 12
- Any other throw is a “carry-over”
If the first throw is a carry-over, then the player throws the dice repeatedly until:- Win: throw the same carry-over again
- Lose: throw 7
Find the probability of winning the game.
Sharif University of Technology 32
Game of Craps - Solution
kkkkk
PkCBPkCPkCBPBPP
TPTPTPQ
TPTPP
10
7,4
10
7,42
1
1
)|()()|()(
9
1
36
1
36
2
36
1)12()3()2(
9
2
36
2
36
6)11()7(
B: winning the game by throwing the carry-overC: throwing the carry-over
P1: the probability of a win on the first throw
Q1: the probability of loss on the first throw
Sharif University of Technology 33
Game of Craps - Solution
3/15/211/511/55/23/1
1098654
)|(
,...,3,2,1,
6/11
1
1
1
k
j
jkkk
jkk
kk
a
k
rpkCBPa
jrp
pr
The probability of win by throwing the number of Plays that do not count is:
The probability that the player throws the carry-over k on the j-th throw is:
Sharif University of Technology 34
Game of Craps - Solution
492929.0495
134
9
2
495
134
36
3
3
1
36
4
5
2
36
5
11
5
36
5
11
5
36
4
5
2
36
3
3
1
21
10
7,42
PP
paPkk
kk
the probability of winning the game of craps is 0.492929 for the player.
Thus the game is slightly advantageous to the house.
Sharif University of Technology 35
Game Of Craps Using Biased Dice
Now assume that for each dice,
- Faces 1, 2 and 3 appear with probability
- Faces 4, 5 and 6 appear with probability
If T represents the combined total for the two dice, we get
16 0,
16
24
2 25
2 26
7
16
1 136 6
1 136 6
{ 4} {(1,3),(2,2),(1,3)} 3( )
{ 5} {(1,4),(2,3),(3,2),(4,1)} 2( ) 2( )
{ 6} {(1,5),(2,4),(3,3),(4,2),(5,1)} 4( ) ( )
{ 7} {(1,6),(2,5),(3,4),(4,3),(5,2),(6,1
p P T P
p P T P
p P T P
p P T P
2
2 28
2 29
210
11
136
1 136 6
1 136 6
16
16
)} 6( )
{ 8} {(2,6),(3,5),(4,4),(5,3),(6,2)} 4( ) ( )
{ 9} {(3,6),(4,5),(5,4),(6,3)} 2( ) 2( )
{ 10} {(4,6),(5,5),(6,4)} 3( )
{ 11} {(5,6),(6,5)} 2(
p P T P
p P T P
p P T P
p P T P
2) .
Sharif University of Technology 36
Game Of Craps Using Biased Dice
This gives the probability of win on the first throw to be
and the probability of win by throwing a carry-over to be
Thus
Although perfect dice gives rise to an unfavorable game,
T = k 4 5 6 7 8 9 10 11
pk = P{T = k} 0.0706 0.1044 0.1353 0.1661 0.1419 0.1178 0.0936 0.0624
1 ( 7) ( 11) 0.2285P P T P T
210
24 77
0.2717k
k kk
p
p pP
1 2{winning the game} 0.5002P P P
Sharif University of Technology 37
Game Of Craps Using Biased Dice
Even if we let the two dice to have different loading factors and (for the situation described above), similar conclusions do follow.
For example,
gives:
1 2
1 20.01 and 0.005
{winning the game} 0.5015.P
Euler, Ramanujan and Bernoulli Numbers
Sharif University of Technology 39
Euler’s Identity
S. Ramanujan in (J. of Indian Math Soc; V, 1913): if are numbers less than unity where the subscripts are the series of prime numbers,
then
The terms above are arranged so that the product obtained by multiplying the subscripts are the series of all natural numbers
This follows by observing that the natural numbersare formed by multiplying
primes and their powers.
2 3 5 7 11, , , , ,a a a a a 2,3,5,7,11,
2,3,4,5,6,7,8,9, .
2 3 2 2 52 3 5 7
2 3 7 2 2 2 3 3
1 1 1 11
1 1 1 1
.
a a a a aa a a a
a a a a a a a a
Sharif University of Technology 40
Euler’s Identity
This is used to derive a variety of interesting identities including the Euler’s identity.
By letting This gives the Euler identity
The right side can be related to the Bernoulli numbers (for s even). Bernoulli numbers are positive rational numbers defined through the power
series expansion of the even function
Thus if we write Then
2 3 51/ 2 , 1/ 3 , 1/ 5 ,s s sa a a
1
prime 1
1(1 ) 1/ .ss
p np n
2 cot( / 2).x x
1 2 3 4 51 1 1 1 16 30 42 30 66, , , , , .B B B B B
2 4 6
1 2 3cot( / 2) 12 2! 4! 6!x x x x
x B B B
Sharif University of Technology 41
Euler’s Identity
Alse we can obtain :
This is another way to define Bernoulli numbers Further
which gives
Thus
62 431 21
1 2 2! 4! 6!x
B xx x B x B xe
2 1 2 1 2 420 0
2 2 2 2 2
14 ( )
2(2 )! 1 1 1 1(2 ) 1 2 3 4
n n x xxn
n n n n n
xe
B n dx x e e dx
n
2 42 4
1 1
1/ ; 1/ etc.6 90k k
k k
22
21
(2 )1/
2(2 )!
nn n
nk
BS k
n