Discrete Mathematics 6th edition, 2005
Chapter 6
Counting methods and the pigeonhole principle 1. Basic Principles 2. Permutations and Combinations 3. Algorithms for Generating Permutations and Combinations 4. Introduction to Discrete Probability* 5. Discrete Probability Theory* 6. Generalized Permutations and Combinations 7. Binomial Coefficient and Combinatorial Identities
2
6.1 Basic Principles
Menu for Quick Lunch 2 appetizers, 3 main courses,
4 beverages
24 possible dinners consisting of
1 appetizer, 1 main course and
1 beverage
NHT, NHM, NHC, NHR,
NCT, NCM, NCC, NCR,
NFT, NFM, NFC, NFR,
SHT, SHM, SHC, SHR,
SCT, SCM, SCC, SCR,
SFT, SNM, SFC, SFR
APPETIZERS
Nachos …………… 2.15 Salad ……………… 1.90 MAIN COURSES
Hamburger ……. 3.25 Cheeseburger... 3.65 Fish Filet ……….. 3.15 BEVERAGES
Tea ………………… .70 Milk ……………….. .85 Cola ………………. .75 Root Beer ……… .75
3
Multiplication Principle
Multiplication principle
If an activity can be performed in k successive
steps,
Step 1 can be done in n1 ways
Step 2 can be done in n2 ways
…
Step k can be done in nk ways
Then: the number of different ways that
the activity can be performed is the product
n1n2…nk
4
Addition Principle
How many 8-bit strings that begin either 101
or 111?
8-bit strings that begin 101
can be constructed in five successive steps
select the 4-th bit; select the 5-th bit;…
2 2 2 2 2 = 25 = 32 8-bit strings
8-bit strings that begin 111
can be constructed in five successive steps
select the 4-th bit; select the 5-th bit;…
2 2 2 2 2 = 25 = 32 8-bit strings
8-bit strings that begin either 101 or 111
32 + 32 = 64 8-bit strings
5
Addition Principle
Addition principle
Let X1, X2,…, Xk be a collection of k pairwise
disjoint sets, each of which has nj elements,
1 < j < k, then the union of those sets
k
X = Xj j =1
has n1 + n2 + … + nk elements
6
6.2 Permutations and Combinations
A permutation of n distinct elements x1, x2,…, xn
is an ordering of the n elements.
Example: there are 3! = 6 permutations of three
elements a, b, c:
abc bac cab
acb bca cba
Theorem 6.2.3
There are n! permutations of n elements
7
Theorem 6.2.10:
The number of r-permutations of a set of
n distinct objects is
P(n,r) = n(n-1)(n-2)…(n-r+1), rn
r-permutations
An r-permutation of n distinct elements is
an ordering of an r-element subset of
{x1, x2,…, xn}
Example: The number of 2-permutations of X={a,b,c} is 32 = 6
ab, ac, ba, bc, ca, cb
8
Combinations
Let X = {x1, x2,…, xn} be a set containing n distinct
elements
An r-combination of X is an unordered selection of r elements of X, for r < n
The number of r-combinations of n distinct element is denoted
C(n,r) or ( ) n r
Theorem 6.2.17:
The number of r-combinations of a set of n distinct
objects is
C(n,r) = = = , rn
P(n,r) n(n-1)(n-2)…(n-r+1) n! r! r! (n-r)!r!
9
6.6 Generalized Permutations and Combinations
Example
Problem
How many strings can be formed using the following letters?
M I S S I S S I P P I
Solution
11! ? no. why? duplication of letters!
Fill 11 blanks with the letters given . There are C(11,2) ways to choose positions for the two P’s.
There are C(9,4) ways to choose positions for the four S’s.
There are C(5,4) ways to choose positions for the four I’s.
There are 1 position to be filled by M.
By the multiplication principle, the number of ways of ordering the letters
C(11,2) C(9,4) C(5,4) = = = 34,650 11! 9! 5! 11!
2!9! 4!5! 4!1! 2!4!4!1!
2 Ps, 4 Ss, 4 Is and 1 M
10
Generalized Permutations and Combinations
Theorem 6.6.2:
Suppose that a sequence S of n items has n1 identical objects of type 1,
n2 identical objects of type 2, … and
nt identical objects of type t.
Then the number of orderings of S n!
n1!n2!...nt!
C(n, n1)C(n-n1, n2)C(n-n1-n2, n3) … C(n-n1-
…-nt-1, nt)
= …
=
n! (n-n1)! (n-n1- … -nt-1)!
n1!(n-n1)! n2!(n-n1-n2)! nt0!
n!
n1!n2!...nt!
Proof
11
Generalized Permutations and Combinations
Example
Problem Consider 3 books
a computer science book, a physics book, and a history book.
Suppose that the library has at least six copies of each of these books.
How many ways can we select six books?
Solution The problem is to choose unordered, repetitions allowed.
We use two bookends to separate 3 kinds of books.
By the bookends’ positions, the selection is determined. There are 8 positions: 6 books’ positions + 2 bookends’ positions
The possible number of bookends’ positions : C(8,2)
3 2 1 0 4 2 3 0 3
12
Generalized Permutations and Combinations
Theorem 6.6.5:
If X is a set containing t elements,
the number of unordered,
k-element selections from X,
repetitions allowed, is
C(k+t-1, t-1) = C(k+t-1, k)
Let X = {a1,…,at}
Consider k+t-1 slots and k+t-1 symbols (k ’s and (t-1) /’s)
A selection: each placement of these symbols into the slots
The number n1 of ’s up to the 1st /: the selection of n1 a1’s.
The number n2 of ’s between the 1st and 2nd /’s: the selection of n2 a2’s.
And so on.
There are C(k+t-1, t-1) way to select the positions for the /’s and
= The number of way to select the position for the ’s: C(k+t-1, k)
Proof
13
6.3 Algorithms for Generating Permutations and Combinations
Lexicographic order
Given two strings = s1s2…sp and = t1t2…tq
over {1,2,…,n}
Define < if
p < q and si = ti for all i = 1, 2,…, p
Or for some i, si ti and for the smallest i, si < ti
Example: if = 1324, = 1332, = 132,
then < and < .
14
Algorithm (1)
R-combination from set {x1, … ,xn}
String s1 … sr where s1 < s2 < s2 < s3 …
5 combination of { 1 2 3 4 5 6}
12345 ~ 23456
Find next r-combination, t, given a number, s
Find the right most element sm that is not at its max
ti = si for i = 1, .. , m -1
tm = sm + 1
tm+1 = sm + 2, … until tr
15
Pseudo code
Combination (r, n) { for i=1 to r, si = i; print(s1, …, sr); for i=2 to C(n,r) { m = r max_val = n while (sm == max_val) { m = m – 1; max_val = max_val – 1} sm = sm + 1; for j = m+1 to r sj = sj-1 + 1 print (s1, …, sr); } /* for loop */ }
1234 (first)
i = 2 to C(n,r) %exclude first one
m = 4, max_val = 6
skip while 1235
skip while 1236
sm = 6 (max)
while m = 3, max = 5
1245
m = 4 max = 6 again
skip while 1246
m = 4 max = 6
while m = 3, max = 5
1256
while m = 2, max = 4
1345
m = 4 max = 6 again
skip while
1346
…
16
Algorithm (2)
E.g. Permutation of {1 2 3 4 5 6} that follows 163542 Can it be 1635 _ _ ? No (only “4” and “2” left, “163524” is less)
Can it be 163 _ _ _ ? No (“4” “2” “5” left)
Remaining digits, 42 or 542, largest permutation
Must find the first digit d whose right neighbor r that satisfies d < r
d is the third digit 3 < 5
Digit following 16 must exceed 3 that is 4
164 _ _ _ 164 “2 3 5”
1234, 1243, 1324, 1342, 1423, … , 4231, 4312, 4321
17
function perm (n, S) % print first perm disp(S); for (i = 2:factorial(n)), disp('=== Iteration:'); m = n - 1; while (S(m) > S(m+1)) m = m -1; end; disp('index of right most digit that is less than its right'); disp(m); k = n; while (S(m) > S(k)) k = k - 1; end; disp('index of right most digit that is greater than pivot'); disp(k);
18
% swap sm and sk
temp = S(k);
S(k) = S(m);
S(m) = temp;
p = m+1;
q = n;
while (p<q)
temp = S(q);
S(q) = S(p);
S(p) = temp;
p = p + 1;
q = q - 1;
end;
disp(S);
end; % big for loop
19
S = [5 6 7 8], n = 4
5 6 7 8 === Iteration: index of right most digit that is less than its right 3 index of right most digit that is greater than pivot 4 Swapping sm and sk 7 8 5 6 8 7
Who should change?
With who? Why is it this one?
20
5 6 8 7 === Iteration: index of right most digit that is less than its right: 2 index of right most digit that is greater than pivot: 4 Swapping sm and sk: 6 and 7 5 7 8 6 Sorting the rest 5 7 6 8 === Iteration: index of right most digit that is less than its right: 3 index of right most digit that is greater than pivot: 4 Swapping sm and sk: 6 and 8 5 7 8 6
7 is smallest that is greater than 6!
Until before 6, it is sorted from big to small
So once we make the switch start from beginning
And sort back to small to big
21
5 7 8 6 === Iteration: index of right most digit that is less than its right: 2 index of right most digit that is greater than pivot: 3 Swapping sm and sk: 7 and 8 5 8 7 6 Sorting the rest 5 8 6 7 === Iteration: index of right most digit that is less than its right: 3 index of right most digit that is greater than pivot: 4 Swapping sm and sk: 6 and 7 5 8 7 6
22
6.4 Introduction to Discrete Probability
An experiment is a process that yields an
outcome
An event is an outcome or a set of outcomes
from an experiment
The sample space is the event of all possible
outcomes
23
Probability
Probability of an event is the number of outcomes in the event divided by the number of outcomes in the sample space.
If S is a finite sample space and E is an event (E is a subset of S) then the probability of E is
P(E) = |E| / |S|
Example 2 fair dice are rolled. What is the probability that
the sum of the numbers on the dice is 10?
There are 3 ways: (4,6), (5,5), (6,4)
The size of the event is 3
The probability is 3/36 = 1/12
24
6.5 Discrete Probability Theory
When all outcomes are equally likely and
there are n possible outcomes, each one
has a probability 1/n.
BUT this is not always the case. When
all probabilities are not equal, then some
probability (possibly different numbers)
must be assigned to each outcome.
25
Probability Function
A probability function P assigns to each outcome x
in a sample space S a number P(x)
so that 0P(x)1, for all xS, and
P(x) = 1 x S
Example 2 though 6 of die are equally likely appear
P(2) = P(3) = P(4) = P(5) = P(6)
1 is 3 times as likely as any other number to appear
P(1) = 3P(2)
1= P(1) + P(2) + P(3) + P(4) + P(5) + P(6)
= 3P(2) + P(2) + P(2) + P(2) + P(2) + P(2) = 8P(2)
P(2) = P(3) = P(4) = P(5) = P(6) = 1/8
P(1) = 3/8
26
Probability of an Event
The probability of an event E is defined as the
sum of he probabilities of the outcomes in E
P(E) = P(x) x E
Example probability of an odd number is
P(1)+P(3)+P(5) = 3/8+1/8+1/8 = 5/8
27
Probability of an Event
Suppose that E={x1, …, xk} and E={xk+1, …, xn}
Then
k n
P(E) = P(xi) and P(E) = P(xi) i=1 i=k+1
Now
k n P(E) + P(E) = P(xi) + P(xi)
i=1 i=k+1
n
= P(xi) = 1 i=1
Theorem 6.5.5
Let E be an event. The probability of E, the complement of E, satisfies
P(E) + P(E) = 1
Proof
28
Probability of an Event
Example: Birthday Problem
Find the probability that among n persons, at least two people have birthdays on the same month and date. Assume that all month and dates are equally likely, and
ignore February 29 birthdays.
E: the event “at least two persons have the same birthday”
E: the event “no two persons have the same birthday”
the size of the sample space: 365n
|E| = 365 364 … (365-n+1)
P(E) = [ 365 364 … (365-n+1) ] / 365n
P(E) = 1 - P(E)
n=22 P(E) = 0.475695, n=23 P(E) = 0.507297 > ½
29
Probability of Two Events
Theorem 6.5.9
Given any two events E1 and E2 in
a sample space S. Then
P(E1E2) = P(E1) + P(E2) – P(E1E2)
Let E1 = {x1, …, xi}
E2 = {y1, …, yj}
E1 E2 = {z1, …, zk}
Then in the list
x1, …, xi, y1, …, yj,
z1, …, zk occurs twice
Proof It follows that i j k P(E1E2) = P(xt) + P(yt) - P(zt)
t=1 t=1 t=1
= P(E1) + P(E2)
– P(E1E2)
30
Probability of Two Events
Example
Among a group of students, some take art and some take computer science.
A student is selected at random.
Let A be the event “the student takes art,” and
let C be the event “the student takes computer
science.”
Then AC is the event “the student takes art
or computer science or both,” and
AC is the event “the student takes art and
computer science.”
31
Probability of Two Events
Example
Two fair dice are rolled.
What is the probability of getting doubles or a sum of 6?
E1: get doubles
P(E1) = 6/36 = 1/6
E2: get a sum of 6
P(E2) = 5/36
E1 E2: get doubles and get a sum of 6
P(E1 E2) = 1/36
P(E1 E2) = 1/6 + 5/36 - 1/36 = 5/18
[(1,1), (2,2), (3,3), (4,4), (5,5), (6,6)]
[(1,5), (2,4), (3,3), (4,2), (5,1)]
32
Mutually Exclusive Events
Events E1 and E2 are mutually exclusive
if and only if E1E2 = .
Example
E1: get doubles
P(E1) = 6/36 = 1/6
E2: get a sum of 5
P(E2) = 4/36 = 1/9
E1 E2: get doubles and get a sum of 5
P(E1 E2) = 1/6 + 1/9 = 5/18
Theorem 6.5.11
If E1 and E2 are mutually exclusive events,
P(E1 E2) = P(E1) + P(E2)
[(1,1), (2,2), (3,3), (4,4), (5,5), (6,6)]
[(1,4), (2,3), (3,2), (4,1)]
33
Conditional Probability
Conditional probability is the probability of an event E,
given that another event F has occurred.
In symbols P(E|F).
If P(F) > 0 then
P(E|F) = P(EF) / P(F)
Example Weather records show that the probability of high barometric
pressure is 0.80, and
the probability or rain and high barometric pressure is 0.10.
The probability of rain given high barometric pressure is
P(R|H) = P(RH) / P(H) = 0.10 / 0.80 = 0.125
R: the event “rain”
H: the event “high barometric pressure”
34
Independent Events
If the probability of event E does not depend
on event F
P(E|F) = P(E)
E and F are independent events
P(E) = P(E|F) = P(EF) / P(F)
P(EF) = P(E)P(F)
Two events E and F are independent if
P(EF) = P(E)P(F)
35
Independent Events
Example
Intuitively, if we flip a fair coin twice, the outcome of
the second toss does not depend on the outcome of
the first toss
H: the event “head on first toss”
T: the event “tail on second toss”
HT: the event “head on first toss and
tail on second toss”
P(HT) = P(H)P(T) = ½ ½ = ¼
36
Pattern Recognition
Pattern recognition places items into classes,
based on various features of the items.
Given a set of features F we can calculate the
probability of a class C, given F: P(C|F)
Place the item into the most probable class, i.e.
the one C for which P(C|F) is the highest. Example: Wine can be classified as Premium (R), Table wine (T)
or Swill (S). Let F {acidity, body, color, price}
Suppose a wine has feature F, and P(T|F) = 0.5, P(R|F) = 0.2
and P(S|F) = 0.3. Since P(T|F) is the highest number, this wine
will be classified as table wine.
37
Bayes’ Theorem
Theorem 6.5.20
Given pairwise disjoint classes C1, C2,…, Cn and
a feature set F, then
P(Cj|F) = P(F|Cj)P(Cj)
n P(F|Ci)P(Ci)
i=1
P(Cj|F) = P(CjF)/P(F)
P(F|Cj) = P(FCj)/P(Cj)
P(Cj|F) = P(CjF)/P(F)
= P(F |Cj)P(Cj)/P(F)
Proof F = (FC1)(FC2)… (FCn)
Ci are pairwise mutually exclusive
(FCi) are pairwise mutually exclusive
P(F) = P(FC1)P(FC2)… P(FCn)
P(FCi) = P(F|Ci)P(Ci) n P(F) = P(F|Ci)P(Ci)
i=1
38
Example - Bayes’ Theorem (1/4)
The ELISA test is used to detect
antibodies in blood and indicates
the presence of the HIV virus.
15% of the patients at on clinic have
the HIV virus.
Among those that have the HIV virus,
95% test positive on the ELISA test.
Among those that do not have the
HIV virus, 2% test positive on the
ELISA test.
Problem: Find the probability that
a patient has the HIV virus if the
ELISA test is positive.
P(H)=0.15
P(Pos|H)=0.95 P(Pos|H)=0.02
P(H)=0.85
39
Example - Bayes’ Theorem (2/4)
Problem: Find the probability that a patient has the HIV
virus if the ELISA test is positive.
P(H)=0.15
P(Pos|H)=0.95 P(Pos|H)=0.02
P(H)=0.85
A
B
P(H)=0.15
P(A)=0.15*0.95 P(B)=0.85*0.02
P(H)=0.85
A
B
40
Example - Bayes’ Theorem (3/4)
Problem: Find the probability that a patient has the HIV
virus if the ELISA test is positive.
P(A) / P(A B)
P(A)=0.1425 P(B)=0.017
A
B
P(A | (A B)) = 0.1425 / (0.1425+0.017)
= 0.893
A
B
41
Example - Bayes’ Theorem (4/4)
The ELISA test
is used to detect antibodies in blood and indicate the presence of the HIV virus.
15% of the patients at on clinic have the HIV virus.
Among those that have the HIV virus, 95% test positive on the ELISA test.
Among those that do not have the HIV virus, 2% test positive on the ELISA test.
Problem
Find the probability that a patient has the HIV virus if the ELISA test is positive.
Solution
H: the classes that are “has the HIV virus”
H: the classes that are “does not have the HIV virus”
Pos: the feature that is “test positive”
P(H)=0.15, P(H)=0.85, P(Pos|H)=0.95, P(Pos|H)=0.02
P(H|Pos) = { P(Pos|H) P(H) } / {P(Pos|H) P(H) + {P(Pos|H) P(H) }
= (0.95)(0.15) / {(0.95)(0.15) + (0.12)(0.85)} = 0.893
Bayes’ Theorem
42
Example: Telemarketing (1/4)
At the telemarketing firm SellPhone, Dale, Rusty, and Lee make calls.
What are P(D|H), P(R|H), P(L|H)?
Caller
Dale Rusty Lee
Percent of Calls 40 25 35
Percent of hang-ups 20 55 30
43
Example: Telemarketing (2/4)
P(D) = 0.4
P(R) = 0.25
P(L) = 0.35
P(H|D) = 0.2 P(H|R) = 0.55
P(H|L) = 0.30
44
P(D) = 0.4
P(R) = 0.25
P(L) = 0.35
Example: Telemarketing (3/4)
P(H|D)P(D)
= 0.4*0.2
= 0.08
P(H|R)P(R)
= 0.25*0.55
= 0.1375
P(H|L)P(L)
= 0.35*0.30
= 0.105
P(H)
= P(H|D)P(D)+
P(H|R)P(R)+
P(H|L)P(L)
= 0.08 +
0.1375+0.105
= 0.3225
45
Example: Telemarketing (4/4)
P(R|H) = = 0.426
P(L|H) = = 0.326
P(D|H) = P(H|D)P(D)
P(H)
P(R|H) = P(H|R)P(R)
P(H)
P(L|H) = P(H|L)P(L)
P(H)
0.08
0.3225 0.1375
0.3225
0.105
0.3225
P(D|H) + P(R|H) + P(L|H) = 1
P(D|H) = = 0.248
46
6.8 The Pigeonhole Principle
First form: If n pigeons fly into k pigeonholes and
k < n, some pigeonhole contains at least two
pigeons.
1 2 3
4 5 6
7 8 9
pigeonhole
1 2 3
4 5 6
7 8 9
1 2
3 4
5 6 7
8 9 10
11
12
47
Combinatorial Identities and Combinatorial Argument
Combinatorial Identity
An identity that results from some counting process
Combinatorial Argument
The argument that leads to its formulation
48
Second Form of the Pigeonhole Principle
If X and Y are finite sets with |X| > |Y| and
f : X Y is a function,
then f(x1) = f(x2) for some x1, x2 X, x1 x2.
a
b
c
d
e
f
1
2
3
4
5
X Y
49
Third Form of the Pigeonhole Principle
Let f be a function from a finite set X into a finite set Y.
Suppose |X| = n, |Y| = m.
Let k = n/m.
Then there are at least k values a1, a2,…, ak X
such that f(a1) = f(a2) = … f(ak).
Example:
n = 5, m = 3
k = n/m = 5/3 = 2.
a b c d e
1
2
3
X Y
50
Example 6.8.1
Ten persons have
first names Alice, Bernard, and Charles and
last names Lee, McDuff, and Ng.
Show that at least two persons have the same first and last
names.
There are nine possible names.
By the Pigeonhole principle, at least two persons have
the same last and first names.
Proof
51
Example 6.8.3
Show that if we select 151 distinct computer science courses numbered between 1 and 300 inclusive, at least two are consecutively numbered.
Let the selected course numbers be
C1, C2 , C3 , C4 , ... , C151
The 302 numbers consisting of the above together with
C1+1, C2 +1, , ... , C151+1
range in value between 1 and 301. By the second form of the
Pigeonhole Principle, at least two of these values coincide. It must
be that one of and one of are equal. Thus,
Ci = Cj +1, and course Ci follows course Cj
Proof
1
2
1 2