Chapter 2

Probability

Sections 2.2-2.3

Sample Spaces and the Algebra of Sets

The Probability Function

“The most important questions of life are, for the most part, really only

problems of probability.”

Pierre Simon de Laplace, in his

"Theorie Analytique des Probabilites"

Casino: A House of Probability

What is Probability Theory?

Probability theory is the mathematics of randomness.

We shall be concerned with determining or quantifying the exact or estimated chance that a random event will occur.

Definition

An experiment is any procedure thatCan be repeated, theoretically, an infinite

number of timesHas a well-defined set of possible

outcomes

Examples of Experiments

Rolling a pair of dice Measuring a hypertensive’s blood

pressure Selecting a 3rd year BS ME major

Definition

Each possible result of an experiment is referred to as a sample outcome s. The set of all possible outcomes is called the sample space, and is usually denoted by S.

Examples

Roll a die and observe the number that comes up: S = {1,2,3,4,5,6}

Roll a die repeatedly and count the number of rolls it takes until the first 6 appears: S = {1,2,…}

Turn on a light bulb and measure its lifetime: S = [0,+)

Examples

Flip two coins and observe the sequence of heads and tails: S = {HH, TH, HT, TT}

Choose real numbers a, b, and c such that the equation ax2 + bx + c = 0 has imaginary roots: S = {(a,b,c)| b2 – 4ac < 0}

Example

(2.2.14) A probability-minded despot offers a convicted murderer a final chance to gain his release. The prisoner is given twenty chips, ten white and ten black. All twenty are to be placed into two urns, according to any allocation scheme the prisoner wishes, provided each urn has at least one chip. The executioner will then pick one of the two urns at random and from that urn, one chip at random. If the chip selected is white, the prisoner will be set free, otherwise, he “buys the farm.” Characterize the sample space describing the prisoner’s possible allocation options.

Definition

A subset A of the sample space S of an experiment is called an event.

Example

If we roll a die and observe the number that comes up, two possible events that can be defined are A: the outcome is odd, and B: the outcome is at least 4.

These can be viewed as subsets of the sample space S, with A = {1,3,5} and B = {4,5,6}.

Definitions

Let A and B be any two events defined over the same sample space S. Thena) The intersection of A and B, A B, is the event

whose outcomes belong to both A and B.

b) The union of A and B, A B, is the set of all outcomes in A or B (or both).

c) If A B = , then the events A and B are said to be mutually exclusive.

Remark 1

The notions of union and intersection can be extended to more than two sets. Let A1, A2,…, An be sets. We write

n

ini AAAA

121 ...

n

ini AAAA

121 ...

Remark 2

A useful property for sets is the Distributive Law:

A (B C) = (A B) (A C)

A (B C) = (A B) (A C)

Definition

Let A be any event defined on a sample space S. The complement of A, denoted by AC, is the set of outcomes in S not in A.

Remark 1

By De Morgan’s Law in logic, we can write the complement of a union or intersection of two events more simply as follows:

(A B)C = AC BC

(A B)C = AC BC

Remark 2

We can express unions, intersections, and complements in terms of Venn diagrams:

This can be done even for three or more events.

A BA B AC

Example 1

Let A1, A2,…, Ak be any set of events defined on a sample space S. Determine the outcomes that belong to the event CkCC

k AAAAAA ...... 2121

Example 2

Suppose that the events A1, A2,…, Ak are intervals of real numbers such that

Ai = {x| 0 x < 1/i}, i = 1,2,…,k.

Describe the sets and . What

happens as k ?

k

iiA

1k

iiA

1

Example 3

An internist has 520 patients, of which (1) 230 are hypertensive, (2) 185 are diabetic, (3) 35 are hypochondriac and diabetic, (4) 25 are all three, (5) 150 are none, (6) 140 are only hypertensive, and finally, (7) 15 are hypertensive and hypochondriac but not diabetic. How many of the internist’s patients are hypochondriac, but neither diabetic nor hypertensive?

Definitions of Probability

Classical Probability Empirical Probability Axiomatic Probability

Classical Probability

Suppose that an experiment has n possible outcomes, each outcome being equally likely to occur. If some event A were satisfied by m of these n, then the probability of A is m/n.

Empirical Probability

We can estimate the probability of an event A by repeating the same experiment over and over (say n times), and observing the number of times n(A) the event occurs.

The probability of A is then defined as

.n

Ann

)(lim

Definition (Probability Axioms)

Suppose that to each event A of a sample space S, a number denoted by P(A) is associated with A. If P satisfies the following axioms, then it is called a probability function and the number P(A) is said to be the probability of A. P(A) 0 P(S) = 1 If A1, A2, A3, … is a sequence of pairwise disjoint

events (that is, Ai Aj = for i j), then

11 kk

kk APAP

Some Implications

Let P be a probability function. Thena) P() = 0

b) If A1, A2,…, An is a sequence of pairwise disjoint events, then

n

kk

n

kk APAP

11

Theorem

Let P be a probability function on a sample space S and let A and B be events defined on S. Then

a) P(AC) = 1 – P(A)

b) P(A B) = P(A) + P(B) – P(A B)

c) If A B, then P(A) P(B).

d) P(A) 1.

Example 1

(2.3.1) According to a family-oriented lobbying group, there is too much crude language and violence on television. Forty-two percent of the programs they screened had language they found offensive, 27% were too violent, and 10% were considered excessive in both language and violence. What percentage of programs did comply with the group’s standards?

Example 2

(2.3.11) If State’s football team has a 10% chance of winning Saturday’s game, a 30% chance of winning two weeks from now, and a 65% chance of losing both games, what are their chances of winning exactly once?

Example 3

Let A and B be two events. Show that P(A)+P(B)–1 P(A B) P(A)+P(B).

Exercises

Ex. 2.2, #s 3,6,13 Ex. 2.3, #s 2,12,13,16

Section 2.4

Conditional Probability

Idea of Conditional Probability

Obtaining partial information about the outcome of a random experiment may, in many instances, change the probabilities of events.

A Simple Illustration

Suppose you roll a fair die and observe the number that faces up.

Obviously, the probability that the number that comes up is a 3 is 1/6.

However, if we are given that the result is odd, the above probability becomes 1/3.

A Simple Illustration

Define the ff. events:

T=“The number that comes up is a 3”

O=“The number that comes up is odd” Then P(T|O)=1/3, read as “the probability

of rolling a 3 given that an odd number is rolled.

A Simple Illustration

Note that P(T|O) = n(TO)/n(O) Dividing the numerator and denominator

by n, the number of sample points, gives

P(T|O) = P(TO)/P(O). Generalizing to any two such events leads

to the following definition.

Definition

Let A and B be any two events defined on a sample space S such that P(B)>0. Then the conditional probability of A given that B has occurred is

P(A|B) = P(AB)/P(B).

Example 1

(2.4.2) Find P(AB) if P(A)=0.2, P(B)=0.4, and P(A|B)+P(B|A)=0.75.

Example 2

(2.4.12) A fair coin is tossed three times. What is the probability that at least two heads will occur given that at most two heads have occurred?

Example 3

(2.4.18) Two fair dice are rolled. What is the probability that the sum of the two dice is greater than or equal to eight, given that at least one of the dice is a 5?

Theorem (Generalized Multiplicative Rule)

If in an experiment, the events A1, A2,…,Ak can occur, then

P(A1A2…Ak) = P(A1) P(A2|A1)

P(A3|A1A2) … P(Ak|A1A2…Ak-

1)

Example

Three cards are drawn in succession, without replacement, from an ordinary deck of playing cards. Find the probability that the first card is a red ace, the second card is a ten or jack, and the third card is greater than 3 but less than 7.

Theorem (Law of Total Probability)

Let A be an event with P(A)>0 and P(AC)>0. Then for any event B,

P(B) = P(B|A)P(A) + P(B|AC)P(AC)

Example

An insurance company rents 35% of the cars for its customers from agency I and 65% from agency II. If 8% of the cars of agency I and 5% of the cars of agency II break down during the rental periods, what is the probability that a car rented by this insurance company breaks down?

Definition

Let {A1,A2,…,An} be a set of nonempty subsets of the sample space S of an experiment. If the events A1,A2,…,An

are mutually exclusive and ,

the set {A1,A2,…,An} is called a partition of S.

SAn

ii

1

Theorem (Law of Total Probability)

If {A1,A2,…,An} is a partition of the sample space of an experiment and P(Ai) > 0 for i=1,2,…,n, then for any event B of S,

n

iii APABPBP

1)()|()(

Example

Suppose that 80% of the seniors, 70% of the juniors, 50% of the sophomores, and 30% of the freshmen of a college use the library of their campus frequently. If 30% of all students are freshmen, 25% are sophomores, 25% are juniors, and 20% are seniors, what percent of all students use the library frequently?

Theorem (Bayes’ Theorem)

Let {A1,A2,…,An} be a partition of the sample space S of an experiment. If for for i=1,2,…,n, P(Ai) > 0, then

n

iii

kkk

APABP

APABPBAP

1)()|(

)()|()|(

Example 1

(2.4.46) Brett and Margo have each thought about murdering their rich Uncle Basil in hopes of claiming their inheritance a bit early. Hoping to take advantage of Basil's predilection for immoderate desserts, Brett has put rat poison in the cherries flambe; Margo, unaware of Brett's activities, has laced the chocolate mousse with cyanide. Given the amounts likely to be eaten, the probability of the rat poison being fatal is 0.60; the cyanide, 0.90. Based on the other dinners where Basil was presented with the same dessert options, we can assume that he has a 50% chance of asking for the cherries flambe, a 40% chance of ordering the chocolate mousse, and a 10% chance of skipping dessert altogether. No sooner are the dishes cleared away when Basil drops dead. In the absence of any other evidence, who should be considered the prime suspect?

Example 2

A box contains seven red and 13 blue balls. Two balls are selected at random and are discarded without their colors being seen. If a third ball is drawn randomly and observed to be red, what is the probability that both of the discarded balls were blue?

Example 3

Example 3

(Monty Hall problem) On the game show Let’s Make a Deal, the host, Monty Hall, gives you a choice of three doors. Behind one door is the Grand Prize; behind the others, worthless prizes, called “zonks”. You pick a door, say Door A, and the host, who knows what is behind each door, opens another door (say Door B), revealing a zonk. The host then offers you the opportunity to change your selection to the remaining unopened door (say Door C). Should you stick with your original choice or switch? Does it make any difference?

Remark

This problem was published as a letter to Marilyn vos Savant’s “Ask Marilyn” column in Parade magazine in 1990.

Marilyn's response caused an avalanche of correspondence, mostly from people who would not accept her solution. Eventually, she issued a call to Math teachers among her readers to organize experiments and send her the charts. Some readers with access to computers ran computer simulations, which validated her answer.

Exercises

Ex. 2.4, #s 4, 9, 11, 19, 24, 29, 37, 41, 45, 53

Section 2.5

Independence

Definition

Two events A and B are said to be independent if P(AB) = P(A)P(B).

Example 1

Suppose we roll two dice (one red, one green) and observe the numbers that face up. Define the following events:

A: “The red die shows an even number of spots.”

B: “The number of spots on the two dice have the same parity.”

Are events A and B independent?

Example 2

Suppose that A and B are independent events. Show that AC and BC are also independent. (In fact, A and BC can also be shown to be independent.)

Definition

Events A1, A2,…, An are said to be independent if for every set of indices i1, i2,…, ik between 1 and n, inclusive,

kk iiiiii APAPAPAAAP ......

2121

Example

(2.5.14) In a roll of a pair of fair dice (one red and one green), let A be the event that the red die shows a 3, 4, or 5; let B be the event that the green die shows a 1 or 2; and let C be the event the dice total is 7. Show that A, B, and C are independent.

Probabilities of Intersections

Independence of events implies that the probability of one event is not affected by the probabilities of the other events.

In many cases, independence of events A1, A2,…, An follows immediately from physical considerations. In these cases, the definition provides an easy method of evaluating probabilities of intersections of several events.

Example 1

(2.5.4) Urn I has three red chips, two black chips, and five white chips; urn II has two red, four black, and three white. One chip is drawn at random from each urn. What is the probability that both chips are the same color?

Example 2

(2.5.6) Three points, X1, X2, and X3, are chosen at random in the interval (0,a). A second set of three points, Y1, Y2, and Y3, are chosen at random in the interval (0,b). Let A be the event that X2 is between X1 and X3. Let B be the event that Y1 < Y2 < Y3. Find P(A B).

Example 3

(2.5.22) According to an advertising study, 15% of television viewers who have seen a certain automobile commercial can correctly identify the actor who does the voiceover. Suppose that 10 such people are watching TV and the commercial comes on. (a) What is the probability that at least one of them can name the actor? (b) What is the probability that exactly one can name the actor?

Example 4

(2.5.27) An urn contains w white chips, b black chips, and r red chips. The chips are drawn out at random, one at a time, with replacement. What is the probability that a white appears before a red?

Exercises

Ex. 2.5, #s 2,5,19,24,26 Two one-peso coins, one with P(Head)=p and

one with P(Head)=q, are to be tossed together independently. Define

p0 = P(0 heads occur)p1 = P(1 head occurs)p2 = P(2 heads occur)

Can p and q be chosen such that p0 = p1 = p2? Justify your answer.

Sections 2.6, 2.7

Combinatorics

Combinatorial Probability

Recall: Classical Probability

Suppose that an experiment has n possible outcomes, each outcome being equally likely to occur. If m of these n satisfy an event A, then P(A)=m/n.

Counting Techniques

Fundamental Principle of Counting or Multiplication Rule

Permutations Combinations

Fundamental Principle of Counting If operation Ai, i=1,2,…,k can be

performed in ni ways, i=1,2,…,k, respectively, then the ordered sequence (operation A1, operation A2,…, operation Ak) can be performed in n1n2…nk ways.

Example 1

(2.6.4) Suppose that the format for license plates in a certain state is two letters followed by four numbers.(a) How many different plates can be made?(b) How many different plates are there if the letters can be repeated but no two numbers can be the same?(c) How many different plates can be made if repetitions of numbers and letters are allowed except that no plate can have four zeros?

Example 2

(2.6.9) A restaurant offers a choice of four appetizers, fourteen entrees, six desserts, and five beverages. How many different meals are possible if a diner intends to order only three courses? (Consider the beverage to be a “course.”)

Permutations

An ordered arrangement of r objects from a set A containing n objects (0rn) is called a permutation of the elements of A taken r at a time, and is denoted by nPr.

Theorem

The number of permutations of length r that can be formed from a set of n distinct elements is

where n! = n(n-1)(n-2)…(2)(1). (Note that as a convention, 0! = 1.)

)!(

!

rn

nPrn

Corollary

The number of ways to permute an entire set of n distinct objects is nPn = n!.

Example 1

(2.6.31) The crew of Apollo 17 consisted of a pilot, a copilot, and a geologist. Suppose that NASA had actually trained nine aviators and four geologists as candidates for the flight. How many different crews could they have assembled?

Example 2

(2.6.32) Uncle Harry and Aunt Minnie will both be attending your next family reunion. Unfortunately, they hate each other. Unless they are seated with at least two people between them, they are likely to get into a shouting match. The side of the table at which they will be seated has seven chairs. How many seating arrangements are available for those seven people if a safe distance is to be maintained between your aunt and your uncle?

Example 3 (Birthday Problem)

Suppose that k people are selected at random from the general population. What are the chances that at least two of those k were born on the same day?

Remark:

The following are the values of the probability for k=15, 23, 30, 40, 60: k P(at least two have same birthday) 15 0.253 23 0.507 30 0.706 40 0.891 60 0.995

Permutations of Non-distinct Objects The number of ways to arrange n

objects, n1 being of one kind, n2 of a second kind,…, nr of an rth kind, is

where .

!!...!

!

21 rnnn

n

r

ii nn

1

Example 1

(2.6.36) An interior decorator is trying to arrange a shelf containing eight books, three with red covers, three with blue covers, and two with brown covers. Assuming the titles and sizes of the books are irrelevant, in how many ways can she arrange the eight books?

Example 2

A delivery truck has to go from point X to point Y and make a stop at point O. How many different routes are possible, assuming the driver never wants to go out of her way?

Example 3

(2.7.12) If the letters in the phrase

A ROLLING STONE GATHERS NO MOSS

are arranged at random, what are the chances that not all the S’s will be adjacent?

Example 4

(2.6.46) Show that (k!)! is divisible by (k!)(k-1)!.

Exercises

Ex. 2.6, #s 6, 8, 9, 17, 23, 26, 29 Ex. 2.7, #s 10, 11, 14

Combinations

A selection of r objects from a set A containing n objects (0rn) without regard to order is called a combination of the elements of A taken r at a time, and is denoted by or nCr.

r

n

Theorem

The number of ways to form combinations of size r from a set of n distinct objects, where repetitions are not allowed, is given by

)!(!

!

rnr

n

r

nCrn

Example 1

In how many ways can two math and three biology books be selected from eight math and six biology books?

Example 2

A poker hand consists of 5 cards from a standard deck of 52. Find the number of different poker hands that are full houses. (Note: A full house consists of three cards of one denomination and two of another.)

Example 3

(2.7.4) A bridge hand (13 cards) is dealt from a standard 52-card deck. Let A be the event that the hand contains four aces; let B be the event that the hand contains four kings. Find P(A B).

Example 4

(2.6.58) Prove that

.n

n

nnn2...

10

Remark

The previous example implies that the number of subsets of a set with n (distinct) elements is 2n.

Example 5

(Vandermonde’s Identity) Use a combinatorial argument to show that for all positive integers m, n, and r,

r

i r

nm

ir

n

i

m

0

Exercises

Ex. 2.6, #s 37, 42, 51, 53, 56, 59