Introduction to Probability and Counting

SJSU

August 15, 2016

Probability- Chapter 3

Population and Sample

Statistics: a branch of mathematics dealing with the collection,analysis, interpretation, and presentation of masses of numerical data

Probability: the relative frequency with which an event occurs or islikely to occurProbability allows us to assess the uncertainty in experimental resultsor random events

Population - overall group of objects about which conclusions are tobe drawn

Sample - a subset of the population that is actually obtained

Example: population - every student at SJSU, sample - a subset of 100students

2 / 1

Probability- Chapter 3

Probability- 3.2

Probabilities are numbers between 0 and 1 which is the chance of aphysical event occurring

Probability is denoted as P(A) where A is an event

Percentages are probabilities multiplied by 100

Example: The probability that a fair coin will yield heads is 0.50 or there isa 50% chance of heads.

3 / 1

Sample Spaces and Events

Sample Spaces and Points

sample space: a sample space for an experiment is a set S with theproperty that each physical outcome of the experiment corresponds toexactly one element of S.sample point: an element of Sevent: any subset A of a sample space is called an event. The emptyset ⊘ is called the impossible event (null or empty set); the subset Sis called the certain event.

Example: During a space flight there are three computers (two for backup)that operate independently. We want to discuss the operability of thecomputer at launch time. Computer 1: yes or no, computer 2: yes or no,computer 3: yes or no.Sample space: S = {yyy , yyn, yny , yyn, nny , nyn, ynn, nnn}, “yny” is asample point. Event A: computer 1’s yes A = {yyy , yyn, yny , ynn}, EventB: computer 2’s yes B = {yyy , yyn, nyn, nyy}, Event C: computer 3’s yesC = {yyy , nyy , nny , yny}.

4 / 1

Sample Spaces and Events

Union and Intersection and Complement- 3.2 and 3.4

union: A ∪ B (A or B)

intersection: A ∩ B (A and B)

compliment: C’ (not)

5 / 1

Sample Spaces and Events

Union and Intersection and Complement

Example:Event A: computer 1’s yes A = {yyy , yyn, yny , ynn}Event B: computer 2’s yes B = {yyy , yyn, nyn, nyy}Event C: computer 3’s yes C = {yyy , nyy , nny , yny}

A ∪ B computer 1 or 2 are operable

A ∩ B computer 1 and 2 are operable

(A ∪ B) ∩ C ′ computer 1 or 2 are operable but computer 3 isinoperable

6 / 1

Sample Spaces and Events

Mutually Exclusive

mutually exclusive (disjoint): two events A1 and A2 are mutuallyexclusive if and only if A1 ∩ A2 = ⊘ or P(A1 ∩ A2) = 0. EventsA1,A2,A3, . . . are mutually exclusive if and only if Ai ∩ Aj = ⊘ fori 6= j .

Example: Mutually exclusive is defined for events (which have probabilitiesassociated with them) not for components. The computers cannot bedefined as mutually exclusive. Event A (computer 1 is operable) can bemutually exclusive of the event A’ (computer 1 is inoperable). Both A andA’ cannot happen at once. A={yyy , yyn, yny , ynn} andA’={nnn, nyn, nny , nyy} are mutually exclusive, they do not share samplepoints. A ∩ A′ = ⊘.

Event A: computer 1’s yes A = {yyy , yyn, yny , ynn}Event B: computer 2’s yes B = {yyy , yyn, nyn, nyy}Question: Are events A and B mutually exclusive?

7 / 1

Sample Spaces and Events

Trees

Example: Computer 1 is inoperable 51% of the time, computer 2 isinoperable 30% and computer 3 is inoperable 10%.

What is the probability that all three computers are operable?

What is the probability that computer 1 is operable but computers 2 and 3are inoperable?

8 / 1

Permutations and Combinations - 3.5

Multiplication Principle

Multiplication Principle: Suppose a procedure can be broken into msuccessive (ordered stages, with r1 outcomes in the first stage, r2 in thesecond stage,..., and rm in the mth stage. If the number of outcomes ateach stage is independent of the choices in previous stages and if thecomposite outcomes are all distinct, then the total procedure hasr1 × r2 × ...× rm different composite outcomes.

1. DNA is formed from four nucleotides (A,T,G,C). How many differentchains of 6 nucleotides is possible? A-T-T-G-C-A, A-T-T-G-C-C ...

2. How many three letter “words” can be created (26 letters in thisalphabet)?

9 / 1

Permutations and Combinations - 3.5

Factorial!

Factorial n!

n is a positive integer

n! = n(n − 1)(n − 2)...3 · 2 · 1 (n factorial)

By convention the empty product: 0! = 1

Examples:

7! = 7 · 6 · 5 · 4 · 3 · 2 · 1 = 5040

7!5! =

7·6·5·4·3·2·15·4·3·2·1 = 7 · 6 = 42

10 / 1

Permutations and Combinations - 3.5

Permutation

Permutation - is an arrangement of objects in a definite order:nPr =

n!(n−r)!

ex. twenty amino acids commonly form peptides. Five amino acidsA-V-G-C-T ordering gives a different pentapeptide than A-G-V-C-T.

Permutation - is an arrangement of objects in a definite order

Given 5 amino acids how many pentapeptides can be made?There is no repetition of amino acids in this pentapeptide.

5 4 3 2 1

5! = 120

Extra: Given 20 amino acids how many pentapeptides can be made?20*19*18*17*16= 1,860,480

11 / 1

Permutations and Combinations - 3.5

Combination

Combination - is a selection of objects without regard to order: nCr

ex. There are 20 components, 3 are selected at random to be tested forquality.

We only care about which three are selected and not their order. Ifcomponents (1,6,10) are selected, it is the same as components(6,1,10) being selected.

20C3 =

(

20

3

)

12 / 1

Permutations and Combinations - 3.5

Multiplication Principle

1. Two dice are rolled, one green and one red(a) How many different outcomes of this procedure are there?

(b) What is the probability that there are different values on the two dice(no doubles)?

13 / 1

Permutations and Combinations - 3.5

Permutation

How many ways can we arrange the seven letters in the word SYSTEMS?

14 / 1

Permutations and Combinations - 3.5

Combination

A committee of k people is to be chosen from a set of 7 women and 4 men(a) The committee has 5 people: 3 women and 2 men

(b) The committee can be any positive size but must have equal men andwomen

(c) The committee has 4 people and one of them must be Mr. Baggins

(d) The committee has 4 people and at least 2 are women

15 / 1

Permutations and Combinations - 3.5

Overview

sample space: a sample space for an experiment is a set S with theproperty that each physical outcome of the experiment corresponds toexactly one element of S.

sample point: an element of S

event: any subset A of a sample space is called an event. The emptyset ⊘ is called the impossible event (null or empty set); the subset Sis called the certain event.

mutually exclusive (disjoint): two events A1 and A2 are mutuallyexclusive if and only if A1 ∩ A2 = ⊘ or P(A1 ∩ A2) = 0. EventsA1,A2,A3, . . . are mutually exclusive if and only if Ai ∩ Aj = ⊘ fori 6= j .

union: A ∪ B (A or B)

intersection: A ∩ B (A and B)

compliment: C’ (not)

16 / 1

Axioms of Probability

Axioms of Probability

Let S denote a sample space for an experiment: P(S)=1

Let P(A) ≥ 0 for every event A

Let A1,A2,A3, . . . be a finite or an infinite collection of mutuallyexclusive events. ThenP(A1 ∪ A2 ∪ A3 . . .) = P(A1) + P(A2) + P(A3) + . . ..

Example: There are 4 blood types (A,B,AB,O) with percentages(41%,9%,4%,46%). What is the probability that a person will have typeA, B, or AB blood?

17 / 1

Axioms of Probability

Theorems

P(⊘) = 0 the probability assigned to the empty set is zero.

P(A′) = 1− P(A) the probability an event will not occur A’ is equalto one minus that it will occur

Addition rule: P(A ∪ B) = P(A) + P(B)− P(A ∩ B)

if events A and B are mutually exclusive then: P(A ∩ B) = 0

Example: Lead and mercury can be found in water; near industrial plants32% of streams have toxic levels of lead and 16% have toxic levels ofmercury. 38% of streams have toxic levels of lead or mercury. What is theprobability that a randomly selected sample will contain toxic levels of leadonly?

18 / 1

Conditional Probability

Conditional Probability

Let A and B be events such that P(A) 6= 0. The conditionalprobability of B given A, denoted by P(B |A), is defined by:

P(B |A) = P(A∩B)P(A)

Example: 49% of all infections involve anaerobic bacteria. 70% of allanaerobic infections are polymicrobic. What is the probability that agiven infection involves anaerobic bacteria and is polymicrobic?

Multiplication rule: P(A ∩ B) = P(B |A)P(A)Example: Suppose we toss a die once. Find the probability of a 1,given that an odd number was obtained.

19 / 1

Independence - 3.6

Independence - 3.6

Independent - informally - events/objects acting irrespective of each other.

Independent events: events A and B are independent if and only ifP(A ∩ B) = P(A)P(B)Independent events: events A,B,C. . . are independent if and only ifP(A ∩ B ∩ C ∩ . . .) = P(A)P(B)P(C ) . . .Let A and B be events such that at least one of P(A) or P(B) isnon-zero. A and B are independent if and only if P(B |A) = P(B) ifP(A) 6= 0 and P(A|B) = P(A) if P(B) 6= 0

Example: Draw a card from a deck. Event A: spade is drawn and Event B:a 10,J,Q,K,A is drawn. (If I knew the card was a spade (event A) wouldthat give me any information about event B?) and vice-versa

Example: Two children have 3/4 chance of having brown eyes. Are theseevents independent? Yes

20 / 1

Bayes’ Rule - 3.8

Bayes’ Theorem

Let A1,A2,A3, . . . ,An be a collection of mutually exclusive events wholeunion is S. Let B be an event such that P(B) 6= 0. Then for any of theevents Aj , j = 1, 2, 3, . . . , n,

P(Aj |B) =P(B |Aj)P(Aj)

∑ni=1 P(B |Ai )P(Ai )

We usually use the simpler case when there are only two events A and A’:

P(A|B) =P(B ∩ A)

P(B)=

P(B |A)P(A)

P(B |A)P(A) + P(B |A′)P(A′)

21 / 1

Bayes’ Rule - 3.8

Bayes Theorem

Example: 40% of highway accidents involve speeding and 30% involvealcohol. If alcohol is involved there is a 60% chance that there is alsospeeding, otherwise this probability is only 10%. An accident involvesspeeding, what is the probability that alcohol is involved?

22 / 1

Bayes’ Rule - 3.8

Bayes Theorem

Example: The original blood test for HIV was called ELISA; according to astudy (Weiss et al. 1985) the conditional probability that a person wouldtest positive given they have HIV was 0.977 and the conditional probabilitythat a person would test positive given they did not have HIV was 0.926.The World Almanac at the time cited that the probability of having HIV inNorth America was 0.0026. Suppose a random person is tested and theytest positive; what is the conditional probability they have HIV?

23 / 1

Bayes’ Rule - 3.8

Bayes Theorem

Do the results surprise you? 3.3% chance of having HIV given a positivetest result.Let’s go a bit further:

Out of 10000 people we would expect 26 to have HIV.

Of the 26 people with HIV, 26*.977 =25.4 would get a positive test

Of those 9974 without HIV, 9974*.074= 738 would get a positive test

Note that this phenomenon of large unexpected changes in conditionalprobabilities is not unusual when dealing with rare events/diseases. Whathappens is the number of false positives is much larger than the number oftrue positives. Public policy: how does this effect mandatory testing of thepopulation? How about just testing at risk groups?

24 / 1

Extra Problems

Multiplication Principle

How many ways are there to form a three letter sequence using the lettersa, b,c,d,e,f:(a) with repetition of letters allowed?Answer: we have six choices for each letter in the sequence, there are6× 6× 6 = 216 three-letter sequences(b) without repetition of any letter?Answer: six choices for the first letter, five choices for the second letter,four choices for the third letter: 6× 5× 4 = 120 three-letter sequences.(c) without repetition and containing the letter e?Answer: e . . . e . . . eIn each case, there are 5 choices for which of the other 5 letters (excludinge) goes in the first remaining position and 4 choices for which of theremaining 4 letters goes in the other positions. Thus there are3× 4× 5 = 60 three-letter sequences containing an e.(d) with repetition and contain the e?Answer: All possible ways (63 = 216) minus and sequences with no e’s(53 = 125), so 216-125=91 25 / 1

Extra Problems

Multiplication rule

Assume there is a 50% chance of hard drive damage if a powerline is hit inan electric storm. There is a 5% chance that an electrical storm will occur.If there is a .1% chance that the line will be hit during a storm what is theprobability that the line will be hit and there will be hard drive damageduring the next storm?Answer:H: power line hitD: hard drive damagedP(H ∩ D) = p(D|H)p(H) = .5 ∗ .001 = .005

26 / 1

Extra Problems

Combinations

(a) How many different 8-digit binary sequences are there with six 1s andtwo 0s?Answer: the position of the 1s and 0s matters. There are 8 places the 1scan go. The zeros fill in the remaining 2 spots: 8C6 = 28(b) What is the probability of getting from a random generator an 8-digitbinary sequence with six 1s and two 0s?Answer: 8C6

28= 28/256 = 0.109

27 / 1

Extra Problems

Combinations

Questions:(a) How many 5-card hands (subsets can be formed from a standard52-card deck?Answer: 52C5 =

52!5!47! = 2, 598, 960 different 5 card hands

(b) If a 5-card hand is chosen at random, what is the probability ofobtaining a flush (all 5 cards in the hand are in the same suit- 13 cards persuit)Answer: There are 4 suits, and a subset of 5 cards from the 13 cards in asuit can be chosen b13C5 =

13!5!8! = 1287 ways. So there are

4× 1287 = 5148 flushes or in probability terms 51482598960 = 0.00198

(c) What is the probability of obtaining exactly 3 aces?Answer: First how many ways are there to choose 3 of the 4 aces?

4C3 = 4. Then fill out the hand with the other 2 cards from the 48 thatare not aces (48C=1128) ways. There are 4× 1128 = 4152 hands withexactly 3 aces. In probability terms that is 4152

2598960 = 0.00174

28 / 1

Extra Problems

Bayes Theorem

It is reported that 50% of all computer chips are defective. Inspectionensures that only 5% of chips legally marketed are defective.Unfortunately, some chips are stolen before inspection. If 1% of all chipsare stolen, find the probability that a given chip is stolen given that it isdefective.Answer: D: chip is defective T: chip is stolenP(D) = .50 and P(T ) = .01

P(T |D) = P(D/T )P(T )P(D|T )P(T )+P(D|T ′)P(T ′) =

.5∗.01.50∗.01+.05∗.99 = 0.0917

29 / 1

Extra Problems

Bayes Theorem

Type A: 41%, Type B 9%, Type AB 4%, Type O 46%. During WWII 4%of inductees with type O blood were typed as having type A, 88% of thosewith type A were correctly typed, 4% with type B were typed as A and10% with type AB were typed as A. A soldier was tested and typed as A,what is the probability that his true blood type is A?Answer:P(A1—E)? A1=he has type A, A2=he has type B, A3=he has type AB,A4=he has type O and E=he is typed as type A.P(A1)=.41, P(A2)=.09, P(A3)=.04, P(A4)=.46 P(E |A1) = .88P(E |A2) = .04 P(E |A3) = .10 P(E |A4) = .04P(A1|E ) = .88∗.41

.88∗.41+.04∗.09+.10∗.04+.04∗.46 = 0.933If a person was typed with type A blood, there was a 93.3% chance thathis true type was in fact type A.

30 / 1