Date post: | 03-Apr-2018 |
Category: |
Documents |
Upload: | katsuyouri |
View: | 225 times |
Download: | 0 times |
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 1/66
Statistics Probability
Introduction to probability
STA 113 Chapter 2 of Devore
Aug 31, Sep 2, 2010
Introduction to probability
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 2/66
Statistics Probability
Table of contents
1 Statistics
2 ProbabilityGeneral discussions
Basic conceptsSet theoryAxiomsPropertiesCombinatorics – countingConditional probabilityIndependence
Introduction to probability
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 3/66
Statistics Probability
Statistics
Definition from Wikipedia:Statistics is a mathematical science pertaining to the collection, analysis, interpretation or explanation, and presentation of data.
Statistics arose between 17th to 18th century, from the needof states to collect data on their people and economies, inorder to administer them.
Mathematical methods of statistics emerged from probabilitytheory.
Key element: account for (model) randomness anduncertainty in the observations.
Two current main schools: frequenists and Bayesian.
Introduction to probability
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 4/66
Statistics Probability
Examples
Famous story - does smoking cause lung cancer?
How to model and predict freeway traffic flows?
How to detect cancerous genes from microarray data?
How to estimate the effect of air pollution on public health
from observational environmental data?
How to determine if new drugs and medical devices help fromclinical trials?
Does lowering the drink age increase car accident rate (DUI
among youngsters)?U.S. presidential election: do incumbents have advantage?
...
Introduction to probability
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 5/66
Statistics Probability
Example: Needle Exchange Program
Needle exchange programs (NEP): give free clean needles inexchange of used ones to injection drug users. Hope to reduceHIV incidence.
Scientific question: Will NEP help in the long run?Q1: What data should we collect? - Design.
Q2: What information statements can we make when we havedata? - Inference.
Q3: What action to make? - Decision.In this course, we will focus on “inference”.
Introduction to probability
S i i P b bili G l di i B i S h A i P
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 6/66
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Probability: some discussion
From Encyclopaedia Britannica: Probability theory is the
branch of mathematics concerned with analysis of randomphenomena.
Probability theory is a mathematical foundation for statistics.
Introduction to probability
St ti ti P b bilit G l di i B i t S t th A i P
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 7/66
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Probability: some discussion
Useful strategy used in much of science:For a given experiment
attribute all that is known or theorized to a mechanistic model(mathematical function)
attribute everything else to randomness, even if the process under study is known not the be “random” in any sense of the word Use probability to quantify the uncertainty in your conclusionsEvaluate the sensitivity of your conclusions to the assumptionsof your model
Probability has been found extraordinarily useful, even if truerandomness is an elusive, undefined, quantity
Introduction to probability
Statistics Probabilit General disc ssions Basic concepts Set theor Axioms Pro
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 8/66
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Two types of Probability
Two types
1 Discrete – male or female, heads or tails, number of radioactive particles, how many buses go by, counts of nucleotides
2 Continuous – height of people, size of the head of a crab,length of time between volcanic eruptions, images (?), speechsignals
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 9/66
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Experiment
DefinitionBroadly, an experiment is any action or process whose outcome is subject to uncertainty.
Examples of the outcome of an experiment:a collection of measurement from a sampled population.
measurement from a laboratory experiment
the result of a clinical trial
survival time of a cancer patientthe result of a simulated computer experiment
...
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 10/66
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Sample space
Definition
A sample space S is the set of all possible outcomes of anexperiment.
Examples:
1 Experiment is flipping one coin – Sample space is {H , T }.
2 Experiment is flipping two coins – Sample space is{HH , TT , HT , TH }.
3 Experiment is flipping n coins – How many elements in thesample space ?
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 11/66
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Sample space
Definition
A sample space S is the set of all possible outcomes of anexperiment.
Examples:5 Experiment is playing five rounds of Russian roulette – Sample
space is {D , LD , LLD , LLLD , LLLD }. Why is this differentthan coin flipping.
6 Experiment is sequencing three nucleotides – Sample space is{AAA, CCC , GGG , TTT , AAC , AAT , AAG , ..., }. How big isthis sample space ? (Hint: There are four nucleotides.)
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 12/66
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
An event
Definition
A event is any collection of outcomes contained in the sample space, S .
Examples:
1 Flipping one coin – Getting heads.
2 Flipping two coins – Getting two heads.
3 Playing Russian roulette – Getting shot on the fourth spin.
4 Sequencing nucleotides – Your genome.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 13/66
y p y
Set theory
An event is a subset of the sample space S . Normal set operationshave particular interpretations in this setting
Definition
1 A ⊂ B implies that the occurrence of A implies the occurrence of B
2 Intersection A ∩ B: implies the event that both A and B occur
3 Union A ∪ B: implies the event that at least one of A or B occur
4 The complement of an event A denoted A (also notated Ac
or A): the set of all outcomes in S not contained in A,A = S\A - the event that A does not occur
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 14/66
More set theory
Definition
Two events A and B are mutually exclusive or disjoint if they
have no outcomes in common, i.e. A ∩ B = Ø.
Being a Duke fan and a Carolina fan is mutually exclusive.What is the event and what is the experiment ?
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 15/66
Fun aside: Barber paradox
A laymen version of the famous Russell’s paradox, discovered byBertrand Russell in 1901.
Consider a barber who shaves men if and only if they do notshave themselves.
Should the barber shave himself or not?If yes, then the barber should not shave himself, by his rule.If no, then the barber should shave himself, by his rule.
Generally, consider R , the set containing all sets that do not
contain themselves as members.Does R contains itself? Contradiction
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 16/66
What is probability?
Given an experiment and a sample space S , the objective of probability is to assign to each event A a number Pr(A), called theprobability of the event A, which will give a precise measure of thechance that A will occur.
1 Event: A = Duke wins next year’s NCAA basketball title?.What is Pr(A) ?
2 Event: A = someone here wins the lottery. What is Pr(A) ?
3 Event: A = I spin a quarter and it comes up heads. What is
Pr(A) ?
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 17/66
Probability: Frequentist interpretation
There are several possible interpretation of probability.
There is not agreement, at all, in how probabilities should beinterpreted
There is (nearly) complete agreement on the mathematical
rules probability must follow (axioms)Frequentist interpretation: A probability is the frequency of anevent will occur in repeated identical repetitions of anexperiment.
Often associated with Jerzy Neyman and Egon Pearson whodescribed the logic of statistical hypothesis testing.
The single main stream school until recently.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 18/66
Probability: Bayesian interpretation
An alternative interpretation of probability is so-called“Bayesian”,
Named after the 18th century Presbyterian minister andmathematician Thomas Bayes.
A Bayesian interprets probability as a subjective degree of belief: For the same event, two separate people could have differing probabilities.
Bayesian interpretations of probabilities avoid some of the
philosophical difficulties of frequency interpretations.Largely popularized by revolutionary advance in computationaltechnology and methods during the last twenty years.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 19/66
Axiomatic Foundations of Probability
The foundations of modern probability theory, the axiomaticbasis, are laid by Andrey Kolmogorov in 1933.
The axiomatic approach is not concerned with theinterpretation of probabilities.
Concerned only that probabilities are defined by a functionsatisfying the axioms.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 20/66
Axioms of probability
DefinitionAxiom 1: For any event A, Pr(A) ≥ 0 .Axiom 2: Pr(S ) = 1.Axiom 3: If A1, A2, ..., Ak are mutually exclusive events, then
Pr
A1
A2
· · ·
Ak
=
k i =1
Pr(Ai ).
Let k → ∞, A1, A2, A3, ... are mutually exclusive events
Pr
A1
A2
· · ·
=∞i =1
Pr(Ai ).
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 21/66
Axioms of probability
Example: coin flipS = {H , T } and H
T = S
1 = Pr(S )
1 = Pr(H ) + Pr(T ) = Pr(S )
Pr(T ) = 1 − Pr(H ) = p
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 22/66
Properties of probability – Venn diagrams
Following properties can all be derived from the axioms, andunderstood from Venn diagrams.
Proposition
For any event A, Pr(A) = 1 − Pr(A)
Sometimes A is easier to compute than A.Example: Flip n coins with Pr(H ) = p . What is the probability of the event one or more heads ?
Pr(A) = Pr(TTTTT · · · ) = (1 − p )n
Pr(A) = 1 − (1 − p )n.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 23/66
Properties of probability – Venn diagrams
Proposition
If A and B are mutually exclusive, then Pr (A
B ) = 0 and A
B = ∅.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
P i f b bili V di
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 24/66
Properties of probability – Venn diagrams
Proposition
For any two events A and B,
Pr
A
B
= Pr(A) + Pr(B ) − Pr
A
B
.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
P i f b bili V di
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 25/66
Properties of probability – Venn diagrams
For any three events A, B , C ,
Pr AB C = Pr(A) + Pr(B ) + Pr(C )
− Pr
A
B
− Pr
A
C
− Pr
B
C
+ Pr
A
B
C
.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
P i f b bili V di
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 26/66
Properties of probability – Venn diagrams
For any three events A, B , C ,
Pr AB C = Pr(A) + Pr(B ) + Pr(C )
− Pr
A
B
− Pr
A
C
− Pr
B
C
+ Pr
A
B
C
.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
P ti f b bilit V di
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 27/66
Properties of probability – Venn diagrams
In an experiment with N outcomes that are equally likely theprobability of any outcome A is
Pr(A) = p =1
N .
Roll a six sided dice. What is p ?
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
C ti 101
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 28/66
Counting 101
When you have to count, go through the following checklist
1 Does order matter ?
2
Can objects be chosen only once ?3 Can I break the problem into simpler computations ?
4 Is the complement simpler to work with ?
5 Can I simplify computations by conditioning ?
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Order matters with repetition
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 29/66
Order matters with repetition
Given a set of objects for example A, B , C , D .Assume
1 Order matters: ABC is different from CBA.
2 Objects can be selected more than once, i.e. AAA is possible.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Order matters with repetition
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 30/66
Order matters with repetition
An ordered collection of k objects is a k -tuple.
Your genome is a k -tuple, {AAACTGATTTCCC · · · }, with k ≈ 3billion.
A fixed price meal is a k -tuple, {app, main, dessert}, with k = 3.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Order matters product rule
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 31/66
Order matters - product rule
Proposition
A set consists of ordered collections of k-tuples with n1 choices for element 1, n2 choices for element 2 ,..., and nk choices for element k. The number of possible k-tuples are n1n2 · · · nk .
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Fixed price menu
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 32/66
Fixed price menu
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Fixed price menu
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 33/66
Fixed price menu
Appetizers: n1 = 6Mains: n2 = 6Desserts n3 = 6Number of meals: n1 × n2 × n3 = 216.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
An argument for intelligent design
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 34/66
An argument for intelligent design
Your genome is a k -tuple, with k ≈ 3 billion.William A. Dembski makes the following argument.
1 Assume that A, C , T , G are equally likely.
2 Assume that order matters and an amino acid in one positiondoes not effect others.
3 The probability of your genome is 1/430,000,000,000.
4 Astronomically small chance so cannot be random, or evolve
from random mutations.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Order matters without repetition - permutation
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 35/66
Order matters without repetition permutation
Given a set of objects for example A, B , C , D .Assume
1 Order matters: ABC is different from CBA.2 Objects cannot be selected more than once: AAA is not
possible, neither is AAB . ACB is possible.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Order matters without repetition - permutation
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 36/66
Order matters without repetition permutation
Example: Pick a starting lineup for Duke given the 12 people onthe roster.
P 5,12 = 12 × 11 × 10 × 9 × 8.
More generally
P k ,n =n!
(n − k )!,
where n! = n(n − 1)(n − 2)(n − 3) · · · 1 and 0! = 1.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Order doesn’t matter without repetition - combinations
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 37/66
Order doesn t matter without repetition combinations
Given a set of objects for example A, B , C , D .Assume
1 Order does not matter: ABC is the same as CBA.2 Objects cannot be selected more than once: AAA is not
possible, neither is AAB . ACB is possible.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Order doesn’t matter without repetition - combinations
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 38/66
Order doesn t matter without repetition combinations
Definition
Given n distinct objects, any unordered subset of size k objects is a
combination. The number of combinations is nk
or n choose k.
A bridge hand is a combination.
So is a poker hand.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Order doesn’t matter without repetition - combinations
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 39/66
p
An example: Given the set {A, B , C , D , E } how manycombinations of size three ?
1 First observation: P 3,5 > 53, why ?
Permutations{ABC }, {ACB }, {BCA}, {BAC }, {CBA}, {CAB }Combination {ABC }.
2 Second observation: for any combination of size 3 there are
3! = 6 permutations.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Order doesn’t matter without repetition - combinations
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 40/66
p
An example: Given the set {A, B , C , D , E } how manycombinations of size three ?
3 Third observation:
P 3,5 =5
3
× 3!5
3
=
P 3,5
3!=
5!
3!(5 − 3)!.
4
Induction:
n
k
=
n!
k !(n − k )!.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Poker hands
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 41/66
52 playing cards and you are dealt 5 cards (a hand).4 suits : ♣, ♥, ♠, ♦.Each suit has 13 cards 2 − 9, J , Q , K , A.You want the best hand.
Some poker handsNone the samePair
Two pairsThree of a kind
Full houseFour of a kind
Royal flush
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Poker hands
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 42/66
Number of total poker hands:
525
= 2, 598, 960.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
None the same
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 43/66
Number of none the same:Counting strategy 1: break into cases and then multiplyCases: face cards and suits.Choices of faces
135
: 5 different faces out of 13 for example
{3 8 5 2 9}.Choices of suits 4 × 4 × 4 × 4 × 4: each card can be one of foursuits for example {♣♥♠♦♥}Total:
135
× 45 = 1, 317, 888
Pr(none the same) =1, 317, 888
2, 598, 960 = 50.7%.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Flush
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 44/66
Choices of faces
135
: 5 different faces out of 13.
Choices of suits 4.Total: 13
5× 4 = 5148
Pr(flush) =5148
2, 598, 960= 0.2%.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
None the same no flush
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 45/66
This example uses the idea that sometimes the complement iseasier to count.The set {None the same and no flush} can be written as
{None the same and no flush} = {None the same}−{None the same and
Observe: {None the same and flush} = {flush}.Total:
135
× 45 −
135
× 4 = 1, 312, 740
Pr(none the same no flush) =
5148
2, 598, 960 = 50.5%.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Conditional probability, motivation
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 46/66
The probability of getting a one when rolling a (standard) dieis usually assumed to be one sixth
Suppose you were given the extra information that the die rollwas an odd number (hence 1, 3 or 5)
Conditional on this new information, the probability of a oneis now one third
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Conditional probability
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 47/66
Definition
Given two events A and B with Pr(B ) > 0, the conditional
probability of A given B has occurred is defined as
Pr(A|B ) =Pr(A
B )
Pr(B ).
When B is the sample space: P (B |B ) = 1
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Example
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 48/66
Consider our die roll example
B = {1, 3, 5}
A = {1}
P (one given that roll is odd) = P (A | B )
=P (A ∩ B )
P (B )
=P (A)
P (B )
=1/6
3/6=
1
3
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Conditional probability
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 49/66
Proposition (Multiplication rule)
Pr(B
A) = Pr(B |A) × Pr(A).
Proposition (Law of total probability)
Let A1, ..., Ak be mutually exclusive and exhaustive events (i.e.,k i =1 Ai = S ). Then for any other event B
Pr(B ) =k
i =1
Pr(B |Ai ) × Pr(Ai ).
Applies when k go to ∞.Such collection of sets A1, ..., Ak is also called a partition of samplespace.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Conditional probability
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 50/66
Proof of Law of total probability
By logic
B = (A1 and B ) or (A2 and B ) or ... or (Ak and B ).
In equation form
B = (A1
B )
(A2
B )
...
(AK
B )).
This implies
Pr(B ) = Pr
(A1
B )
(A2
B )
...
(AK
B )
.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Conditional probability
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 51/66
Proof of Law of total probability
Pr(B ) = Pr
(A1
B )
(A2
B )
...
(AK
B )
.
Since Ai are exclusive and exhaustive
Pr(B ) =k
i =1
Pr(Ai
B ) =
k i =1
Pr(B |Ai ) × Pr(Ai ).
The last step is by the multiplication rule.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Bayes’ theorem
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 52/66
Theorem
Let A1, ..., Ak be mutually exclusive and exhaustive events withPr(Ai ) > 0 for all i = 1, ..., k. Then for any other event B withPr(B ) > 0
Pr(A j |B ) =Pr(A j
B )
Pr(B )=
Pr(B |A j )Pr(A j )k i =1 Pr(B |Ai )Pr(Ai )
, j = 1, ..., k .
P (Ai ) is often called prior probability; P (Ai |B ) is called posterior
probability.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Bayes’ theorem
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 53/66
A common application of Bayes’ theorem is the following:
Pr(model|data) =Pr(data|model) Pr(model)
Pr(data).
The important quantities are:
1 Likelihood: Pr(data|model)
2 Prior: Pr(model)
3 Posterior: Pr(model|data)
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
The reverend Thomas Bayes
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 54/66
Stigler’s Law of Eponymy: no scientificdiscovery is named after its original discovers
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Monty Hall
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 55/66
http://en.wikipedia.org/wiki/Monty_Hall_problem
Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats.
You pick a door, say No. 1, and the host, who knows what’sbehind the doors, opens another door, say No. 3, which he knowshas a goat. He then says to you, “Do you want to pick door No.2?” Is it to your advantage to switch your choice?
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Monty Hall
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 56/66
A shortcut answer:
Event A: You first pick the right doorEvent B : You switch to the correct door given the host open agoat doorBy the law of total probability
Pr(B ) = Pr(B |A)Pr(A) + Pr(B |A
)Pr(A
)= 0 × 1/3 + 1 × 2/3
= 2/3
Good to switch. What notational shortcut did I take here?In fact, event B is a combination to two events: (1) you switch(B 1), (2) the host open a goat door (B 2).Write out the above equation in terms of A, B 1, B 2 yourself.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Example: diagnostic tests
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 57/66
Let + and − be the events that the result of a diagnostic testis positive or negative respectively
Let D and D c be the event that the subject of the test has or
does not have the disease respectivelyThe sensitivity is the probability that the test is positivegiven that the subject actually has the disease, P (+ | D )
The specificity is the probability that the test is negativegiven that the subject does not have the disease, P (− | D c )
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
More definitions
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 58/66
The positive predictive value is the probability that thesubject has the disease given that the test is positive,P (D | +)
The negative predictive value is the probability that thesubject does not have the disease given that the test isnegative, P (D c | −)
The prevalence of the disease is the marginal probability of disease, P (D )
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
More definitions
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 59/66
The diagnostic likelihood ratio of a positive test, labeledDLR +, is P (+ | D )/P (+ | D c ), which is the
sensitivity /(1 − specificity )
The diagnostic likelihood ratio of a negative test, labeledDLR −, is P (− | D )/P (− | D c ), which is the
(1 − sensitivity )/specificity
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Example
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 60/66
A study comparing the efficacy of HIV tests, reports on anexperiment which concluded that HIV antibody tests have asensitivity of 99.7% and a specificity of 98.5%
Suppose that a subject, from a population with a .1%
prevalence of HIV, receives a positive test result. What is theprobability that this subject has HIV?
Mathematically, we want P (D | +) given the sensitivity,P (+ | D ) = .997, the specificity, P (− | D c ) = .985, and the
prevalence P (D ) = .001
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Using Bayes’ formula
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 61/66
P (D | +) =P (+ | D )P (D )
P (+ | D )P (D ) + P (+ | D c )P (D c )
=P (+ | D )P (D )
P (+ | D )P (D ) + {1 − P (− | D c )}{1 − P (D )}
= .997 × .001.997 × .001 + .015 × .999
= .062
In this population a positive test result only suggests a 6%probability that the subject has the disease
(The positive predictive value is 6% for this test)
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Statistical independence
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 62/66
Definition
A and B are independent if and only if
Pr(A
B ) = Pr(A) × Pr(B ).
If A and B are independent, then
P (A | B ) =P (A)P (B )
P (B )= P (A).
The above is equivalent to the definition (alternative definition).
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Statistical independence
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 63/66
Knowing B does not provide any info about event A.
Independence is not mutually exclusive (P (A ∩ B ) = 0).
Independence is usually on assumption, usually hard to prove.
Many gambling games provide models of independent events.
If A and B are independent events,A and B , A and B , A and B are also independent.
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Example
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 64/66
Volume 309 of Science reports on a physician who was ontrial for expert testimony in a criminal
Based on an estimated prevalence of sudden infant deathsyndrome (SIDS) of 1 out of 8, 543, Dr Meadow testified thatthat the probability of a mother having two children with
SIDS was
18,8543
2
The mother on trial was convicted of murder
What was Dr Meadow’s mistake(s)?
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Example: continued
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 65/66
For the purposes of this class, the principal mistake was toassume that the probabilities of having (SIDS) within a familyare independent
That is, P (A1 ∩ A2) is not necessarily equal to P (A1)P (A2)
Biological processes that have a believed genetic or familiarenvironmental component, of course, tend to be dependentwithin families
In addition, the estimated prevalence was obtained from anunpublished report on single cases; hence having noinformation about recurrence of SIDs within families
Introduction to probability
Statistics Probability General discussions Basic concepts Set theory Axioms Pro
Mutual independence
7/28/2019 Lecture 1 - Chapter 2 of Stats Textbook
http://slidepdf.com/reader/full/lecture-1-chapter-2-of-stats-textbook 66/66
Extend the concept of independence into more than twoevents.
Is the condition P (A ∩ B ∩ C ) = P (A)P (B )P (C ) enough?
Is the condition “all the pairs are independent” enough?
Definition
A collection of events A1, ..., An are mutually independent if for any subcollection Ai 1 , ..., Ai k (k > 1), we have
Pr(Ai 1
Ai 2
...Ai k ) = Pr(Ai 1 ) × Pr(Ai 2 ) × ... Pr(Ai k ).
Introduction to probability