School of InformationUniversity of Michigan
Discrete and continuous distributions
Where does the binomial coefficient come from?
A B C D E F G
Suppose I 7 blue and pink balls, each of them uniquely marked so that I can distinguish them
How many different samples can I draw containing the same balls but in a different order?7!
I have 7 choices for the first spot, 6 choices for the second (since I’ve picked 1 and now have only 6 to choose from),5 choices for the third, etc.
7! = 7 * 6 * 5 * 4 * 3 * 2 * 1
ABC DE FG
A B CD
Now if I am just counting the number of blue and pink balls, I don’t care about the order.So all possible arrangements (3!) of the pink balls look the same to me
A B CD EF G
A B CD EG F
A B CD FE G
A B CD FG E
A B CD GF E
A B CD GE F
So instead of having 7! combinations, we have 7!/3! combinations, because where before we had 6 different possibilities of uniquely ordering different pink balls – they are equivalent
FE G
The same goes for the blue balls, if we can’t tell them apart, we lose a factor of 4!
Binomial coefficient =C(n,k)= -----------------------------------------------------------------number of ways of arranging n different things
(# of ways to arrange k things)*(# ways to arrange n-k things)
= -----------------n!
k! (n-k)!
Note that the binomial coefficient is symmetric – there are the same number of ways of choosing k or n-k things out of n
We’ve got the coefficient, what is the distribution about?
Suppose your sample of 7 is actually drawn from a very large population (so large that it is basically unaffected by the removal of a
measly 7 balls) p = probability that ball is pink (1-p) = probability that ball is not pink (blue)
The probability that you draw a sample with 3 pink balls and 4 blue balls in a particular order e.g. (two pink followed by 3 blues, followed by a pink followed by a blue) is
prob(pink)*prob(pink)*prob(blue)*prob(blue)*prob(blue)*prob(pink)*prob(blue)
= p3*(1-p)4
We’ve got the coefficient, what is the distribution about?
But the binomial distribution just tells us what the probability is of drawing e.g. 3 pink balls, not 3 pink balls at a particular point in the draw
The probability that you draw a sample with 3 pink balls and 4 blue balls in no particular order is
= C(7,3) p3*(1-p)4
+
….
Probability distribution
A probability distribution lists all the possible outcomes and their probabilities
Outcomes are mutually exclusive e.g. drawing 0, 1, 2, 3… pink balls
Outcome probabilities sum to one e.g. when drawing 7 balls, the probability has to be
one of {0,1,2,3,4,5,6,7}
Denote p(x) to mean P(X=x), that is the probability that the outcome is x
Binomial distribution
The binomial distribution tells us the probability of drawing k pink balls out of n
It depends on n = the number of trials (draws) k = the number of pink balls (successes) p = the probability of drawing a pink ball (success)
knk
knk
ppknk
n
ppk
nknp
−
−
−−
=
−⎟⎟⎠
⎞⎜⎜⎝
⎛=
)1()!(!
!
)1(),(
the binomial distribution in R
dbinom(x, size, prob) if blue and pink balls
are equally likely> dbinom(3,7,0.5)
[1] 0.2734375
>barplot(dbinom(0:7,7,0.5),names.arg=0:7)
0 1 2 3 4 5 6 7
0.00
0.05
0.10
0.15
0.20
0.25
what if p ≠ 0.5?
> barplot(dbinom(0:7,7,0.1),names.arg=0:7)
0 1 2 3 4 5 6 7
0.0
0.1
0.2
0.3
0.4
What is the mean?
mean of a binomial distribution is just n*p in general = E(X) = x p(x)
0 * + 1 * + 2 * + 3 * + 4 * + 5 * + 6 * + 7 *
0.00
0.05
0.10
0.15
0.20
0.25
probabilities that
sum to 1
= 3.5
What is the variance?
variance of a binomial distribution is justn*p*(1-p)
in general 2 = E[(X-)2] = (x-)2 p(x)
(-3.5)2 *+ + + + + +
0.00
0.05
0.10
0.15
0.20
0.25 probabilities that
sum to 1
(-2.5)2 *
+
(-1.5)2 *
(-0.5)2 * (0.5)2 *
(1.5)2 *
(2.5)2 *
(-3.5)2 *
Which distribution has greater variance?
0 1 2 3 4 5 6 7
0.00
0.05
0.10
0.15
0.20
0.25
0 1 2 3 4 5 6 7
0.0
0.1
0.2
0.3
0.4
p = 0.5 p = 0.1
var = n*p*(1-p) = 7*0.1*0.9=7*0.09var = n*p*(1-p) = 7*0.5*0.5 = 7*0.25
briefly comparing an experiment to a distribution
experiments = 1000
tosses = 7
for (i in 1:experiments) {
x = sample(c("H","T"), tosses, replace = T)
y[i] = sum(x=="H")
}
hist(y,breaks=-0.5:7.5)
lines(0:7,dbinom(0:7,7,0.5)*1000)
theoretical
distributionresult of 1000 trialsHistogram of y
y
Fre
quen
cy
0 2 4 6
050
100
150
200
250
300
cumulative distribution
aka CDF = cumulative density function the probability that x is less than or equal to
some value a
( ) ( ) ( ) ( )∑=∑ ==≤=≤≤ axax
x xpxXaXaF PrPr
cumulative distribution
P(X=x)
0 1 2 3 4 5 6 7
cumulative distribution
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5 6 7
probability distribution
0.0
0.2
0.4
0.6
0.8
1.0
P(X≤x)
> barplot(dbinom(0:7,7,0.5),names.arg=0:7) > barplot(pbinom(0:7,7,0.5),names.arg=0:7)
cumulative distribution
0 1 2 3 4 5 6 7
prob
abili
ty d
istr
ibut
ion
0.0
0.2
0.4
0.6
0.8
1.0
0 1 2 3 4 5 6 7
cum
ulat
ive
dist
ribut
ion
0.0
0.2
0.4
0.6
0.8
1.0
P(X=x) P(X≤x)
example: surfers on a website
Your site has a lot of visitors 45% of whom are female
You’ve created a new section on gardening Out of the first 100 visitors, 55 are female. What is the probability that this many or more of
the visitors are female? P(X≥55) = 1 – P(X≤54) = 1-pbinom(54,100,0.45)
another way to calculate cumulative probabilities
?pbinom P(X≤x) = pbinom(x, size, prob, lower.tail = T) P(X>x) = pbinom(x, size, prob, lower.tail = F)
> 1-pbinom(54,100,0.45)[1] 0.02839342
> pbinom(54,100,0.45,lower.tail=F)[1] 0.02839342
female surfers visiting a section of a website
0 6 13 21 29 37 45 53 61 69 77 85 93
probability distribution
0.00
0.02
0.04
0.06
what is the area under the curve?
cumulative distribution
0 6 13 21 29 37 45 53 61 69 77 85 93
cumulative distribution
0.0
0.2
0.4
0.6
0.8
1.0
<3 %
> 1-pbinom(54,100,0.45)
[1] 0.02839342
Another discrete distribution: hypergeometric
randomly draw n elements without replacement from a set of N elements, r of which are S’s (successes) and (N-r) of which are F’s (failures)
hypergeometric random variable x is the number of S’s in the draw of n elements
⎟⎟⎠
⎞⎜⎜⎝
⎛
⎟⎟⎠
⎞⎜⎜⎝
⎛−−
⎟⎟⎠
⎞⎜⎜⎝
⎛
=
n
N
xn
rN
x
r
xp )(
hypergeometric example
fortune cookies there are N = 20 fortune cookies r = 18 have a fortune, N-r = 2 are empty What is the probability that out of n = 5 cookies, s=5
have a fortune (that is we don’t notice that some cookies are empty)
> dhyper(5, 18, 2, 5) [1] 0.5526316
So there is a greater than 50% chance that we won’t notice.
hypergeometric and binomial
When the population N is (very) big, whether one samples with or without replacement is pretty much the same
100 cookies, 10 of which are empty
1 2 3 4 5
0.0
0.1
0.2
0.3
0.4
0.5binomial
hypergeometric
number of full cookies out of 5
code aside
> x = 1:5
> y1 = dhyper(1:5,90,10,5)
> y2 = dbinom(1:5,5,0.9)
> tmp = as.matrix(t(cbind(y1,y2)))
> barplot(tmp,beside=T,names.arg=x)
hypergeometric probability
binomial probability
Poisson distribution
# of events in a given interval e.g. number of light bulbs burning out in a building in a year # of people arriving in a queue per minute
!)(
x
exp
x λλ −
=
λ = mean # of events in a given interval
Example: Poisson distribution
You got a box of 1,000 widgets. The manufacturer says that the failure rate is 5
per box on average. Your box contains 10 defective widgets. What
are the odds?> ppois(9,5,lower.tail=F)[1] 0.03182806 Less than 3%, maybe the manufacturer is not
quite honest. Or the distribution is not Poisson?
Poisson approximation to binomial
If n is large (e.g. > 100) and n*p is moderate (p should be small) (e.g. < 10), the Poisson is a good approximation to the binomial with λ = n*p
0 1 2 3 4 5 6 7 8 9 11 13 15
binomialPoisson
0.00
0.05
0.10
0.15
Continuous distributions
Normal distribution (aka “bell curve”) fits many biological data well
e.g. height, weight
serves as an approximation to binomial, hypergeometric, Poisson
because of the Central Limit Theorem (more on this later) is important to inference problems
sampling from a normal distribution
x <- rnorm(1000)
h <- hist(x, plot=F)
ylim <- range(0,h$density,dnorm(0))
hist(x,freq=F,ylim=ylim)
curve(dnorm(x),add=T)
Histogram of x
x
Density
-4 -2 0 2 4
0.0
0.1
0.2
0.3
0.4
plotting on log axes
First of all, this is what a log function looks like
0 200 400 600 800 1000
0
1
2
3
4
5
6
7
x
y
> x = 1:1000
> y = log(x)
> plot(x,y)
y = log(x) is equivalent to
x = exp(y) = ey
plotting the function y = e-x
> x = 1:20 > y = exp(-x) > plot(x,y)
5 10 15 20
0.0
0.1
0.2
0.3
x
y
hard to tell what’s going on here, all the values are so close to 0
changing the axes
1 2 5 10 20
1 e-09
1 e-07
1 e-05
1 e-03
1 e-01
x
y
both x and y on a log scale
5 10 15 20
1 e-09
1 e-07
1 e-05
1 e-03
1 e-01
x
y
just y on a log scale
> plot(x,y,log="xy")> plot(x,y,log="y")
from PS: CO2 levels over last ~ 50 years
CO2 levels over last ~ 400,000 years