Analysis of Categorical and Ordinal Data: Binomial and Logistic Regression
Lecture 6
Analysis of Binary Data
Binomial Regression
- Used when the individual “trial” is not the unit of study, but rather when there are replicates of a set of trials (i.e. seedlings in a quadrat)
» In the past, folks often analyzed this type of dataset by converting the response variable to a percentage, and then doing regression on the percentages (after doing ugly transformations…)
- Model predicts the underlying Binomial probability that would produce the observed number of successes given a number of trials
Logistic Regression
- Used when the individual Bernoulli trial is the unit of study (i.e. did the seedling die…)
- Model predicts the probability of “success” of a given trial
Steps in a likelihood analysis for binomial regression
In R:1.Specify the “scientific model” that predicts the probability of “success” as a function of a set of independent variables…
-- Note that your scientific model should predict expected values bounded by 0 and 1 (since the predicted value is a probability)
2.Define the likelihood function (using dbinom)
binom_log_lh_function <- function(successes,trials,p){ dbinom(x=successes,size=trials, prob = p, log = TRUE) }
3.Set up optimization to find the parameters of the scientific model that maximize likelihood across the dataset
Logistic Regression Example: analysis of windthrow data
Traditionally: Summarize variation in degree and type of damage, across species and tree sizes, from the storm, as a whole...
A likelihood alternative: Use the spatial variation in storm intensity that occurs within a given storm to estimate parameters of functions that describesusceptibility to windthrow, as a function of variationin storm severity and individualtree attributes...
Types of Response Variables(with examples from analysis of windthrow data)
BINARY: Only two possible outcomes (yes, no; lived, died; etc.)- This is termed a “Bernoulli trial”
CATEGORICAL: Multiple categories (uprooted, snapped,...)
ORDINAL: Ordered categories (degree of damage): none, light, medium, heavy, complete canopy loss {usually estimated visually}
CONTINUOUS: just what the term implies, but rarely used in analyses of wind damage because of the difficulties of quantifying damage accurately...
Analysis of Binary Data:Traditional Logistic Regression
Definition: Logit = log of an odds ratio (i.e log[p/(1-p)])
Benefits of logits• A logit is a continuous variable• Ranges from negative when p < 0.5 to positive when p > 0.5
...)1/(log 21 cXbXapp
Standard logistic regression involves fitting a linear function to the logit:
Consider a sample space consisting of two outcomes (A,B) where the probability that event A occurs is p
What if your terms are multiplicative?
...)1/(log 21 XbXapp
sisjissisjisj
bDBHScapp **))1/(log(
Example: Assume that the probability of windthrow is a joint (multiplicative) function of
(1) Storm severity, and(2) Tree size
In addition, assume that the effect of DBH is nonlinear.... A model that incorporates these can be written as:
A little more detail....
Pisj is the probability of windthrow of the jth individual of species s in plot i
DBHisj is the DBH of that individual
as, bs, and cs are species-specific, estimated parameters, and
Si is the estimated storm severity in plot i
- NOTE: storm severity is an arbitrary index, and was allowed to range from 0-1
- NOTE: you can think of this as a hierarchical model, with trees nested in plots, and S is the plot term
sisjissisjisj
bDBHScapp **))1/(log(
But don’t you have to measure storm severity (not estimate it)?
Likelihood Function for Logistic Regression
isj isjisj
isjisj
n windthrownot wastree if p
n windthrow wastree if plikelihood-log
)1log(
)log(
It couldn’t be any easier... (since the scientific model is already expressed as a probabilistic equation):.
sisjissisjisj
bDBHScapp logit **))1/(log(
logit
logit
isj e1
ep
loglikelihood <- function(pred,observed){ ifelse(observed == 1, log(pred), log(1-pred)) }
Example: Windthrow in the Adirondacks
Highly variable damage due to:• variation within storm• topography• susceptibility of species within a stand
Reference: Canham, C. D., Papaik, M. J., and Latty, E. F. 2001. Interspecific variation in susceptibility to windthrow as a function of tree size and storm severity for northern temperate tree species. Canadian Journal of Forest Research 31:1-10.
The dataset
Study area: 15 x 6 km area perpendicular to the storm path
43 circular plots: 0.125 ha (19.95 m radius) censused in 1996 (20 of the 43 were in oldgrowth forests)
The plots were chosen to span a wide range of apparent damage
All trees > 10 cm DBH censused
Tallied as windthrown if uprooted or if stem was < 45o from the ground
Critical data requirements
Variation in storm severity across plots
Variation in DBH and species mixture within plots
sisjijssisjisj
bDBHScapp **))1/(log(
The analysis...
7 species comprised 97% of stems – only stems of those 7 species were included in the dataset for analysis
# parameters = 64 (43 plots + 3 parameters for each of 7 species)
Parameters estimated using simulated annealing
sisjijssisjisj
bDBHScapp **))1/(log(
Predicted Probability of Windthrow
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0Obs
erve
d P
ropo
rtio
n W
indt
hrow
n
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
279787
527
312
174
123
111
70
67
62
Model evaluation
Numbers above bars represent the number of observations in the class
The solid line is a 1:1 relationship
Estimating Storm Severity
Plot-Level Windthrow
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Sto
rm S
ever
ity I
ndex
0.0
0.2
0.4
0.6
0.8
1.0
densitybasal area
Results: Big trees...
Storm Severity
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Pro
babi
lity
of M
orta
lity
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
ACRU ACSA BEAL FAGR PIRU PRSE TSCA ALL
70 cm DBH
Little trees...
Storm Severity
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Pro
babi
lity
of M
orta
lity
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
ACRU ACSA BEAL FAGR PIRU PRSE TSCA ALL
10 cm DBH
New twists
Effects of partial harvesting on risk of windthrow to residual trees
Effects of proximity to edges of clearings on risk of windthrow
Research with Dave Coates in cedar-hemlock forests of interior B.C.
Effects of harvest intensity and proximity to edge…
(2) )100/*(*))1/(log( hBAbDBHcSapp sisjsisisjisj
(3) )100/(**))1/(log( hBAbDBHcSapp sisjsisisjisj
Equation (2) introduces the effect of prior harvest removal to equation (1) by adding basal area removal and assumes the effect is independent and additive
Equation (3) assumes the effects of prior harvest interact with tree size:
Models 1a – 3a: test models where separate c coefficients are estimated for “edge” vs. “non-edge” trees (edge = any tree within 10 m of a forest edge)
Equation (1): basic model – probability of windthrow is a species-specific function of tree size and storm severity:
(1) sisjsisisjisj
bDBHcSapp **))1/(log(
Other issues…
Is the risk of windthrow independent of the fate of neighboring trees? (not likely)
- Should we examine spatially-explicit models that factor in the “nucleating” process of spread of windthrow gaps?…
Analysis for ORDINAL Response Variables
The categories in this case are ranked (i.e. none, light, heavy damage)
Analysis shifts to cumulative probabilities...
The “Parallel Slopes” form of ordinal logistic regression
The Challenge: Since the response categories are ordinal, and the model predicts cumulative probabilities, we need a scientific model that generates predictions that keep the categories in order (i.e. the cumulative probability that a response should be in or less than level k needs to be greater than the predicted cumulative probability for level k-1
The Parallel Slopes solution:
- Just allow the intercept term in the equation for the logit to vary among the k ordinal responses, while the slope stays constant (Note that you only need k-1 intercepts…)
Simple Ordinal Logistic Regression
)Pr( ik XYy p
xbxba plogit( k ...) 2211
(i.e. the probability that an observation y will be less than or equal to ordinal level Yk (k = 1.. n-1 levels) , given a vector of X explanatory variables),
)1/( logitlogit eep
Then simple ordinal logistic regression fits a model of the form:
If
Remember:
)Pr()Pr( 1 ikikk XYyXYy p
The probability that an event will fall into a single class k (rather than the cumulative probability) is simply
In our case...
sbisjisks DBHScaplogit )(
and where aks, cs and bs are species specific parameters (s = 1.. S species), and Si are the estimated storm severities for the i = 1..N plots.
)Pr( kYy p where
# of parameters: N + (K-1+2)*S, where N = # of plots, K = # of ordinal response levels, and S = # of species
The Likelihood Function Stays the Same
isj isjk
isjk
not did tree if p
k level damage dexperience tree if plikelihood-log
)1log(
)log(
Again, since the scientific model is already expressed as a probabilistic equation:
)Pr()Pr( 1 ikikk XYyXYy p
The probability that an event will fall into a single class k (rather than the cumulative probability) is simply
Hurricane Damage in Puerto Rico
Storm damage assessment in the permanent plot at the Luquillo LTER site- Hurricane Hugo - 1989
- Hurricane Georges – 1998 Combined the data into a single analysis: 136 plots, 13 species
(including 1 lumped category for “other” species), and 3 damage levels:- No or light damage
- Partial damage
- Complete canopy loss Total # of parameters = 188 (15,647 trees)
Canham, C. D., J. Thompson, J. K. Zimmerman, and M. Uriarte. 2010. Variation in susceptibility to hurricane damage as a function of storm intensity in Puerto Rican tree species. Biotropica 42:87-94.
Parameter Estimation with Simulated Annealing
-11460
-11440
-11420
-11400
-11380
-11360
-11340
-11320
-11300
0 2500000 5000000
Iteration
Likelihood
Solving simultaneously for 188 parameters in a dataset containing > 15,000 trees takes time!
Model Evaluation
Goodness of FitCombined Hugo and Georges Analysis
Predicted Probability
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Obs
erve
d P
ropo
rtio
n
0.0
0.2
0.4
0.6
0.8
1.0
Complete Damage Partial Damage No Damage 1:1 Line
Comparison of the two storms...
Figure 1. Relative Variation in Severityof Hurricanes Hugo and Georges
Storm Severity Index
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Pro
port
ion
of P
lots
0.00
0.05
0.10
0.15
0.20
0.25
0.30
HugoGeorges
Statistics on variation in storm severity from Hurricanes Hugo and Georges
Severity Hugo Georgesn 96 40minimum 0.20 0.02maximum 1.00 0.69mean 0.63 0.44S.D. 0.17 0.16
ALCLAT
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
Storm Severity
Pro
babi
lity
of D
amag
e
NoneMediumComplete
BUCCAP
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1Storm Severity
Pro
babi
lity
of D
amag
eNoneMediumComplete
CASARB
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1Storm Severity
Pro
babi
lity
of D
amag
e
NoneMediumComplete
CECSCH
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1Storm Severity
Pro
ba
bili
ty o
f Da
ma
ge
NoneMediumComplete