Propensity Scores
October 2014
Alexander M. Walker MD, DrPHExtensive parts of this presentation incorporate the work of John D. Seeger, PharmD, DrPH
“In mathematics, you don't understand things.
You just get used to them.”
- John von Neumann
1903-1957
Research Goal
Compare two treatments with respect to a health or economic outcome
“Counterfactual” ideal If the same people had received B instead of A, how
would their outcomes have differed? What is achievable: “similar” not “same”
Comparable treatment groups … insofar as you can tell!
4
Pictures for Confounding
Comparison of Heterogeneous Groups
5
E1 E2
6
E1 E2
Internal Composition May Differ
7
Affectedindividuals
E1 E2
Risks that Depend on Subgroup Status
8
Affectedindividuals
50%
15%
50%
15%
E1 E2
Risks that Depend on Subgroup Status
Note that the proportions of affected individuals are different in the yellow and gray groups …
9
50%
15%
50%
15%
E1 E2
Risks that Depend on Subgroup Status
Note that the proportions of affected individuals are different in the yellow and gray groups …
… and that the group-specific risks are identical in the two exposed populations E1 and E2.
10
The differences in risk are due to the covariate
structure of compared
populations, not to the effects of
E1 and E2
E1 E2
Internal Risk Factor Heterogeneity Creates an Differences in Group Risk
11
Propensity Scores to Create Populations with Similar Covariate Structure
12
E1 E2
Covariate Heterogeneity
E1 has more
Yellow
E2 has more
Gray
13
E2
E1
E1
E2
Gray
predicts E2
Yellow predicts
E1
Covariate Status as a Predictor of Treatment
14
Propensity Scores
PS is the predicted probability of treatment, given all the covariates
Matching on the PS creates study populations that have balance on the covariates
Perfect for a single, dichotomous covariate Not perfect, but very good for multiple covariates
15
E1 E2
Propensity for Covariate PatternsThink of orange and green as two distinct covariate patterns that have the same predicted Pr(E1).
Pr(
E1
|Ora
ng
e)=
xP
r(E
1|G
ree
n)
=x
16
E1 E2
Gathering subjects with identical propensity puts all individuals with covariate patterns orange and green into the same stratum.
Conditioning on Propensity Permits Unconfounded Comparisons
At a given propensity level x, there is no association between treatment and covariate patterns.
Pr(
E1
|Ora
ng
e)=
xP
r(E
1|G
ree
n)
=x
Formal Expression
Propensity(x) P(T=1|x) = E(T|x)
The propensity associated with level x of the covariate X is the probability that treatment is present (equivalently, is “B” as opposed to “A”), given level x, and this is in turn equal to the expected value of treatment, given x.
Note that the definition does not specify the parametric form of the Propensity(x) . The examples in this talk use a logistic function; others -- including nonparametric functions -- are also used.Notation. A single capital letter denotes a variable, a single
lower case letter denotes a particular value for that variable.
Probability Calculus
Under propensity matching, how do X (covariate status) and T (treatment status) relate to one another?
1.Pr( x, t | p ) = Pr( x | p ) Pr( t | x, p ) Probability Theory
2.Pr( t | x, p ) = Pr( t | p ) p incorporates all information about t
that is in x
Pr( x, t | p ) = Pr( x | p ) Pr( t | p )
Pr( x, t | p ) = Pr( x | p ) Pr( t | p )
Given a particular value of the propensity score variable, that is at P=p, the covariates X and T are uncorrelated.
At particular levels of P individually and therefore collectively (i.e. “conditionally on P”), the lack of correlation guarantees that X cannot confound the association between T and any outcome.
20
Matching on Propensity Scores
Propensity Matching: Method
Identify candidate predictors of treatment B v A Perform a logistic regression of B v A Obtain from the regression a “predicted” probability of B v
A Sort all members of A and B according to this propensity Match A patients to B patients on the propensity
Duragesic and Long-Acting Opioids
Duragesic LA Opioids
N 504 2,201
65+ years 29% 10%
Male 35% 49%
Periph Vasc Disease 4% 1%
Sx of Abd or Pelvis 18% 10%
> 2 hospitalztns 6 mo 9% 3%
30 days NonRx Costs $1,136 $746
Straightforward Regressionproc logistic data = mother.propensity2 descending; model DuragesicUser = DischCostIndex EncCostIndex RxCostIndex OtherCostIndex RxCostPrior1 OtherCostPrior1 AnyRx OneDisch TwoDisch ThreePlusDisch AnyICD443 AnyICD719 AnyICD724 AnyICD787 AnyICD789 q3_95_new q4_95 q1_96 q2_96 q3_96 q4_96 q1_97 q2_97 q3_97 q4_97 q1_98 q2_98 q3_98 q4_98 hmo men young old /rl; where enrbaseflag = 1 and validindex = 1 and sameday = 0 and medicare = 0 and malignant = 0; output out = mother.propensity3 p = score ; run;
Propensity Output
Obs PATIENT score 1 2909 0.57475 2 3438 0.328993 3841 0.013244 5674 0.484115 5734 0.068926 8573 0.117807 10210 0.346928 13056 0.097379 13376 0.3735010 16026 0.1563511 16865 0.0610612 16949 0.10568
25
E1
Pr(E1)=x
Pr(E1)=x
E2(sample) E2 (residual)
Choose from E2 a sample that matches E1 in size.
Matching on Propensity
26
E1
Pr(E1) = 0.5
Pr(E1) = 0.5
E2At every level of propensity in the constructed cohorts, Pr(E1) = 0.5. Therefore, treatment is uncorrelated with propensity, and you can collapse all the propensity-matched groups together to form a cohort in which all covariate patterns are uncorrelated with treatment, and there will be no confounding bias.
Matching on Propensity
Stratum I II III IV V
Stratum I II III IV V
Duragesic and Long-Acting Opioids
Duragesic LA Opioids
N 504 2,201
65+ years 29% 10%
Male 35% 49%
Periph Vasc Disease 4% 1%
Sx of Abd or Pelvis 18% 10%
> 2 hospitalztns 6 mo 9% 3%
30 days NonRx Costs $1,136 $746
Propensity-Matched Cohorts
Duragesic LA Opioids
N 478 478
65+ years 26% 25%
Male 36% 33%
Periph Vasc Disease 4% 3%
Sx of Abd or Pelvis 17% 18%
> 2 hospitalztns 6 mo 8% 8%
30 days NonRx Costs $1,084 $1,043
Pharmacoepidemiol Drug Saf. 2005 Jul;14(7):465-76.
Do Statins Affect Risk of AMI?
The purpose of the study was to assess whether statins affect the risk of risk of acute myocardial infarction (AMI)
Strong predictors for statin use that affect risk of AMI
How to design an observational study?
Note: we would not ordinarily use observational data for efficacy questions, but this serves as a suitable test case because there is a known gold standard
+Risk Factors: age (45M, 55F), diabetes, smoking, HTN, low HDL,family history of premature CHD-Risk Factor: high HDL
Risk Category LDL to initiate drug Tx
LDL Goal of drug Tx
No CHD and <2 Risk Factors
³ 190 <160
No CHD and ³ 2 Risk Factors
³ 160 <130
With CHD >130 £100
NCEP ATP II guidelines (1993)
Good Clinical Practice Creates Confounding
Gold Standard for the Effect of StatinsC
AR
E T
rial R
esu
lts
Sack
s FM
, et
al
N E
ngl J M
ed
. 19
96;3
35:1
001-9
Data Source
• Fallon Community Health Plan• Central Massachusetts HMO• ~200,000 members• Claims Data available on:
– Enrollment (age, sex, date)– Ambulatory care visits– Hospitalization– Pharmacy dispensings (drug & quantity)– Laboratory tests (tests & results)
Patient Entry, Analytic Sequence
1993 1994 1995 1996 1997 1998 1999
1 of 9 Blocks
1) Apply eligibility criteria• FCHP member for at least 1 year• At least one physician visit in last year• LDL, HDL, TG levels in last 6 months• At least one physician visit in cohort accrual block• No PAD diagnosis before index date• Not current statin user
2) Estimate propensity score (statin initiation)3) Match statin initiators with non-initiators4) Repeat for all blocks of time5) Follow matched groups for diagnosis of MI
2nd/94
~35,000Members
All Fallon members with any LDL > 130 mg/dl
Require 1 yearEnrollment
Current Statin Users (1501)
Statin Initiators, Eligible (77)
Statin Initiators, Not Eligible (34)
Non Statin Users, Not Eligible (24,799)
Non Statin Users, Eligible (9,639)
Month of1/1/94
PropensityScoreMatching
Total subjects in cohort (36,050)
“Typical” Statin Initiator and Non-Initiator Variable Init iators Non- Init iators P-value Lipid-related labs 26.04 13.58 <0.0001 Different Prescription Drugs 5.02 2.88 <0.0001 LDL level (mg/dl) 180.25 155.08 <0.0001 Triglyceride level (mg/dl) 202.66 166.91 <0.0001 Cardiovascular-related Prescription Drugs 0.59 0.26 <0.0001 Cardiovascular-related vis its 1.11 0.27 <0.0001 Age (years) 62.04 58.02 <0.0001 Physician Visits 7.69 6.31 <0.0001 Ischemic Heart Disease 20.27% 5.57% <0.0001 HDL level (mg/dl) 43.29 46.60 <0.0001 Cardiovascular-related diagnoses 0.28 0.12 <0.0001 Cardiovascular-related hospitalizations 0.56 0.13 <0.0001 MI 11.58% 2.92% <0.0001 Angina 11.92% 3.14% <0.0001 Unstable Angina 10.59% 2.22% <0.0001 Smoking 25.80% 18.10% <0.0001 Hypertension 19.96% 12.96% <0.0001 Labs 10.48 10.74 0.0074 Hospitalizations 0.22 0.08 <0.0001 Male 53.14% 47.61% <0.0001
111% (46%-204%)Risk Increase
Statin Non-Initiators
Statin Initiators
Months of Follow-Up
Cu
mula
tive Inci
den
ceMI Outcome (Unmatched)
HR=2.11 (1.46-3.04)
Calculate Propensity Score
Predict treatment Statin initiation vs. not In each 6-month period of cohort accrual
Using baseline covariates Obtain fitted values from regression Fitted value for each study subject is the Propensity
Score
Construct Rich Model
More than 8 events per covariate leads to unbiased estimates
Many more persons exposed to drug of interest than study outcomes
In Drug Safety studies, usually the outcome is rare
Therefore can control for more covariates when exposure is dependent variable than when outcome is
Cepeda S, et al. Am J Epidemiol 2003;158:280-287.
Propensity modeling permits the introduction of many, many covariates
*build model for 9501;proc logistic descending data=new1; model statin = male smok obes age9501 ang9501 usa9501 chf9501 isch9501 ath9501 cva9501 usa9501 mi9501 olmi9501 htn9501 tia9501 afib9501 ascv9501 hth9501 ost9501 cvs9501 htdx9501 circ9501 cond9501 rvsc9501 hhd9501 dysr9501 hrt9501 ns9501
ins9501 diab9501 skca9501 depr9501 adj9501 schz9501 deb9501 rheu9501 days9501 lres9501 tres9501 hres9501 hbac9501 cvhp9501 ekg9501 cvrx9501 cvvs9501 llab9501 lab9501 cvdg9501 hosp9501 rx9501 vist9501 diag9501; output out=psmodel pred=PROPSCORE; run;
Propensity Regression Parameter Estimates
Obs ID STATIN PROPSCORE 1 1909 0 0.57475 2 2438 0 0.338993 3841 0 0.013244 4674 0 0.484115 4734 0 0.068926 5573 0 0.117807 6210 1 0.346928 7056 1 0.097379 7376 1 0.3735010 8026 1 0.1563511 8865 1 0.0610612 9949 1 0.10568...
Output File – Propensity Scores
45
Obs ID STATIN PROPSCORE 1 1909 0 0.57475 2 2438 0 0.338993 3841 0 0.013244 4674 0 0.484115 4734 0 0.068926 5573 0 0.117807 6210 1 0.346928 7056 1 0.097379 7376 1 0.3735010 8026 1 0.1563511 8865 1 0.0610612 9949 1 0.10568...
Match exposed to non-exposed according to Propensity Score values
Obs ID STATIN PROPSCOR 1 1909 0 0.57475 2 2438 0 0.338993 3841 0 0.013244 4674 0 0.484115 4734 0 0.068926 5573 0 0.117807 6210 1 0.346928 7056 1 0.097379 7376 1 0.3735010 8026 1 0.1563511 8865 1 0.0610612 9949 1 0.10568...
Match exposed to non-exposed according to Propensity Score values
Obs ID STATIN PROPSCOR 1 1909 0 0.57475 2 2438 0 0.338993 3841 0 0.013244 4674 0 0.484115 4734 0 0.068926 5573 0 0.117807 6210 1 0.346928 7056 1 0.097379 7376 1 0.3735010 8026 1 0.1563511 8865 1 0.0610612 9949 1 0.10568...
Match exposed to non-exposed according to Propensity Score values
Propensity Score Distribution (After Matching)
0
50
100
150
200
250
300
350
400
450
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
Propensity Score
Nu
mb
er o
f P
erso
ns
Statin Initiators Non-Initiators
Balance Achieved by Matching Matched at 0.01 Propensity Score Variable Initiators Non-Initiators N=2901 N=2901 P-value Lipid-related labs 24.90 24.64 0.4987 Different Prescription Drugs 4.57 4.54 0.7639 LDL level (mg/dl) 177.84 177.58 0.7837 Triglyceride level (mg/dl) 200.34 200.50 0.9626 Cardiovascular-related Prescription Drugs 0.51 0.51 0.9367 Cardiovascular-related visits 0.74 0.83 0.1249 Age (years) 61.47 61.68 0.5030 Physician Visits 7.25 7.27 0.8732 Ischemic Heart Disease 15.13% 15.48% 0.7428 HDL level (mg/dl) 43.51 43.55 0.9079 Cardiovascular-related diagnoses 0.21 0.23 0.2145 Cardiovascular-related hospitalizations 0.40 0.39 0.7929 MI 7.89% 8.69% 0.3169 Angina 8.51% 8.72% 0.8151 Unstable Angina 7.14% 7.31% 0.8393 Smoking 23.85% 24.27% 0.7355 Hypertension 16.58% 17.99% 0.1649 Labs 10.45 10.48 0.7874 Hospitalizations 0.16 0.16 0.6915 Male 52.33% 52.09% 0.8747
Only 1 of 52 variables sig. different at P<0.05
31% (7%-48%)Risk Reduction
Statin Non-Initiators
Statin Initiators
Months of Follow-Up
Cu
mu
lati
ve I
ncid
en
ce
MI Outcome (After Matching)
HR=0.69 (0.52-0.93)
Interpreting Propensity Coefficients
53
Odds Ratio for Non-Statin Lipid Lowering Therapy as a Predictor of Statin Initiation
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
9402 9501 9502 9601 9602 9701 9702 9801 9802
Matching Block (Year and Half)
Od
ds
Ra
tio
LDL as a Predictor of Statin Initiation
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
190+175-190160-175145-160130-145<130
LDL Category
Od
ds
Ra
tio
fo
r In
itia
tio
n
Age as a Predictor of Statin Initiation
0.00
2.00
4.00
6.00
8.00
10.00
12.00
75+65-7555-6545-5534-45<35
Age Category
Od
ds
Ra
tio
fo
r In
itia
tio
n
When Is the Model Sufficient?
57
Early Matching Results
New Variables Suggested post hoc for the Propensity Score
Cardiac Disease Cardiovascular
Diagnoses Hospitalizations Outpatient visits Medications
EKGs Number of labs Number of lipid labs
Other Causes of “Medicalization”
Schizophrenia Adjustment Disorder Depression Non-Skin CA Skin CA Debility Rheumatic Disease
Imbalance on Non-Included Variables
NIVs are Predictors of Statin Initiation
New Ranking of Predictors# Variable c-stat OR P-value
Lipid lab tests 0.82 1.14 <0.011 Rxs 0.72 1.20 <0.012 LDL 0.71 1.02 <0.013 Triglycerides 0.61 1.00 <0.01
CVRxs 0.60 1.70 <0.01CVVisits 0.59 1.25 <0.01
4 Age 0.58 1.02 <0.015 Visits 0.57 1.04 <0.016 Ischemic Heart Ds. 0.57 4.20 <0.017 HDL 0.57 0.98 <0.01
CVDiagnoses 0.56 1.94 <0.01CVHospitalizations 0.55 1.23 <0.01
8 MI 0.54 4.63 <0.019 Angina 0.54 3.88 <0.0110 Unstable Angina 0.54 5.13 <0.0111 Smoking 0.54 1.55 <0.0112 Hypertension 0.54 1.77 <0.01
Balance on New Variables Original Matching Revised Matching Variable Statin Init.
(N=3579) Non-Init. (N=3579)
P-value Statin Init. (N=2901)
Non-Init. (N=2901)
P-value
Lipid lab tests 26.3 16.4 <0.01 24.9 24.6 0.50 CVRxs 0.51 0.53 0.09 0.51 0.51 0.94 CVVisits 0.96 0.54 <0.01 0.74 0.83 0.12 CVDiagnoses 0.25 0.17 <0.01 0.21 0.23 0.21 CVHospitalizations 0.46 0.31 <0.01 0.40 0.39 0.79 Lab tests 10.3 11.3 <0.01 10.5 10.5 0.79 EKGs 0.48 0.48 0.77 0.48 0.51 0.35 Schizophrenia 2.2% 3.6% 0.03 2.1% 2.2% 0.99 Depression 2.4% 2.9% 0.14 1.9% 2.7% 0.05 Non-skin cancer 5.3% 5.7% 0.47 5.2% 5.5% 0.60 Adjustment dis. 2.7% 3.6% 0.03 2.8% 3.1% 0.59 Skin cancer 2.5% 2.7% 0.71 2.2% 2.4% 0.73 Debility 1.7% 1.5% 0.78 1.6% 1.3% 0.44 Rheumatic dis. 1.3% 1.8% 0.07 1.2% 1.7% 0.19
Thank You!