Propensity Scores
October 2012
Alexander M. Walker MD, DrPHExtensive parts of this presentation incorporate the work of John D. Seeger, PharmD, DrPH
“In mathematics, you don't understand things. You just get used to them.”
- John von Neumann1903-1957
Research Goal Compare two treatments with respect to a health or
economic outcome “Counterfactual” ideal
If the same people had received B instead of A, how would their outcomes have differed?
What is achievable: “similar” not “same” Comparable treatment groups … insofar as you can tell!
4
Pictures for Confounding
Comparison of Heterogeneous Groups
5
E1 E2
6
E1 E2
Internal Composition May Differ
7
Affectedindividuals
50%
15%
50%
15%
E1 E2
Risks that Depend on Subgroup Status
8
These differences in risk are due to the covariate
structure of compared
populations, not to the differential effects of
E1 and E2
E1 E2
Internal Risk Factor Heterogeneity Creates an Differences in Group Risk
9
Propensity Scores to Create Populations with Similar Covariate Structure
10
E1 E2Covariate Heterogeneity
E1 has more Yellow
E2 has more Gray
11
E2
E1E1
E2Gray predicts
E2Yellow predicts
E1
Covariate Status as a Predictor of Treatment
12
Propensity Scores PS is the predicted probability of treatment, given
all the covariates Matching on the PS creates study populations that
have balance on the covariates Perfect for a single, dichotomous covariate Not perfect, but very good for multiple covariates
13
E1 E2
Propensity for Covariate PatternsThink of orange and green as two distinct covariate patterns that have the same predicted Pr(E1).
Pr(E1)=x
Pr(E1)=x
14
E1 E2
Pr(E1)=x
Pr(E1)=x
Gathering subjects with identical propensity puts all individuals with covariate patterns orange and green into the same stratum.
Conditioning on Propensity Permits Unconfounded Comparisons
At a given propensity level, there is no association between treatment and covariate patterns.
Formal Expression
Propensity(x) P(T=1|x) = E(T|x)The propensity associated with level x of the covariate X is the probability that treatment is present (equivalently, is “B” as opposed to “A”), given level x, and this is in turn equal to the expected value of treatment, given x.Note that the definition does not specify the parametric form of the Propensity(x) . The examples in this talk use a logistic function; others -- including nonparametric functions -- are also used.Notation. A single capital letter denotes a variable, a single lower case letter denotes a particular value for that variable.
Probability Calculus
Under propensity matching, how do x (covariate status) and t (treatment status) relate to one another?
1.Pr( x, t | p ) = Pr( x | p ) Pr( t | x, p ) Probability Theory
2.Pr( t | x, p ) = Pr( t | p ) p incorporates all information about t
that is in x Pr( x, t | p ) = Pr( x | p ) Pr( t | p )
Pr( x, t | p ) = Pr( x | p ) Pr( t | p )
Given a particular value of the propensity score variable, that is at P=p, the covariates X and T are uncorrelated.
At particular levels of P individually and therefore collectively as well (“conditionally on P), X cannot confound the association between T and any outcome.
18
Matching on Propensity Scores
Propensity Matching: Method
Identify candidate predictors of treatment B v A Perform a logistic regression of B v A Obtain from the regression a “predicted” probability of B v
A Sort all members of A and B according to this propensity Match A patients to B patients on the propensity
Duragesic and Long-Acting Opioids
Duragesic LA OpioidsN 504 2,20165+ years 29% 10%Male 35% 49%Periph Vasc Disease 4% 1%Sx of Abd or Pelvis 18% 10%> 2 hospitalztns 6 mo 9% 3%30 days NonRx Costs $1,136 $746
Straightforward Regressionproc logistic data = mother.propensity2 descending; model DuragesicUser = DischCostIndex EncCostIndex RxCostIndex OtherCostIndex RxCostPrior1 OtherCostPrior1 AnyRx OneDisch TwoDisch ThreePlusDisch AnyICD443 AnyICD719 AnyICD724 AnyICD787 AnyICD789 q3_95_new q4_95 q1_96 q2_96 q3_96 q4_96 q1_97 q2_97 q3_97 q4_97 q1_98 q2_98 q3_98 q4_98 hmo men young old /rl; where enrbaseflag = 1 and validindex = 1 and sameday = 0 and medicare = 0 and malignant = 0; output out = mother.propensity3 p = score ; run;
Propensity Output
Obs PATIENT score 1 2909 0.57475 2 3438 0.328993 3841 0.013244 5674 0.484115 5734 0.068926 8573 0.117807 10210 0.346928 13056 0.097379 13376 0.3735010 16026 0.1563511 16865 0.0610612 16949 0.10568
23
E1
Pr(E1)=x
Pr(E1)=x
E2(sample) E2 (residual)
Choose from E2 a sample that matches E1 in size.
Matching on Propensity
24
E1
Pr(E1) = 0.5
Pr(E1) = 0.5
E2At every level of propensity in the constructed cohorts, Pr(E1) = 0.5. Therefore, treatment is uncorrelated with propensity, and you can collapse all the propensity-matched groups together to form a cohort in which all covariate patterns are uncorrelated with treatment, and there will be no confounding bias.
Matching on Propensity
Stratum I II III IV V
Stratum I II III IV V
Duragesic and Long-Acting Opioids
Duragesic LA OpioidsN 504 2,20165+ years 29% 10%Male 35% 49%Periph Vasc Disease 4% 1%Sx of Abd or Pelvis 18% 10%> 2 hospitalztns 6 mo 9% 3%30 days NonRx Costs $1,136 $746
Propensity-Matched Cohorts
Duragesic LA OpioidsN 478 47865+ years 26% 25%Male 36% 33%Periph Vasc Disease 4% 3%Sx of Abd or Pelvis 17% 18%> 2 hospitalztns 6 mo 8% 8%30 days NonRx Costs $1,084 $1,043
Pharmacoepidemiol Drug Saf. 2005 Jul;14(7):465-76.
Do Statins Affect Risk of AMI? The purpose of the study was to assess whether
Statins affect the risk of risk of acute myocardial infarction (AMI)
Strong predictors for statin use that affect risk of AMI
How to design an observational study? Note: we would not ordinarily use observational
data for efficacy questions, but this serves as a suitable test case because there is a known gold standard
+Risk Factors: age (45M, 55F), diabetes, smoking, HTN, low HDL,family history of premature CHD-Risk Factor: high HDL
Risk Category LDL to initiate drug Tx
LDL Goal of drug Tx
No CHD and <2 Risk Factors
³190 <160
No CHD and ³2 Risk Factors
³ 160 <130
With CHD >130 £100
NCEP ATP II guidelines (1993)
Good Clinical Practice Creates Confounding
Gold Standard for the Effect of StatinsCA
RE Tr
ial R
esul
tsSa
cks F
M, e
t al
N E
ngl J
Med
. 199
6;33
5:10
01-9
Data Source
• Fallon Community Health Plan• Central Massachusetts HMO• ~200,000 members• Claims Data available on:
– Enrollment (age, sex, date)– Ambulatory care visits– Hospitalization– Pharmacy dispensings (drug & quantity)– Laboratory tests (tests & results)
Patient Entry, Analytic Sequence
1993 1994 1995 1996 1997 1998 1999
1 of 9 Blocks
1) Apply eligibility criteria• FCHP member for at least 1 year• At least one physician visit in last year• LDL, HDL, TG levels in last 6 months• At least one physician visit in cohort accrual block• No PAD diagnosis before index date• Not current statin user
2) Estimate propensity score (statin initiation)3) Match statin initiators with non-initiators4) Repeat for all blocks of time5) Follow matched groups for diagnosis of MI
2nd/94
~35,000Members
All Fallon members with any LDL > 130 mg/dl
Require 1 yearEnrollment
Current Statin Users (1501)
Statin Initiators, Eligible (77)
Statin Initiators, Not Eligible (34)
Non Statin Users, Not Eligible (24,799)
Non Statin Users, Eligible (9,639)
Month of1/1/94
PropensityScoreMatching
Total subjects in cohort (36,050)
“Typical” Statin Initiator and Non-Initiator Variable Init iators Non- Initiators P-value Lipid-related labs 26.04 13.58 <0.0001 Different Prescription Drugs 5.02 2.88 <0.0001 LDL level (mg/dl) 180.25 155.08 <0.0001 Triglyceride level (mg/dl) 202.66 166.91 <0.0001 Cardiovascular-related Prescription Drugs 0.59 0.26 <0.0001 Cardiovascular-related vis its 1.11 0.27 <0.0001 Age (years) 62.04 58.02 <0.0001 Physician Visits 7.69 6.31 <0.0001 Ischemic Heart Disease 20.27% 5.57% <0.0001 HDL level (mg/dl) 43.29 46.60 <0.0001 Cardiovascular-related diagnoses 0.28 0.12 <0.0001 Cardiovascular-related hospitalizations 0.56 0.13 <0.0001 MI 11.58% 2.92% <0.0001 Angina 11.92% 3.14% <0.0001 Unstable Angina 10.59% 2.22% <0.0001 Smoking 25.80% 18.10% <0.0001 Hypertension 19.96% 12.96% <0.0001 Labs 10.48 10.74 0.0074 Hospitalizations 0.22 0.08 <0.0001 Male 53.14% 47.61% <0.0001
111% (46%-204%)Risk Increase
Statin Non-Initiators
Statin Initiators
Months of Follow-Up
Cum
ulat
ive
Incid
ence
MI Outcome (Unmatched)
HR=2.11 (1.46-3.04)
Calculate Propensity Score Predict Treatment
Statin Initiation vs Not In Each 6-month Period of Cohort Accrual
Using Baseline Covariates Obtain Fitted Value From Regression Fitted Value is the Propensity Score
Construct Rich Model More than 8 events
per covariate leads to unbiased estimates
Many more persons exposed to drug of interest than study outcomes
In Drug Safety studies, usually the outcome is rare
Therefore can control for more covariates when exposure is dependent variable than when outcome is
Cepeda S, et al. Am J Epidemiol 2003;158:280-287.
*build model for 9501;proc logistic descending data=new1; model statin = male smok obes age9501 ang9501 usa9501 chf9501 isch9501 ath9501 cva9501 usa9501 mi9501 olmi9501 htn9501 tia9501 afib9501 ascv9501 hth9501 ost9501 cvs9501 htdx9501 circ9501 cond9501 rvsc9501 hhd9501 dysr9501 hrt9501 ns9501
ins9501 diab9501 skca9501 depr9501 adj9501 schz9501 deb9501 rheu9501 days9501 lres9501 tres9501 hres9501 hbac9501 cvhp9501 ekg9501 cvrx9501 cvvs9501 llab9501 lab9501 cvdg9501 hosp9501 rx9501 vist9501 diag9501; output out=psmodel pred=PROPSCORE; run;
Propensity Regression Parameter Estimates
Obs ID STATIN PROPSCORE 1 1909 0 0.57475 2 2438 0 0.338993 3841 0 0.013244 4674 0 0.484115 4734 0 0.068926 5573 0 0.117807 6210 1 0.346928 7056 1 0.097379 7376 1 0.3735010 8026 1 0.1563511 8865 1 0.0610612 9949 1 0.10568...
Output File – Propensity Scores
43
Obs ID STATIN PROPSCOR 1 1909 0 0.57475 2 2438 0 0.338993 3841 0 0.013244 4674 0 0.484115 4734 0 0.068926 5573 0 0.117807 6210 1 0.346928 7056 1 0.097379 7376 1 0.3735010 8026 1 0.1563511 8865 1 0.0610612 9949 1 0.10568...
Output File – Propensity Scores
Propensity Score Distribution (After Matching)
050
100150200250300350400450
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
Propensity Score
Num
ber o
f Per
sons
Statin Initiators Non-Initiators
Balance Achieved by Matching Matched at 0.01 Propensity Score Variable Initiators Non-Initiators N=2901 N=2901 P-value Lipid-related labs 24.90 24.64 0.4987 Different Prescription Drugs 4.57 4.54 0.7639 LDL level (mg/dl) 177.84 177.58 0.7837 Triglyceride level (mg/dl) 200.34 200.50 0.9626 Cardiovascular-related Prescription Drugs 0.51 0.51 0.9367 Cardiovascular-related visits 0.74 0.83 0.1249 Age (years) 61.47 61.68 0.5030 Physician Visits 7.25 7.27 0.8732 Ischemic Heart Disease 15.13% 15.48% 0.7428 HDL level (mg/dl) 43.51 43.55 0.9079 Cardiovascular-related diagnoses 0.21 0.23 0.2145 Cardiovascular-related hospitalizations 0.40 0.39 0.7929 MI 7.89% 8.69% 0.3169 Angina 8.51% 8.72% 0.8151 Unstable Angina 7.14% 7.31% 0.8393 Smoking 23.85% 24.27% 0.7355 Hypertension 16.58% 17.99% 0.1649 Labs 10.45 10.48 0.7874 Hospitalizations 0.16 0.16 0.6915 Male 52.33% 52.09% 0.8747
Only 1 of 52 variables sig. different at P<0.05
31% (7%-48%)Risk Reduction
Statin Non-Initiators
Statin Initiators
Months of Follow-Up
Cum
ulat
ive
Incid
ence
MI Outcome (After Matching)
HR=0.69 (0.52-0.93)
Interpreting Propensity Coefficients
49
Odds Ratio for Non-Statin Lipid Lowering Therapy as a Predictor of Statin Initiation
00.20.40.60.8
11.21.41.6
9402 9501 9502 9601 9602 9701 9702 9801 9802
Matching Block (Year and Half)
Odd
s R
atio
LDL as a Predictor of Statin Initiation
0.00
2.00
4.00
6.00
8.00
10.00
12.00
14.00
190+175-190160-175145-160130-145<130
LDL Category
Odd
s R
atio
for I
nitia
tion
Age as a Predictor of Statin Initiation
0.00
2.00
4.00
6.00
8.00
10.00
12.00
75+65-7555-6545-5534-45<35
Age Category
Odd
s R
atio
for I
nitia
tion
When Is the Model Sufficient?
53
Early Matching Results
New Variables Suggested post hoc for the Propensity Score
Cardiac Disease Cardiovascular
Diagnoses Hospitalizations Outpatient visits Medications
EKGs Number of labs Number of lipid labs
Other Causes of “Medicalization” Schizophrenia Adjustment Disorder Depression Non-Skin CA Skin CA Debility Rheumatic Disease
Imbalance on Non-Included Variables
NIVs are Predictors of Statin Initiation
New Ranking of Predictors# Variable c-stat OR P-value
Lipid lab tests 0.82 1.14 <0.011 Rxs 0.72 1.20 <0.012 LDL 0.71 1.02 <0.013 Triglycerides 0.61 1.00 <0.01
CVRxs 0.60 1.70 <0.01CVVisits 0.59 1.25 <0.01
4 Age 0.58 1.02 <0.015 Visits 0.57 1.04 <0.016 Ischemic Heart Ds. 0.57 4.20 <0.017 HDL 0.57 0.98 <0.01
CVDiagnoses 0.56 1.94 <0.01CVHospitalizations 0.55 1.23 <0.01
8 MI 0.54 4.63 <0.019 Angina 0.54 3.88 <0.0110 Unstable Angina 0.54 5.13 <0.0111 Smoking 0.54 1.55 <0.0112 Hypertension 0.54 1.77 <0.01
Balance on New Variables Original Matching Revised Matching Variable Statin Init.
(N=3579) Non-Init. (N=3579)
P-value Statin Init. (N=2901)
Non-Init. (N=2901)
P-value
Lipid lab tests 26.3 16.4 <0.01 24.9 24.6 0.50 CVRxs 0.51 0.53 0.09 0.51 0.51 0.94 CVVisits 0.96 0.54 <0.01 0.74 0.83 0.12 CVDiagnoses 0.25 0.17 <0.01 0.21 0.23 0.21 CVHospitalizations 0.46 0.31 <0.01 0.40 0.39 0.79 Lab tests 10.3 11.3 <0.01 10.5 10.5 0.79 EKGs 0.48 0.48 0.77 0.48 0.51 0.35 Schizophrenia 2.2% 3.6% 0.03 2.1% 2.2% 0.99 Depression 2.4% 2.9% 0.14 1.9% 2.7% 0.05 Non-skin cancer 5.3% 5.7% 0.47 5.2% 5.5% 0.60 Adjustment dis. 2.7% 3.6% 0.03 2.8% 3.1% 0.59 Skin cancer 2.5% 2.7% 0.71 2.2% 2.4% 0.73 Debility 1.7% 1.5% 0.78 1.6% 1.3% 0.44 Rheumatic dis. 1.3% 1.8% 0.07 1.2% 1.7% 0.19
Thank You!