Designs to determine the impact of Ab resistance

How do we correctly measure the outcomes of antibiotic resistance?

Measuring impact of resistance -methodological issues :

• Which study designs are appropriate?

• Case control vs. Cohort studies.

• what is the right control? • What are we measuring?

• Confounders. What do we have to adjust for?

• Defining the outcomes?

Study designs used to define the impact of Ab resistance:

• Descriptive studies

• Case-control studies

• Cohort studies

• Meta-analysis

• Mathematical models

Descriptive studiesEmergence of MDR19A pneumococcal strain…

Pichichero JAMA 2007

• What type of a study is this?• Prospective cohort study.

• Cohort study: – A cohort – a group of people who share a

common experience / condition

– Compare exposure of interest and follow to measure outcomes in exposed vs. non-exposed.

Emergence of MDR19A / Pichichero JAMA 2007

Trend in time:– More non-VT– More non-VT resistance– More total Sp

resistance– More 19A among non-

VT

1816

AOM

375 AOM

included

211

tympanocentesis

59

S. pneumonia

9

19A strain

Emergence of MDR19A / Pichichero JAMA 2007

• What are the bad outcomes of the resistant strain?

• Can we determine that the bad outcome is a result of the resistance?

19A MDR S. pneumoniae; an emerging pathogen

• Resistant to all FDA approved antibiotics (for AOM).• Susceptible only to FQ.• Caused bad outcomes (compared to what?)

• But this is just a case series…• Can we conclude that the bad outcomes are related to

it being resistant?• Can we conclude that due to the emergence of this

MDR clone we will observe worse outcomes?

• Yet, it is an important descriptive study.

CA-MRSA with no identified predisposing risk / Herold et al. JAMA 1998

• Design: Retrospective review of medical records

• Objective: To determine whether community-acquired MRSA infections in children with no identified predisposing risks are increasing and to define the spectrum of disease associated with MRSA isolation.

• Case series and descriptive studies may be of great importance.

• Controlled studies should follow.

Study designs used to define the impact of Ab resistance:

• Descriptive studies

• Case-control studies

• Cohort studies

• Meta-analysis

• Mathematical models

Case-control / Cohort studiesconfusing terms

Health / economical outcomes

Resistance

Risk factors: Ward

Time

Time in hospital comorbidities

Ab rx

IC measures

OutcomeExposure

OutcomeExposure

Vs.

Who?

Resistant bacteria compared to what???

what is the counterfactual?• The control group

– Patients infected by resistant strains (cases) vs. susceptible strains (controls)

– Assesses the independent impact of the acquisition of a resistance determinant.

– Patients infected by resistant strains (cases) vs. patients w/o resistant strains (controls)

– Assesses the burden of having resistant infection

Two different questions!

0

10

20

30

40

50

60

70

1a 2a 1b 2b

MRSA

MSSA

Confounders

Health / economical outcomes

Resistance

Risk factors: Ward

Time

LOS

Comorbidities

Ab rx

IC measures

OutcomeExposure

Confounders: direction of bias they induce

Time

• Bias in either direction

0123456789

10

# c

ase

s

1 3 5 7 9 11

Time (months)

VRE

VSE

LOS / comorbidities

• Bias towards worse outcome (away from the null)

VRE Mortality

Very sick (comorbidities)

Which outcomes?Defining the outcomes

• Mortality– Distinguish all cause mortality from disease-

specific mortality– In-hospital vs. post-discharge mortality

• Morbidity - more difficult to define– Clinical failure– Length of stay in hospital– Need for surgery

• Economic outcomes (hospital costs vs. charges vs. resource used)

Health & Economic outcomes of VRE / Carmeli Arch Inter Med 2002

• Design: Retrospective Matched cohort study

• 233 cases of VRE infections

• 647 controls matched by (randomly picked from the cohort):– Place (Hospital ward)– Time (calendar day)– Duration of hospital stay till infection

• Outcomes– Mortality - ICU admission– LOS - Surgery– Hospital Costs - discharge to institution

• To control for confounding:– Propensity score (RF for being VRE)– CVD, ID, DM, transplantation, MRSA, C. difficile,

Ab Rx: 3rd cephalosporins

Health & Economic outcomes of VRE / Carmeli Arch Inter Med 2002

Health & Economic outcomes of VRE / Carmeli Arch Inter Med 2002

Outcome Cases

(n=233)

Controls

(n=647)

RR (95%ci)

Attributable to VRE

P-value

Mortality % 17 6 2.13

(1.05-4.37)

6 .4

LOS mean d 15.1 8.5 1.73

(1.43-2.1)

6.2 <.001

Cost mean $ 52449 31915 1.4

(1.26-1.59)

12766 <.001

Surgery % 18 10 2.74

(1.52-4.92)

10 .001

ICU admission %

25 14 3.47

(1.75-6.85)

11 <.001

Discharge to institution %

51 35 2.01

(1.34-3.02)

16 .001

But they did not compare to VSE

• What is the right control?

• What does this tell us?

• What would be the counterfactual of not having VRE?

0

10

20

30

40

50

60

70

1a 2a 1b 2b

VRE

VSE

Study designs used to define the impact of Ab resistance:

• Descriptive studies

• Case-control studies

• Cohort studies

• Meta-analysis

• Mathematical models

Meta-analysis“an objective review of the literature”

• A statistical method for combining information from independent studies to derive an overall estimate (a summary estimate).

• Great appeal: response to treatments may vary among individuals and different studies

• Main problems:– Publication bias– Varying quality of the studies– Are we not adding apples and oranges? Did all

studies measure the outcome similarly? Did they adjust for the same confounders?

Comparison of mortality associated with MRSA vs. MSSA / Cosgrove et al. CID 2002

• Included studies that did not exclude CA-MRSA.

• Most studies did not adjust for LOS.

• Subset of 11 studies with adjustment to LOS - no different results.

Meta-analysis VRE vs. VSE / DiazGranados et al. CID 2005

• Limitations:• Most studies retrospective

(leading to inaccurate/incomplete data)

• Most studies in pre-linezolid, dalfopristin-quinupristin (Synercid) era (no appropriate treatment for VRE): external validity for the current environment?

• Did not adjust for LOS, but only for comorbidities

Meta-analysis PNSSP vs. PSSP CID 2006 Tleyjeh et al

• Raised a lot of controversy.• In contrast to “conventional

wisdom”

• Same criticism: not all studies adjusted for confounding factors.

• How was quality of the studies assessed?

• Resistance & virulence not linked in PRSP

•Resistance & virulence not linked in PRSP

•Concordant vs. non-concordant Rx. analysis resulted in same RR.