AIFA EUROPEAN CONFERENCE ON CLINICAL RESEARCH FOR DECISION MAKING

1

AIFA EUROPEAN CONFERENCE ON CLINICAL RESEARCH FOR DECISION MAKING

External validity of clinical trials and transferability of

their results to medical practice

March 30, 2007

Luigi Pagliaro, Professor of Medicine, University of Palermo, Palermo, Italy

2

External validity of clinical trials and transferability of their results to medical

practice

1.External validity or applicability

2.Applicability: information needed

3.Problems of applicability

4.Help for evaluating applicability

Final remarks

References

3

In an RCT:

Internal validity refers to the comparability between the intervention (I) and the control group (C), and to the care for minimizing the risk of bias External validity refers to the availability of information to guide the applicability of the trial results to the target population in a specific setting (1)

I. group C. group

Trial results

Target population and

setting

External validity,

applicability

External validity or applicability

4

There is consensus that RCTs with adequate internal validity most reliably assess the efficacy of clinical treatments (2) The statistical efficiency of RCTs is maximized in the megatrials that enroll very large numbers of pts belonging to different categories to detect “moderate but still worthwhile treatment effects” on major outcomes (3)

A fundamental idea underlying the logic of the megatrials is that the direction (although not the size) of a treatment effect is similar throughout the different categories of pts (4)

1. External validity or applicability

5

From 5

“The modest benefit ascribed to many treatments in clinical trials can be misleading because modest average effects may reflect a mixture of substantial benefit for some, little benefit for many, and harm for a few”


6

RCT

Megatrials are the best method to answer the question: “can this treatment work”? (6)

Individual pts

……But the applicability of the treatment to the practice must take into account the characteristics of the individual patients (7)


7


practice

1.External validity or applicability

2.Applicability: information needed



Final remarks

References

8

1. Characteristics of randomised

pts

Eligibility; inclusion & exclusion criteria; ratio of randomised pts to eligible non-randomised pts; baseline risk; stage in the natural history and severity of disease; comorbidity

2. Setting

Recruitment from primary, secondary or tertiary care; characteristics & expertise of participating centres and clinicians

2 Applicability: information needed (from 1, modified)

9

3. Differences of trial

protocol vs practice

Doses and administration of the trial treatment; adequacy of non- trial treatments; therapeutic or diagnostic advances since the trial

4. Outcome measures

and follow-up

Clinical relevance of the end points and effect size; effect of the intervention on the most relevant components of a combined end point

5. Adverse effects of treatment

Rates of adverse effects and of treatment discontinuations; were pts at risk of complications excluded from the trial?


10

Lack of this information is one of the main reasons for the underuse in practice of treatments that were beneficial in trials (1), and/or of the use of a treatment in the “wrong” patients,sometimes with harmful consequences

Many RCTs don’t report the information needed to evaluate the applicability of their results to the practice, and/or they exclude specific subgroups of pts that could benefit from the treatment (8-11)


11


practice


2. Applicability: information needed


4. Help for evaluating applicability

Final remarks

References

12

RCTs lacking information needed to apply the

results in groups of patients; some examples:

Treatment for patients with hypertension and comorbidities

Peg-interferon in HCV-related cirrhosis without knowledge of esophagogastric varices

Tolvaptan for hyponatremia in cirrhosis

Warfarin for patients requiring caution

3. Problems of applicability

13

Inappropriate use of trial results to “wrong” patients and risk of harmful consequences; some examples

Endarterectomy for asymptomatic carotid stenosis in centres and surgeons with lesser expertise than those participating in the trial

Spironolactone prescribed to pts with heart failure more vulnerable to hyperkalemia than those in the trial


14

RCT Main comorbidities excluded

Appel & al Renal damage &/or diabetes

Hansson & al None reported

Wing & al Plasma creatinine >2.5 mg/dl

ALLHAT, pts≥ 55-year-old

History of HF or EF < 35%

Sacks & al Heart disease, diabetes+insulin, renal failure

Comorbidities were present in 89-100% of the Fortin’s pts potentially eligible to

each RCT


Comorbidity in hypertensive pts in primary care (12)

15

Heathcote EJ & al. Peginterferon alfa-2° in patients with chronic hepatitis and cirrhosis. N Engl J Med 2000; 343: 1673-80 (13) The paper does not report whether patients

with esophago-gastric varices were included in the study This information would have been of help for decision making: esophagogastric varices (prevalence about 40% in Child A cirrhosis, target of the study) suggest an advanced stage of disease (14), with a lower probability of interferon response (15)


Lack of information

16

Schrier RW & al (SALT trials). Tolvaptan, a selective oral vasopressin V2-receptor antagonist, for hyponatremia. N Engl J Med 2006; 355: 2099-112 (16); letters: 2007; 356: 961-3 The trial excluded cirrhotics with Child-Pugh score >10

The information about these pts would have been of paramount relevance: at least 50% of cirrhotics who develop significant hyponatremia have Child-Pugh score >10 (17, 18)

Only 67 cirrhotics were treated with Tolvaptan

The trial cannot provide any information on the risk of adverse events of the drug in cirrhotics


Lack of information

17

Schrier RW & al (SALT trials). Tolvaptan, a selective oral vasopressin V2-receptor antagonist, for hyponatremia. N Engl J Med 2006; 355: 2099-112 (16); letters: 2007; 356: 961-3 The trial excluded cirrhotics with Child-Pugh score >10; only 67 cirrhotics received Tolvaptan


Lack of information

The information about these pts would have been of paramount relevance: at least 50% of cirrhotics who develop significant hyponatremia have Child-Pugh score >10 (17, 18) The trial cannot provide any information on the risk of adverse events of the drug in cirrhotics

18

Hart RG & al. Antithrombotic therapy to prevent stroke in pts with atrial fibrillation: a meta-analysis. Ann Intern Med 1999; 131: 492-501 (19)

Exclusion criteria of the RCTs are often ill-defined (eg: “medical disorders” [20]; “contraindications for Warfarin or aspirin therapy” [21]), and information on co-treatments for pts with comorbidities is generally omitted


Lack of information

Clinicians may be uncertain about the risk/benefit of Warfarin in a substantial number of pts: a reason for its underuse (30-60% [22])?

19

Heterogeneity X2: p < 0.0001

Baseline riskEndoscopic sclerotherapy for prevention of first variceal bleeding in cirrhosis: only RCTs with bleeding rate > 40% in untreated controls show a significant benefit (data adjusted for the duration of follow up); data subsequently confirmed (2)

1. From: Pagliaro L & al. Ann Intern Med 1992; 117: 59-70

2. D'Amico G, Hepatology 1995; 22: 332-54

3.1 External validity and applicability to subgroups

and individuals

20

The ACAS (Asymptomatic Carotid Artery Surgery) trial of carotid endarterectomy vs medical treatment rejected 40% of applicants,subsequently barring those with adverse operative outcomes (1)

However: in USA, most carotid endarterectomies are performed by surgical teams whose complication rates would have rendered them ineligible for ACAS (1,23): results of the ACAS trial and of 8 published series

ACAS 8 series

Stroke & death, % (CI)

1.50 (0.6-2.4)

4.30 (3.5-5.2)

Inappropriate use of RCT results


21

The RALES trial (24): Spironolactone significantly reduced all-cause & cardiac mortality in pts with severe congestive heart failure (NYHA class III-IV)

Pts with serum creatinine concentration > 2.5 mg/dl were excluded Mean age 65 years; females 27%; proportion of diabetics not reported Mean doses of spironolactone 26 mg/day; pts co- treated with β-blockers 10%

No hyperkalemia-related deaths in the trial; however, in the years following the publication of the RALES:

Inappropriate use of RCT results


22

Hyperkalemia-related deaths per 1000 pts

Prescriptions of Spironolactone per 1000 pts

RALES

RALES


After the RALES publication, spironolactone prescriptions greatly increased, even for pts more vulnerable to hyperkalemia than those in the trial (eg for elderly, diabetics, and for pts co-treated with β-blockers), or for pts excluded from the trial (ie HF in NYHA class I and II; mild renal failure); higher doses were often given (25, 26)

Inappropriate use

of RCT results

23


practice


2. Applicability: information needed


4. Help for evaluating applicability

Final remarks

References

24

The drug X is effective in the disease X1

Evidence from an unbiased RCT

The drug X can work in my pts with the disease X1, provided that:

They have the characteristics a, b, c….m

However, there is no evidence to support the drug X if they have the characteristics n, o, p…..z

An approach requiring time

and competence

Interpretive medicine (27, 28)


25

Guidelines built on the global evidence of many RCTs may fill in the gaps

of information of the individual RCTs

0

20

40

60

80

100

Individual RCTs

Pre-appraised summaries Guidelines

5

72 84

% preferences of 302 GP(From 29)

Guidelines


26

Potential heterogeneity in risks of treatment or in risk without treatment, eg baseline risk

Potential heterogeneity of treatment effect related to pathophysiology, eg multiple pathologies underlying a clinical syndrome; or genetic variation

Underuse of treatment in clinical practice due to uncertainty about benefit, eg underuse in specific groups of pts such as in elderly


A subgroup analysis can be worthwhile when established a priori from pathophyisiologic principles in trial populations probably heterogeneous (30,31), eg:

Subgroup analysis

From 31

27

Pharmacogenomic polymorphism

Conclusions. “VKORC1haplotypes can be used to stratify patients into low-, intermediate-,and

high- dose warfarin groups and may explain

differences in dose requirements among

pts of different ancestries”(32) {VKORC1: Vit K

epoxide reductase complex 1}

Subgroup analysis


28

Pharmacogenomic

polymorphism

Conclusions: in pts with IBD,

very high TPMT activity predicts

treatment failure (33)

{TPMT: Thiopurine

methyltransferase}

Subgroup analysis


29

Some final remarks

External validity refers to the information needed to promote the applicability of trial results to clinical practice

Main issues of external validity are the characteristics of randomised pts, comorbidities and co-treatments, outcome measures, and setting

Information gaps due to inadequate external validity may lead to clinical decisions unsupported by adequate knowledge on the most appropriate applicability of the treatments

1

30

Overviews and guidelines gathering the global evidence of individual RCTs and analysis of predefined subgroups may aid the transferability of clinical research to practice

A more effective remedy would be the inclusion of the most relevant issues of external validity in the design and reports of the RCTs submitted for approval or publication

Some final remarks

2

31

References

1. Rothwell PM. Lancet 2005; 365: 82-932. Guyatt GH & al for the EBM Working Group. JAMA 1993; 270: 2958-603. Collins R & al. In: Maynard A & Chalmers I Eds. Non-random

reflections on Health Services Research. BMJ Publ Group, 1997, p.197-230

4. Peto R. J Clin Epidemiol 1995; 48: 23-405. Kravitz RL & al. Milbank Quarterly 2004; 82: 661-876. Haynes B. BMJ 1999; 319: 652-37. Rothwell PM & al. Lancet 2005; 365: 256-658. Hejat A& al. Arch Intern Med 2002; 162: 1682-88 (HF)9. Herland K & al. Respir Med 2005; 99: 11-19 (COPD)10.Uijen AA & al. J Clin Epidemiol 2007; 60: 330-5 (Hypertension)11.Van Spall HGC & al. JAMA 2007; 297: 1233-40 (Eligibility)12.Fortin M, & al. Ann Fam Med 2006; 4: 104-10813.Heathcote EJ & al. N Engl J Med 2000; 343: 1673-8014.Pagliaro L & al. Portal hypertension in cirrhosis: natural history. In: Portal hypertension. Pathophysiology and treatment, J Bosch & RJ Groszmann Eds. Blackwell Scientific Publ, 1994: p.72-9215. Poynard T & al Gastroenterology 2002; 122: 1303-1316. Schrier RW & al. N Engl J Med 2006; 355: 2099-2112

32

References

17. Angeli P & al. Hepatology 2006; 44: 1535-4218. Biggins SW & al. Hepatology 2005; 41: 32-919. Hart RG & al. Ann Intern Med 1999; 131: 492: 50120. Ezekowitz MD & al. N Engl J Med 1992; 327: 1406-1221. GullovAL & al. Arch Intern Med 1998; 158: 1913-2122. Choudhry NK & al. BMJ 2006; 332: 141-323 Hartling L & al. Ann Intern Med 2005; 142: 1100-1124. Pitt B & al (RALES). N Engl J Med 1999; 341: 709-1725 Juurlink DN & al. (Ontario Study) N Engl J Med 2004; 351: 543-5126. Bozkurt B & al. JACC 2003; 41: 211-427. Horton R. Stat Med 2000; 19: 3149-6428. Horton R. Can Med Ass J (CMAJ) 1998; 158: 245-9

29. Guyatt GH & al. BMJ 2000; 320: 954-530. Feinstein AR. J Clin Epidemiol 1998; 51: 297-9931. Rothwell PM, Lancet 2005; 365: 176-8632. Rieder MJ & al. N Engl J Med 2005; 352: 2285-9333. Ansari A& al. Aliment Pharmacol Ther 2002; 16: 1743-50

33

stop

Date post:	20-Jan-2016
Category:	Documents
Upload:	natala
View:	34 times
Download:	5 times

AIFA EUROPEAN CONFERENCE ON CLINICAL RESEARCH FOR DECISION MAKING

Documents