Post-Approval Clinical
Epidemiological Studies
(PACES)
• John Sampalis, BSc, BA (Hon.), MSc, PhD, F.A.C.E.
• Associate Professor Faculty of Medicine
• McGill University
The views and opinions expressed in the following PowerPoint slides are
those of the individual presenter and should not be attributed to Drug
Information Association, Inc. (“DIA”), its directors, officers, employees,
volunteers, members, chapters, councils, Special Interest Area
Communities or affiliates, or any organization with which the presenter is
employed or affiliated.
These PowerPoint slides are the intellectual property of the individual
presenter and are protected under the copyright laws of the United States of
America and other countries. Used by permission. All rights reserved. Drug
Information Association, DIA and DIA logo are registered trademarks or
trademarks of Drug Information Association Inc. All other trademarks are
the property of their respective owners.
2www.diahome.orgDrug Information Association
Definitions
Controlled Clinical Trials Routine Clinical Practice
OUTCOMES
EFFICACYIDEAL/CONTROLLED CONDITIONS
• Selected patient population:
• Narrow spectrum of target population
• Controlled treatment administration
• Defined patient follow up &
assessments
• University / Research setting
EFFECTIVENESSREAL LIFE CONDITIONS
• No or minimal selection of patients:
• Broad spectrum of target population.
• Minimal or No Control of Treatment &
Follow Up
• Treatment as per routine care
• Access to care as per existing policies
Risk of Serious Adverse Events
• Potentially Life Threatening
• Increase Morbidity
• High Detection Threshold
Risk of Non Serious Adverse Events
• High Detection Threshold
Risk of Serious Adverse Events
• Potentially Life Threatening
• Increase Morbidity
• Comorbidity & Concomitant Medication
Related
• Lower Detection Threshold
Risk of Non Serious Adverse Events
• Comorbidity and Concomitant Medication
Related
• Low Detection Threshold
SAFETY
TOLERABILITY
Risk of Nuisance Adverse Events
• Comorbidity and Concomitant Medication
Related
• High & Low Likelihood
JSS Medical Research ® 2013
Post-Approval Clinical & Epidemiological Studies close
the safety & efficacy gaps
CONTROLLED CLINICAL TRIAL• Highly Selective Patient Population
• Structured Treatment Administration
• Protocol Based Follow Up and Assessment
• High Safety Signal Detection Threshold
• No Impact of Access to Care
EXPECTED
EFFICACY
from RCT
EXPECTED
RISK
from RCT
REAL-LIFE• Minimum Selection of Patient Population
• Routine Care Treatment Administration
• Non Structured Safety Assessment
• Physician Discretion
• Impact of Access to Care
OBSERVED
EFFICACY
OBSERVED
RISK
PROBLEMS
• Increased Cost to Society
• Compromised Return on Investment
• High Burden of Illness and Societal Impact
• Safety: Iatrogenic Adverse Effects
SOLUTIONS
• Describe Treatment and
Safety Gaps
• Identify and Segregate
Determinants & Contributing
Factors to the GAPS
• Design & Implement
Interventions to Alter
Modifiable Determinants and
Factors
• Assess Impact of Interventions
& Repeat until Effectiveness &
Safety are Optimized and
Gaps are Minimized
• Evolution to Optimal Disease
Management through ongoing
Evaluation
Treatment
Gap
Safety
Gap
JSS Medical Research ® 2013
PACES should be an integral and essential phase of the drug
development life cycle with implications comparable to that of
Phase II-III studies.
D r u g d e v e l o p m e n t
Pre-clinical Clinical I-III Post-Approval
• Basic science
JSS Medical Research ® 2013
• Safety PK/PD,
• Proof of Concept & Optimal Regimen
• Efficacy & Safety
• Effectiveness & Safety
Hig
h
R
ea
l-L
ife
Ge
ne
raliz
atio
n
Hig
hest
Pre & Post Approval Epidemiological Surveys(Disease Epidemiology, Burden of Illness, Treatment Patterns, Treatment Gap, Knowledge Gap)
PMOS & Registries(real life conditions)
Target Populations
Community MDs
Phase IV(Emulate real life
non-ideal conditions)
Target Populations
Patient Subgroups
Community MDs
Chart Review
Database data mining
Epidemiological Survey (pre & post-approval)
OBJECTIVE
To conduct a true observational study:
Assessment / Description of
• Disease Epidemiology
• Burden of Illness: Morbidity, Mortality, Quality
of Life, Economics
• Variance in Epidemiological Profile: Temporal,
Regional
• Practice Patterns: Medical Decision Making,
Physician / Patient Preferences
• Treatment Gap: Determinants and
Contributing Factors:
Adherence
Patients - treatment
Physicians - practice guidelines
Demographics
Comorbidity
Access to Care
Safety & Tolerability
Hypothesis Generating
Assess existing Interventions aimed at
improving effectiveness
DESIGN
• Prospective Cohort: Single & Multiple Cohorts
• Retrospective Cohort: Single & Multiple
• Cross Sectional Survey
+ −• Prevent Selection bias:
Ensure generalization to
target population
• Avoid Halo effect:
Responses must be valid
Large sample size: Ensure
adequate response rates and
unbiased responder sample
Relatively easy to conduct
• Short time to execute
Large sample size
• Random selection to ensure
representation of the target
population (External validity)
• Allows sub group analyses
JSS Medical Research ® 2013
Phase IV (post-approval)
OBJECTIVE
To Assess Real - Life / Routine Care• Effectiveness & Safety
• Patient and Physician Satisfaction
• Health Care Utilization
• Health Economics
• Patient Outcomes: Quality of Life, Utility, Satisfaction
with Treatment, Productivity, Health Care Costs
To expand physician and patient experience and
comfort with treatment
Treatment Gap• To identify profile of patients responding / not
responding to treatment (Responder analyses in real
life setting)
• Assess New Interventions aimed at reducing
treatment gap & optimizing effectiveness
Assess effectiveness and other outcomes in
patient subgroups• Patient Characteristics: Comorbidity & Concomitant
Medication Use
• Disease Severity or Phenotype
DESIGN
• Single prospective cohort treated with study intervention
• 2 or more prospective cohorts to compare treatments
Open label or Blinded
Random or Non-Random Treatment Allocation
• Sample Size & Follow Up:
Determined by Objectives and Hypotheses
Limited to provide valid answers to scientific questions
• All costs for patient care covered by the study
+ −Time to set up and execute
• Typically 1 -3 years
Attention
• Not to Confuse / Conduct a
Seeding Trial
• Prevent Selection Bias:
Physicians and Patients
• Select Outcomes that are
Relevant to Real-Life,
Routine Care
Emulates Real – Life Setting
• Effectiveness not affected by
access to care and adherence
to treatment
• Patient Outcomes can be
assessed
Physician / Patient Education
• Familiarity with the treatment
JSS Medical Research ® 2013
Post-Marketing Observational Study (PMOS)
OBJECTIVE
To conduct an observational study on the treatment and
outcomes of patients
• Usually asses the effect of specific treatment or
interventions observed in routine care
Typically follow all patients with a disease or
condition or could restrict to specific treatments(s)
Patient treatment and management is NOT
affected by the study
All assessments including follow up is as per
routine care
Decision to treat patient with study
intervention MUST be independent of
enrolment in the study
• Specific Hypotheses and Objectives are ESSENTIAL
Assess / Describe
• Real - Life / Routine Care:
Effectiveness & Safety
Compliance / Adherence
Patient and Physician Satisfaction
Health Care Utilization
Health Economics
Patient Outcomes: Quality of Life, Utility, Satisfaction
with Treatment, Productivity, Health Care Costs
Treatment Gap
• Determinants of response / non response to treatment
• Assess Existing Interventions aimed at reducing treatment gap
& optimizing effectiveness
DESIGN
• Single prospective cohort treated with study intervention
• 2 or more prospective cohorts having different treatments
• Sample Size & Follow Up:
Determined by Objectives and Hypotheses
Limited to provide valid answers to scientific questions
• All costs for patient care NOT provided by study;
• Payment is as per existing Insurance or by the patient.
+ −Relatively high cost to maintain
Complicated Design and Statistical
Analysis to adjust for potential bias
(Selection, Indication,
Confounding) and Observational –
Non – Structured Treatment
Schedules
Attention:
• Avoid Seeding Trials
Valid scientific questions,
study objectives
Regulatory requirements
with Ethics Review & Patient
Consent
Longitudinal follow up of patients
in real – life setting
Patient outcomes can be
assessed
Validity of Results
• Low risk for selection bias
• Hard outcome data collected
JSS Medical Research ® 2013
Patient Registry (post-approval)
OBJECTIVE
Similar to PMOS
Exception
• Specific Hypotheses and Objectives are not
required: General Purpose must be specified
(e.g. Effectiveness or Safety)
• Larger Sample Size (Indefinite)
Not determined / limited by specific
Hypotheses and Objectives.
In theory: Could include 100% of target
population
• Longer / Indefinite Follow Up
Not determined by specific Hypotheses
and Objectives
In theory: Could be as long as patient is
undergoing treatment and qualifies for
the registry
DESIGN
• Single prospective cohort with disease or treated with
specific intervention
• 2 or more prospective cohorts to compare treatments
• All costs for patient care NOT provided by study;
• Payment is as per existing Insurance or by the patient.
+ −Relatively high maintenance
cost
Complicated Design and Statistical
Analysis to adjust for potential bias
(Selection, Indication,
Confounding) and Observational –
Non – Structured Treatment
Schedules
Attention:
• Avoid Seeding Trials
Valid scientific overall
purpose
Regulatory requirements
with Ethics Review &
Patient Consent
Low cost to implement
Create a longitudinal & unbiased
database
• No limit or restrictions on
questions to be addressed
• Foundation for PMOS studies
Patient outcomes can be
assessed
Validity of Results
• Low risk for selection bias
• Hard outcome data collected
JSS Medical Research ® 2013
Chart Review (pre & post-approval)
OBJECTIVE
To conduct an observational study on the
treatment and outcomes of patients
• Effectiveness
• Safety
• Health care utilization
DESIGN
• Typical case-control (referent)
• Retrospective cohort (trohoc) design based on treatments
+ −Possible bias
• Indication
• Confounding
• Information
Data Quality
• Validation
• Missing data
Short duration
Lower cost than prospective
studies
JSS Medical Research ® 2013
Administrative Database
OBJECTIVE
To conduct an observational study on the
treatment and outcomes of patients
• Usually asses the effect of specific treatment
or interventions observed in routine care
Typically follow patients with a disease
or condition or could restrict to specific
treatment
• Specific Hypotheses and Objectives are
recommended to avoid unfocused data mining
Assess / Describe
• Effectiveness & Safety
• Compliance / Adherence
• Health Economics
• Health Care Utilization
• Treatment Gap
Determinants of response / non
response to treatment
Patient Subgroups
Treatment Patterns
Access to Care
DESIGN
• Prospective Cohort: Single & Multiple Cohorts
• Retrospective Cohort: Single & Multiple
• Case – Control Studies: Retrospective
Sample size and f/u: limited to target population in database
+ −Validity and Accuracy of the
data limited to quality of
administrative databases
Outcomes limited to those
recorded in databases:
• Mortality
• Morbidity
• Health care utilization
• Costs
No lab / clinical
assessment results
No patient subjective
outcomes
Short time to completion: 6-9
months
Less costly than chart review and
prospective studies
Unrestricted scope: Address
several questions and objectives
Generalization to Target
Population: Study based on almost
100% of local target populations
JSS Medical Research ® 2013
Epidemiological
survey
Phase IVPMOS &
Registry
Chart ReviewAdministrative
Database
Using PACES to assess different outcomes
Treatment Patterns
Disease Epidemiology
Clinical Outcomes
Safety / Tolerability
Effectiveness
Medical Decision
Knowledge Gap
Patient OutcomesPatient.
Outcomes
Health Economics Health Economics
JSS Medical Research ® 2013
Interventional or Observational
Interventional
Patient management is affected by study
participation:
Treatment(s) used are determined by the
study protocol.
Schedule of assessments is according to
the study protocol.
Decision making on management is
affected by study procedures or tests that
are not part of routine care .
Observational (Non Interventional)
JSS Medical Research ® 2013
Patient management is NOT affected by study
participation:
Treatment(s) used are determined
independently of the study.
Schedule of assessments is according to
routine care.
Recommendations regarding
assessment schedule are possible.
Decision making on management is NOT
affected by study procedures or tests that
are not part of routine care .
PACES: The challenges and the red flags
CHALLENGES
RED FLAGS Poorly Written Protocol: Rationale, Research Questions, Design
Patient incentives: Payment, Support programs, call in numbers, access to services, lottery
Study Execution: Poor Protocol Adherence, Consent Form
Physician Selection: Marketing and prescription driven
Reimbursement: Per prescription or Patient Enrolled
Who will do the study? Marketing Companies, Limited or No Scientific Support, No KOL Endorsement.
Avoid Seeding or Marketing Trials
Ensure Scientific Credibility and Valid Research Questions
Random representative physician and patient samples
No interference with routine care
Academic Participation
Sponsor - Arm’s length for execution
Dissemination of Results: Publications / Presentations
Core REQUIRED Expertise:
Clinical Epidemiology, Biostatistics, Health Economics, Patient Outcomes, Clinical Knowledge
Highest Quality in Project Management
Adherence to Regulatory Requirements (PATIENT CONSENT)
JSS Medical Research ® 2013
Necessary ingredients Things to look for
PACES: The necessary ingredients;
things to look for
Not a
market
research
Publication
of results
Scientific
rationale
Scientific
sound
protocol
Statistical
analysis
High
quality
study
execution
• Credibility
• Relationship with KOLs
• Transparency
• Independence: Arm’s length from Sponsor
• Clinical Epidemiologist
• Biostatisticians
• Clinicians
• Medical Writing team
• KOLs collaboration
• Good Clinical Operations Team
• Data Management System
• Quality Assurance
• GCP / Physician Qualification
• Independent Ethics Review Board
• Patient Consent
JSS Medical Research ® 2013
Regulatory
Compliance
PACES: The Target Optimization of Effectiveness
& Safety Through Ongoing Evaluation
OptimalEffectiveness
& Safety
Hypotheses regarding the causes of the treatment gap
Testing of Hypotheses
Design and Implementation of
Interventions
Initial Realization of treatment gap
&
Follow up Assessments
Description of the treatment
gap
JSS Medical Research ® 2013
PACES: Why do we need them?
Benefits: Integral Part of the Drug Development Program
Similar importance as Phase III studies
Only source of Real – Life Assessment of:
Effectiveness and Safety
Benefit Risk Ratio
Health Economics
Utilization and Practice Patterns
Medical Decision Making
Continuous Evaluation of:
Treatment Gap
Safety Signals
Measurements of Impact of Treatments:
Burden of Illness
Health Care Costs
Detection of Barriers to Treatment and Safety Optimization:
Patient
Health Care Provider
Health Care System
Access to Care
Evaluation of Interventions aimed at improving:
Benefit – Risk
Cost - Effectiveness
JSS Medical Research ® 2013
Disadvantages: Cost
Follow us
JSS Medical Research ® 2013
John Sampalis B.Sc., B.A. (hon.), M.Sc., Ph.D., F.A.C.E.
Professor of Surgery and Medicine McGill University, University of Montreal & University of Laval Director, Surgical Research Jewish General Hospital.
Head Surgical Epidemiology, Division of Surgical Research McGill University
CEO and Chief Scientific Officer JSS Medical Research Inc.
(514) 934-6116 ext. 232
THANK YOU