53
The PROTECT project Progress Status: February 2011 An Innovative Public-Private Partnership for New Methodologies in Pharmacovigilance and Pharmacoepidemiology
The PROTECT project
Progress Status: February 2011
An Innovative Public-Private Partnership for New Methodologies in Pharmacovigilance and Pharmacoepidemiology
2
PROTECT is receiving funding from the European Community's Seventh
Framework Programme
(FP7/2007-2013) for the Innovative Medicine Initiative
(www.imi.europa.eu).
3
PROTECT Goal
These methods will be tested in real-life situations.
To strengthen the monitoring of benefit-risk of medicines in Europe by developing
innovative methods
to enhance early detection and assessment of adverse drug reactions from different data
sources (clinical trials, spontaneous reporting and
observational studies)
to enable the integration and presentation of data
on benefits and risks
4
Clinical trials Observational studies
Electronic health records
Spontaneous ADR reports
Risks
Benefit-risk integration and representation – WP5
Signal detectionWP3
Benefits
Validation studies
WP6
Training and education
WP7
Signal evaluationWP2
Data collection from consumers – WP4
5
Partners
Public PrivateRegulators:EMA (Co-ordinator)DKMA (DK)AEMPS (ES)MHRA (UK)
Academic Institutions:University of MunichFICF (Barcelona)INSERM (Paris)Mario Negri Institute (Milan)Poznan University of Medical Sciences University of GroningenUniversity of UtrechtImperial College LondonUniversity of Newcastle Upon Tyne
EFPIA companies:GSK (Deputy Co-ordinator)
Sanofi-
Aventis
Roche
Novartis
Pfizer
Amgen
Genzyme
Merck Serono
Bayer Schering
Astra Zeneca
Lundbeck
NovoNordisk
Takeda
SMEs:Outcome EuropePGRx
Others:
WHO UMC
GPRD
IAPO
CEIFE
6
Objectives:
To create and maintain the conditions needed to achieve the objectives and deliverables of the
PROTECT project.
WP 1: Project Management and Administration
Financial monitoring and accountancy
Track of work progress in line with the work programme
Knowledge management
tools and strategies
Administrative, organisational and financial
support
Quality control and assurance measures
Scientific steer towards the
overall project objectives and
strategy
7
WP 2: Framework for pharmacoepidemiological studies
To:•
develop
•
test
•
disseminate
of pharmacoepidemiological studies applicable to:
•
different safety issues
•
using different data sources
methodological standards for the:•
design
•
conduct
•
analysis
Objectives:
8
Art is made to disturb. Science reassures.Georges Braque
Is it always true ?
9
Two studies on the use of statins and the risk of fracture done in GPRD around the same period by two different groups.
10
Why such a difference ?
•
Different patients (source population, study period, exclusion criteria)
•
Study design (e.g. matching criteria for age)
•
Definition of current statin
use (last 6 months vs. last 30 days)
•
Possibly different outcomes (mapping)
•
Possibly uncontrolled/residual confounding
11
Work Package 2
Work plan•
Three Working Groups (WG1-WG3)–
Databases
–
Confounding
–
Drug Utilisation
12
Work Package 2 – WG1: Databases
Work Plan
Conduct of adverse event -
drug pair studies in different
EU databases–
Selection of 5 key adverse event -
drug pairs
–
Development of study protocols for all pairs
–
Compare results of studies
–
Identify sources of discrepancies
Databases–
Danish national registries
–
Dutch Mondriaan
database
–
British GPRD database
–
British THIN databases
–
Spanish BIFAP project
–
German Bavarian claims database
13
Work Package 2 – WG1: Databases
Progress status (1/3)
Selection of key adverse events and drugs
•
Selection criteria:–
Adverse events that caused regulatory decisions
–
Public health impact (seriousness of the event, prevalence of drug exposure, etiologic fraction)
–
Feasibility
–
Range of relevant methodological issues
14
Work Package 2 – WG1: Databases
Progress status (2/3)
Selection of 5 key adverse events and drugs
•
Initial list of 55 events and >55 drugs
•
Finalisation based on literature review and consensus meeting
Antidepressants (incl. Benzodiazepines)
- Hip Fracture
Antibiotics -
Acute liver injury
Beta2 Agonists
- Myocardial infarction
Antiepileptics
-
Suicide
Calcium Channel Blockers -
Cancer
15
Work Package 2 – WG1: Databases
Progress status (3/3)
Development of study protocols
•
Descriptive studies for the Drug AE pairs in all databases
•
5 different study designs in selected databases–
Cohort design
–
Nested case control design
–
Population based case control
6 Final protocols in Feb 2011 (separate protocols for antidepressants and benzodiazepines versus hip fracture)
–
Case crossover
–
Self controlled case series
16
Work Package 2 – WG2: Confounding
Work Plan
•
Objective
–
To evaluate and improve innovative methods to control confounding
•
Method
–
Creation of simulated cohorts
–
Use of methods to adjust for observed and unobserved confounding
e.g. time-dependent exposure, propensity scores, instrumental variables, prior event rate ratio (PERR) adjustment, evaluation of measures of balance in real-life study
17
Work Package 2 – WG2: Confounding
Progress status •
Finalisation
of protocol to conduct simulation studies–
Propensity score methods
–
Instrumental variable methods
–
Time-dependent confounding
•
First results on propensity scores (PS)/balance measures–
Usefulness of measures for balance for reporting of the amount of balance reached in PS analysis and selecting the final PS model
–
Recommendation of methods to quantify balance of confounder distributions when applying PS methods:
standardised difference
Kolmogorov-Smirnov distance, or
overlapping coefficient
18
Work Package 2- WG3: Drug Utilisation
Work Plan
•
Use of national drug utilisation data (incl
IMS)
•
Inventory of data sources on drug utilisation data for several European countries
•
Evaluation and dissemination of methodologies for drug utilisation studies in order to estimate the potential public health impact of adverse drug reactions
•
Collaboration with EuroDURG
agreed
19
Work Package 2- WG3: Drug Utilisation
Progress Status
Inventory on Drug Use data “Drug consumption databases in Europe”
(last version January 2011)
−
11 research working groups across Europe identified
−
Databases heterogeneous, administrative focus and influenced by the national health system structure
•
Collecting DU data (in/out hospital) –
from public databases (for 6 selected drugs)
–
from IMS (Antibiotics, Antidepressants and Benzodiazepines. Explored
for other drugs)
20
Work Package 3: Signal Detection
Objective:
To improve early and proactive signal detection from spontaneous reports, electronic health records, and
clinical trials.
21
Work Package 3: Signal Detection
Scope•
Develop new methods for signal detection in Individual Case Safety Reports.
•
Develop Guidelines for signal detection and strengthening in Electronic Health Records.
•
Implement and evaluate concept-based Adverse Drug Reaction terminologies as a tool for improved signal detection and strengthening.
•
Evaluate
different methods
for signal detection
from clinical
trials.
•
Recommendations for good signal detection practices.
22
Work Package 3: Sub-projects
1.
Merits of disproportionality analysis
2.
Structured database of known ADRs
3.
Concordance with risk estimates
4.
Signal detection recommendations
5.
Better use of existing ADR terminologies
6.
Novel tools for grouping ADRs
7.
Other information to enhance signal detection
8.
Signal detection based on SUSARs
9.
Subgroups and risk factors
10. Signal detection in Electronic Health Records
11. Drug-drug interaction detection
12. Duplicate detection
23
•
Objective:Making available, in a structured format, already known ADRs to allow for
–
Triaging out known ADRs
–
Automatic
reduction of masking effects
•
Approach:– Manual identification
– Pooling of existing structured information (?)
– Free text extraction!
•
Progress to date:– All 375 SPCs of CAPs
(substances). Addition of non-CAPs
under discussion.
Work Package 3 – Structured database of SPC 4.8
24
•
Proof-of-concept analysis of free text extraction algorithm– Initial match rate increased from 72% to 98%
Drug SPC Term Verbatim match Fuzzy matching algorithm
Aclasta FLU-LIKE SYMPTOMS Flu
symptoms
Advagraf OTHER ELECTROLYTE ABNORMALITIES - Electrolyte abnormality
Advagraf PAIN AND DISCOMFORT - Pain and discomfort NEC
Advagraf PRIMARY GRAFT DYSFUNCTION - Primary graft dysfunction*
Advagraf PRURITUS PRURITUS Pruritus*
Advagraf PSYCHOTIC DISORDER PSYCHOTIC DISORDER Psychotic disorder*
Advagraf PULSE INVESTIGATIONS ABNORMAL - Investigation abnormal
Advagraf RASH RASH Rash*
Advagraf RED BLOOD CELL ANALYSES ABNORMAL - Red blood cell analyses*
Advagraf RENAL FAILURE RENAL FAILURE Renal failure*
Advagraf RENAL FAILURE ACUTE RENAL FAILURE ACUTE Acute renal
failure, Renal
failure
acute*
Advagraf RENAL IMPAIRMENT RENAL IMPAIRMENT Renal impairment*
Advagraf RENAL TUBULAR NECROSIS RENAL TUBULAR NECROSIS Renal tubular necrosis*
Advagraf RESPIRATORY FAILURES - Respiratory failure, Failure respiratory
Advagraf RESPIRATORY TRACT DISORDERS - Respiratory tract disorders NEC
Advagraf SEIZURES - Seizure, Seizures*
Advagraf SHOCK SHOCK Shock*
Better
option:Red blood cell
abnormal
Work Package 3 – Structured database of SPC 4.8
25
•
Scope–
EudraVigilance, VigiBase
–
National data sets: AEMPS, BFARM, DKMA, MHRA
–
Company data sets: AZ, Bayer, Genzyme, GSK
•
Focus–
# reports, # drugs and # ADR terms
–
Types of reports (AEs or ADRs, Vaccines, Seriousness, ...)
–
Additional information (presence of data elements available for stratification and sub-setting, e.g. demographics)
–
Supporting systems (analytical methods, medical triages)
•
Current status–
Survey deployed and completed by most organisations
Work Package 3 – Database survey
26
•
Proof
of concept–
Temozolomide
–
Not illustrating timeliness
–
VigiBase
as of Feb 2009
Work Package 3 - Better use of existing terminologies
Term Level of terminology
# Reports IC
Erythema Multiforme PT 13 +0.30
Stevens-Johnson Syndrome PT 19 +0.68
Toxic Epidermal Necrolysis PT 6 +0.51
Bullous Conditions HLT 42 -0.01
Severe Cutaneous Adverse Reactions
SMQ 47 -0.04
Erythema Multiforme WHO-ART HLT 35 +0.46
27
•
Approach–
Automatic
generation of groups
of MedDRA terms based
on semantic
information–
Based
on a mapping
of MedDRA to SNOMED CT–
Groups
MedDRA terms based
on semantic
distance
•
Progress –
Evaluation
study
completed–
Comparison
with standard MedDRA SMQs
as gold
standard
•
Next
steps:–
Refinement
of methods–
Use
in signal detection!
Work Package 3 – Novel tools to group ADRs
28
•
Overall scope–
Inform best practices on which data should be used and which methods are optimal
–
Explore novel uses of existing clinical data in ongoing and completed clinical trials for safety signal detection
•
Progress
Draft
protocol
–
Conduct benchmark survey of available methods and processes
–
Create a library of publications on this topic
–
Identify compounds and relevant data sets for retrospective analysis.
–
Conduct analyses and document results.
–
Create recommendations for best practices
Work Package 3 - Signal detection from clinical trials
29
•
Overall scope
–
EHRs
versus
ICSRs
for early
signal detection
–
Confirmatory
vs exploratory
data analysis
•
Focus so far has been
on the adaptation of an existing
analytical
platform
to THIN
•
Detailed
protocols
under development
(completion by Aug 2011)
Work Package 3 - Signal detection in Electronic Healthcare Records (EHRs)
30
•
Subpackage
11: Drug-drug
interaction detection
reference
set under construction
•
Subpackage
12: Duplicate
detection
completed
in VigiBase
•
Study protocols agreed for–
Subpackage
1: Merits of disproportionality
analysis
–
Subpackage
2: Concordance with risk estimates
–
Subpackage
5: Better use of existing terminologies
Work Package 3 - Other
31
Work Package 4: Data collection from consumers
Objectives:
To assess the feasibility, efficiency and usefulness of modern methods of data collection including using web-based data collection and computerised, interactive voice responsive
systems (IVRS) by telephone
32
Work Package 4 - Project Definition
•
Prospective, non interventional study which recruits pregnant women directly without intervention of health care professional
•
Collect data from them throughout pregnancy using either web based or interactive voice response systems (IVRS):–
medication usage, lifestyle and risk factors for congenital malformation
•
Compare data with that from other sources and explore differences
•
Assess strengths and weaknesses of data collection and transferability to other populations
33
Using health care professionals to capture data
•
Expensive and data capture relatively infrequent
•
Will miss drug exposure before comes to attention of HCP
•
Patients may not tell truth about “sensitive”
issues
Work Package 4 - Issues with current methods
34
Using EHR records
•
non prescription medicines, homeopathic and herbal medicines not captured
–
? Women switch to “perceived safer”
medicines
•
Medicines prescribed/dispensed may not be medicines consumed –
problem with p.r.n.
medicines (i.e. dosage as needed)
•
EHR may miss lifestyle and “sensitive”
information
Work Package 4 - Issues with current methods
35
Work package 4 - Study population
•
4 countries:
•
1400 pregnant women per country
–
Self identified as pregnant
–
Volunteers may not be “typical”
of pregnant population –
can characterise
United-Kingdom
Poland
Denmark
The Netherlands
36
Study subject picks up a leaflet in a pharmacy or browses specific web sites to find out about the
study in one of 4 countries.
Study subject enrolls for the web or phone (IVRS) method of data collection.
Work Package 4: Patient workflow overview
Final outcome survey is completed at the end of pregnancy.
Web
n = 1200 per country
Study subject completes the surveys online.
IVRSn = 200 per country
Study subject completes the surveys via an outbound reminder
or by inbound call she initiates.
37
Work Package 5: Benefit-Risk Integration and Representation
Objectives:
•
To assess and test methodologies for the benefit-risk assessment of medicines
•
To develop tools for the visualisation of benefits and risks of medicinal products
Perspectives of patients, healthcare prescribers, regulatory agencies and drug manufacturers
From pre-approval through lifecycle of products
38
Work Package 5: Workstreams
WorkstreamsA Develop framework for benefit-risk analysis
B Review of methodologies used, elicitation of preferences and integrating effects and preferences
C Criteria for case study selection & case study selection
D Determine data to be gathered from case studies and format required
E Develop software to support application of methodology and graphical representation
F Application of methodology and graphical methodology to case studies wave 1
39
Work Package 5: Work Plan
1.
Review of methodologies used to model effects of medicines, elucidation of patients’
preferences and
integrating effects and preferences.
Review of methodologies for graphical representation and visualisation techniques.
2.
Selection of case studies (waves 1 and 2)3.
Data selection/requirements for case studies
4.
Identification/development of software for B/R.
5.
Application of methodology, recommendations, finalisation of tools, protocols for validation studies.
40
Work Package 5: Workstream A - completed
•
Framework for B-R analysis: achieved through a Charter (SC approved)–
Large scope covering principally post-approval setting, individual and population-based decision making, various perspectives (patients, prescriber, regulators, industry)
–
Address possible interdependencies with other PROTECT WPs–
Review of B-R methodologies and graphical representation tools–
Selection of candidate methodologies based on specified criteria–
Process for selection of case studies, according to selection criteria–
Implementation of case studies using relevant methodologies and including preferences of various stakeholders
–
Test available representation technologies applied to above mentionned
case studies and B-R methodologies–
Publication and presentation of case studies in various settings
41
Work Package 5: Overview
WS B Methods
WS B Methods
WS C Case studies
WS C Case studies
WS D Framework /
Data
WS D Framework /
Data
WS E Software / graphics
WS E Software / graphics
WS F Application
WS F Application
• Review of existing methods not inventing new methods.
• Emphasis on graphical representation.• Methods estimating(1) magnitude /
incidence of events and (2) value elicitation of benefits and risks, from a patient and regulator perspective and how combine them into a single measure.
• Review of existing methods not inventing new methods.
• Emphasis on graphical representation.• Methods estimating(1) magnitude /
incidence of events and (2) value elicitation of benefits and risks, from a patient and regulator perspective and how combine them into a single measure.
• PrOACT-URL framework for performing benefit-risk analysis.
• Oversee working parties for extracting objective measures of magnitude / incidence of benefits and risks.
• PrOACT-URL framework for performing benefit-risk analysis.
• Oversee working parties for extracting objective measures of magnitude / incidence of benefits and risks.
• Not developing software, but explore suitable existing software (possibly with adaptation).
• Not developing software, but explore suitable existing software (possibly with adaptation).
• Apply the methodology to the case studies using the data
• May also elicit the subjective value data for the benefits and risks.
• Apply the methodology to the case studies using the data
• May also elicit the subjective value data for the benefits and risks.
• Wave 1: has 4 case studies: Raptiva, Tysabri, Ketek, and Acomplia.
• Drugs which have data readily available from EPARs.
• Not revisiting EMA decisions, but use to demonstrate and test methodologies.
• Wave 1: has 4 case studies: Raptiva, Tysabri, Ketek, and Acomplia.
• Drugs which have data readily available from EPARs.
• Not revisiting EMA decisions, but use to demonstrate and test methodologies.
42
Work Package 5: Workstream B
•
Protocol for evidence synthesis endorsed by all members
•
34 items to review have been identified through literature search
•
List of evaluation criteria has been generated
•
Focus on their potential for graphical representation
Collabor ators
Literature search
Other initiative
sMethodological review
Elicitation of suitable methods
Literature search
External meeting
s
Other initiativ
esVisual representations review
Elicitation of suitable graphics
Integration of methodologies and visual representations
Develop visual representations
add-ons and software
Application to case studies
Present case-studies results emphasising on communication of, and use of graphical representations for, understanding benefits and risks
43
Work Package 5: Workstream C
•
Progress
–
Criteria for
wave 1 case studies and drugs for case
studies (Acomplia
®, Raptiva
®, Tysabri
®, Ketek ®)
–
Draft criteria for
wave 2 and library of possible
candidates (more challenging)
Uncertainty about what the main benefits and risks are.
Uncertainty about the population who has the disease.
Different time for Benefit and for Risk (long term risks).
Individual benefit-risk, or subgroups of benefit risk.
New drugs vs. long marketed drugs.
44
Work Package 5: Workstream C
•
Next steps
–
Discussion with other workstreams
for appropriate
data identification and extraction (WS D), applicability of case studies for WS F to run.
–
Identify potential presentations and publications.
45
Work Package 5: Workstream D
•
Scope–
Data Collection dependent from Framework used:
Using PrOACT-URL (generic framework for decision making), identification of data sources to be used depend on detailed description of each of the steps of the framework (see back up slide)
–
Lead to a draft “Guidelines for preparing a Case Study Report”
–
Based on Acomplia®
experience, most data/information necessary for B-R assessement
at time of market authorisation
and of market withdrawal were included into EPAR (Regulators perspective)
–
In addition to EPAR, additional data sources for other drugs or for other perspectives will require
Additional data collection from existing data sets (PSURs, formal B-R reviews)
Creation of new data (e.g. questionnaires for patient preferences elicitation)
46
Work Package 5: Workstream D
•
Next steps
–
Prepare identification of data sources to be used/created for other Wave 1 case studies (Raptiva
®, Tysabri®, Ketec®)
–
Actual supply of data
47
Work Package 5: Workstream F
•
Scope
–
Workstream
(WS) F is:
applying the methodology from WS B
to the case studies selected from WS C
using the data collected in WS D
with the software and graphical methods selected by WS E
–
Done by four interdisciplinary teams in four locations
–
More than one method will be applied to each case study, and several methods explored overall
–
The aim of the first wave is to test the application of the methods and framework on relatively simple case studies
–
This then feeds back into the second wave to refine the tools
48
Work Package 6: Validation
Started in September 2010
Objectives:
•
To validate and test the transferability and feasibility of methods developed in PROTECT to other data sources and population groups
•
To determine
the added
value of using
other
data
sources as a supplement
or alternative to those
generally
used
for drug
safety
studies,
in
order
to
investigate
specific
aspects or issues.
4949
Work Package 6 - Inventory of data sources
•
Creating a comprehensive list of data sources (ongoing)–
Review of European databases (electronic healthcare records, cohorts, registries)
–
ENCePP
–
EFPIA
•
Outcomes of other Work Packages (2-5) will be evaluated in light of the inventory of data sources (e.g. type of data, covariate information, mode of collection, type of prescription data, etc)
50
Work Package 7: Training & communication
Objective:
To identify training opportunities and support training programmes to disseminate the results achieved in
PROTECT.
51
•
Development
of
a platform
of
training opportunities.
•
Regular interaction
with
EU2P Consortium.
•
Communication
Plan: draft
list
of
conferences
and other
international
forums
suitable
for
the
presentation
of
the
results
of
PROTECT.
Work Package 7: Scope
52
Work Package 7: Training Platform
https://w3.icf.uab.es/trainingopp(under development)
