Patient and Citizen Engagement in HTA Decision Making
E4 Panel session,
CADTH Symposium,
May7th, 2013
1
Agenda• Moderation
o Paul Oh
• Patient perspective
o Durhane Wong-Reiger
• Citizen perspective
o Janet Parsons and The Ontario Citizen‟s Council
• Decision maker/ HTA Body perspectives
o CDR - Elaine MacPhail
o pCODR - Mona Sabharwal
• Discussion (All)
Questions for Reflection and Response
• What is currently being done around patient
and citizen engagement in HTA?
• What has been the impact of patient and
citizen engagement in HTA?
• What does “good” or “better” look like?
• How can Canada become a leader in this
area?
ADVOCARE
Durhane Wong-Rieger, PhD
Chair, Consumer Advocare Network
Thanks to Janssen for some of the slides and analysis
What are “Best” Outcomes in Healthcare?
• Defined as attaining best individual health
possible and optimal health for population
• Individual makes healthcare choices that lead
to best personal outcomes, based on personal
suitability and shared social values
• System provides healthcare options that allow
all patients to get right care, at right time with
optimal use of healthcare and social
resources
8
How can system Allocate Resources to Achieve “Best” Outcomes?
• Allocation based on shared social values
(necessary, equitable, accessible,
affordable, compassionate)
• Allocation defined based on achieving
optimal individual needs (personal good)
• Allocation to optimize population impact
(accountable, appropriate, sustainable)
9
How does Patient Engagement Improve Health Outcomes?
• For self, patients choose optimal healthcare: make evidence-based cost-effective choice;, adhere to care plan; engage in self-management; give feedback
• For healthcare system, patients help decide care options: identify patient outcomes, define patient values, use assessment and decision-making framework
• For funders, patients take part in allocation of resources: define societal values, develop assessment and decision-making framework, evaluate outcomes
10
Model of Current Patient Input in HTAConsultee, Informant
Input thru Council, Task Force, Collect Info: Survey, Poll, Focus Group
Form of Info: Answers, Opinions, Deliberation
Impact: Advise, DiscretionaryExamples: NICE Citizens Council, IQWiG, Ontario Citizens Council
Patient/Public Member
Input thru Committee, Board, CouncilCollect Info: Experts, Deliberation
Form of Info: Analytical, GuidelinesImpact: Varied, Based on GuidelinesExamples: NICE, AU MASC, CEDAC,
pERC Ontario CED
Individual Patients
Input thru Clinical Trials, TestimonyCollect Info: QoL, PROs, Impact
StatementForm of Info: Ratings, Qualitative Impact: Varied, Emotional Suasion
Examples: SMC, IQWiGQuebec conseil, BC Pharmacare,
Patient Groups
Input: SubmissionCollect Info: Written, Oral, MeetingsForm of Info: Qualitative Statement
Degree of Impact: ResponseExamples: NICE, SMC AU MASC,
CADTH, pCODR, Ontario CED
Patient Representativeness 11
Citizen engagement in HTA:A Brokered Dialogue with the
Ontario Citizens’ Council
Janet Parsons, PhDJim Lavery, PhD
Advisory body to the Executive Officer of the Ontario Public Drug Programs
Provides advice on the values that reflect the needs, culture and attitudes of Ontario’s citizens about government drug policy
Ontario Citizens’ Council
http://www.health.gov.on.ca/en/public/programs/drugs/councils/
13
CADTH patient and public involvement Launched in May 2010
Seeks input via Canadian patient groups
Uses template, guide, online submission form
Respects existing CDR timelines
Incorporates patient input systematically
throughout drug review and
recommendation-making process
Must meet the needs of patients, reviewers
and CDEC
Patient Input
Process
Public members on: Canadian Drug Expert
Committee & Health Technology Expert Review
Panel with full voting rights
16
How patient input is used
Patient input collated, summarized, used to inform protocol & included
review report
Patient input presented, used in deliberations & reflected in
recommendations
CADTH ReviewTeam
Expert Committee
Drug PlansPatient input
shared
Patient Input SubmissionCanadian Patient Groups
Reasons & recommendations
published
Original input & summary sent to
CDEC
Patient input
shared
Patient input
shared
Patient input
shared
17
Applied Lessons in Patient Engagement
• Must be imbedded in organizational culture• one of pCODR‟s Guiding Principles (Collaborative and
Representative) and part of pERC‟s Deliberative Framework
• Begin at the beginning• engagement of stakeholders front and centre since Day 1
• Genuine consultation and interaction, based
on best practices of HTA agencies, nationally
and internationally• inclusive, responsive, transparent follow-up actions (even if it
means not being able to do something)
• Take “ownership” of what you can • on-going investment in providing information and education,
provide guidance but respect who owns specific steps/issues
© 2013 pan-Canadian Oncology Drug Review19
pCODR Review Process
Updated March 31, 2011
1. Conduct Pre-
Submission Planning activities including
getting input from PAG and notifying Patient
Advocacy Groups
2. Prepare & submit
Request for Drug
Review
4.2Conduct
Economic Review
5. Summarize & Review
with pERC
6. Prepare & Publicly
Post Initial Recomm,
Post Reviews
8. Summarize & Review
with pERC
3.1 Screen Submission and Initiate
Review Process
End‡
Ind
us
try/
Tu
mo
ur
Gro
up
pC
OD
R*
Variable 5 business days 70-90 business days 12 business days 10 business days 20 business days
7.1 Get Feedback
from Submitter
(and impacted manufacturer)
7.3 Get Feedback from
Patient Advocacy
Group
7.2 Get Feedback from PAG
Pati
en
tA
dv
ocac
y
Gro
up
s
9. Prepare & Publicly Post
Final Recomm & Post Input
12 business days
*Includes pCODR Secretariat, Clinical
Guidance Panel, Economic Guidance Panel,
pCODR Expert Review Committee (pERC)
and Provincial Advisory Group (PAG)
4.1.1/4.2.2Clarify info
with Submitter
during review
4.1Conduct Clinical Review
3.2 Collect Patient
Advocacy Group Input
Estimated
99 – 149
business days
7.4
Eligible for
Early
Conversion?
No Yes
‡Next steps could include
Recommendation implementation,
Procedural Review or Resubmission
Direct impact of patient
perspective
Indirect impact of patient
perspective
© 2013 pan-Canadian Oncology Drug Review
pERC Patient Members: 2 voting, 1 alternate
Selection criteria/process:
• Demonstration of personal knowledge of, experience with and
understanding of issues related to cancer and its management
(among other qualifications)
• Broad advertising, independent 3rd party to screen applicants, diverse
panel for decision making
Training:
• On-line training modules, orientation, on-going educational supports
• All members forward suggestions anonymously for specific training
topics, demonstrating lots of overlap in needs
Committee participation:
• Present materials, deliberate and vote at all meetings
• Participate in working groups, regardless of topic
• Present at conferences upon invitation and as available
21© 2013 pan-Canadian Oncology Drug Review
Detailed Description of Each Element of the pERCDeliberative Framework (1)
22
Criteria Sub-Criteria Sub-Criteria Definitions
Overall Clinical Benefit Effectiveness
(systematic review in the Clinical Guidance
Report)
The potential health impact of the
drug compared to the other drug and
non-drug alternatives, measured in
terms of relevant patient outcomes
such as mortality, morbidity, quality
of life. Magnitude, direction and
uncertainty of effect should be
considered.
Safety
(systematic review in the Clinical Guidance
Report)
Frequency and severity of adverse
effects associate with the new drug
compared to other drug and non-drug
alternatives.
Burden of Illness
(Clinical Guidance Report, patient advocacy group
input)
Incidence, prevalence or other
measure of disease burden on the
population.
Need
(Clinical Guidance Report, patient advocacy group
input)
Availability of an effective alternative
to the drug technology.
© 2013 pan-Canadian Oncology Drug Review
Detailed Description of Each Element of the pERCDeliberative Framework (2)
Criteria Sub-Criteria Sub-Criteria Definitions
Alignment with Patient
Values
Patient Values
(patient advocacy group input)
Patient based values which bear on the
appropriate use and impact of the drug.
Cost effectiveness Economic Evaluations
(Economic Guidance Report and pharmaco-
economic model review)
A measure of the net cost or efficiency
of the drug and companion technology
compared to other drug and non-drug
alternatives. The uncertainty of results
should be considered.
Feasibility of Adoption into
Health Systems
Economic Feasibility (evaluation of budget
impact assessment in Economic Guidance
Report)
The net budget impact of the new drug
on other drug and health system
spending, including companion testing
technology.
Organizational Feasibility
(Provincial Advisory Group input)
The ease with which the new drug can
be adopted, with an assessment of
health system enablers and barriers to
implementation, inclusive of all
elements: operational, capital, human
resources, legislative and regulatory
requirements
CDR Recommendations (2011-12 w/Patient Submissions)
N=58 Total With Patient Submission
No Patient Submission
Overall100% (58) 72% (42/58) 28% (16/58)
List12% (7/58) 71% (5/7) 29% (2/7)
List Conditional45% (26/58) 85% (22/26) 15% (4/26)
Not List43% (25/58 ) 60% (15/25) 40% (10/25)
26
pCODR Recommendations (2012 w/Patient Submissions)
N=8 Total With Patient Submission
No Patient Submission
Overall 100% (8) 100% (8/8) 0% (0/8)
List 0% (0/8) 0% (0/8) 0% (0/8)
Not List 13% (1/8 ) 100% (1/1) 0% (0/1)
List Conditional 88% (7/8) 100% (7/7) 0% (0/7)
27
CDR/pCODR (2011-2012)Reasons for Recommendations
List(n = 4)
List Conditional
(n = 22)
Not List(n = 16)
Clinical Trial Outcomes(effectiveness, sufficient,similar, certain)
100% 91% 88%
Safety (adverse effects, similar risks, certain)
25% 28% 31%
Cost-effectiveness (more, less, certain)
100% 87% 56%
Quality of Life (studied, better, specific)
25% 59% 44%
Patient submissions (reference, substantiated
0% 82% 63%28
Impact of Patient Submissions: CDR/pCODRRecommendations
• Patient submissions in 72% of CDR and 100% of pCODR
cases
• Average # submissions: 2.2 CDR and 1.3 pCODR per case
• CT outcomes (effectiveness, comparison to standard,
certainty) most important and frequently cited reason for
recommendation
• Cost-effectiveness (more costly, uncertain CE analysis,
$/QALY) cited in all “list”; most “list conditional”; half of
“do not list
• Safety (adverse effects, uncertain risks) cited in 30% of
cases
29
Impact of Patient Submissions: CDR/pCODRRecommendations
• Patient submissions always discussed; factor in
almost all recommendations to “list
conditional”; three-fifths of “not to list” and not
cited in “to list” recommendations.
• Patient submissions discounted when factors
not included in CTs or not substantiated by
submitted QOL measures; request for “balance
of positive and negative” opinions.
30
Patient/Public Member Impact on HTA
Feedback from Patient/Public Members
• Patient and Public Members believe their participation is valuable but neither equal nor impactful
• Patient submissions gives added credibility to patient/public role; have something to contribute
• Other HTA members not always receptive to patient submissions; qualitative information difficult to integrate; some would prefer quantitative but not sure what would be included
• Patient submissions have little impact on discussion or decision; however, do increase understanding of rare or unusual conditions
• Patient relations with pharmaceutical companies, even if declared, raises question of conflict of interest and diminution of input
• Patient representatives in meetings would increase understanding of the patient submission
Purported and Real Impact of Patient Engagement in HTA
Looks Like Really Happens
Patient perceived outcomes builtinto clinical trials (QOL, PROs)
Scales unable to capture patient experience; no long-term impact
Patient submissions to HTA give qualitative experience; own words; collective group input
No process to integrate patient submissions into quantitative approach; ignore, discount
Patient or public members on HTA bodies assure perspective of patient or society at table
Patient/public member roles poorly defined; lack technical expertise; public ≠ patient
Multi-stakeholder forums, Citizens Councils, consultations are opportunities for input
Most public/patients unaware of processes; technical oracademic
Plain language reports increase patient understanding and acceptance of HTA outcomes
Patients want full access;don’t trust unless present:want right to appeal
Citizen engagement in HTA:A Brokered Dialogue with the
Ontario Citizens’ Council
Janet Parsons, PhDJim Lavery, PhD
What is the impact of citizen engagement?
Overall, OCC is a success in the making: reasons
Nevertheless, important concerns identified:
• The mandate of the OCC is vague
• As a consequence, challenging to assess overall
impact
• Participants are activated to participate in what they see as
important opportunity for political participationo An opportunity to speak directly to government
o Profound experience of engagement
• If their contributions are not recognized/acknowledged, will
lead to disappointment/disillusionment
What is the impact of citizen engagement?
The Evaluation Challenge in Citizen Engagement:
• What impacts should we expect from such initiatives?
• Narrow views of impact are insufficient
• Which outcomes and processes matter and why?
• Making room for unanticipated outcomes
• Recognition that a multiplicity of potential impacts should be assessed:
o Not only on council members (e.g. degree to which their expectations are met)
o Decision-makers (e.g. how are CC recommendations incorporated into decisions by expert committees)
o Broader public (e.g. level of awareness of CC activities, increased understanding of how decisions are made)
Impact of patient input on decision-making
Informs formulary listing recommendations
Identifies outcomes and issues about condition and
its treatment of importance to patients
Identifies unmet needs
Describes expectations for the new product
Reflected in Final Recommendation document
Results in more relevant recommendation
Identifies where there is no evidence to support the
outcomes or issues of importance to patients
Need better tools to measure impact
Patient groups willing to contribute
74% submissions receive
patient inputAv. 2 patient groups
provide input per drug
Data from May 2010 to April 30 2013
75 different groups
across Canada have
contributed
Patient Advocacy Group Engagement - on pCODR Reviews with pERCFinal Recommendations (as of Dec. 31, 2012)
Generic Name / Brand
Name
Cancer Type Type Patient
Input
Patient
Feedback
Pazopanib (Votrient) Metastatic Renal Cell Carcinoma 1 1
Ipilimumab (Yervoy) Advanced Melanoma 2 1
Sunitinib (Sutent) Pancreatic Neuroendocrine Tumours 1 0
Vemurafenib (Zelboraf) Advanced Melanoma 1 1
Eribulin (Halaven) Metastatic Breast Cancer 2 0
Everolimus (Afinitor) Pancreatic Neuroendocrine Tumours 1 0
Crizotinib (Xalkori) Advanced Non-Small Cell Lung Cancer 1 1
Bendamustine (Treanda) Non-Hodgkins Lymphoma (NHL) 2 1
Bendamustine (Treanda) Chronic Lymphomatic Leukemia
(relapsed/refractory)
2 1
Pazopanib (Votrient) Soft Tissue Sarcoma 1 0
Total (%) 14 (100%) 6 /14 (43%)
•43© 2013 pan-Canadian Oncology Drug Review
Providing Feedback on an Initial Recommendation
• Contributing parties can share their interpretations and
perspective on a pERC recommendation
• pERC reviews feedback to redeliberate and reconsider, if
appropriate, the initial recommendation
• Brings greater transparency to process, improving legitimacy
and confidence that pERC really has chance to consider
“everything”
Do you agree with the Initial Recommendation – why or why not?
Did we interpret your input accurately?
Is there contextual information about the patient experience which needs to be
re-considered?
Is there something we could have worded more clearly?
Are Patients/carers/public happier with NICE?
• Lack of research evidence on patient/carer
views, experiences and preferences
• Quality of life measures often don‟t reflect
issues of most importance to patients
• Weighting on evidence from patients
• Process doesn‟t take account wider societal costs
• Technical language and economics are difficult to
engage with/challenge
• NICE „blight‟/ variability in access to NICE
recommended technologies
*from Victoria Thomas (NICE)
47
Recommendations for Patient Engagement with HTA
o Dialogue with patient groups to define submission: what information, how to collect, how to present
o Engage patients in CT design to ensure patient values
included in measures
o Train potential HTA patient-public members on technical processes of HTA and decision making
o Provide means for patient-public members of HTA
committees to dialogue with patient representatives
o Train all HTA committee members on methods for integrating qualitative information
o Promote transparent decision-making (records of
deliberation as well as outcomes); open meetings.
o COMING: Patient Experts in Health Technology!
Contact:
Durhane Wong-Rieger
Consumer Advocare Network
www.consumeradvocare.org
416-969-7435
49
Citizen engagement in HTA:A Brokered Dialogue with the
Ontario Citizens’ Council
Janet Parsons, PhDJim Lavery, PhD
What does better look like?• Overall, evaluation reveals that OCC is a success in the making
and an important step forward for the province
• Opportunities for improvement
Recommendations:
• Current mandate is vague, needs clarification
• Inviting citizens to the HTA table – be prepared
• Relationship management is critical
• More explicit accounts of how OCC reports are used
• KEY will be: Recognition that these outcomes and processes are
important, and social and political will to encourage governments to
adopt this framework for public engagement
What does better look like?
• Multiple points where communication and process are
important – points where members‟ confidence in the
process is put at risk:
• Selection of members
• Frequency of meetings
• Responses to reports
• Turnover of members
What does better look like?
• Value of the evaluation
• It helps identify what kinds of outcomes
matter, what kinds of processes might help to
improve these outcomes, and WHY
Acknowledgments
St. Michael’s Hospital
• Jim Lavery
• Wendy Rowland
• Muhammad Mamdani
• Andreas Laupacis
• Natalie Baker
• Applied Health Research Centre
OCC
• Study participants
Ontario MOHLTC/OPDP
• Diane McArthur
• Blair Parsons
• Ontario Citizens‟ Council Advisory Committee
Funders
• Ontario Drug Policy Research Network and Ontario MOHLTC
55
What does “better” look like: 2011-2012 CDR Patient Input Evaluation
Ensure all stakeholders are aligned on purpose and values of patient input Describe use of patient input in CDR deliberative process (done)
Consider input from individuals and caregivers
Enhance processes Update template
Provide greater advance notice (done)
Dedicate employee to patient engagement (done)
Obtain patient group feedback on summary of patient input
Provide opportunity to rank importance of outcomes
Provide more training opportunities
Goals
Design
Process
SECOR Evaluation now available at www.cadth.ca
What does “better” look like?
Patient engagement a way of life*
“Good” is evolving
Studies that include more outcomes and issues of importance to
patients
Processes that include greater consultation, participation and
communication
Evidence equally valued: patient, clinical and economic
Greater support and training
Increased transparency
* Canadian Foundation for Healthcare Improvement
What does “better” look like
CADTH has a phased approach for implementing
changes
Beginning in 2013
Beginning in 2014 and beyond
CADTH is piloting patient input in therapeutic
reviews
Beginning with: Comparative Clinical and Cost-
Effectiveness of Drug Therapies for Relapsing-Remitting
Multiple Sclerosis
What does “Better” Patient Engagement look like?
For Patients:
• Seeing/feeling that input has impact when it
matters
• Knowing what information to provide and what to
leave out
• Broadening the patient perspectives gathered
• particularly poor prognosis, rare or uncommon
cancers
• geographically remote cancer patients and
families
• cultural and/or socioeconomic barriers to
patient participation61
© 2013 pan-Canadian Oncology Drug Review61
What does “Better” Patient Engagement look like?
For pCODR:• Patients are aware of pCODR, what we do and what
we don’t do
• Patients are interested to participate in review
process
• Patient input and feedback has clear and credible
key messages
• Clinical trials capture outcomes and issues of
importance to patients
62© 2013 pan-Canadian Oncology Drug Review
62