PCORI Methodology Standards:

Academic Curriculum

© 2016 Patient-Centered Outcomes Research Institute. All Rights Reserved.

Prepared by Albert Wu, MD, MPH

Clifton Bingham, MD

Presented by Albert Wu, MD, MPH

Clifton Bingham, MD

Module 3: Overview

Category 2: Patient-Centeredness

What do we mean by patient (person)-centeredness?

Rationale for patient-centeredness standards

Introduction to stakeholders and engagement

Introduction to representativeness, recruitment, and retention

Introduction to patient-reported outcomes (PROs)

Introduction to dissemination

Outline

3

Patient-centered outcomes research (PCOR) is focused on questions and outcomes that

matter to patients and their caregivers

PCOR helps people and their caregivers to communicate and make informed health

decisions

PCOR addresses one of the “key questions” identified by PCORI:

1. “Given my personal characteristics, conditions, and preferences, what should I

expect will happen to me?”

2. “What are my options, and what are the potential benefits and harms of those

options?”

3. “What can I do to improve the outcomes that are most important to me?”

4. “How can clinicians and the care-delivery systems they work in help me make the

best decisions about my health and healthcare?”

What Is Meant by Patient (Person)–Centered Outcomes Research?

4

“Research done differently”

New framework and orientation

Methods in evolution

Provide guidance for investigators and reviewers

Part of the PCORI Merit Review Process

Why Do We Need Patient-Centeredness Standards?

5

Engaging patients, caregivers, and other stakeholders can contribute to several stages

of research

Defining topics and formulating research questions

Identifying a study population

Choosing interventions, comparators, and outcomes

Developing strategies for recruitment and retention

Conducting the study and analyzing results

Disseminating the research findings into clinical practice

Evidence is still limited on the impact of patient involvement on research quality

PCORI patient-centeredness standards aim for optimal approaches to engage

patients and other stakeholders

Understand how engagement affects study design and outcomes

Rationale for Patient-Centeredness in Research

6

Standards provide a framework to ensure that:

Appropriate stakeholders have been engaged throughout the process

Appropriate groups of people have been selected to participate in the research

Outcomes are those that are important to patients, caregivers, and other

stakeholders

Outcomes have been appropriately validated

Research provides information that will help to inform health decision making

Study results will be meaningful and understandable to different groups that may

use the research

Rationale for Patient-Centeredness Standards

7

Criterion 4: Patient-centeredness

Would the research improve processes to address questions about outcomes of

interest to patients and their caregivers?

Does the research address one or more of the key questions mentioned in PCORI’s

definition of PCOR?

PCORI Merit Review Criteria

8 Source: PCORI. Merit Review Criteria. Available at: http://www.pcori.org/funding-opportunities/merit-review-process/merit-review-criteria.

Accessed July 26, 2015.

Criterion 5: Patient and stakeholder engagement

Are patients and other stakeholders engaged meaningfully in appropriate phases of

the research? Are the roles and the decision-making authority of all research

partners clearly stated?

Does the proposal demonstrate the principles of reciprocal relationships; co-

learning; partnership; and trust, transparency, and honesty?

If engagement is deemed inappropriate in some or all aspects of the proposed

research, does the application justify why it is not?

PCORI Merit Review Criteria

9 Source: PCORI. Merit Review Criteria. Available at: http://www.pcori.org/funding-opportunities/merit-review-process/merit-review-criteria.

Accessed July 26, 2015.

PC-1: Engage appropriate stakeholders

PC-2: Identify, select, recruit, and retain representative study participants

PC-3: Use patient-reported outcomes when possible

PC-4: Plan for research dissemination and implementation

Patient-Centeredness Standards

10

What is a stakeholder?

People or groups that have an interest in a particular

clinical decision and the data that have been developed

to support that decision

Communities that may be affected by or use the results

of research

The people who need to be “around the table”

Stakeholders provide unique and valuable perspectives

beyond those of the researcher

Stakeholder involvement may ultimately lead to greater

use and uptake of research results by the patient and

broader healthcare community

What Is a Stakeholder?

11 Image: AlphaZeta. (September 27, 2014). Via Wikimedia Commons: https://commons.wikimedia.org/wiki/File:Helpinghands.svg. Licensed under the Creative

Commons CC0 1.0 Universal Public Domain Dedication (https://creativecommons.org/publicdomain/zero/1.0/deed.en). Accessed October 7, 2015.

PCORI stakeholder groups

Patients

Clinicians

Purchasers

Payers

Industry

Hospitals and health systems

Policy makers

Training institutions

A person can belong to more than one group

What Are Examples of Stakeholders?

Source: Concannon, T. W., Meissner, P., Grunbaum, J. A., et al. (2012). A new taxonomy for stakeholder engagement in patient-centered outcomes research.

Journal of General Internal Medicine, 27(8), 985–991. http://doi.org/10.1007/s11606-012-2037-1 12

What Are

Examples of

Stakeholders?

Source: Concannon, T. W., Meissner, P.,

Grunbaum, J. A., et al. (2012). A new taxonomy for

stakeholder engagement in patient-centered

outcomes research. Journal of General Internal

Medicine, 27(8), 985–991.

http://doi.org/10.1007/s11606-012-2037-1

Category Description

Patients and the

public

Current and potential consumers of patient-centered health care and

population-focused public health, their caregivers, families, and

patient and consumer advocacy organizations

Providers Individuals (e.g., nurses, physicians, mental health counselors,

pharmacists, and other providers of care and support services) and

organizations (e.g., hospitals, clinics, community health centers,

community-based organizations, pharmacies, EMS agencies, skilled

nursing facilities, schools) that provide care to patients and

populations

Purchasers Employers, the self-insured, government and other entities

responsible for underwriting the costs of health care

Payers Insurers, Medicare and Medicaid, state insurance exchanges,

individuals with deductibles, and others responsible for

reimbursement for interventions and episodes of care

Policy makers The White House, Department of Health and Human Services,

Congress, states, professional associations, intermediaries, and other

policy-making entities

Product makers Drug and device manufacturers

Principal

investigators

Other researchers and their funders

13

Engagement is meaningful involvement of patients, caregivers, clinicians, and other

relevant stakeholders

Involvement can occur at multiple points throughout the research process

Engagement can take place in many different ways and at different levels

Engagement varies according to the particular research project

What Is Engagement?

14

Stakeholder

Involvement in

Phases of

Research

Adapted from: Agency for Healthcare Research and

Quality (AHRQ). (February 2014). Stakeholder

Guide 2014. Available at:

http://www.ahrq.gov/sites/default/files/wysiwyg

/research/findings/evidence-based-

reports/stakeholderguide/stakeholdr.pdf. Accessed

October 7, 2015.

Prioritize research topics

Refine research topics

Conduct research

Comment on draft research results

Create audience-specific

information

Disseminate information

Use research

Suggest a research

topic

Research project

1.

Planning

2.

Conducting 3.

Disseminating

15

Available at http://www.pcori.org/sites/default/files/Engagement-Rubric.pdf

Provides guidance to applicants, reviewers, awardees

Variety of options and examples

Planning the study

Conducting the study

Disseminating the study results

Engagement principles

The PCORI Engagement Rubric

16

Reciprocal relationships

Co-learning

Partnership

Trust

Transparency

Honesty

PCORI Engagement Principles

17

PC-1: Engage appropriate stakeholders

PC-2: Identify, select, recruit, and retain representative study participants

PC-3: Use patient-reported outcomes when possible

PC-4: Plan for research dissemination and implementation

Patient-Centeredness Standards

18

Will the results of the research be applicable only to a small group of individuals with

certain characteristics, or will the conclusions be generalizable to a larger population?

Who needs to be included?

Stakeholder input can be helpful

Are there groups that are being missed that may be important?

• Multiple chronic conditions, low literacy, low socioeconomic status, poor access

to health care, racial and ethnic minorities, rural populations, etc.

• Are there other considerations within a disease that need to be considered

(e.g., disease duration, level of disease activity, prior treatment)?

Representativeness Is Important

19

How to identify, recruit, and retain patients

Timelines and milestones and modifications

Are there any anticipated challenges?

Strategies to address these, especially concerning populations of special interest

Recruitment and Retention

20

What is the necessary final sample size?

Are there any potential barriers for recruitment and strategies to overcome?

What will be done to maximize retention of subjects over the course of the study?

Why might people drop out? How many might not complete the study?

Are there any additional considerations for particular populations?

How can the study be publicized, if necessary? How can the patient or stakeholder

communities help to do this?

Recruitment and Retention of Study Participants

21

PC-1: Engage appropriate stakeholders

PC-2: Identify, select, recruit, and retain representative study participants

PC-3: Use patient-reported outcomes when possible

PC-4: Plan for research dissemination and implementation

Patient-Centeredness Standards

22

A patient-reported outcome (PRO) …

Can be a report coming directly from a patient about their health condition,

without interpretation from clinicians

Can also come from an appropriate proxy (e.g., parent for children, caregiver for

those unable to provide an accurate report)

Reflects both symptoms and impacts of the condition

Includes health-related quality of life

When patients or people at risk of a condition are the best source of information about

outcomes of interest, then the study should use PRO measures in lieu of, or in addition

to, measures derived from other sources

What Are Patient-Reported Outcomes?

23

Kinds of PROs

Image: OpenClips. (2014). Via Wikimedia Commons:

https://commons.wikimedia.org/wiki/File:Umbrella

-158164.svg. Available under the Creative

Commons CC0 1.0 Universal Public Domain

Dedication

(https://creativecommons.org/publicdomain/zero/

1.0/deed.en). Accessed October 7, 2015. 24

Health-related

quality of life Satisfaction

Behavior

Utilization

Utility

Symptoms

Value

PROs should reflect the appropriate concept

These will ideally align with those areas of health, symptoms, and impacts that

patients have identified as being important to them

Identify the relevant domains or aspects of the concept you need to measure

Multiple sources (patients, experts, literature review, etc.)

Identify or develop a series of questions or items that relate to the domain or concept

Develop a score for the questions

Pilot-test the instrument

Measurement Properties of PROs

25

Evaluating a PRO

Conceptual and measurement model

Reliability

Validity

Responsiveness

Interpretability

Respondent and administrative

burden

Alternative forms and cultural and

language adaptations

COSMIN Checklist

Source: Mokkink, L. B., Terwee, C. B., Patrick, D. L., et al. (2010). The COSMIN study reached international consensus on taxonomy, terminology, and definitions of

measurement properties for health-related patient-reported outcomes. Journal of Clinical Epidemiology, 63(7), 737–745.

http://doi.org/10.1016/j.jclinepi.2010.02.006 26

It is essential to know how to interpret the results of a PRO

What is a detectable difference?

What is clinically relevant or actionable?

A variety of methods are used to identify and define these

What Do the Results Mean?

27

PC-1: Engage appropriate stakeholders

PC-2: Identify, select, recruit, and retain representative study participants

PC-3: Use patient-reported outcomes when possible

PC-4: Plan for research dissemination and implementation

Patient-Centeredness Standards

28

How the results of the research are communicated

Results need to be understandable and usable for different audiences

Traditionally, communication is to other researchers and content experts (e.g., peer-

reviewed scientific journals, presentations at specialty meetings)

Scientists do a poor job of communicating results to others

Patients and stakeholders can help to identify “key messages” from research

“What resonated for me was …”

What Is Research Dissemination?

29

Patients and stakeholders can help to put research into terms that are understandable

by others in their community

“Lay” summaries, eighth-grade level or less

Additional methods of communication, beyond journal articles, may be important to

consider for effective dissemination

Internet, videos, magazines, radio, television

May require additional expertise and support to enable

What Is Research Dissemination?

30

Results of research need to be available

Report results in a timely manner

Prevent publication bias by reporting negative as well as positive findings

Facilitate reproduction of original analyses to increase the integrity of findings

Promote data sharing to enable conduct of additional analyses, thereby augmenting

the knowledge generated from the original study

Develop plans to achieve these

How can the research results be translated into action?

What does it mean in terms of making health decisions?

Stakeholders can play a role here

Dissemination and Implementation

31