Engineering a Learning Healthcare System

Mary T. Brophy, MD MPH

mary.brophy@va.gov

Point of Care Research

The State of Clinical Evidence Generation

• Randomized Controlled Trials Gold standard Resource intense Relatively few in number

• Observational studies Affordable Bigger n’s Contend with bias

Medical Evidence: Where Is It?

• Lots of guidelines – not much evidence

• Guidelines not implemented

• How can we develop more and better evidence?

Are Guidelines Based on Evidence?

• ACC/AHA Guidelines 1984-2008• Number of recommendations increased from 1330 to 1973

(+48%)• Only 314 of 2711 recommendations were evidence level A

(11%)• 1246 were level C (48%)• Only 245 of 1305 class I recommendations were level of

evidence A (19%) Too complex Barriers in process of care

Tricoci P et al. JAMA 2009;301:831-841

Do Practitioners Follow Guidelines?

• Multiple studies show that practitioners’ adherence to guidelines is often poor

• Reasons Not based on level A evidence Do not apply to individual patients

Calvin JE et al. Congest Heart Fail 2012;18:73-78; Westergaard B et al. Clin Toxicol 2012;50:129-135; Burns SM Clin Nurse Spec 2012;26:22-28

Traditional clinical trials are too expensive

• Require site and coordinating center study staff• Informatics infrastructure• Regulatory oversight

Informed consent , safety reporting and engagement in research

• Financing not aligned with needs of healthcare system

• Result: Not scalable for Clinical Effectiveness Research

Health R&D as a Percentage of Health Costs

Sources: NIH Data Book; Research!America, Investment in U.S. Health Research 2001, 2002, 2004-2009

Problem Statement

• Healthcare system’s research needs are not fully met by the current research enterprise Designed for basic science inquiry and drug discovery

• There is no scalable model for: Clinical Effectiveness Research

research comparing known treatments Biomarker validation (Personalized Medicine)

For diagnostics, prognostic or therapeutic indications

A Solution

• Creation of a Learning Healthcare System that creates locally applicable knowledge Identifies its’ own needs Uses its’ own infrastructure Adapts available research methodology Directly implements research results

• The knowledge gained is thus not generalizable (thus not ‘research’) but rather is ‘locally selfish’.

A powerful tool of the learning healthcare system

Point of Care Research(POCR)

POCR- Clinical Trial Example

• A clinical trial with a substantial portion of its operations conducted by clinical staff in the course of providing patient/subject’s routine clinical care and where the choice of treatment is between two “equivalent” options

POCR Clinical Trials - Approach

• Clinical research performed with minimal perturbation of the clinical care “ecosystem”

– Subject identification and randomization occurs at usual health care encounter

– Minimal study-defined procedures or visits (mostly directed via EMR)

– Outcome data captured passively

– Convert to decision support and implementation

Cohort Identification

Enroll & Consent

Randomize Intervention

Care providers using EMR

Data Capture ClinicalDecision Support

Study DB Analysis

Study team using traditional scientific tools

POCR Schema

POCR Requirements

• Next-generation EMR Incorporate study logic and workflows Data and knowledgebase connectivity Decision support sophistication

• Culture change for patients and providers• Change in regulatory requirements

Informed consent Engagement in research Adverse event reporting

POCR - Pilot Study Goals

A pilot study was adapted to assess the following:

1. Ability to modify the of EMR (VistA) to conduct the research

2. Physician and patient acceptance

3. IRB and regulatory acceptance of approach

4. Identification of logistical obstacles

POCR – Use Case

• Initiated as a Boston VA initiative• Insulin protocol

Sliding scale insulin regimen Weight based insulin regimen

• Both regimens are approved and in use at VA Boston

• No published data comparing outcomes

POCR – Insulin Protocol

• Open label RCT comparing the regimens• Inclusion: inpatients who require an insulin

regimen and are not in the ICU• Exclusion: inability to give informed consent• Endpoints:

Primary - LOS Secondary - inpatient glycemic control and

readmission within 30 days for glycemic control

POCR – Insulin Protocol

• Methods No modification of the current sliding scale or weight

based regimens as they exist in the VA Boston VistA system

Interface with clinicians entirely through the VistA packages

Data collection and follow-up is done passively through the VistA system

POCR in the VA

• VistA - VA EMR Use existing features within EMR

Alert System Order sets Consult System

Customize features Identify, enroll and randomize patients Embed processes in dialogue boxes

Recruitment Workflow

Select Option 1

View of the Endocrine Medication Menu in VistA: Consider Patient for Study Enrollment

Study Information andInstructions(select Yes or No)

Study Information Page

Dialog template for note(decision to enroll)

Enrollment Note Entry

Consent Process for the Study

• Randomization for insulin treatment introduced by clinician; patient willing to talk with study nurse

• Study nurse explains study and seeks consent• Signed Informed Consent – short, three-paged ICF with

three options: 1) randomization and chart surveillance, 2) no randomization and permit chart surveillance, 3) decline participation

• Signed HIPAA authorization

Closing the Implementation Gap

• Hybrid Bayesian/frequentist approach “adaptive learning” Use of Bayesian posterior probability to reset the

randomization (adaptive randomization) Use of conventional (frequentist) error rate calculations

to evaluate the evidence

• “Learning” promotes automated implementation of the winning strategy

POCR Recruitment Summary

Recruitment N (%)

Number of Eligible Patients 146

No response from clinician 17 (11.6%)

Clinician refusal 28 (19.2%)

Clinician participation 101 (69.2%)

Number of Eligible Patients Approached 101

Patients refusal 4 (4%)

Patients enrolled 97 (96%)

Randomized 88 (87%)

Chart review only 9 (9%)

From the Specific to the General

And

Future Direction

Point of Care Research(POCR)

Optimal Study Attributes for POCR

• Limited to questions of the type: which “approved” treatment works better?

• Interventions with well described toxicity

• Broad inclusion criteria; limited exclusion criteria

• Objectively identifiable endpoints

• Minimal need for study specific visits

Requirements For POC Adoption

• Rethink relationship between clinical care and research

• Revisit engagement in research, consent, adverse event reporting

• Educate to create a cultural change in ‘responsibilities’ of patients

• Reengineer the EMR infrastructure

• To allow questions to be asked and answered

• To facilitate decision-support adoption

• Develop novel analytical approaches

• New ideas for supporting research

Alternative to the investigator-initiated approach

Support of research by clinical care dollars

POCR - Advantages

• Pragmatic qualities address issues of Clinical Effectiveness• Results directly relevant to healthcare system (‘locally

selfish research’)• Ability to assess long-term clinically relevant outcomes

(lower cost)

• Faster (immediate) Integration of results into practice thereby lowering the T2 translation barrier• Enhanced acceptance by providers• Adaptive randomization • Conversion to a decision support node

• Improved logistics: • Facilitation of economic analysis• Accommodates device and strategy studies

POCR Trials Will Not:

• Explain how biological, etiologic, or behavioral mechanisms interact to produce the observed clinical outcomes

• Test new therapies where side effects are not well established

From the Present to the Future State

• Early introduction of Point of Care Research to regulatory/oversight thought leaders to plan the way forward

• Consider implementation requirements - from single site with highly engaged research personnel using in-person informed consent to national rollout

• Conduct focus groups of providers and patients to better understand acceptability

Ethical and Regulatory Issues in POCR

• Informed consent

• Engaged in research

• Serious adverse event reporting

Individual vs Collaborative Experiment

“…physicians make therapeutic experiments daily on their patients….medicine by its nature is an experimental science but it must apply the experimental method systematically.”

Claude Bernard, 1865

“This is what I regard as the University spirit, not simply diagnosing a patient and deciding what to do for him in order to earn our fee, but what we can get out of this case in order to do better next time.”

Obituary of Sir FrancisFrasier, BMJ 1964 in Baum, Lancet, 1986

“Little progress has been made resolving the paradox of informed consent being unnecessary for the uncontrolled experimentation of normal practice but required for clinical research.”

Hutton, 2001

Ethical “Benefits” of POCR

• Individualized clinical care linked with systematic research• Honest admission of uncertainty in medical decisions • Wider clinician and patient participation – thus, wider

applicability of results• Option for adaptive randomization – favoring more

participants receiving the beneficial intervention • Potentially shorter implementation times – thus,

potentially earlier results and benefits to healthcare system

Ethical “Risk” of POCR

• Harm to doctor–patient relationship and respect for doctor’s knowledge and authority

• Patient-centered care compromised

• Inadequate informed consent process– time-pressured– patient vulnerable to coercion – waiver of documentation

• Clinical care setting may impede providing patient with ongoing information relating to study participation

Informed Consent Debate

• All trial protocols must be reviewed by award-granting bodies and local ethics committees (IRBs) – that said….

• For Informed Consent: patients’ rights to self determination – study information improves patient understanding of nature of disease and treatment – signed consent provides physician and institutional protection

• Against Informed Consent: difficulty for patient to comprehend nature of disease, uncertainty of treatment and need for randomization – undermines confidence in doctor – bias created by refusals – delays inherent in consent process

from Baum “Do we need informed consent” The Lancet, 1986

Informed Consent Debate

• Fully informed consent is unobtainable ideal, concepts too abstract and details too technical; hence, all trials might be considered as unethical.

• One approach - abandon requirement of informed consent, relying on safeguards such as ethics committees (IRBs).

• Another approach - retain spirit of informed consent; take measures to maximize patient understanding; rely on ethics committees as a further level of protection.

• Authors favor last approach – allowing patients to bring values and knowledge into play, to understand concept of equipoise and to make expectations of intervention more realistic.

British Health Technology Assessment, 1998

Cluster Randomization

• Intact units are allocated different interventions. • Consent for intervention from a “guardian” – e.g. head of a

group practice or firm – best having several guardians to minimize conflicts of interest.

• Must always guard against “authority” submitting persons to unwarranted and/or unsafe experimentation.

• Must also consider validity of design and analysis, freedom to leave a trial, implications of early stopping of a trial.

Hutton, in “Statistics in Medicine” 2001; Sabin et al., Hastings Center Report 2008

Waiver of Informed Consent

• The research involves no more than minimal risk to the subjects

• The waiver or alteration will not adversely affect the rights and welfare of the subjects

• The research could not practicably be carried out without the waiver or alteration

• Whenever appropriate, the subjects will be provided with additional pertinent information

McCrae AD et al, Trials 2011;12:202; Weijer C et al, Trials 2011;12:100

Criteria for HIPAA Waiver

• Use or disclosure involves no more than minimal risk

• Research could not be practically done without waiver

• Privacy risks are reasonable vis. a vis. benefits to individuals or importance of research

• Plan to destroy identifiers exists

• Assurance that data will not be reused or disclosed to others except for research that would also qualify for a waiver.

OHRP Guidance on Engagement in Research

• Not engaged in research if: Provide services that would typically be performed as

part of routine clinical monitoring and/or follow-up of subjects

Do not administer the study interventions being tested or evaluated under the protocol

Do not obtain informed consent

SAE Reporting

• FDA is encouraging risk based monitoring• Require reporting AEs only if unexpected,

serious and would have implications for conduct of study

• What is unexpected? Single occurrence of serious event that is uncommon

and strongly associated with drug exposure or uncommon in study population

More frequent than expected More severe than expected

POCR – IRB Decision?

At what point in the spectrum of medical knowledge discovery are assumptions like these reasonable?

1) POC research comparing two approved interventions introduces no risks beyond those normally encountered by patients receiving such treatments, and thus, having provided patients with appropriate information and choices, informed consent can be waived.

……and

2) It is impractical to obtain HIPAA authorization in that informed consent was not required and since well-proven procedures were in place to ensure the confidentiality and security of any identifiers and protected health information obtained from medical records.

3) The treating clinicians, within their scope of practice, are simply offering patients either treatment A, treatment B or a roughly 50:50 chance of treatment A or B. Thus, they need not be identified or credentialed as members of the research team.

ExampleJune 2011

Dear Veteran,

We would like to inform you of a change in your prescription for mesalamine tablets or capsules. We will be changing your medication to mesalamine SA CAPSULES (APRISO®). Both products contain the same drug and work in a similar way. Both medications are safe and effective for your condition. Mesalamine is used for the treatment of ulcerative colitis and other conditions as determined by your provider.

Mesalamine SA CAPSULES (APRISO®) have the advantage of using one common dose and are only taken once a day.

Please finish all of your remaining supply of mesalamine EC TABLETS (ASACOL®) before starting your new mesalamine SA CAPSULES (APRISO®) prescription. Do NOT use both mesalamine products at the same time.

Read your prescription label carefully as the number of tablets and how often you take mesalamine may be different.

The doctors and pharmacists at your VA clinic have authorized this change. If you have any questions regarding this letter please call the pharmacy.

Sincerely,

The Boston VA Healthcare System Pharmacy Department

Discussion

• What level of patient and provider implicit or explicit “consent” is needed? Notification +/- opt out General enrollment into a program (non-study specific) Consent

verbal +/- witness written

Point of Care Research Team

• Principal Investigators: Louis Fiore and Philip Lavori• Co-Investigators: Mary Brophy, James Kaufman, Mike Gaziano and

Matt Liang and Ryan Ferguson • Informatics: Leonard D’Avolio and Chester Conrad• CPRS Engineers: Gus O’Neil and Tom Sabin• Ethics and Informed Consent: John Hermos• Pilot Content Expert: Stephen Swartz• Data Management: Lauren Weil• Statisticians: Philip Lavori, Robert Lew, Gheorghe Doros and Sarah

Leatherman

Sponsored by the

VA Cooperative Studies ProgramOffice of Research & Development

Department of Veteran’s Affairs

Additional NIH support to Stanford University is gratefully acknowledged

CTSA UL1 RR025744

CCSG P30 CA124435

Point of Care Research