PCORI Methodology Standards:

Academic Curriculum

© 2016 Patient-Centered Outcomes Research Institute. All Rights Reserved.

Prepared by Hadi Kharrazi, MD, PhD

Dan Ford, MD, MPH

Presented by Hadi Kharrazi, MD, PhD

Module 9: Case Study (Others)

Category 7: Data Networks as Research-Facilitating Structures

DN-1 (requirements for the design and features of data networks) is discussed in

Modules 3, 4, 5, 6, and 7

A. Data Integration Strategy—Module 4

B. Risk Assessment Strategy—Module 6

C. Identity Management and Authentication of Individual Researchers—Module 6

D. Intellectual Property Policies—Module 7

E. Standardized Terminology Encoding of Data Content—Module 4

F. Metadata Annotation of Data Content—Module 4

G. Common Data Model—Module 5

DN-2 (selection and use of data networks) is discussed in Modules 8 and 9

Data Network Methodology Standards Mapping With Content

4

Mini-Sentinel

DARTNet: Distributed Ambulatory Research in Therapeutics Network

HMORN: HMO Research Networks

SCANNER: SCAlable National Network for Effectiveness Research

SHRINE/i2b2: Shared Health Research Information Network

eMERGE: Electronic Medical Records and Genomics

caBIG: Cancer Biomedical Informatics Grid

Regenstrief

Other Data Networks: Examples

4

Mini-Sentinel is a pilot project sponsored by the U.S. Food and Drug Administration

(FDA) to create an active surveillance system

The Mini-Sentinel project aims to inform and facilitate development of a fully

operational active surveillance system for monitoring the safety of medical products

Mini-Sentinel uses preexisting electronic healthcare data from multiple sources

Data partners for this network include (#18) Group Health, Harvard Pilgrim Health Care

Institute, Health Partners, Henry Ford, Lovelace, Marshfield, Myers / Fallon, Kaiser

Permanente (Colorado, Hawaii, North Carolina, Northwest, South Carolina),

Vanderbilt, Healthcore / Wellpoint, Humana, and Aetna

Information obtained through Mini-Sentinel is intended to complement other types of

data and information compiled by FDA scientists

Mini-Sentinel

5 Source: http://minisentinel.org.

“Mini-Sentinel uses a distributed data approach in which Data Partners maintain

physical and operational control over electronic data in their existing environments.”

“The Mini-Sentinel Common Data Model standardizes administrative and clinical

information across Data Partners.”

“Data Partners execute, within their own institutions’ firewalls, standardized

computer programs (e.g., modular programs) provided by the Operations Center or

project workgroups.”

“Data Partners then share the output of these programs with the Operations Center

and project workgroups, typically in aggregated form.”

“A key benefit of the distributed approach is that it minimizes the need to share

identifiable patient information.”

Mini-Sentinel: Architecture

6 Source: Mini-Sentinel Data Activities. Available at: http://www.mini-sentinel.org/data_activities/. Accessed September 2, 2015.

178 million members (assuming no double counting due to health plan change)

358 million person-years of observation time

48 million individuals currently enrolled, accumulating new data

35 million individuals have over 3 years of data

Age distribution:

35% 22-44 years

26% 45-64 years

<10% for other age ranges

3.9 billion dispensings

Accumulating over 45 million dispensings per month

4.1 billion unique encounters, including 42 million acute inpatient stays

Accumulating over 51 million encounters per month

30 million members with at least 1 laboratory result

Mini-Sentinel: Data Specification

7 Source: Mini-Sentinel Distributed Database “At A Glance.” Available at: http://www.mini-sentinel.org/about_us/MSDD_At-a-Glance.aspx.

Accessed September 2, 2015.

Mini-Sentinel: Additional Information

8

Distributed Database

Architecture and

Specification

http://www.mini-

sentinel.org/work_products/Data_Activities/Mini-Sentinel_Year-4-

Distributed-Database-Summary-Report.pdf

Common Data Model

http://www.mini-

sentinel.org/work_products/Data_Activities/Mini-

Sentinel_Common-Data-Model.pdf

Data Quality Review http://minisentinel.org/work_products/About_Us/Mini-

Sentinel_SOP_Data-Quality-Review-and-Characterization.pdf

Governance, Principles,

and Policies

http://www.mini-sentinel.org/work_products/About_Us/Mini-

Sentinel-Principles-and-Policies.pdf

“DARTNet is a rapidly growing collaboration of PBRNs [practice-based research

networks] that are working to build a national collection of EHR [electronic health

record] data, claims data, and patient-reported outcomes data”

“The networks seek to blend quality improvement, effectiveness, and translational

research with a data-driven learning system”

“The learning system includes advanced performance measures and assistance with the

development and deployment of clinical decision support systems”

“All DARTNet members benefit from the DARTNet Learning Community, which provides

benchmarked data to practices to identify high performers in clinical care”

DARTNet pilot studies include the comparative-effectiveness research (CER) and safety

of oral medications for type 2 diabetes, MRSA, therapies for major depression and

chronic kidney disease

Distributed Ambulatory Research in Therapeutics Network (DARTNet)

9 Source: Distributed Ambulatory Research in Therapeutics Network (DARTNet). Available at: http://www.aafp.org/patient-care/nrn/studies/all/dartnet.html.

Accessed September 2, 2015.

DARTNet is a federated network of EHR data and other clinical information from

multiple organizations

Databases reside in multiple member practices and are linked through a secure

web-based system so they can be searched and queried

Three models of data extraction and sharing:

1. OMOP XML Data Extract

• Import via ROSITA, which handles patient index matching, removing HIPAA PHI,

harmonization and standardization of the data, and minimal outputs for local

querying

2. Third-party EHR data extraction tools

• Local federated databases + ROSITA

3. Manual data extraction programs

• Local federated databases + ROSITA

DARTNet Project: Architecture

10 Source: DARTNet Institute. Data extraction. Available at: http://www.dartnet.info/technology.htm. Accessed September 2, 2015.

The 12 distinct research networks that make up DARTNet Institute offer access to

approximately 12.5 million patient visits per year, 5 million patient lives, and

approximately five billion data points

Partners types include:

Academic Partners: Northwestern University, Ohio State University, University of

Alabama at Birmingham, University at Buffalo, University of California San Diego,

University of Colorado Denver, University of Kansas, University of Minnesota,

University of Texas San Antonio, University of Vermont, and University of

Washington

Private Partners: ABC Crimson Care Registry, AAFP, and QED CINA Inc

CTSA Partners: 9 university-based Clinical and Translational Science Award centers

Network Partners: AppNet, eNQUIRNet, CoNNECT, CCPC, FREENET, LA_NET,

MAFPRN, NCNC, SAFINet, STARNet, UNYNet, and WPRN

DARTNet Project: Data Specification

11 Source: DARTNet Institute. Networks. Available at: http://www.dartnet.info/networks.htm. Accessed September 2, 2015.

Architecture and Data

Extraction Specification

http://www.dartnet.info/Technology.htm

Common Data Model (OMOP) http://www.dartnet.info/OMOPwork.htm

Reusable OMOP and SAFTINet

Interface Adaptor (ROSITA)

http://www.dartnet.info/Rosita.htm

The Scalable Architecture for

Federated Translational

Inquiries Network (SAFINet)

http://www.ucdenver.edu/academics/colleges/medicalschool/

programs/outcomes/coho/saftinet/Pages/default.aspx

DARTNet Datasets http://www.dartnet.info/datasets.htm

DARTNet Project: Additional Information

12

HMORN brings together the research departments of a number of healthcare systems

Collectively, HMORN represents more than 1,400 scientists and research staff with

methodological and content expertise from an array of disciplines

HMORN will be renamed as Health Care Systems Research Network (HCSRN)

HMORN’s VDW Data Area Workgroups:

Define, maintain, and interpret data file specifications

Propose new variables

Identify site-specific issues with data standards

Provide scientific input for each data area

Health Maintenance Organization Research Network (HMORN)

Source: Health Care Systems Research Network. Available at: http://www.hmoresearchnetwork.org. Accessed September 2, 2015. 13

HMORN Members

Baylor Scott & White Center for Applied

Health Research

Catholic Health Initiatives

Essentia Institute of Rural Health

Geisinger Center for Health Research

Group Health Research Institute

Harvard Pilgrim Health Care Institute

HealthPartners Institute for Education and

Research

Henry Ford Health System Research

Centers

Kaiser Permanente Division of Research

Kaiser Permanente Institute for Health

Research

Kaiser Permanente Center for Clinical and

Outcomes Research

Kaiser Permanente Department of Research

& Evaluation

Maccabi Institute for Health Services

Research

Marshfield Clinic Research Foundation

Meyers Primary Care Institute

Mid-Atlantic Permanente Research Institute

Palo Alto Medical Foundation Research

Institute

The Center for Health Research, Kaiser

Permanente Hawaii

The Center for Health Research, Kaiser

Permanente Northwest

14

HMORN Members

15

The VDW is a federated database in which each site stores its data locally in identical

data structures

Each site creates a series of datasets across selected content areas based on common

definitions and concepts

The purpose is to enhance efficiencies in performing multisite research

Each site retains ownership and control of its respective data

Periodic data quality checks “look at ranges, cross-field agreement, implausible data

patterns, and cross-site comparisons”

“Each institution’s VDW data remain at their site until a study-specific need arises.

The minimum necessary required data are extracted after ethical, contractual and

HIPAA requirements are met”

HMORN Virtual Data Warehouse (VDW) Architecture

16 Source: HCSRN. Data Resources. Available at: http://www.hcsrn.org/en/About/Data/. Accessed September 2, 2015.

More info: http://www.hcsrn.org/en/Tools%20&%20Materials/VDW/

VDW data include …

Enrollment/demographics

Census

Pharmacy

Utilization (procedure and diagnosis codes; inpatient and outpatient events)

Vital signs

Laboratory

Tumor registry

HMORN Virtual Data Warehouse (VDW) Data Specification

17

HMORN Virtual Data Warehouse (VDW) Data Specification

18

HMORN VDW Data Types

“Patient-centered SCAlable National Network for Effectiveness Research (pSCANNER) is

one of 11 clinical data research networks that comprise PCORnet”

“pSCANNER is designed to be a stakeholder-governed federated network that will

utilize a distributed architecture to integrate data from three existing networks

covering over 21 million patients”

Data sites in California include Veterans Health Administration system; University of

California, Davis; University of California, Irvine; University of California, Los Angeles;

University of California, San Diego; University of California, San Francisco; AltaMed;

Queenscare Clinics; and the Children’s clinics

Task forces include Data Harmonization, Technical and Analytics, IRB Coordination and

Streamlining, and Patient Engagement and PPRN

pSCANNER

Source: http://pscanner.ucsd.edu. Accessed September 2, 2015. 19

“The vision of pSCANNER is to provide a scalable and flexible distributed infrastructure

for collaborative comparative effectiveness research, while engaging patients and

clinicians”

“pSCANNER will be used for conducting multiple studies within the same computer

network, where each study might focus on different clinical domains, have varying

data needs and data models, and might have different data-sharing policies”

“SCANNER is comprised of a set of SSL-encrypted Web Services that allow a user to

perform distributed statistical analysis on data hosted on remote sites. Multiple

statistical analysis methods are supported via a plug-in architecture …”

pSCANNER uses OMOP CDM v4

pSCANNER Architecture

Source: http://pscanner.ucsd.edu/about/architecture. Accessed September 2, 2015. 20

pSCANNER

Architecture

Source:

http://pscanner.ucsd.edu/about/architecture.

Accessed September 2, 2015. 21

pSCANNER: OMOP

Mapping (Sample

UCSD Data)

Source:

http://pscanner.ucsd.edu/about/semantic-

interoperability. Accessed September 2, 2015. 22

Shared Health Research Informatics Network (SHRINE) is a federated query tool for

i2b2 databases

“… Harvard Medical School (HMS) proposed creating a web-based sotware network

(based on the prototype model SPIN, Shared Pathology Informatics Network) that

would alllow the participating Harvard hospitals to link their respective i2b2 instances

for the sharing of obfuscated, aggregated counts of patients meeting selected inclusion

and exclusion criteria for demographics, diagnoses, medications, and labs”

“It was envisioned that this network, called SHRINE (Shared Health Research

Informatics NEtwork), would greatly enable population-based research, assessment of

potential clinical trials cohorts, and hypothesis formation for followup study by

combining the EHR assets across the hospital system”

SHRINE/i2b2

Source: https://www.i2b2.org/work/shrine.html. Accessed September 2, 2015. 23

SHRINE/i2b2

“i2b2 (Informatics for Integrating Biology

and the Bedside) is an NIH-funded

National Center for Biomedical

Computing based at Partners HealthCare

System”

“The i2b2 Center is developing a scalable

informatics framework that will enable

clinical researchers to use existing

clinical data for discovery research and,

when combined with IRB-approved

genomic data, facilitate the design of

targeted therapies for individual patients

with diseases having genetic origins”

Source: https://www.i2b2.org/. Accessed September 2, 2015.

Central

“aggregator”

broadcasts

query to local

hospital

“adaptors”,

which return

aggregate

counts only

24

SHRINE/i2b2

25

i2b2 interface running on SHRINE

“eMERGE is a national network organized and funded by the National Human Genome

Research Institute (NHGRI) that combines DNA biorepositories with electronic medical

record (EMR) systems for large scale, high-throughput genetic research in support of

implementing genomic medicine”

“Each center participating in the consortium is uniquely situated to provide critical

resources to this highly collaborative and productive network”

“Each site combines a biobank or study cohort with extensive genomic data and access

to clinical data derived from electronic medical records”

“Sites are geographically dispersed and have diverse patient populations, including two

sites focusing specifically on pediatrics”

Electronic Medical Records and Genomics (eMERGE)

Source: https://emerge.mc.vanderbilt.edu. Accessed September 2, 2015. 26

Electronic

Medical Records

and Genomics

(eMERGE)

27

Electronic

Medical Records

and Genomics

(eMERGE)

Source:

https://emerge.mc.vanderbilt.edu/emerge-sites/.

Accessed September 2, 2015. 28

Other Data

Networks:

Available

Information

29

Summary of Research Networks: Architecture

30

Practice caBIG DARTNet HMORN Mini-Sentinel SCANNER Regenstrief SHRINE

i2b2 eMERGE

Architecture

Paradigm Distributed Distributed Distributed Distributed Distributed Central Distributed Unknown

Query

distribution/

data

request

process

Federated

queries

with local

storage

Federated with

local storage in

common data

model format

Ad hoc

(until

2011)

Publish and

subscribe with

local storage in

common data

model format

Multiple

Direct from

central

Repository

Federated Unknown

Data

integration

Strategy

Multiple

shared

data models

Common Data

Model

Common

Data Model

Common Data

Model Multiple

Common

Data Model

I2b2 Star

Schema Ad hoc

Security

standards

Tight Element-

based Access

Defense in

Depth Ad hoc

Defense in

Depth

NIST Level

3 Unknown Unknown Unknown

Summary of Research Networks: Governance

31

Guideline caBIG DARTNet HMORN Mini-

Sentinel SCANNER eMERGE

Timely research data sharing Yes Yes Yes Yes Yes Yes

Identity management and authentication of

individual researchers Yes Yes Unknown Unknown Yes Unknown

Healthcare and research network audits Unknown Unknown Unknown No Yes Unknown

Specific consent for data sharing with

informed consent for research Sometimes Unknown Yes NR

Per IRB

request Yes

Network trust, business associate

agreements

Contract

Agreements Unknown Yes Yes Yes Yes

Data use agreement Yes Unknown Yes No Yes Yes

Intellectual property policies No Unknown Yes Yes No Unknown

Regenstrief and SHRINE were not reviewed.

Summary of Research Networks: Governance

32

Guideline caBIG DARTNet HMORN Mini-Sentinel SCANNER eMERGE

Governing body NCI Board of Dir. Gov. Board FDA Board No Steering C.

Data access committees No Unknown Unknown Proposal

Review No Yes

Stakeholder engagement Workgroups Unknown Stakeholder

Council Privacy Panel

Patient

Focus Panel

Stakeholder

Engagement

Workgroup

Centralized coordinating center NCI University No FDA University University

Other committees and

workgroups for operational

performance

Architecture,

Clinical Trial,

Research,

Training…

Admin,

Technical,

Research…

Asset

Stewardship,

IRB

Coordinate,

KM group

Safety,

Operations,

Project

Management

Architec-

ture,

Policy, CER

Genomics,

Informatics,

Consent,

Oversight

Regenstrief and SHRINE were not reviewed.

Summary of Research Networks: Semantic Interoperability

33

Guideline caBIG DARTNet HMORN Mini-Sentinel SCANNER

Standardized

terminology encoding of

data content

VCDE Workspace

Terminologies

ICD9-CM,

RxNorm,

SNOMED-CT…

CPT, HCPCS, ICD9-

CM, Insurance

Claims…

ICD9-CM,

HSPCS/CPT,

NDC

ICD9-CM,

LOINC,

RxNorm, CPT…

Metadata annotation of

data content

ISO/IEC 11179

Standards

Continuity of

Care Records

(CCR)

No Limited meta-

data No

Common data model

Biomedical

Research

Integrated

Domain Group

(BRIDG)

No

Virtual Data

Warehouse (VDW)

specifies data

structure

Mini-Sentinel

Common Data

Model (MSCDM)

Observational

Medical Outcomes

Partnership

(OMOP)

*Regenstrief, SHRINE, and eMERGE were not reviewed.