Date post: | 16-Dec-2015 |
Category: |
Documents |
Upload: | sophie-miles |
View: | 216 times |
Download: | 0 times |
November 7, 2014
Health IT Implementation, Usability and Safety Workgroup
David Bates, chairLarry Wolf, co-chair
Workgroup Members
•David W. Bates, Brigham and Women’s Hospital
(Chair)
•Larry Wolf, Kindred Healthcare (Co-Chair)
• Joan Ash, Oregon Health & Science University
• Janey Barnes, User-View Inc.
• John Berneike, St. Mark's Family Medicine
• Bernadette Capili, New York University
• Michelle Dougherty, American Health Information Management Association
• Paul Egerman, Software Entrepreneur
• Terry Fairbanks, Emergency Physician
• Tejal Gandhi, National Patient Safety Foundation
• George Hernandez, ICLOPS
• Robert Jarrin, Qualcomm Incorporated
• Mike Lardieri, North Shore-LIJ Health System
• Bennett Lauber, The Usability People LLC
• Alisa Ray, Certification Commission for Healthcare Information Technology
• Steven Stack, American Medical Association
Ex Officio Members • Svetlana Lowry, National Institute of
Standards and Technology • Megan Sawchuck, Centers for Disease
Control and Prevention• Jeanie Scott, Department of Veterans
Affairs• Jon White, Agency for Healthcare
Research and Quality-Health and Human Services
ONC Staff • Ellen Makar, (Lead WG Staff)
1
Meeting Schedule
Meetings Task
Monday, September 22, 2014 2:00 PM-4:00 PM Eastern Time
• Review charge• Work to date=- background / history• Preliminary goals discussion of deliverable
Friday, October 10, 2014 1:00 PM-3:00 PM Eastern Time
• Presentation of usability research MedStar and NIST
Friday, October 24, 2014 1:00 PM-3:00 PM Eastern Time
• ECRI and TJC results of adverse event database analysis• Usability Testing • Implementation Science (field reports)• Certification – Alicia MortonFriday, November 7, 2014
1:00 PM-3:00 PM Eastern Time
Friday, December 12, 2014 1:00 PM-3:00 PM Eastern Time
• Post-implementation Usability & Safety, Risk Management & Shared Responsibility
• Safety Center Report Out • Realignment of timeline/ goals for 2015
2
Agenda
Objective: ONC Health IT Certification Program
1:00 p.m. Call to Order/Roll Call • Michelle Consolazio, Office of the National Coordinator
1:05 p.m. Context: Usability and Safety Criteria for the ONC Health IT
Certification Program• Larry Wolf, co-chair
1:20 p.m. ONC Health IT Certification• Alicia Morton, Office of the National Coordinator
1:40 p.m. How Usability of EHRs and Workflow Impact Patient Safety• Alicia Morton, Office of the National Coordinator• Lana Lowry, National Institute of Standards and Technology
2:55 p.m. Public Comment 3:00 p.m. Adjourn
4
ONC Health IT Certification
• 2014 Edition EHR Certification Criteria on “safety-enhanced design” (using UCD processes)
- Identify what is being done - Increased transparency based on information available through certification. See ONC’s CHPL site.
• ONC Authorized Certifying Body (ACB) can conduct surveillance in live environments.– ACB’s are “health oversight agencies” under HIPAA– See ONC FAQ #45
• CMS Fact sheet “flexibility rule” http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/Downloads/CEHRT2014_FinalRule_QuickGuide.pdf 5
Safety- Enhanced Design
6
New: Safety-enhanced design. User centered design processes must be applied to each capability an EHR technology includes that is specified in the following certification criteria:
§ 170.314(a)(1), (2), (6) through (8), (16) and (18) through (20) and (b)(3), (4), and (9).
http://www.gpo.gov/fdsys/pkg/FR-2014-09-11/pdf/2014-21633.pdf
Current: Safety-enhanced design. User-centered design processes must be applied to each capability an EHR technology includes that is specified in the following certification criteria:
§ 170.314(a)(1), (2), (6) through (8), and (16) and (b)(3) and (4).
UCD processes in 2014 Edition R2 Electronic Health Record (EHR) Certification Criteria
§170.314(g)(3) Safety-enhanced design. User -entered design processes must be applied to each capability an EHR technology includes that is specified in the following certification criteria:
§170.314(a)(1) Computerized provider order entry§170.314(a)(2) Drug-drug, drug-allergy interaction checks§170.314(a)(6) Medication list§170.314(a)(7) Medication allergy list§170.314(a)(8) Clinical decision support§170.314(a)(16) Inpatient setting only – electronic medication administration record§170.314(a)(18) Optional – computerized provider order entry – medications§170.314(a)(19) Optional – computerized provider order entry – laboratory§170.314(a)(20) Optional – computerized provider order entry – diagnostic imaging§170.314(b)(3) Electronic prescribing§170.314(b)(4) Clinical information reconciliation§170.314(b)(9) Optional – clinical information reconciliation and incorporation
Certification test procedure: http://www.healthit.gov/sites/default/files/170.314g3safetyenhanceddesign_2014_tp_approved_v1.3_0.pdf 7
Quality Management System
§170.314(g)(4) Quality management system. For each capability that an EHR technology includes and for which that capability's certification is sought, the use of a Quality Management System (QMS) in the development, testing, implementation and maintenance of that capability must be identified.– (i) If a single QMS was used for applicable capabilities, it would only need to be
identified once.– (ii) If different QMS were applied to specific capabilities, each QMS applied would
need to be identified. This would include the application of a QMS to some capabilities and none to others.
– (iii) If no QMS was applied to all applicable capabilities such a response is acceptable to satisfy this certification criterion.
Test procedure: http://www.healthit.gov/sites/default/files/170.314g4qms_2014_tp_approvedv1.2.pdf
8
ONC Health IT Certification ProgramCAPT Alicia Morton, DNP, RN-BC
9
ONC Health IT Certification Program
Workgroup Information Request /Items for today’s discussion:
• Brief overview of the ONC Health IT Certification Program– Current, 2014 Edition certification requirements and how
they are tested
• ONC CHPL site overview and revision plans• Surveillance program for certified products
10
Both cars meet baseline safety standards, functional conformance testing, vehicle emissions testing
Yet, those baseline standards are not the differentiator.
• Safe• Useful• Usable• Satisfying
11
How Does the ONC Health IT Certification Program Work?
Regulatio
n
• ONC issues a regulation that includes certification criteria (and associated standards) for health IT products and corresponding certification program requirements
Develope
rs
• Create health IT products that, at a minimum, meet the standards and certification criteria adopted by HHS in regulation
ONC-ATLs
• Test health IT products based on the standards and certification criteria adopted by HHS
ONC-ACBs
• Issue certification to tested health IT products; Conduct surveillance • Submit product information to ONC for posting on the Certified Health IT Product list (
CHPL)
Providers
& Hospitals
• Have assurances products meet specific certification criteria and associated standards• In the case of the EHR Incentive Program (aka MU), certified health IT in a specified
manner to attest/report to the program to receive an incentive and avoid a payment adjustment
12
ONC Health IT Certification Program Structure / Process
13
13
ONC reviews and posts certified
product to CHPL
ONCapproves
ONC-AAApproved Accreditor
accredits
NIST NVLAPNational Voluntary Laboratory
Accreditation Program
accredits
Developer
performs testing against criteria
ACBAuthorized
Certification Body*
ACBAuthorized
Certification Body*
ONC-ACBONC-Authorized
Certification Body
certifies tested products
Authorized Testing Body*
Authorized Testing Body*
ATLAccredited Testing
Laboratory
performs testing against criteria
ISO/IEC 17011
ISO/IEC 17025
NIST 150
NIST 150-31
auth
oriz
es
Product successfully passes testing
Product successfully achieves certification
ISO/IEC 17065
Seven Certification Criteria Categories in 2014 EditionClinical Care
CoordinationCQM Privacy &
SecurityPatient
EngagementPublic Health Utilization
CPOETransitions of Care
(1) Capture & exportAuthentication, access control, authorization
VDT Immunization InfoAutomated MU
Numerator Calculation
Drug-drug, drug-allergy checks
Transitions of Care (2) Import & calculate Auditable Events Clinical summary Immunization
TransmissionAutomated MU
Measure Calculation
Demographics E-prescribing Electronic reporting Audit report Secure messagingSyndromic
surveillance Transmission
Safety Enhanced Design
Vital signsClinical Info
ReconciliationAmendments
(HIPAA privacy)Reportable labs
transmissionQuality
management system
Problem listIncorporate lab
results Auto-log off Cancer info
Medication listSend labs to ambulatory
providersEmergency Access Cancer transmission
Med allergy list Data portability End-user device encryption
CDS Integrity
E-notesAccounting of
disclosures (HIPAA Privacy)
Drug-formulary
Smoking statusImage results Base EHRFamily HHx
Patient List CreationEMAR
Advance Directives
14
Helpful Clarifying Terminology
15
2014 Edition FR: Redefining Certified EHR Technology and Related Terms
1. Certified EHR Technology (CEHRT)—SPECIFIC to the EHR Incentive Program (MU)
2. Base EHR Definition (this subsumes the term “qualified EHR”)
3. Complete EHR Definition (going away with next rule-making as noted in 2014 Edition Release 2 FR. No effect on certification to the 2014 Edition.)
MU Measure
CMS sets specific provider performance
metric related to the use of certified capabilities
+Certified Capability in EHR Demonstration of MU=
ONC adopts certification criteria that specify
technical capabilities for EHR technology
An eligible provider reports their performance on each MU metric to CMS in order to receive an incentive or
avoid a penalty
An EP must e-prescribe more than 40% of their Rx’s
EHR technology required to be able create standardized e-Rx file
EP attests 73%
An EP must implement 5 CDS interventions related to 4 or more CQMs
EHR technology required to enable multiple CDS related functionalities
EP attests “yes”
An EP must record problems in standardized data for more than 80% of patients
EHR technology required to be able to record problems in SNOMED CT
EP attests 93%
ONC Health IT Certification and CMS Meaningful UseRelationship
16
Certified HIT Product List
• All products at www.healthit.gov/chpl • Test Result Transparency: The final rule
requires that ONC-ACBs submit a hyperlink of the test results used to issue a certification to a Complete EHR or EHR Module.
• Includes information on what was tested, write-ups on the “usability” assessments performed, and more.
17
Certified Health IT Surveillance
• ONC-AA performs surveillance & technical assessment of the ONC-ACBs– According to ONC program requirements and standards governing certification
bodies • ONC-ACBs perform surveillance of certified Health IT products
– Proactive and reactive• Proactive: ONC priority areas of Exchange, Safety, Security , Population Management
(quality measurement)• Reactive: product complaints, large number of inherited certified status request, etc.
– ONC Surveillance guidance/ priority areas:• First/ CY 2014
http://www.healthit.gov/sites/default/files/onc-acb_2013annualsurveillanceguidance_final_0.pdf
• Report for CY 2014 due to ONC late Feb 2015• Second, CY 2015
http://www.healthit.gov/sites/default/files/onc-acb_cy15annualsurveillanceguidance.pdf • Plans for CY 2015 just received and under review by ONC
• NIST NVLAP performs surveillance of the ONC-ATLs 18
Questions/Discussion /Contacts
ONC Health IT Certification Program: www.healthit.gov/certification
Certified Health IT Product List: www.healthit.gov/chpl
ONC Certification team mailbox: [email protected]
19
Lana Lowry NIST HIT Usability Project Lead
How Usability of EHRs and Workflow Impact Patient Safety
• Only safety related usability must be evaluated
• Helps vendors, hospitals, and other stakeholders to ensure that EHR use errors are minimized
• Provides technical guidance for summative usability evaluations prior to deployment or implementation of an HER
• The summative usability testing evaluation is meant to be independent from factors that engender creativity, innovation, or competitive features of the system
• Examples of safety-related usability issues that have been reported by healthcare workers include poorly designed EHR screens that slow down the user and might sometimes endanger patients, warning and error messages that are confusing and often conflicting, and alert fatigue (both visual and audio) from too many messages, leading users to ignore potentially critical messages.
EHR Usability Protocol (EUP) – a Core Validation Tool
21
Focus of the EUP
• Clearly distinguishes between usability aspects that pertain to user satisfaction and usability features that impact clinical safety
• Limited critical usability aspects that pertain to the clinical safety must be embedded into the system and must be required as a core functionality not a competition feature; a "barrier to entry" to the marketplace on safety is an expected outcome.
• Typical measures for clinical safety are adverse events (wrong patient, wrong treatment, wrong medication, delay of treatment, unintended treatment).
• Accepted usability/safety standards should be considered industry standard practices. Any company is free to go above and beyond the basic standard, however the minimum standards for usability in safety enhanced design should be established and articulated to address patient safety.
22
EUP is a model for understanding relationship between usability issues and patient safety outcomes through:
• Step I. Usability Application Analysis led by the development team, which identifies the characteristics of the system’s anticipated users, use environments, scenarios of use, and use related usability risks that may induce medical errors
• Step II. Expert Review/Analysis of EHR Application, an independent evaluation of the critical components of the user interface in the context of execution of various use case scenarios and usability principles
• Step III. Usability Testing, involving creation of a test plan and then conducting a test that will assess usability for the given EHR application including use efficiency and presence of features that may induce potential medical errors
EUP Key Areas
23
Three-step process for design evaluation and human user performance testing for the EHR
24
• EUP emphasis is on ensuring necessary and sufficient usability validation and remediation has been conducted so that
(a) use error is minimized and (b) use efficiency is maximized.
• The EUP focuses on identifying and minimizing critical usability issues
• The intent of the EUP is to validate that the application’s user interface is free from critical usability issues and supports error-free user interaction
Objective of the EUP
25
• Elimination of “never events”
• Identification and mitigation of critical use errors
• Identification of areas for potential UI improvement and record user acceptance / satisfaction
• Report summative testing results in CIF (Common Industry Format) www.nist.gov/customcf/get_pdf.cfm?pub_id=907312
Goal of the EUP
Relationship between EUP and NIST UCD Guidelines
27
UCD EUP
User Needs, Workflows & Environments Application Analysis
Engage Users Expert Review/Analysis of EHRs
Set Performance Objectives Application Analysis
Design Application Analysis
Test and Evaluate Usability Testing
EUP is a Key Component of UCD
28
• Performance is examined by collecting user performance data that are relevant indicators of the presence of safety risks
• These measures may include, but are not limited to, objective measures of successful task completion, number of errors and corrected errors, performance difficulties, and failures to complete the task successfully or in proper sequence
• Performance is also evaluated by conducting post-test interviews focused on what users identify as risks based on confusion or misunderstanding when carrying out directed scenarios of use
• The goal of the validation test is to make sure that critical interface design issues are not causing patient safety-related use error; in other words, that the application’s user interface supports error-free user interaction
How to Measure Performance?
29
Theoretical Example of EUP Scenario
30
Theoretical Example of EUP Scenario
31
Theoretical Example of EUP Findings
N Use Errors in Usability Test Severity Rating
6/18 When reviewing a patient chart, the clinician does not detect that new lab critical results are available
2
17/18 User copies the wrong (older) data from one note to the current note being written
3
1/18 When mcg/kg/min is ordered, the user incorrectly selects units of mcg/hr and accepts the wrong dose
4
15/18 A medication schedule is changed, the user fails to select the “refresh” button in the menu and does not comply with the new medication schedule
2
Scale is 1 (least) to 4 (most)32
Relating Usability and Patient Safety
33
Example: Wrong Patient Record Open
EHR: Patient A EHR: Patient B
Imaging: Patient A
34
Example: Wrong Mode for Action
Direct Dose Mode (mcg/min)Weight Dose Mode (mcg/kg/min)
35
Example: Incomplete Data Displayed
Lidocaine Hydrochlor
36
Example: Data not Readily Available
80 mg
37
Definitions
Usability: How useful, usable, and satisfying a system is for the intended users to accomplish goals in the work domain by performing certain sequences of tasks
Workflow: A set of tasks – grouped chronologically into processes – and the set of people or resources needed for those tasks that are necessary to accomplish a goal
Workaround: Actions that do not follow explicit rules, assumptions, workflow regulations, or intentions of system designers
38
Relating Workflow and Patient Safety
Poorly supported work processes suboptimal nonstandard care, poor decision support, dropped tasks
Missed information delays in diagnosis, missed/redundant treatment, wrong patient
Inefficient clinical documentation copy/paste, “smart text”, templates, scribes
Provider dissatisfaction workarounds, slower adoption rates in specialty areas
High rates of false alarms ignored alarms, alerts, reminders
39
Methods: Modeling with SMEs
Ambulatory care physicians; collegial discussions
Interdisciplinary team meetings – human factors, informatics, physicians
Process maps
Goal-means decomposition diagram
Insights for moving towards “patient visit management system”
40
Workflow “Buckets” in Ambulatory Care
Before patient visit
During patient visit
Physician encounter
Discharge
Visit documentation
41
Balance workload
Does pt have significant complexity?
Clinical overview and review new findings/labs
Review prior history and physical
Before Patient Visit
yes
no
42
Balance workload
Does pt have significant complexity?
Clinical overview and review new findings/labs
Review prior history and physical
yes
no
Before Patient Visit
43
Balance workload
Does pt have significant complexity?
Clinical overview and review new findings/labs
Review prior history and physical
yes
no
Before Patient Visit
44
Balance workload
Does pt have significant complexity?
Clinical overview and review new findings/labs
Review prior history and physical
yes
no
Before Patient Visit
45
Patient Checkin, Vital
Signs and Chief
Complaint
Get History, Signs and
Symptoms, review
Review of Systems,
make Presumptive
Diagnosis
Initiate intent to order medications, procedures, labs,
consults
Verify medications and allergies
Discharge
Patient education
Document relevant history, physical, assessment, plan
Documentation to support bil ling
Documentation for others (legal,
research, compliance, MU)
Does patient need education?
Yes
Does patient need a summary?
No
NoIs referral needed?
Explicit Orders: Medications, procedures, labs, imaging, consults/
referral
Does patient need a clinical procedure?
No
Yes Clinical Procedure
Warm up and
remember pertinent
information
Form initial treatment plan
Examine patient, physical
Yes Give patient summary
Physician and/or others tells/reviews
patient initial assessment, plan,
and “to do” activities, motivates
following plan
Pick ICD/CPT codes, verify insurance, investigate requirement for public
reporting
Is more information
needed?No
YesReview chart/
research guidelines, sideline consult
Collect Medicatio
n Reconciliation data
and Review of Systems
data
Verify dosage for some medications
Document medications reconciled
During Patient Visit
Physician Encounter
Before Patient Visit
Discharge Visit Docm
46
Recommendations for EHR Developers
• Increase efficiency: – Reviewing results with the patient– Drafting pre-populated orders to be formally executed later– Supporting drafting documentation with shorthand notations without a keyboard
• Design for empathetic body positioning/eye contact
• Support dropping tasks and delaying task completion
• Verification of alarms and alerts and data entry without “hard stops”
47
Recommendations for Ambulatory Care
• Moderate organizational design flexibility
• Design room to support patient rapport & EHR access
• Minimize redundant data entry via interoperability
• Reduce clinic pace or increase flexibility of pace
• Ensure functionality that supports continuity in task performance in the case of interruption
• Relax requirements to enter detailed data for others during fast-paced patient visits
48
49
Next Meeting: Friday, December 12, 2014 1:00 PM-3:00 PM Eastern Time