Joint Exchange / Interop Work Group Test Workgroup
John Donnelly/Judith Huntman/Joe LamyAug 15, 2012
Roll
2
Name Organization RepresentationNick VanDuyne NYeC StateIvan Handler ILJohn Donnelly NJ Task Group Co-Chair - IWGMarty Prahl SSA
Federal agencyDennis Peterson, Curtus Browning, Elaine Hunolt, Taimur Aslam, Willie Singletary, Monica VanDyke VHA
Kevin Brady NIST NISTDavid Tao, Dan Huber Siemens
EHR VendorDave Cassel, Rob Klootwyk Epic Bill Howard, Charles Parisot GEJeff Cunningham, Tim Dunnington ICA
HIE vendor
Mike LaRocca Intersystems
Vladi Reznikov, Jennifer Puyenbroek CONNECT PMO
Matt Bell, Tom Wilson, Kathy Lin AxolotlTeddy Gedamu MedfxMelissa Owens Marshfield Clinic Health SystemSteve Leighty MedVirginia
Regional HIE Brian Bonnington, Dave Trepanier, Dave Smith, Steve Michael IHIE
Anuj Desai Interop WG Interop WGEric Heflin THSA Task Group Co-Chair - ExchangeMariann Yeager Exchange Exchange Ed O'Connor, Judith Hutman Nitor ONC Testing Contractor
Dennis Wilson, Alisa Ray, Sue Rieber CCHIT
ATCBs and Testing SMEs
Jim St. Clair IHE USA
Amit Trivedi, Kevin Brown, Michella Knighton ICSA Labs
Traci Mapps SLI Global SolutionsKyle Meadors, Timothy Bennett Drummond Group
Milton Padilla, Steve Wilson, Mark Shin, Doublas Biggs, Les Biggs InfoGuard
• Roll Call – Cayla• Announcements – Eric• Review of Phases – Judith (10 mins)• Vote on Harmonized Phase I Test Cases – John (10 mins)• Execution of Test Cases – Judith/Joe (10 mins)• Test Case Metrics – Judith/Joe (7 mins)• Open Discussion – All• Wrap Up - John• Schedule Review - John
Agenda
3
• Harmonized PD/QD/RD/MP/AF (aka Patient Record Lookup) Tests Are Ready for Vote!• Future changes are possible, but will be incorporated into the next
phase of test cases design and implementation (post Oct 2012)
• RFP• Has 5 respondents• Contracting is expected to begin this week
Announcements
4
Harmonization Phases
Phase 1 – SOAP Send/Receive
Phase 2 – Direct Send/Receive
Phase 3 – Secondary Methods, Service
Discovery, & Provider Directories
•Parent Use Cases (Done)•Test Case harmonization (Done)•Tests: Mess & Auth Framework (Done)•Tests: Patient Discovery (XCPD, PIX, PDQ), Query for Documents, Retrieve Documents (Done)
•Test Execution Guides (Done)
•Parent Use Cases•Test Case harmonization•Tests: Direct Applicability Statement (Done)
•Tests: Direct XDM/XDR-Bridge (Done)•Tests: Direct Certificate Discovery (Done)
•Test Execution Guide
•Parent Use Cases•Test case harmonization•Tests: Web Services Registry (Done)•Tests: HPD+•Tests: Document Submission (Done)•Tests: Administrative Distribution (Done)•Test Execution Guides
Unified Test Process – Has Been Proposed
Vote on Harmonized Artifacts
• PD, QD, RD, Messaging & Auth (Phase I)• Vote to ratify this workgroups decision that the current written test
cases for PD, QD, RD, MP, AF (aka Patient Record Lookup) are in “final” status
• ‘round-the-room vote:• Approve• Abstain – with reason stated• Do not approve
• Artifacts may be found at: http://exchange-iwg.wikispaces.com/Test+Artifacts+Harmonization
Phase I: SOAP send/receive Status
Parent Use Cases Done
Test Cases: Patient Discovery (XCPD, PIX, PDQ) Done
Test Cases: Query for Documents Done
Test Cases: Retrieve Documents Done
Test Cases: Messaging Platform & Authorization Framework Done
Test Case harmonization Done
Test Execution Guides Done
Harmonization Activity Status
8
Phase II: Direct send/receive Status
Parent Use Cases Done
Test Cases: Direct Done
Test Case harmonization In progress
Test Execution Guides (deprecate Mod Spec reference, document self-guided process through multiple modular testing artifacts)
In progress
Harmonization Activity Status
Harmonization Activity Status
Phase III: Secondary Specs Status
Parent Use Cases
Test Cases: Document Submission Done
Test Cases: Administrative Distribution Done
Test Cases: Web Services Registry Done
Test Cases: HPD+
Test Cases: Content Not started
Test Case harmonization Not started
Test Execution Guides In progress
Direct Harmonization: Deployment Models
• Deployment models:• EHR-HIE Test Spec identifies actors: assumes specific deployment models• ONC Test package: agnostic• Direct Project specifications do not specify a model
• Issues:• Specifying a deployment model may restrict testing or add requirements outside the scope
of the specs
• Solution:• Present EHR-HIE Test Spec deployment models and actors as examples of
implementation models• Map EHR-HIE Test Spec deployment models to our use cases• Show the outliers (what they are testing that is not in our scope)
Harmonization: Modular Approach (PD-QD-RD)
End User Auth
TC1
TC2
TC3
PD
PD1
PD2
PD3
QD
QD1
QD2
QD3
RD
RD1
RD2
RD3
Message
Render
MR1 MR2
MR3
EHR-HIE ApplicantProduction Participant
Messaging & Auth
TC1
TC2
TC3
Even
t-Bas
ed
Inte
rop
Test
ing TC
1TC2
TC3
PD1
PD2
QD1 RD
2
TC1
TC2
MR1
MR3
MR2
Harmonization: Modular Approach (Direct)
End User Auth
TC1
TC2
TC3
Direct Messagi
ng
DM1 DM2
DM3
Cert Discove
ryCD1
CD2
CD3
Message
Render
MR1 MR2
MR3
EHR-HIE ApplicantProduction Participant
Deplmnt A Reqts
A1 A2
A3
Even
t-Bas
ed
Inte
rop
Test
ing TC
1TC2
TC3
DM1
DM2
CD1
CD2
A1
A2
MR1
MR3
MR2
Deplmnt B Reqts
B1 B2
B3
DM3
CD3
A3
Deplmnt X Reqts
X1 X2
X3
Automating Test Case Execution
Messaging & Auth• Modular: New scripts for each web service• Some test cases implicitly covered by test cases in core specs
• Testing invalid SAML assertions• Critical to exchange security• Probably can’t be executed using CONNECT as test tool
Automating Test Case Execution
Direct• NIST: Direct messaging tool in development• ONC Test: Prototype Direct Cert Discovery tool• September 2012: Incorporated into NIST/ONC-Cert MU2 Test Procedures
Automating Test Case Execution: soapUI scripts complete
Priority PD QD RD TOTALSHigh 7 18 4 29Med 3 16 1 20Low 3 0 1 4TOTALS 13 34 6 53
‘round-the-room
What risks or other concerns do you have at this point in the process related to the harmonized test cases or automation approach?
Open Discussion
•RFP released — July 2nd •Questions received by 6 organizations•Answers distributed — July 13th•Bids due — July 20th•Expected award — August 15th•Contract start — TBD
Schedule Review
18
We are here