SNOW SHRINE metadata release notes€¦ · SNOW SHRINE metadata release notes Page 9 of 13...

SNOW SHRINE metadata release notes

A service of the Greater Plains Collaborative Research Network

Date 20190430 Version 20190430

SNOW SHRINE metadata release notes Page 2 of 13

Amendment History

Version Date Editor Comments

20180330 20180330 James Campbell

Jay Pedersen First release of SNOW SHRINE metadata

20181001 20181207

James Campbell

Jay Pedersen

Yeshwanth Narayana

Second release of SNOW SHRINE

metadata; revisions to ICD-10-CM, RxNorm,

NDC, SNOMED CT, LOINC

20190430 20190501 James Campbell

Jay Pedersen

Third release to include RXNORM and NDC

metadata from NLM for spring 2019

Review Timetable

Review date Responsible owner Comments

20180330 James R. Campbell

© University of Nebraska Medical Center

Nebraska Medicine


Table of Contents

1 Purpose ................................................................................................................................. 5

2 Audience of interest for this document .................................................................................. 5

3 Content scope and history ..................................................................................................... 5

4 Quality assurance, versioning and maintenance ................................................................... 6

5 Description of Release Datasets and Instructions for use ..................................................... 7

5.1 Reference copy of SCILHS 3.1b; renamed Accessible Research Commons for Health (ARCH)

Model Ontology ................................................................................................................................ 7

5.2 SNOW SHRINE metadata files; FULL_YYYYYMMDD release (versions 20180330 and

20181001) ......................................................................................................................................... 7 5.2.1 Pcornet_Demo_oracle.zip: ARCH (SCILHS 3.1b) metadata for DEMOGRAPHICS ............................. 7 5.2.2 Pcornet_diag.zip: ARCH (SCILHS 3.1b) metadata for DIAGNOSES; modified v20181001 to include

124503 SNOMED CT conditions, 93522 ICD-10-CM codes from 20181001 release; SNOMED CT metadata

searching supported via transitive closure tables ............................................................................................. 7 5.2.3 Pcornet_enc.zip: ARCH (SCILHS 3.1b) metadata for ENCOUNTERS .................................................. 7 5.2.4 Pcornet_enroll.zip: ARCH (SCILHS 3.1b) metadata for ENROLLMENT ................................................ 7 5.2.5 Pcornet_lab.zip for LABORATORY and PATHOLOGY .......................................................................... 7 5.2.6 Pcornet_med.zip for MEDICATIONS ...................................................................................................... 7 5.2.7 Pcornet_proc_nocpt.zip: ARCH (SCILHS 3.1b) metadata for PROCEDURES ...................................... 8 5.2.8 Pcornet_vital.zip: ARCH (SCILHS 3.1b) metadata for VITALS ............................................................... 8 5.2.9 KUMC_NACCR.zip: ................................................................................................................................. 8 5.2.10 AdapterMappings.csv: ........................................................................................................................... 8

5.3 SNOW SHRINE metadata files; DELTA_YYYYMMDD release ................................................... 8 5.3.1 Pcornet_lab_delta_{Oracle/Sqlserver/Postgres}.zip: .............................................................................. 8 5.3.2 Pcornet_med_delta_{Oracle/Sqlserver/Postgres}.zip: ............................................................................ 8 5.3.3 README_for_{Oracle/Postgres/Sqlserver}.txt ........................................................................................ 8 5.3.4 AdapterMappings.csv .............................................................................................................................. 8

6 Deployment Instructions ........................................................................................................ 8

6.1 About I2b2 metadata sets; for the beginner ................................................................................. 8

6.2 FULL release step-by-step .......................................................................................................... 9 6.2.1 Installing i2b2 client metadata ................................................................................................................. 9 6.2.2 Installing metadata to support SHRINE ................................................................................................. 10 6.2.3 Java memory parameters for the TomCat server which runs SHRINE ................................................. 11

6.3 DELTA release .......................................................................................................................... 11 6.3.1 SNOW_SHRINE_20180330 procedures ............................................................................................... 11

7 Acknowledgements ............................................................................................................. 12

7.1 Grant Support ........................................................................................................................... 12

7.2 SNOMED CT; IHTSDO Observables project ............................................................................. 12

7.3 College of American Pathologists .............................................................................................. 13

7.4 LOINC ....................................................................................................................................... 13

7.5 National Library of Medicine ...................................................................................................... 13


7.6 Grant Funding ........................................................................................................................... 13


1 Purpose

This document summarizes the datasets, use cases and deployment of i2b2 metadata to

support interoperation of the SNOW SHRINE research data network and in service to the

larger community of SHRINE affiliated datamarts.

2 Audience of interest for this document

Interested parties may include:

SNOW SHRINE and collaborating SHRINE datamart information managers,

programming staff and administrators

I2b2 user community members with interest in interoperation of queries with others

employing Office of the National Coordinator (ONC) for Health IT terminology

standards

Research network data managers evaluating resources for use by their collaborators

3 Content scope and history

Dr Isaac Kohane, Dr Shawn Murphy and a team at Harvard University led the project which developed

i2b2. Staff at Harvard developed the protocols of the SHRINE network which allows i2b2 datamarts to

share queries across research networks. In order for those queries to interoperate, i2b2 concept

metadata and concept coding of observation facts must be aligned or extensively mapped at every

site supporting a datamart. In view of the fact that a large fraction of i2b2 research data originate in

the electronic health record (EHR), the semantic interoperability framework imposed by ONC

pragmatically defines a major portion of the terminology standards that could contribute to rapid

development of interoperability in any US research data network.

Harvard promoted a metadata set for SHRINE networks which it named in reference to their

information management scheme “Scalable Collaborative Infrastructure for a Learning Healthcare

System” (SCILHS). In fall of 2017 the Greater Plains Collaborative (GPC) network evaluated SCILHS

metadata for support of interoperation within the SNOW SHRINE network it was deploying with hub at

the University of Wisconsin, Madison. GPC identified opportunities for enhancing SCILHS 3.1b

metadata content for medications, laboratory results and diagnoses(problems/conditions) to assure

comprehensive use of ONC data standards. GPC consulted with Harvard in 2017 to propose an

expanded metadata set which UNMC would support on a regular maintenance cycle in order to

maintain currency with ONC terminology standards evolution and provide versioning and

synchronization of SHRINE networks for interoperation.


UNMC began publishing SNOW_SHRINE metadata 20180330 and supports FULL implementations

consisting of pipe delimited TXT or comma delimited CSV which can build all version-specific tables.

First release is also accompanied by a DELTA dataset which consists of TXT files of SQL commands

that may be executed to convert an existing deployment of (previous) version to the new release

version. The first DELTA to be published 20180330 consisted of version changes to move from

SCILHS(ARCH) 3.1b to SNOW_SHRINE_20180330.

Publication date

Version Content

3/30/2018 20180330

SCILHS 3.1b content AND enhancements which include

added content:

a) MEDICATIONS includes historically

comprehensive RXNORM

(IN/SCD/GPCK/SBD/BPCK) and NDC codes

indexed by ingredient

b) LABORATORY with expanded LOINC 2.61 lab

metadata covering 2484 tests in lab, anatomic and

molecular pathology

12/7/2018 20181001 New content development:

a) ENCOUNTER DIAGNOSES: ICD-10-CM updated

with additional 1485 concepts from CMS to

20181001; 30 code revisions this release

b) PROBLEM LIST: SNOMED CT conditions added

with US extension 20180930

c) LABORATORY expanded to include 218 more

molecular pathology and microbiology

d) MEDICATIONS updated with 1461 new RxNorm

and 9781 NDC from NLM release files 20181001

5/1/2019 20190501 New content development:

a) MEDICATIONS updated with 115879 RXCUIs and

559664 NDC codes updated from NLM 20190423

4 Quality assurance, versioning and maintenance

Relevant International and ONC data standards included within each SNOW SHRINE version follows:

SNOW SHRINE version ONC Reference Standard versions

20180330 LOINC 2.61; RXNORM 20180130; ICD-10-CM 2017; ICD-9-

CM 20141001; HCPCS(includes CPT4® but discussion

ongoing with American Medical Association)


20181001 LOINC 2.61; RXNORM 20181001; NDC 20181001; ICD-10-

CM 20181001; ICD-9-CM 20141001

5 Description of Release Datasets and Instructions for use

5.1 Reference copy of SCILHS 3.1b; renamed Accessible Research

Commons for Health (ARCH) Model Ontology

May be accessed at https://github.com/ARCH-commons/arch-ontology

5.2 SNOW SHRINE metadata files; FULL_YYYYYMMDD release (versions 20180330 and 20181001)

Contains all metadata tables, adapter mappings and TABLE_ACCESS modifications to install

SNOW_SHRINE i2b2 node including the NAACCR data tables developed at KUMC.

5.2.1 Pcornet_Demo_oracle.zip: ARCH (SCILHS 3.1b) metadata for DEMOGRAPHICS

Metadata for DEMOGRAPHICS in Oracle and SQL Server formats includes:

a)Pcornet_demo_{Oracle/SQLServer_metadata.txt: metadata for DEMOGRAPHICS table,

b)Pcornet_demo_Oracle_schemes.txt, c)Pcornet_demo_oracle_table_access.txt: additions to

Table_Access for including demographics in deployment scheme

5.2.2 Pcornet_diag.zip: ARCH (SCILHS 3.1b) metadata for DIAGNOSES; modified

v20181001 to include 124503 SNOMED CT conditions, 93522 ICD-10-CM codes

from 20181001 release; SNOMED CT metadata searching supported via transitive

closure tables

5.2.3 Pcornet_enc.zip: ARCH (SCILHS 3.1b) metadata for ENCOUNTERS

5.2.4 Pcornet_enroll.zip: ARCH (SCILHS 3.1b) metadata for ENROLLMENT

5.2.5 Pcornet_lab.zip for LABORATORY and PATHOLOGY

Expanded laboratory metadata set which includes 1880 common tests for laboratory, anatomic and

molecular pathology. 20181001 release file is I2B2_OBSERVABLES.txt for lab metadata.

5.2.6 Pcornet_med.zip for MEDICATIONS

Expanded metadata schema for orderable, dispensed and administered medicinal substances meant

to be comprehensive for the history of clinical drug use in the US pharmacopoeia. The schema offers

classification by VA drug therapeutic classes as well as alphabetically by ingredient. The

classifications are linked to orderable medications and then to NDC package codes at the base level.

This release includes 109154 RXCUI( of type = IN,MIN,PIN,SCD,SBD,GPCK or BPCK) and 527401

NDC, each representing a unique combination of manufactured product and package.


5.2.7 Pcornet_proc_nocpt.zip: ARCH (SCILHS 3.1b) metadata for PROCEDURES

5.2.8 Pcornet_vital.zip: ARCH (SCILHS 3.1b) metadata for VITALS

5.2.9 KUMC_NACCR.zip:

University of Kansas development of NAACCR tumor registry datasets; includes a)

Shrine_ont.naaccr_ontology.zip: metadata files in csv format, b) Adapter_mappings_naaccr_1to1.zip:

adapter mappings for NAACCR metadata

5.2.10 AdapterMappings.csv:

Adapter mappings for all other metadata types required for deployment of SHRINE network query

functionality using SNOW SHRINE metadata, available only for the 20180330 release

5.3 SNOW SHRINE metadata files; DELTA_YYYYMMDD release

This first DELTA release consists of SQL executables in TXT files which can update the

implementation of the SCILHS 3.1b i2b2 metadata to SNOW_SHRINE_20180330.

5.3.1 Pcornet_lab_delta_{Oracle/Sqlserver/Postgres}.zip:

SQL command file which incrementally modifies the LAB table from SCILHS 3.1b to current version

SNOW_SHRINE

5.3.2 Pcornet_med_delta_{Oracle/Sqlserver/Postgres}.zip:

SQL command file which incrementally modifies the MED table from SCILHS 3.1b to current version

SNOW_SHRINE

5.3.3 README_for_{Oracle/Postgres/Sqlserver}.txt

Contains a brief procedural description for deploying the delta release in each of three common

operating systems

5.3.4 AdapterMappings.csv

Contains a copy of full adapter mappings for deployment of SNOW SHRINE

6 Deployment Instructions

6.1 About I2b2 metadata sets; for the beginner

Metadata tables, sometimes referenced by users as ‘ontologies’, support i2b2 aggregate query

functionality in the user client and across SHRINE networks. They do this by organizing code

references stored in OBSERVATION_FACT.CONCEPT_CD in hierarchical arrays in the metadata

tables where the code reference is contained in the attribute C_BASECODE. The hierarchical

structure is configured in the attribute C_FULLNAME which specifies all the hierarchical paths which

may lead to the concept of interest. Steps in the path which may be queried as ‘parent’ concepts –


supporting aggregation queries - are called ‘folders’ in the client and are the ontological supertypes of

all nodes descendant in the ontology.

Metadata are deployed in the i2b2 metadata schema as a set of relational tables, one for each data

type stored in i2b2, such as demographics, diagnoses, medications, etc. Each metadata row in the

table specifies one path to a particular C_BASECODE concept reference and includes

C_METADATAXML which supports query functionality for the concept as well as details of aggregate

searching in C_COLUMNNAME, C_OPERATOR and C_DIMCODE.

The metadata deployment for a datatype is enabled when a row is created in TABLE_ACCESS

specifying the table name in C_TABLE_NAME and the root of the ontology in C_FULLNAME.

Installing an i2b2 metadata schema therefore, consists of creating a metadata table for each datatype

to be supported, adding the table name and metadata details to TABLE_ACCESS and then running a

utility to create the CONCEPT_DIMENSION table which supports client searching. Metadata tables

may specify the use of certain code sets for indexing OBSERVATION_FACT.CONCEPT_CD. The

code sets must be identified as coding systems in a SCHEMES metadata table.

Metadata tables may also contain rows which define modifiers for the datatype. Modifiers are an

additional i2b2 search tool which support query of subsets of concepts, effectively enforcing a data

model in i2b2. Modifier paths are specified in C_BASECODE and are applied to hierarchies in

CONCEPT_DIMENSION as specified in the attribute M_APPLIED_PATH. A table row with

M_APPLIED_PATH not equal to ‘@’ indicates that that row is a modifier definition. Once the

metadata table is installed, modifiers are deployed in the schema by running a utility which creates the

table LOAD_MODIFIER_DIM.

Sites which are participating in SHRINE networks have an additional requirement for installing adaptor

mappings, which support mapping a network query to the specific paths and structures of the site

metadata.

6.2 FULL release step-by-step

6.2.1 Installing i2b2 client metadata

Extract the zipped file SNOW_SHRINE_FULL_YYYYMMDD to expose a folder ‘MetadataFiles’ and a

zipped file ‘AdapterMappings.zip’. Within the ‘MetadataFiles’ folder you will find zipped sets of TEXT

files named pcornet_<type>,datestamp.zip, where type is lab, med, etc. The following table describes

the names and purpose of the enclosed files.

File Description

pcornet_<type>_metadata.txt CSV or ‘|’ delimited file containing the entire

contents for one metadata table. This should be

used to replace any existing content in the

metadata table.


pcornet_<type>_table_access.txt Text file containing the row entries to update the

TABLE_ACCESS entry for the table

pcornet_<type>.

pcornet_<type>_schemes.txt Text file containing entries for the SCHEMES

table in the i2b2 ONT cell associated with the

pcornet_<type> table.

The following should be done if the pcornet_<type> table is not currently defined in the

TABLE_ACCESS table in the i2b2 ONT cell, otherwise omit this step. Insert the entry for the

pcornet_<type> table into TABLE_ACCESS by executing the SQL in the

pcornet_<type>table_access.txt file.

The SNOW_SHRINE_20180330 FULL release replaces the contents of the SHRINE metadata tables

in the i2b2 ONT cell. The pcornet_<type>_metadata.txt file contains the full content for the metadata

table for that type. It is a pipe-delimited, text-in-quotes file. Replace the contents of the metadata

table by truncating the table and loading this file into the table C_TABLE_NAME attribute in the

associated TABLE_ACCESS row entry. Do this for demographics, diagnoses, encounters, enrollment,

laboratory, medications, procedures and vitals.

The ‘KUMC_NAACCR.zip’ metadata set is tumor registry metadata for SNOW SHRINE datamarts but

is free for use by any site wishing to use it. As you create each metadata table, load the associated

code system reference data into a SCHEMES table that you create.

Note that empty strings should be imported as NULLs otherwise m_exclusion_cd='' breaks the

modifiers!

Your i2b2 system should have utilities: ‘LOAD_CONCEPT_DIM’ and ‘LOAD_MODIFIER_DIM’.

These utilities should be run now to create ‘CONCEPT_DIMENSION’ and ‘MODIFIER_DIMENSION’

tables after metadata refresh. They should be rerun everytime any change in made to ontology

metadata or modifier metadata respectively.

6.2.2 Installing metadata to support SHRINE

SHRINE datamarts have a separate scheme for SHRINE functionality. The release includes a

compressed file in FULL\MetadataFiles.shrine.zip. The compressed file is ‘shrine.csv’ and should be

extracted and installed as a replacement for the corresponding table in the SHRINE scheme.

The compressed file FULL\AdapterMappings.zip contains the file AdapterMappings.csv. Extract the

‘AdapterMappings.csv’ file and use it to replace the existing AdapterMappings.csv file in the tomcat/lib

folder on your Shrine Spoke machine. This is the library folder associated with the TomCat web

server software which runs the SHRINE software. For example, the path to the file is

/opt/shrine/tomcat/lib/AdapterMappings.csv at a known site. After installing the file, the Tomcat server


software should be restarted to cause the SHRINE software to load the updated mappings in the

AdapterMappings.csv file.

6.2.3 Java memory parameters for the TomCat server which runs SHRINE

The SNOW_SHRINE_20180330 release significantly increases the number of paths defined in the

AdapterMappings.csv configuration file used by the SHRINE software. This may increase the amount

of memory needed by the TomCat server. We suggest that the server should be allowed at least 2

gigabytes of memory.

Within the bin directory of the TomCat server (e.g. /opt/shrine/tomcat/bin), the file setenv.sh defines

Java configuration parameters for the TomCat server, including memory settings. The following is an

example (sentenv.sh) script that sets the memory allocation for the TomCat server to 2 gigabytes, it

does this through the “-Xms” parameter.

export CATALINA_OPTS=" -XX:MaxPermSize=256m -Dakka.daemonic=on -Xms2g”

6.3 DELTA release

6.3.1 SNOW_SHRINE_20180330 procedures

Files in the MetadataFiles folder which relate to the incremental updating of SHRINE metadata tables

from SCILHS 3.1b to SNOW_SHRINE_20180330 include:

#1. pcornet_lab_delta_oracle.zip -- for updating lab metadata (pcornet_lab table).

#2. pcornet_med_delta_oracle.zip -- for updating medications metadata (pcornet_med table).

There are README.txt files in the release folder which have instructions for installing the DELTA in

systems with Oracle, SQL Server or Postgres. The command structures are somewhat different for

each but the instructions below ONLY RELATE TO ORACLE by way of example.

Example: Update the metadata in the pcornet_lab table using the DELTA dataset

(NOTE: An analogous procedure would update the MED metadata, replace "lab" by "med" for this)

Unzip the file pcornet_lab_delta_oracle.zip in the MetadataFiles folder and copy

pcornet_lab_delta_oracle.sql file to the machine where Oracle resides. This file contains the SQL

commands to modify a 3.1B version of the pcornet_lab metadata table for ARCH to

SNOW_SHRINE_20180330 version.

The SQL assumes that the lab metadata table is named pcornet_lab. It is possible that the table at

your site has a version number in the name, such as pcornet_lab_2_1_2. If that is the case, then you

can still update your metadata table using this SQL by temporarily renaming the table to pcornet_lab,

applying the SQL, and then renaming to the original name. The following SQL commands would

rename a table named pcornet_lab_2_1_2 to pcornet_lab, and shows analogous SQL would be used

to restore the original name after applying the SQL file.


>alter table pcornet_lab_2_1_2 rename to pcornet_lab

At this point, the SQL can be applied to update the pcornet_lab table to the

SNOW_SHRINE_20180330 version. After that, the following could be used to restore the original

table name:

>alter table pcornet_lab rename to pcornet_lab_2_1_2

The following procedure applies the SQL in pcorent_lab_delta_oracle.sql to the pcorent_lab table:

Start the Oracle sqlplus utility, using the Oracle account associated with SHRINE table maintenance.

The following command sequence starts the sqlplus utility on a Linux system running Oracle:

sqlplus "<type Oracle account associated with SHRINE metadata>"

Enter password: <type password for account>

You are prompted by:

SQL>

Enter the following sequence of commands:

SQL> set define off

SQL> set escape off

SQL> start pcornet_lab_delta_oracle

SQL> commit;

SQL> exit

Execution may take some time but will update the lab metadata file to SNOW_SHRINE_20180330

metadata.

7 Acknowledgements

7.1 Grant Support

We gratefully acknowledge support from University of Nebraska Medical Center Departments of

Internal Medicine, Pathology and Emergency Medicine; Nebraska Medicine and the Research

Information Technology Office (UNMC) in managing this work.

7.2 SNOMED CT; IHTSDO Observables project

This material includes SNOMED Clinical Terms® (SNOMED CT®) which is used by permission of the

International Health Terminology Standards Development Organisation (IHTSDO). All rights reserved.

SNOMED CT®, was originally created by The College of American Pathologists. “SNOMED” and

“SNOMED CT” are registered trademarks of the IHTSDO.


We wish to thank Farzaneh Ashrafi, Suzanne Santamaria, Daniel Karlsson and all members of the

project team for their assistance and advice in developing this material.

7.3 College of American Pathologists

We wish to thank Raj Dash, Mary Kennedy, Alexis Carter and Ted Carithers as well as members of

the International Pathology and Laboratory Medicine (IPaLM) Special Interest group of the IHTSDO for

their work and advice in developing this material.

7.4 LOINC

This material contains content from LOINC® (http://loinc.org). The LOINC table, LOINC codes, LOINC

panels and form file, LOINC linguistic variants file, LOINC/RSNA Radiology Playbook, and

LOINC/IEEE Medical Device Code Mapping Table are copyright © 1995-2016, Regenstrief Institute,

Inc. and the Logical Observation Identifiers Names and Codes (LOINC) Committee and is available at

no cost under the license athttp://loinc.org/terms-of-use.

We wish to thank Daniel Vreeman, Katie Allen and Swapna Abhyankar of Regenstrief Institute for the

hours they have spent creating LOINC term sets and advising us on this development.

7.5 National Library of Medicine

We wish to thank Vivian Auld, Susan Roy and James Case of the NLM for their kind advice and

support in this project. We are grateful for support of publication of the Nebraska Lexicon© by

reference on the UMLS Terminology Server.

7.6 Grant Funding

NIH 5U01HG009455-02

Deploying ONC National Standards in Support of Metadata for Big Data Research

Warehouse Management

PCORI CDRN-1501-26643 CDRN 04:The Greater Plains Collaborative

NIH 1U54GM115458-0, Subproject ID 5422 Great Plains Idea-Ctr-Biomedical Informatics and

Cyberinfrastructure Enhancement Core

Date post:	30-May-2020
Category:	Documents
Upload:	others
View:	9 times
Download:	0 times

SNOW SHRINE metadata release notes€¦ · SNOW SHRINE metadata release notes Page 9 of 13...

Documents