Automatic Detection of Policies from Electronic Medical Record Access Logs

TRUST Autumn ConferenceNovember 11, 2008

John M. Paulett †, Bradley Malin†‡

† Department of Biomedical Informatics‡ Department of Electrical Engineering and Computer Science

Vanderbilt University

Privacy in Healthcare

Sensitive Data– Patients speak with expectation of

confidentiality– Socially taboo diagnoses– Employment– HIPAA

11/11/2008 2

TRUST

Language for specifying temporal policies– Barth et al.

Framework for integrating policies with system and workflow models– Werner et al.

Model Integrated Clinical Information System (MICIS)– Mathe et al.

11/11/2008 3

Status

TRUST tool to formally specify, model, and managing policies in the context of existing and evolving clinical information systems

But, where do these policies come from?

11/11/2008 4

External Threat

Success with standard security best-practices

11/11/2008 5

Insider Threat

Motivation– Celebrities– Friends / Neighbors– Coworkers– Spouse (divorce)

Evidence of misuse– 6 fired, 80 re-trained – University of California, Davis– 13 fired for looking at Britney Spears’ record – March

2008– George Clooney – October 2007

11/11/2008 6

Protecting Against Insiders

• Access Control– Limit users to only the set of patients they need to

care for– Stop improper accesses from occurring

• Auditing– Catch improper accesses after the fact

11/11/2008 7

Access Control in Healthcare

Upfront definition of policies is problematic– “Experts” have incomplete knowledge– Healthcare is dynamic: workflows and interactions

change faster than experts can define them

“False Positives” cause a negative impact on clinical workflow and potentially patient harm– “Break the glass”

11/11/2008 8

Auditing in Healthcare

Huge amount of data, every day:– Hundreds to thousands of providers– Millions of patients

Which accesses are improper?

11/11/2008 9

Current Auditing

11/11/2008 10

Current Auditing

Vanderbilt University Medical Center– 1 Privacy Officer– 2 staff

Auditing focus– Monitor celebrities – Monitor employee-employee access– Follow-up on external suspicion– Spot checks

11/11/2008 11

Our Goal

Inform Policy Definition Tools– Werner et al.– Barth et al.

Assist auditing by defining what is normal

11/11/2008 12

Our Approach

Characterize normal operations, workflows, and relationships– Use access logs as proxy for this information

11/11/2008 13

Our Approach

Relational Network– Two providers related if they access the record of the

same patient– Strength of the relationship # records accessed in

common

Association Rules– What is the probability that we see two users or two

departments interacting together?– Head → Body

• Confidence - probability of seeing the Body, given the Head• Support - probability of seeing the Head and the Body

11/11/2008 14

Association Rules

11/11/2008 15

1 patient172 patients

Geriatric Psychology Ob-GynNeonatology

Association Rules

11/11/2008 16

1 patient172 patients

Geriatric Psychology Ob-GynNeonatology

Strong Relationship

Association Rules

11/11/2008 17

1 patient172 patients

Geriatric Psychology Ob-GynNeonatology

Weak Relationship

HORNET

Healthcare Organization Relational Network Extraction Toolkit

11/11/2008 18

Open Source

Easy and informative tool for privacy officials

Rich platform for developers

Design Goals

Easily handle healthcare sized networks– 103 to 104 nodes– 106 to 107 edges

Easily configurable for usersExtendable by developersLog format agnostic

11/11/2008 19

11/11/2008 20

Database APIOracle, MySQL, Etc.

File APICSV…

Task APIParallel & Distributed Computation

Network APIGraph, Node, Edge, Network Statistics

HORNET Core Plugins

Association Rule Mining

Noise Filtering Network Abstraction

Social Network Analysis

Database Network Builder

File Network Builder

…Network Visualization

Plugin Architecture

Plugin Chaining– Plugins use Observer Pattern to notify each other– Allows complex piping of results between plugins– Chains defined in configuration file

11/11/2008 21

Plugin Configuration

11/11/2008 22

Association Rule Mining

Network Abstraction

Social Network Analysis

File Network Builder

Network Visualization

Results from Vanderbilt

5 months of access logs from StarPanel, Vanderbilt’s EMR

> 9000 users> 350,000 patients> 7,500,000 views

11/11/2008 23

Edge Distribution

• Distribution of Relationships per User in 1 week

11/11/2008 24

Decay of Relationships

11/11/2008 25

How long do relationships last?

Healthcare is dynamic!

Department Relationships

11/11/2008 26

Relationships (edges) between departments (nodes)

Department Relationships

11/11/2008 27

20 departments with most relationships labeled

Association Rules

For 16 weeks, 55,944 department-department rules (unfiltered)

11/11/2008 28

Association Rules

Head Body Confidence Support # WeeksEmergency Medicine Emergency Med-Housestaff 1.8E-04 0.0043 16Emergency Med-Housestaff Emergency Medicine 1.7E-03 0.0043 16Ob-Gyn School Of Nursing 7.2E-04 0.0025 16Orthopaedics & Rehab Emergency Medicine 7.1E-04 0.0020 16Emergency Medicine Allergy/Pulm/Critical Care 8.3E-05 0.0019 16Emergency Medicine Nephrology & Hypertension 6.5E-05 0.0015 16Emergency Medicine Cardiovascular Medicine 6.3E-05 0.0015 16Emergency Medicine Anesthesiology 6.1E-05 0.0014 16Nephrology Clinic Nephrology & Hypertension 1.1E-03 0.0010 16Hematology/Oncology Cancer Center 5.5E-04 0.0009 16

11/11/2008 29

Sample of rules with high support

Association Rules

Head Body Confidence Support # WeeksHuman & Organizational Dev School Of Nursing 0.19 8.9E-06 4Psychology & Human Devel Mental Health Center 0.12 5.6E-06 5Radiology-Housestaff Orthopaedics & Rehab 0.10 3.9E-06 6Counseling Center Psychiatry 0.08 4.7E-06 6Counseling Center Psychology 0.07 4.4E-06 6Counseling Center Adult Psychiatry 0.07 4.4E-06 6NICU Neonatology 0.04 8.8E-05 14Sedation Service Anesthesiology 0.04 2.0E-06 6Sedation Service Pediatric Critical Care 0.04 6.1E-06 4Radiology-Housestaff Emergency Medicine 0.03 7.7E-06 4

11/11/2008 30

Sample of rules with high confidence and occurring at least 3 weeks

Future Plans

Temporal relationships – Find if certain users or departments are predictive of

a patient seeing another user or department

Filter Network– Remove noise, keep important relationships

User interface– Tool for privacy officers to examine their

organization’s logs

11/11/2008 31

Future Plans

Evaluation of rules by privacy and domain experts

Integrate with MICIS access control system– Werner et al., Barth et al., Mathe et al.

11/11/2008 32

Acknowledgements

NSF grant CCF-0424422, the Team for Research in Ubiquitous Secure Technologies

Dr. Randolph Miller and Kathleen Benitez

Dr. Dario Giuse and David Staggs

NetworkX, Numpy, Cython, Matplotlib

11/11/2008 33

More Information

http://hiplab.mc.vanderbilt.edu/projects/hornet

john.paulett@vanderbilt.edu

11/11/2008 34

Appendix

11/11/2008 36

Developer Documentation

11/11/2008 37

Writing a Plugin

11/11/2008 38

Configuration File

11/11/2008 39

Care Provider Relationships

11/11/2008 40

Children’s Hospital