Internationally Comparable Diagnosis-Specific Survival Probabilities for Calculation

of the ICD-10 Based Injury Severity Scores

Wellington, New Zealand 29 September 2012

Holly Hedegaard, MD, MSPHOffice of Analysis and Epidemiology

U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICESCenters for Disease Control and PreventionNational Center for Health Statistics

Background

• Desire to develop internationally comparable indicators of injury morbidity using administrative datasets

• Decision to hospitalize can vary over time and from country to country

• A standard method to identify patients of similar injury severity level is needed• Consensus derived vs empirically derived • AIS; ISS vs DSP, “SRR”; ICISS

Background

Diagnosis-specific Survival Probability (DSP; “SRR”)• Determined for each individual ICD diagnosis code• Number of patients with a given injury code who survived

Total number of patients with that injury code • Values range from 0-1

ICD-based Injury Severity Score (ICISS)• The product of the DSPs for each injury• ICISS = DSPinj1 x DSPinj2 x DSP inj3 , etc.

• ICISS used in logistic regression models to predict probability of death

Background

• For a more accurate estimate of the DSP, a large number of observations is needed

• At the 2008 Boston ICE meeting, researchers from several countries agreed to pool data to generate the international DSPs (ICE-DSPs)

Contributors

• Rolf Gedeborg, MD, PhD Sweden• Margaret Warner, PhD USA• Li-Hui Chen, PhD USA• John Langley, PhD New Zealand• Pauline Gulliver, PhD New Zealand• Colin Cryer, PhD New Zealand• Yvonne Robitaille, PhD Canada• Robert Bauer, PhD Austria• Clotilde Ubeda, MD, MSc Argentina• Jens Lauritsen, MD, PhD Denmark• James Harrison, MDDS, MPH Australia

Objectives of the Study

• To develop DSPs from pooled data (ICE-DSPs)

• To compare the performance in predicting inpatient mortality of ICISS based on ICE-DSPs to ICISS based on country-specific DSPs

Methods

Seven countries provided data for creating the ICE-DSPs• Australia• Argentina• Austria• Canada• Denmark*• New Zealand*• Sweden*

*Provided record level data

Methods

• The pooled data included nearly 4 million injury diagnoses• 1168 dx had at least 1 observation in the pooled data• 88% had at least 20 observations to calculate DSP• 66% had at least 100 observations to calculate DSP

Methods

• Four methods were used to calculate ICE-DSPs using the pooled data• Summation• Arithmetic means• Trimmed means• Combined approach

• Summation method is recommended (simplest)

Methods

• For the 3 countries that provided record level data, the performance of a logistic regression model using ICE-DSP-derived ICISS to predict mortality was compared to that of a model using ICISS calculated using the country-specific DSPs• Discrimination: c-statistic• Calibration: Nagelkerke’s R2

Results

• Variability among country-specific DSPs

• Range = the difference between the highest and lowest country-specific DSPs for an injury diagnosis

Diagnoses with the least variability in DSPs between countries

ICD-10 code

Diagnosis Range in DSPs

Mean of DSPs

S807 Multiple superficial injuries of lower leg 0.033 0.974

S799 Unspecified injury of hip and thigh 0.035 0.979

S211 Open wound of front wall of thorax 0.040 0.963

T141 Open wound of unspecified body region 0.041 0.976

S122 Fracture of other specified cervical vertebra 0.044 0.954

S829 Fracture of lower leg, part unspecified 0.049 0.970

T149 Injury, unspecified 0.050 0.968

T589 Toxic effect of carbon monoxide 0.052 0.970

S212 Open wound of back wall of thorax 0.056 0.976

S141 Other/unspecified injuries of cervical spinal cord 0.064 0.917

Diagnoses with the most variability in DSPs between countries

ICD-10 code

Diagnosis Range in DSPs

Mean of DSPs

S271 Traumatic haemothorax 0.263 0.940

S368 Injury of other intra-abdominal organs 0.264 0.925

S027 Multiple fractures involving skull and facial bones 0.294 0.893

T689 Hypothermia 0.294 0.852

S066 Traumatic subarachnoid haemorrhage 0.379 0.816

S361 Injury of liver or gall bladder 0.386 0.932

S064 Epidural haemorrhage 0.391 0.920

T175 Foreign body in bronchus 0.408 0.971

S272 Traumatic haemopneumothorax 0.411 0.944

S065 Traumatic subdural haemorrhage 0.539 0.826

Performance of model using ICE-DSP-derived ICISS: Data from New Zealand

Factors in the model C-statistic (Discrimination)

Nagelkerke’s R2 (Calibration)

ICISS from NZ DSPs 0.876 0.2263ICISS from ICE-DSPs 0.868 0.2088

N= 264,348 Inpatient Mortality Rate = 1.2%

Performance of model using ICE-DSP-derived ICISS:

Data from Sweden

Factors in the model C-statistic (Discrimination)

Nagelkerke’s R2 (Calibration)

ICISS from Swedish DSPs 0.829 0.1678ICISS from ICE-DSPs 0.815 0.1489

Age + Sex + ICISS from Swedish DSPs

0.877 0.2385

Age + Sex + ICISS from ICE-DSPs 0.871 0.2232

N=707,968 Inpatient Mortality Rate = 1.6%

Performance of model using ICE-DSP-derived ICISS:

Data from Denmark (one hospital)

Factors in the model C-statistic (Discrimination)

Nagelkerke’s R2 (Calibration)

ICISS from Danish DSPs 0.725 0.1311ICISS from ICE-DSPs 0.681 0.0756

Age + Sex + ICISS from Danish DSPs

0.822 0.2613

Age + Sex + ICISS from ICE-DSPs 0.816 0.2490

N=23,449 Inpatient Mortality Rate = 10.8%

Next Steps: International DSPs

• Are the ICE-DSPs ready for use or do they need to be further refined or tested?• Include out of hospital deaths?• Include data from more countries?• Create ICE-DSPs for different age groups (pediatric vs

adult vs older adult)• Create ICE-DSPs for comorbidities?• Test discrimination/calibration using data from less

resourced countries?

Next Steps: International DSPs

• Do we need to generate standard methods for how to use the ICE-DSPs? • Post the international DSPs to the web?• Create a toolkit on how to use?

• Multiplicative model vs single worst injury• Include ICE-DSPs for comorbidities when

calculating ICISS?

Next Steps: Other Considerations

• Do we continue on the path of international DSPs or do we consider other methods?• Excess Mortality Ratio-adjusted ISS, Kim et al, 2009• Trauma Mortality Prediction Model, Osler, et al, 2007• ICD-10 to AIS crosswalk, Haas, Nathans, et al, 2012

Questions and Discussion

Thank you!

Next Steps: International DSPs

• Should we use the ICE-DSPs to define broader injury severity categories for international comparisons (ordinal scale)?

• Should we use the ICE-DSPs to identify a “basket of injuries” that could be used when ICD-10 coded data are not available (threshold)?

Next Steps: International DSPs

• Should the ICE-DSPs be updated, and if so, how often?• Include the same countries each time?