AHRQ Quality Validation PilotAHRQ Quality Validation PilotProcess and ToolsProcess and Tools

Patricia Zrelak, RN, PhDPatricia Zrelak, RN, PhD

UC Davis Center for Healthcare Policy and ResearchUC Davis Center for Healthcare Policy and Research

AHRQ QI Users MeetingAHRQ QI Users Meeting

Bethesda, MD; September 28, 2007Bethesda, MD; September 28, 2007

AHRQ Validation Pilot: GoalsAHRQ Validation Pilot: Goals

Gather evidence on the scientific acceptability Gather evidence on the scientific acceptability of the patient safety indicators (PSIs)of the patient safety indicators (PSIs) medical record reviews, data analysis, clinical medical record reviews, data analysis, clinical

panels, evidence reviews panels, evidence reviews

Consolidate the evidence base Consolidate the evidence base Improve guidance on the interpretation and Improve guidance on the interpretation and

use of the datause of the data Evaluate potential refinements to the Evaluate potential refinements to the

specificationsspecifications

Pilot goals continuedPilot goals continued

Develop medical record abstraction toolsDevelop medical record abstraction tools review potentially preventable adverse eventsreview potentially preventable adverse events identify potential opportunities for improvementidentify potential opportunities for improvement

Develop mechanisms for conducting Develop mechanisms for conducting validation studies on a routine basisvalidation studies on a routine basis collaborating with other organizationscollaborating with other organizations data collection and analysisdata collection and analysis ongoing evaluation and refinementongoing evaluation and refinement

Patient Safety IndicatorsPatient Safety Indicators

Phase I Phase II

Accidental puncture and laceration

Foreign body left in during procedure

Iatrogenic pneumothorax Postoperative hemorrhage or hematoma

Postoperative pulmonary embolism (PE) or deep vein thrombosis (DVT)

Postoperative physiologic and metabolic derangement

Postoperative sepsis Postoperative respiratory failure

Selected infection due to medical care

Postoperative wound dehiscence

Pilot timelinePilot timeline

Collaborator recruitmentCollaborator recruitment September 2006 to October 2006September 2006 to October 2006

Collaborator training; protocol and tool Collaborator training; protocol and tool development and testing (Phase 1)development and testing (Phase 1) November 2006 to March 2007November 2006 to March 2007

Data collection application development and Data collection application development and testing; data reporting (Phase 1)testing; data reporting (Phase 1) April 2007 to September 2007April 2007 to September 2007

Analysis and assessment (Phase 1)Analysis and assessment (Phase 1) July 2007 to November 2007July 2007 to November 2007

Data CollectionData Collection

Chart abstraction by collaborators Chart abstraction by collaborators

Trained via webinarTrained via webinar Administrative data Administrative data

Cases were assigned based on a sampling probability using Cases were assigned based on a sampling probability using AHRQ QI softwareAHRQ QI software

Medical record abstraction tools & guidelinesMedical record abstraction tools & guidelines• Accidental puncture and laceration• Iatrogenic pneumothorax• Postoperative pulmonary embolism (PE) or deep vein

thrombosis (DVT)• Postoperative sepsis• Selected infection due to medical care

Guidelines in tool development Guidelines in tool development

Literature search – evidence basedLiterature search – evidence based

Alignment with related QI projects & initiatives Alignment with related QI projects & initiatives Ex. Surgical Care Improvement Project (SCIP) Ex. Surgical Care Improvement Project (SCIP)

Professional & regulatory guidelinesProfessional & regulatory guidelines

Ease of use Ease of use

Guidelines in tool Guidelines in tool developmentdevelopment

Ongoing expert review process Ongoing expert review process – Healthcare and medical practitioners Healthcare and medical practitioners – Quality expertsQuality experts

Consultation with other experts as needed Consultation with other experts as needed Local alpha testing and refinementLocal alpha testing and refinement Feedback from collaborator training Feedback from collaborator training Learnings from national pilot testing Learnings from national pilot testing

Recognizing limitationsRecognizing limitations

Chart review Chart review – Not all data elements of interest are available Not all data elements of interest are available – Ex. hand washing, mask use & environmental factorsEx. hand washing, mask use & environmental factors

Time constraints (burden on collaborators)Time constraints (burden on collaborators)– Some items of interest are too time consuming to Some items of interest are too time consuming to

abstract (e.g., lowest urine output)abstract (e.g., lowest urine output) Reliability of certain data elements/differences in Reliability of certain data elements/differences in

practice practice – Ex. incentive spirometry & sequential compression Ex. incentive spirometry & sequential compression

devicesdevices Variability between healthcare systemsVariability between healthcare systems

– Admission weights, temperature documentation in ORAdmission weights, temperature documentation in OR

Section 1: Abstractor details Section 1: Abstractor details

Section 2: Record identification/validationSection 2: Record identification/validation

Section 3: Ascertainment of the event(s)Section 3: Ascertainment of the event(s) Was the patient eligible for the indicator?Was the patient eligible for the indicator? Did the indicator event happen? Did the indicator event happen?

Generic structure of the data abstraction tool

Structure of collection tool Structure of collection tool continued continued

Section 4: Risk factorsSection 4: Risk factors

Section 5: Evaluation and management Section 5: Evaluation and management Characterization of the eventCharacterization of the event Potential preventability of the eventPotential preventability of the event

Section 6: OutcomesSection 6: Outcomes Impact of the event on the patientImpact of the event on the patient

Section 1: Section 1: Abstractor detailsAbstractor details

1.1 1.1 Date abstraction completed Date abstraction completed

_ _ / _ _ / _ _ _ __ _ / _ _ / _ _ _ _

1.2 Abstractor identifier 1.2 Abstractor identifier

PP AA ZZ _ _ _ _ _ _ _ _

Section 2: Section 2: Record identification/validationRecord identification/validation

Demographics Demographics AHRQ study ID, patient identification AHRQ study ID, patient identification

code, DOB, gender, dates of admission code, DOB, gender, dates of admission and dischargeand discharge

Criterion validation Criterion validation – Was the correct chart abstracted (correct Was the correct chart abstracted (correct

PSI, patient and admission)? PSI, patient and admission)?

– Link to administrative data Link to administrative data

Section 3: Section 3: Ascertainment of eventAscertainment of event

Criterion validationCriterion validation - whether cases flagged - whether cases flagged did or did not have the clinical event did or did not have the clinical event

Inclusion and exclusion criteria from the Inclusion and exclusion criteria from the Patient Safety Indicator Technical Patient Safety Indicator Technical Specifications Specifications

Confirmation of the event and dateConfirmation of the event and date Ascertainment of multiple procedures/events Ascertainment of multiple procedures/events

per dischargeper discharge

Ex. Iatrogenic pneumothoraxEx. Iatrogenic pneumothorax

3.1 Was the patient’s admission associated with any of the 3.1 Was the patient’s admission associated with any of the following conditions or procedures (before the date of the following conditions or procedures (before the date of the pneumothorax diagnosis)? Check all that apply. pneumothorax diagnosis)? Check all that apply.

Pregnancy, childbirth or puerperiumPregnancy, childbirth or puerperium Chest injury or trauma Chest injury or trauma Pleural effusion Pleural effusion Diaphragmatic surgery or repair Diaphragmatic surgery or repair Thoracic surgery (excluding bronchial procedures)Thoracic surgery (excluding bronchial procedures) Cardiac surgery Cardiac surgery Lung or pleural biopsy Lung or pleural biopsy Operations on the esophagus Operations on the esophagus Anterior thoracic spinal fusion or thoracic duct surgery Anterior thoracic spinal fusion or thoracic duct surgery None of the above None of the above

If YES to any of the above conditions, please describe the condition or If YES to any of the above conditions, please describe the condition or procedure that apparently led to a pneumothorax in the TEXT BOX below procedure that apparently led to a pneumothorax in the TEXT BOX below and then END the abstraction.and then END the abstraction.

Pneumothorax continuedPneumothorax continued

3.2 Did the patient have a pneumothorax or suspected 3.2 Did the patient have a pneumothorax or suspected pneumothorax at the time of admission? pneumothorax at the time of admission?

Yes Yes No No

If YES, STOP as this is an exclusion criterion. Please If YES, STOP as this is an exclusion criterion. Please describe the circumstances surrounding this describe the circumstances surrounding this pneumothorax at admission in the TEXT BOX below pneumothorax at admission in the TEXT BOX below and then END the abstraction.and then END the abstraction.

3.3 3.3 Did the patient experience a pneumothorax during this admission?

Yes No

If NO, STOP as this is required for study inclusion. Please describe any abnormality or condition that might have been misinterpreted as a pneumothorax, such as pneumomediastinum or subcutaneous emphysema in the in the TEXT BOX below and then END the abstraction.

3.4 Document the date the pneumothorax was diagnosed. Use the earliest date in the event of multiple pneumothoraces.

_ _ /_ _ /_ _ _ _

Ex. Postoperative Pulmonary Embolism or Ex. Postoperative Pulmonary Embolism or Deep Vein ThrombosisDeep Vein Thrombosis

Section 4: Risk FactorsSection 4: Risk Factors

Confounding - whether there are Confounding - whether there are confounding factors that might be important confounding factors that might be important for improving indicator specifications and for for improving indicator specifications and for interpreting and using the AHRQ PSI ratesinterpreting and using the AHRQ PSI rates

Example: Selected Infections Example: Selected Infections

due to Medical Caredue to Medical Care

4.1 Did the patient have any of the following 4.1 Did the patient have any of the following immunosuppressive conditions on admission? immunosuppressive conditions on admission? Check all that apply.Check all that apply.

CancerCancer HIV/AIDS HIV/AIDS Severe malnutritionSevere malnutrition Lupus or other autoimmune disease Lupus or other autoimmune disease Sickle cell disease Sickle cell disease Nephrotic syndrome or chronic renal failureNephrotic syndrome or chronic renal failure Short gut syndrome Short gut syndrome Immunoglobulin deficiency Immunoglobulin deficiency Transplant Transplant Other immunodeficiency (specify)Other immunodeficiency (specify)

Section 5: Evaluation and Section 5: Evaluation and Management Management

Processes of care/process improvementProcesses of care/process improvement Eligibility for interventions Eligibility for interventions Did the patient receive interventions?Did the patient receive interventions? How was the patient diagnosed?How was the patient diagnosed? How was the complication managed once it How was the complication managed once it

occurred? occurred?

Ex. Postoperative Deep Vein Thrombosis or Ex. Postoperative Deep Vein Thrombosis or

Pulmonary EmbolusPulmonary Embolus

5.7 How was the venous thrombosis diagnosed? 5.7 How was the venous thrombosis diagnosed? Duplex ultrasonography Duplex ultrasonography CT scan CT scan Contrast venographyContrast venography By clinical suspicion aloneBy clinical suspicion alone

Ex. Postoperative Deep Vein Ex. Postoperative Deep Vein Thrombosis or Pulmonary EmbolusThrombosis or Pulmonary Embolus

5.8 What specific segment(s) of the venous system was/were 5.8 What specific segment(s) of the venous system was/were identified to have thrombus? Check all that apply. identified to have thrombus? Check all that apply.

Inferior vena cavaInferior vena cava Iliac veinsIliac veins Femoral veins Femoral veins Popliteal vein Popliteal vein Deep lower extremity veins distal to the popliteal Deep lower extremity veins distal to the popliteal Superficial lower extremity vein(s)Superficial lower extremity vein(s) Superior vena cava Superior vena cava Brachiocephalic (innominate) veins Brachiocephalic (innominate) veins Internal jugular vein Internal jugular vein Superficial neck vein Superficial neck vein Subclavian or axillary vein Subclavian or axillary vein Deep upper extremity veins distal to the axillary vein Deep upper extremity veins distal to the axillary vein Superficial upper extremity vein Superficial upper extremity vein Critical documentation missing Critical documentation missing Other (specify):Other (specify):

Ex. Postoperative Deep Vein Ex. Postoperative Deep Vein Thrombosis or Pulmonary EmbolusThrombosis or Pulmonary Embolus

5.9 Was the thrombus occluding or non-occluding? 5.9 Was the thrombus occluding or non-occluding?

5.10 Based on diagnostic test documentation, how is the 5.10 Based on diagnostic test documentation, how is the acuteness or age of the DVT reported. acuteness or age of the DVT reported.

5.12 ‘…..was the PE or venous thrombosis detected as a 5.12 ‘…..was the PE or venous thrombosis detected as a result of routine screening?’result of routine screening?’

5.13 …. ‘signs and symptoms present 48-hours prior to 5.13 …. ‘signs and symptoms present 48-hours prior to diagnostic studies’. diagnostic studies’.

Section 6: OutcomesSection 6: Outcomes

Impact on the patient Impact on the patient

The abstractor is asked to assess the documentation The abstractor is asked to assess the documentation and render a judgment on the impact of the and render a judgment on the impact of the complicationcomplication– Ex. Causative factor related to death, readmission, Ex. Causative factor related to death, readmission,

increased length of stay, and/or transfer to a higher level of increased length of stay, and/or transfer to a higher level of care. care.

Ex. Selected Infection due to Ex. Selected Infection due to Medical Care Medical Care

6.1 Does the chart suggest that the patient suffered any 6.1 Does the chart suggest that the patient suffered any adverse effects or consequences from this infectious adverse effects or consequences from this infectious or inflammatory process? Check all that apply. or inflammatory process? Check all that apply. Additional pain or discomfort Additional pain or discomfort Extended length of hospital stay Extended length of hospital stay Underwent an operating room procedure to treat infection Underwent an operating room procedure to treat infection

(e.g., incision and drainage, excision) (e.g., incision and drainage, excision) Residual disability or impairment of normal function (at Residual disability or impairment of normal function (at

discharge) discharge) Readmission Readmission DeathDeath None or the above or not specified None or the above or not specified

Last question of every toolLast question of every tool

If there are special circumstances or comments related to this case that you feel are important that were not captured in the survey, please state in the TEXT BOX.

DVT/PE example: The patient experienced a postoperative PE-but is was from a septic emboli.

GuidelinesGuidelines

Guidelines for Validation of

Selected AHRQ Quality Indicators(Version 6.2_04/5/2007)

PSI 6: Guideline for Iatrogenic Pneumothorax