Department of Health and Human Services

Measuring Clinical Lab Ordering Quality: Theory and Practice

Measuring Clinical Lab Ordering Quality: Theory and PracticeSteven M. Asch MD MPH

VA, RAND, UCLAApril 29, 2005

INSTITUTE OF MEDICINEDEFINITION OF QUALITY The degree to which health services for individuals and populations * increase the likelihood of desired health outcomes and * are consistent with current professional knowledge

Lundberg , 1981

Were results used properly to improve care?

Has the right testbeen ordered?

Action

The 9 steps in the performance of any laboratorytest. The brain-to-brain turnaround time loop.

Interpretation

Reporting

Analysis

PreparationTransportation

Identification

Collection

Ordering

WHAT IS POOR QUALITY?WHAT IS POOR QUALITY?

• Too little care – underuse– Failure to provide an effective service when

it could have produced a good outcome

• Too much care – overuse– Providing care when its risks of harm

greater than potential benefit

• The wrong care – misuse– Avoidable complications of appropriate care

CONCEPTUAL FRAMEWORK

STRUCTURE PROCESS OUTCOMES

Technical Excellence

• Right choices

• Effective/skillful

Interpersonal Excellence

• Patient-centered

• Responsive

Functional Status

Satisfaction

Mortality

Biological Status

Health CareOrganization

Characteristics

ProviderCharacteristics

CommunityCharacteristics

PopulationCharacteristics

EXAMPLES OF STRUCTURAL MEASURESEXAMPLES OF STRUCTURAL MEASURES

• Health care organization characteristics- Weekend and night hours and

convenient locations of laboratories- Volume

• Provider characteristics– Number of pathologists– Training of laboratory staff

CONCEPTUAL FRAMEWORK

STRUCTURE PROCESS OUTCOMES

Technical Excellence

• Right choices

• Effective/skillful

Interpersonal Excellence

• Patient-centered

• Responsive

Functional Status

Satisfaction

Mortality

Biological Status

Health CareOrganization

Characteristics

ProviderCharacteristics

CommunityCharacteristics

PopulationCharacteristics

4%

30%33%

29%33%

86%

60%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

All U/A Glu K Cr Chol Trig

% Adherence

HTN NEW DIAGNOSIS LABSHTN NEW DIAGNOSIS LABS

Asch et. al. BMC CV, 2005

QATOOL SCORES BY MODEQATOOL SCORES BY MODE

Visit 73%

Medication 69%

Immunization 66%

Physical Exam 63%

Laboratory/Radiology 62%

Surgery 57%

History 43%

Education 18%McGlynn, Asch et. al. NEJM 2003

CONCEPTUAL FRAMEWORK

STRUCTURE PROCESS OUTCOMES

Technical Excellence

• Right choices

• Effective/skillful

Interpersonal Excellence

• Patient-centered

• Responsive

Functional Status

Satisfaction

Mortality

Biological Status

Health CareOrganization

Characteristics

ProviderCharacteristics

CommunityCharacteristics

PopulationCharacteristics

WHY MEASURE OUTCOMES?WHY MEASURE OUTCOMES?

– Allow innovation in process

– People care about outcomes directly

SAMPLE SIZE PROBLEMSSAMPLE SIZE PROBLEMS

– For mortality, need huge samples:• CHF patients: 12% vs 16%, need 957

patients at each hospital.

– Rarer outcomes• People care, but statistical comparison is

impossible.

DOES SICKNESS OR QUALITY DETERMINE CHF MORTALITY?Sickness at Process

Admission Poor Medium Good Total

Least Sick 1/4 4 7 4 5Middle 1/2 11 8 8 9Most Sick 1/4 37 32 26 32Total 16 14 12 14

ACCOUNTABILITY: IS PROVIDER RESPONSIBLE FOR PROBLEM?

ACCOUNTABILITY: IS PROVIDER RESPONSIBLE FOR PROBLEM?

– Current treatment must have big impact relative to other factors.

– Do not want providers avoiding those who:• have a bigger chance of problems• are less likely to adhere to treatment

CHOOSING MEASURES:PRACTICAL CONSIDERATIONS

– Choosing areas to measure

– Selecting indicators

– Designing specifications

– Testing the measure

CHOOSING AREAS:ASSESSING HEALTH IMPORTANCE

– Mortality

– Morbidity

– Utilization

– Cost

PREVALENCE OF SELECTED ACUTE CONDITIONS AMONG WORKING ADULTS

Condition Work Loss Days/100 Persons

Injuries

Influenza

Infections and parasitic disease

Common cold

Digestive system conditions

Other upper respiratory

Acute ear infections

85.5

53.1

20.6

15.4

12.3

9.3

3.2

CHOOSING AREAS: POTENTIAL FOR IMPROVEMENT

CHOOSING AREAS: POTENTIAL FOR IMPROVEMENT

– What are the key outcomes of interest?

– What processes produce those outcomes?

– How well are key elements of care delivered today?

– How variable is care delivery?

CHOOSING MEASURES: DEGREE OF PROVIDER CONTROL

CHOOSING MEASURES: DEGREE OF PROVIDER CONTROL

– How might the measure be affected by characteristics of the enrolled population?

– What actions can providers or clinical laboratories take to improve performance?

STRENGTH OF SCIENTIFIC EVIDENCESTRENGTH OF SCIENTIFIC EVIDENCE

I: Randomized controlled trialII-1: Nonrandomized controlled trialI-2: Cohort or case control studiesII-3: Multiple time seriesIII: Opinions or descriptive studies

COST-EFFECTIVENESS OF PROCESS

MMR Immunization $14 saved/$1 spent

Cervical cancer screening $21,000 spent/year (ages 20-28)

Cervical cancer screening $11,000 spent/year (ages 29-50)

DESIGNING MEASURE SPECIFICATIONSDESIGNING MEASURE SPECIFICATIONS

– Define indicator– Identify target population– Define eligible population– Determine need for risk adjustment– Identify data sources– Write data collection instructions– Develop scoring rules

Example measureExample measure

• Men with a new diagnosis of prostate cancer, who have not had a serum PSA in the prior three months, should have serum PSA checked within one month after diagnosis or prior to any treatment, whichever comes first.

EVALUATING DATA SOURCES

DATA SOURCE STRENGTHS WEAKNESSES

Medical Record Clinical Detail Expense Missing links

Administrative Use of services Clinical detail

Patient Surveys General Health Expense

Interpersonal Clinical detail

TESTING THE MEASURETESTING THE MEASURE

– Reliability: The proportion of times that repeated use of measure in same population gives the same result

– Validity: The extent to which the measure accurately represents the concept being assessed

– Interpretability: Ease with which target audience can understand and use information generated by measure

WHY SHOULD CLINICIANS CARE ABOUT MEASURING QUALITY?

WHY SHOULD CLINICIANS CARE ABOUT MEASURING QUALITY?

– Internal quality improvement

– External monitoring and evaluation

– Consumer/purchaser decision-making

ADEQUACY OF CASE-MIX CONTROLADEQUACY OF CASE-MIX CONTROL

– Severity of disease– Incidence and prevalence by

demographics•age•race•gender