ESRD Beneficiary Focused Learning
Network Special Project
Performance
Indicators
May 2, 2012
Submitted to:
Sharon Last, Contract Office Technical Representative
Centers for Medicare and Medicaid Services
The Renal Network, Inc. ESRD Network 9
911 E. 86th Street, Suite 202, Indianapolis, IN 46240
Phone: 317.257.8265 Fax: 317.257.8291
HHSM-500-2010-NW 009C
Page 1 of 16
TABLE OF CONTENTS
I. Introduction 2
II. Development Process 2
III. Performance Indicators 3
IV. Technical Specification 6
V. Next Steps 16
Page 2 of 16
I. Introduction
As defined in the scope of work (SOW), the Learning Network “shall deliver services that support the
Quality Incentive Program (QIP), the goals of the CMS Office of Clinical Standards and Quality (OCSQ),
and the agency to identify and characterize data sources and data gaps pertinent for Monitoring and
Evaluation (M&E) purposes within the ESRD Network.
In order to accomplish this objective, an early warning system is being developed to identify meaningful
data types that generate standardized, informative summary statistics of ESRD service’s quality and
access of care performance across all Networks. As the term indicator suggests, performance indicators
provide an early warning signal, and thus suggest further investigation in order to properly determine
the results. It is important to clarify that an indicator is meant to be used as a flag to assist in decision
making, rather than a driver for change. A valid indicator should possess the following attributes:
• Standardized (data type is defined and collected in a uniform manner
• Quantitative (possess measurable attributes of quality and access of care)
• Actionable (influenced by provider whose performance is being measured)
• Meaningful (align with the goals of the special project)be standardized, quantitative actionable
A relevant performance indicator requires a feedback mechanism. They are, therefore, performance
indicators, defined as measurable elements of performance (for which there is a data source) that can
be used to assess the quality and access of care provided to ESRD patients and, if necessary, prompt
actions for change. An important requirement of a good performance indicator is that it can be
influenced by the provider or facility whose performance is being measured.
Therefore, a goal of the Learning Network is to define a small set of practical key performance indicators
that are related to the process of delivering ESRD patient care and to validate these indicators in terms
of quality and access of care.
II. Development Process
The Learning Network established a focus group to develop the performance indicators. The focus
group consisted of project consultants and professionals within the ESRD community that serve on the
executive committee for The Renal Network, Inc. (TRN). A meeting was held in early March of 2012 for
the purpose of identifying and validating meaningful data sets in terms of quality and access of care.
The activities that were considered included:
• Barriers to care
• Involuntary Discharges (IVDs)
• At risk for IVDs
• Beneficiary complaints and grievances
• Facility concerns
• Changes in protocol
Page 3 of 16
Particular attention was placed on recent changes related to provider reimbursement. The focus group
considered relevant data sources to determine the feasibility of establishing a set of performance
indicators that may support future monitoring and evaluating activities. In addition, the focus group
was also provided feedback solicited at the Technical Expert Panel (TEP) meeting that was held in
Baltimore in January of 2012. As a result of the discussion, the following methodology was developed
by the focus group to use as a rationale for selecting potential indicators.
Methodology
The Prospective Payment System (PPS) and The Quality Incentive Program (QIP) provide new
opportunities for maximizing profit to dialysis providers. Providers could attempt to maximize profit by
selecting patients to treat who improve provider profits, and refusing to treat patients who worsen
provider profits (“cherry picking”). Cherry picking is medically unethical according to the principle of
maleficence (a physician should not place their own interests before their patient’s interest).
By “back chaining” from these new opportunities for profit described in the PPS and QIP, indicators can
be constructed that sufficiently suggest the possibility of “cherry picking” to warrant further
investigation.
Two triggers could indicate the need to investigate a facility for possible “cherry picking” patients to
maximize profits. These indicators raise the possibility of “too good to be true” improvement in
performance (i.e., improvement that might be less likely attributed to process improvement and more
likely attributed to “cherry picking”):
1. “Sudden” (“unreasonably1” rapid) changes in a facility’s performance rates rewarded by
increased reimbursement from before to after the PPS and QIP, and
2. “Significant” (“unreasonable2”) improvement in means of those performance measures
rewarded by increased reimbursement from before to after the PPS and QIP.
References 1
± 2 standard deviations from national, regional, or facility-of-interest change in national rate 2 ± 2 standard deviations from national, regional, or facility-of-interest mean performance measures
III. Performance Indicators
Specific performance indicators were chosen to investigate a potential change in performance for access
of care (Table 1) and quality of care (Table 2). Additional criteria for defining the performance indicators
were: (i) the importance of having a valid data source that generates standardized results; (ii) the factors
affecting the frequency of reporting; and (iii) the feasibility to report the data at a local, regional or
national level. The technical specifications for each individual performance indicator are listed
separately in Section IV.
Page 4 of 16
Table 1
Specific performance indicators and their purpose and rationale for the investigation of a potential change in performance in access of care
Reference
Number Performance Indicator Purpose Rationale
AOCPI-1 Decrease In Incident
Catheter Only Percentage
To investigate for potential
barriers to admission related to
incident dialysis patients
presented for admission with
catheter only.
High incident rates, consequently leading to high prevalent rates, would decrease
facility reimbursement under the PPS and/or QIP. A sudden and significant
decrease in incident catheter rates might indicate refusal to admit patients to a
facility who have catheters, thus providing a barrier to appropriate access of care.
AOCPI-2 Involuntary Discharge
Trending
To investigate for potential
patient profiling or "cherry
picking" that contributes to an
increase in involuntary discharges.
Certain justifications for involuntary discharge might indicate a facility’s desire to
reverse “cherry pick” patients who worsen performance indicators associated
with reimbursement.
AOCPI-3 Barrier to Admission
Trending
To investigate for potential
patient profiling or "cherry
picking" that contributes to an
increase in facility refusals of an
ESRD patient.
Refusing admission to patients who would worsen rates of performance
measures in order to enhance facility reimbursement is unethical.
AOCPI-4 Failure Rate in Transition
to Home Dialysis
To investigate for an increase in
patients that train for a home
modality but fail to transition to
the home dialysis program.
Home training is now reimbursed on a per training day basis, rather than on
completion of training. Profits would increase by referring patients for training
with a low probability of successful completion of training and transitioning to
home dialysis. Such patients would justify per training session payment to the
maximum allowed number of training sessions, fail to go home, and be returned
for continued treatment to the in-center facility.
Page 5 of 16
Table 2
Specific performance indicators and their purpose and rationale for the investigation of a potential change in performance in quality of care
Reference
Number Performance Indicator Purpose Rationale
QOCPI-1 Increase in Percentage of Blood
Transfusions
To investigate for possible decreased use of
erythropoietin stimulating agents resulting in
an increased percentage of blood
transfusions.
There may be a reverse incentive to minimize use of bundled
drugs, such as ESAs, thereby increasing the percentage of
patients receiving blood transfusions.
QOCPI-2 Increase in Percentage of
Transplant Candidates
Developing Antibodies
To investigate for possible decreased use of
erythropoietin stimulating agents resulting in
an increased percentage of transplant
candidates developing antibodies (surrogate
for increased blood transfusions).
After correction for multiparous women and people with
previously rejected transplants, this indicator could serve as a
surrogate for increased transfusions required by limiting the
use of ESA because it is included in the bundle.
QOCPI-3 Increase in Bloodstream
Infection Rate
To investigate for possible patient and
healthcare worker safety issues related to the
care of ESRD patients with catheters
Financial incentive to minimize use of anything expensive
included in the bundle such as more expensive, more effective
catheter dressings or IV antibiotics.
QOCPI-4 Decrease In Direct Patient Care
Staffing Ratio
To investigate for potential quality of care
issues related to a reduction in direct patient
care staffing.
Financial incentive to decrease number and type of patient care
staff to maximize dialysis reimbursement [shouldn’t be any
different than it has always been – i.e before bundle and QIP]
QOCPI-5 Facility Maintenance Condition
Level Citation Trends
To investigate for possible reductions in
facility maintenance expenditures that
contributes to an increase in facility citations.
Financial incentive to cut corners and fix citations when and if
they occur (site visits are few and far between).
QOCPI-6 Significant Change in Transplant
Referral Rate
To investigate for a possible decrease in
referral rates for ESRD patients presented as a
transplant candidate.
Bundled payment is per outpatient treatment. Anything that
removes the patient from outpatient facility, such as
transplantation, decreases facility reimbursement without
decreasing facility overhead. Financial incentive to decrease
transplantation by decreasing transplant referrals to transplant
center.
Page 6 of 16
IV. Technical Specifications
DECREASE IN INCIDENT CATHETER ONLY PERCENTAGE
Access of Care Performance Indicator # 1
Outcome Measure
Technical Specifications
Beneficiary Focused Learning Network, Version 1.0, March 2012
Purpose:
To investigate for potential barriers to admission related to incident dialysis patients presented for
admission with catheter only
Status:
Proposed
Numerator:
The number of incident ESRD patients admitted with catheter only to the facility during the reporting
period
Denominator:
The number of all incident ESRD patients admitted to the facility during the reporting period
Data Source(s):
Vascular Access Reporting Tool
Form 2728
Trigger
To be determined
Frequency:
Quarterly
Exclusions:
None
Report Level:
Facility
Page 7 of 16
INVOLUNTARY DISCHARGE TRENDING
Access of Care Performance Indicator # 2
Outcome Measure
Technical Specifications
Beneficiary Focused Learning Network, Version 1.0, March 2012
Purpose:
To investigate for potential patient profiling or "cherry picking" that contributes to an increase in involuntary
discharges
Status:
Proposed
Numerator:
The number of involuntary discharges incurred by an ESRD Network for the reporting period
Denominator:
None
Data Source(s):
Facility Self Reporting
Network Contact Utility
Standard Information Management System (SIMS) Contact Module
Trigger
To be determined
Frequency
Quarterly
Exclusions:
None
Report Level:
Network
Page 8 of 16
BARRIERS TO ADMISSION TRENDING
Access of Care Performance Indicator # 3
Outcome Measure
Technical Specifications
Beneficiary Focused Learning Network, Version 1.0, March 2012
Purpose:
To investigate for potential patient profiling or "cherry picking" that contributes to an increase in facility
refusals of an ESRD patient
Status:
Proposed
Numerator:
1. The number of facility refusals of admission incurred by an ESRD Network during the reporting period
2. The number of facility refusals of admission incurred by an ESRD Network during the reporting period
Denominator:
1. The number of all ESRD incident patients during the reporting period
2. None
Data Source(s):
Facility Self Reporting (recommendation to CMS to collect)
Network Contact Utility
Trigger
To be determined
Frequency
Quarterly
Exclusions:
None
Report Level:
Network
Page 9 of 16
FAILURE RATE IN TRANSITION TO HOME DIALYSIS
Access of Care Performance Indicator # 4
Outcome Measure
Technical Specifications
Beneficiary Focused Learning Network, Version 1.0, March 2012
Purpose:
To investigate for an increase in patients that train for a home modality but fail to transition to the home
dialysis program
Status:
Proposed
Numerator:
The number of patients currently in home program during the reporting period (recommend two months)
Denominator:
The number of patients commencing home dialysis training during the reporting period
Data Source(s):
ESRD Claims information
CROWNWeb or SIMS
Medicare Cost Report (Form 265-94) Worksheet S-1
Trigger
To be determined
Frequency
Quarterly
Exclusions:
None
Report Level:
Facility
Page 10 of 16
INCREASE IN PERCENTAGE OF BLOOD TRANSFUSIONS
Quality of Care Performance Indicator # 1
Outcome Measure
Technical Specifications
Beneficiary Focused Learning Network, Version 1.0, March 2012
Purpose:
To investigate for possible decreased use of erythropoietin stimulating agents resulting in an increased
percentage of blood transfusions
Status:
Proposed
Numerator:
The number of prevalent ESRD patients receiving a blood transfusion not related to specific conditions
causing decreased hemoglobin levels during a reporting period
Denominator:
The number of all prevalent ESRD patients during the reporting period
Data Source(s):
Facility Self Reporting
Claims Data
Trigger
To be determined
Frequency
Quarterly
Exclusions:
ESRD Patients who have blood transfusions because of specific conditions that decrease hemoglobin levels
(i.e. cancer, GI Bleed)
Report Level:
Facility
Page 11 of 16
INCREASE IN PERCENTAGE OF TRANSPLANT CANDIDATES DEVELOPING ANTIBODIES
Quality of Care Performance Indicator # 2
Outcome Measure
Technical Specifications
Beneficiary Focused Learning Network, Version 1.0, March 2012
Purpose:
To investigate for possible decreased use of erythropoietin stimulating agents resulting in an increased
percentage of transplant candidates developing antibodies (surrogate for increased blood transfusions)
Status:
Proposed
Numerator:
The number of prevalent ESRD patients who are awaiting transplants that have developed antibodies
during the reporting period
Denominator:
The number of all prevalent ESRD patients who are awaiting transplants during the reporting period
Data Source(s):
United Network for Organ Sharing (UNOS)
Medicare Cost Report (Form 265-94) Worksheet S-1
Trigger
To be determined
Frequency
Quarterly
Exclusions:
None
Report Level:
Organ Procurement Organization (OPO)
Page 12 of 16
INCREASE IN BLOODSTREAM INFECTION RATE
Quality of Care Performance Indicator # 3
Outcome Measure
Technical Specifications
Beneficiary Focused Learning Network, Version 1.0, March 2012
Purpose:
To investigate for possible patient and healthcare worker safety issues related to the care of ESRD patients
with catheters (12-month rolling average)
Status:
Proposed
Numerator:
Total number of catheter associated blood-stream infections
Denominator:
The number of patients with catheters multiplied by months of catheter use during 12-month period
Data Source(s):
CROWNWeb
National Healthcare Safety Network (NHSN)
Claims Data
Trigger
To be determined (potential data includes prior CDC and USRDS data)
Frequency
Quarterly
Exclusions:
None
Report Level:
Facility
Page 13 of 16
DECREASE IN DIRECT PATIENT CARE STAFFING RATIO
Quality of Care Performance Indicator # 4
Structural Measure
Technical Specifications
Beneficiary Focused Learning Network, Version 1.0, March 2012
Purpose:
To investigate for potential quality of care issues related to a reduction in direct patient care staffing
(includes physicians, RNs, LPNs, nurses aides, technicians, social workers & dieticians)
Status:
Proposed
Numerator:
1. The number of direct patient care staff employed by the dialysis facility
2. The number of direct patient care technicians
3. The number of direct patient care nurses (includes both RNs and LPNs)
Denominator:
1. The number of ESRD patients currently in dialysis program
2. The number of ESRD patients currently in dialysis program
3. The number of ESRD patients currently in dialysis program
Data Source(s):
Medicare Cost Report (Form 265-94) Worksheet S-1
Annual Survey
Trigger
To be determined
Frequency
Annually
Exclusions:
Renal Dialysis Facilities that have a cost reporting period of less than six months
Report Level:
Facility
Page 14 of 16
FACILITY MAINTENANCE CONDITION LEVEL CITATION TRENDS
Quality of Care Performance Indicator # 5
Structural Measure
Technical Specifications
Beneficiary Focused Learning Network, Version 1.0, March 2012
Purpose:
To investigate for possible reductions in facility maintenance expenditures that contributes to an increase
in facility citations (number of facilities in the state being surveyed)
Status:
Proposed
Numerator:
Total number of dialysis facility citations during the reporting period
Denominator:
Total number of state agency surveys conducted during the reporting period
Data Source(s):
State Department of Health Survey Data
Trigger
To be determined
Frequency
Annually
Exclusions:
None
Report Level:
Facility
Page 15 of 16
SIGNIFICANT CHANGE IN TRANSPLANT REFERRAL RATE
Quality of Care Performance Indicator # 6
Outcome Measure
Technical Specifications
Beneficiary Focused Learning Network, Version 1.0, March 2012
Purpose:
To investigate for a possible decrease in referral rates for ESRD patients presented as a transplant
candidate
Status:
Proposed
Numerator:
Number of patients who are awaiting transplants plus number of patients that received successful
transplants during 12 month period
Denominator:
Total number of patients currently in dialysis program
Data Source(s):
Medicare Cost Report (Form 265-94) Worksheet S-1
Transplant waiting lists
Trigger
To be determined
Frequency
Annually
Exclusions:
None
Report Level:
Facility
Page 16 of 16
V. Next Steps
Data collection tools are only as good as the data that is collected and entered into them. Without a
mutually agreed upon set of performance indicators with clearly defined specifications, the validity and
reliability of the data is unsubstantiated and in most cases will not provide standardized results.
The Learning Network has established a comprehensive set of performance indicators that includes
meaningful data types that can be utilized by the ESRD Networks to promote the collection of
standardized data. The next step in the process is to develop a dashboard to collect, analyze and report
the metrics in a uniform and consistent manner. In order to accomplish this objective the following
tasks will need to be completed:
• Submit a detailed list of performance indicators to CMS
• Obtain approval from CMS on final performance indicators
• Obtain access to data and analyze and review the data
• Develop a streamlined process for data collection and accurate metric calculation
• Select a tool that offers the best solution (i.e. Microsoft Access, Excel, etc…)
• Design and develop the dashboard
• Report the results
The development of performance indicators will provide us with meaningful data that that can be used
for developing data types that generate standardized, informative summary statistics and provide
insight for root cause analysis. The Learning Network has developed performance indicators in an effort
to investigate changes in performance as it relates to the quality and access of care provided to ESRD
beneficiaries.
