46
Program Performance Review Broward Addiction Recovery Division (BARC) April 15, 2008 Report No. 08-11 Office of the County Auditor Evan A. Lukic, CPA County Auditor
Program Performance Review
Broward Addiction Recovery Division (BARC)
April 15, 2008 Report No. 08-11
Office of the County Auditor Evan A. Lukic, CPA
County Auditor
2
Table of Contents Topic Page
Executive Summary.......................................................................................................... 3
Purpose and Scope ............................................................................................................ 5
Background ....................................................................................................................... 6
Section 1: Assessment of BARC’s Accountability System ......................................... 13
Section 2: Assessment of BARC’s Performance ......................................................... 28
Section 3: Assessment of BARC Program Alternatives ............................................. 39
Appendix A...................................................................................................................... 45
Appendix B ...................................................................................................................... 46
3
Executive Summary In this Program Performance Review, we (1) assess the quality of BARC’s accountability system, (2) evaluate its performance by analyzing available data, and (3) assess potential alternatives that could increase program revenues and reduce program costs. Section 1: Assessment of BARC’s Accountability System (See pages 13-27)
• Prior to Fiscal Year 2008, BARC lacked an effective accountability system
• In Fiscal Year 2008 BARC made the following improvements
o Added efficiency measures to each section’s set of performance measures, thus meeting Office of Management & Budget requirements, and
o Collected reasonably accurate unit cost data, which enables a comparison of BARC’s service costs to similar agencies and setting appropriate client fees
• However, BARC should further improve its accountability system by
o Fixing problems with its Client Management Information System in order to limit the need for time-consuming manual data collection, which is prone to error,
o Routinely benchmarking against high-performing agencies to compare its performance in areas such as staff productivity and service costs,
o Developing and implementing data reliability controls to correct data reporting problems regarding staff productivity, client counts, and client outcomes, and
o Collecting additional client outcome data, as identified by program supervisors, to better evaluate program effectiveness
Section 2: Assessment of BARC’s Performance (See pages 28-38)
• Due to the data reliability problems discussed in Section 1, we could not evaluate BARC’s effectiveness
o As of April 2008, BARC did not have reliable data on two important outcome
measures required by DCF: (1) whether clients successfully complete treatment and (2) whether clients are employed upon treatment discharge
• However, BARC’s treatment model contains many elements advocated in addiction recovery literature
o BARC has adopted several recommended practices, including continuum of
care, duration of treatment, and provision of culturally and gender specific services
4
• Our analysis of BARC’s unit cost data indicates that inpatient treatment costs are significantly higher than those reported in a 2005 national study
o For example, BARC’s Fiscal Year 2007 daily per client detoxification cost of
$458 is 83% higher than the daily per client national average of $250, o Similarly, BARC’s daily per client residential treatment cost of $261 for
Fiscal Year 2007 is 153% higher than the national average of $103
• Two factors contribute to BARC’s relatively high inpatient treatment costs: low bed utilization and high staffing levels
o BARC’s inpatient facilities are relatively underutilized; for example, the detox
bed utilization rate of 80% and residential bed utilization rate of 67% are both lower than the 91% national average inpatient bed utilization rate
o BARC’s staffing levels are high; for example, BARC’s residential treatment program “bed to staff” ratio of 1.6 is 106% higher than Miami-Dade County’s ratio of 3.3 beds per staff
• Therefore, to reduce costs to meet the daily per client national average detox and residential treatment costs, BARC would need to increase bed utilization and decrease staffing; these actions would save the County approximately $4.7 million annually
Section 3: Alternatives to Current Service Delivery Model (See pages 39-44) To reduce the amount of general and capital funds used for BARC services, the Board of County Commissioners could consider the following alternatives:
• Outsource some or all of BARC’s programs
o Only 3.4% of Florida’s substance abuse treatment facilities are operated directly by local governments,
o Most large Florida counties contract with community providers for the provision of substance abuse treatment services
o If the County contracted with providers that could meet the national average cost for detoxification and residential treatment, it would save the County $4.7 million annually
o In addition, the County could avoid a capital outlay of $21.3 million if it did not construct a new detox facility, as currently planned
• Not fund some BARC programs such as detoxification or outpatient services
o Some other large Florida counties, such as Hillsborough and Orange, do not fund detoxification because local public hospitals provide these services; for Fiscal Year 2007, BARC’s detox program cost $4.8 million
o Miami-Dade and Orange counties do not fund outpatient treatment programs, instead relying on community providers for these services; the cost of BARC’s outpatient program totaled $3.4 million in Fiscal Year 2007
5
• Offset the need for general funds by improving client fee collection and Medicaid reimbursement rates
o BARC’s client service fee collection rate is historically low; the average
collection rate for the past two fiscal years is 4.9% o BARC did not collect available Medicaid reimbursement for the past three
fiscal years; annual revenue projections indicate that uncollected Medicaid revenues total approximately $450,000 during this time
Recommendations
• To address program deficiencies and identify the feasibility of alternative practices we make several recommendations including:
o Providing a detailed justification of the need for a proposed $21.3 million detoxification facility expansion in light of existing bed underutilization (see page 33)
o Conducting further comparative analysis with similar detoxification and residential programs, including Miami-Dade County’s residential program, to analyze service cost, bed utilization, staffing ratios/composition and developing a plan to reduce treatment costs in accordance with results (see page 38)
o Reporting to the Board of County Commissioners within 60 days the potential effects of implementing various service delivery models used in other large Florida counties, such as contracting with community providers for the provision of services, providing less intensive residential treatment options and not funding the existing level of treatment services (see page 41)
6
Purpose and Scope This report presents the results of a Program Performance Review of the Broward Addiction Recovery Division (BARC) within the Human Services Department. The purpose of this review is to assess the quality of BARC’s accountability system, evaluate its performance and identify options for improving services and reducing operating costs. To accomplish our objectives, we:
• Reviewed pertinent performance measurement, general management, and addiction and recovery literature;
• Analyzed available performance data for BARC’s operations; • Analyzed information provided by BARC’s managers and employees relative to
specific management processes and controls; • Reviewed applicable BARC records and documents, including relevant policies
and procedures; • Reviewed applicable Florida Statutes and Broward County Ordinances pertaining
to substance abuse treatment; • Interviewed BARC managers and employees; and • Interviewed substance abuse treatment program officials in other jurisdictions and
Florida Department of Children and Families staff Background Established in 1973, BARC provides medical and clinical treatment, substance abuse and nutrition education and support services to Broward County residents and homeless individuals who are chemically dependent and 18 years or older. In accordance with Section 397.406, Florida Statutes, BARC is licensed and regulated by the State of Florida as a substance abuse treatment provider. BARC provides a range of substance abuse treatment services intended to meet client needs, including:
• Access Services: At the time an individual seeks treatment BARC completes a comprehensive assessment, which involves gathering information about the individual’s physical and mental health and history of drug use, and in conjunction with the individual, makes a determination of the appropriate treatment and level of care needed to meet individual and programmatic goals.
• Detoxification1 (Detox) Services: A medically supervised 35-bed facility operating 24 hours per day, 7 days per week, for clients who need medical stabilization before receiving other program services or referral to other agencies. During detox, clients also receive individualized assessments and addiction and recovery education, and
1 The U.S. Substance Abuse and Mental Health Administration defines detoxification as the process through which a person who is physically dependent on alcohol, illegal drugs, prescription drugs or a combination, is withdrawn from the substance of dependence.
7
may participate in Alcoholics Anonymous (AA) and Narcotics Anonymous (NA) meetings.
• Residential Treatment Services: BARC operates a 92-bed, short-term inpatient program aimed at providing a supportive environment to establish the principle of recovery. Clients are provided counseling, substance abuse and nutrition education and family support and may participate in AA and NA meetings. Typically, residential treatment lasts for 28 days, after which clients may be referred to other BARC programs or community resources.2
• Non-Residential Day Treatment: A three to five week program operating Monday through Friday that generally provides intensive group and individual counseling and support services to clients upon their release from residential treatment and before placement in a less intensive program such as outpatient services.
• Outpatient Services: A three to six month program in which clients participate in individual and group counseling, are taught basic life management and interpersonal skills, receive psychiatric evaluations as needed and HIV testing, education and counseling. Outpatient services are provided mornings and evenings to enable clients to maintain employment while receiving treatment.
• Resources for Recovery: A case management and advocacy program for clients who have had difficulty staying sober in traditional settings and have a greater need for assistance and follow-up.
• Specialized Services: BARC also has programs that are designed to meet the needs of a diverse Broward County population, including: • Co-Occurring Disorders: substance abuse treatment for individuals with both
substance abuse and mental health issues; • Family Involvement Program: educational services for family members of
current clients as deemed appropriate; • Hispanic Track: outpatient services provided in Spanish that take language and
culture into consideration; • Mature Adult Program: specialized treatment for clients who are 55 years of
age and older; and • Perinatal Addiction Services: residential, day treatment and outpatient
services for pregnant women or women with children 7 years or younger.
2 However, pregnant women may stay in residential treatment until their babies are delivered and a 90-day residential treatment program is available to women with children 7 years of age and younger.
8
Service Locations BARC provides services at four facilities located throughout the County:
• BARC Central at 1000 S.W. 2nd Street in Ft. Lauderdale, • BARC Lauderhill at 4200 N.W. 16th Street in Lauderhill, • Stephen R. Booher Building at 3275 N.W. 99th Way in Coral Springs, and • BARC South at 5701 Hollywood Boulevard in Hollywood.
Exhibit 1 below shows the types of services provided at each of these four facilities. Exhibit 1: Availability of BARC services by facility
Type of Service
BARC Central
BARC Lauderhill
Booher Building
BARC South
Access x Detoxification x Residential x Non-Residential Day x x Outpatient x x x Case Management x Specialized Services x x x x
Source: BARC Client Admissions Exhibit 2 below shows the number of client admissions, by type, during Fiscal Year 2007; clients are often admitted to more than one type of service during their treatment. Exhibit 2: Most BARC admissions were for Access and Detoxification services
Type of Service
Number of Admissions
Access 3,430 Detoxification 1,594 Residential 870 Non-Residential Day 1,036 Outpatient 893 Case Management 259 Specialized Services 846
Source: BARC While most clients seek treatment on their own, some clients are referred to BARC by the Florida Department of Children and Families, the judicial system or community health providers. Clients are charged fees for services, on a sliding scale, based on their annual income and family size. Section 397.431(2), Florida Statutes, requires clients receiving treatment to contribute as much of the cost of services as they can.
9
Exhibit 3 below shows an example of BARC’s sliding fee scale for several services. Exhibits 4 thru 7 illustrate the gender, race, age, and addiction type(s) of Fiscal Year 2007 clients. Exhibit 3: BARC’s Fiscal Year 2008 sliding fee scale is based on each client’s annual income and family size Client Income Level3
Access Fee/Hour
Detoxification Fee/Day
Residential Fee/Day
Outpatient (Group) Fee/Hour
$0 - $7,499 $7.50 $18.50 $13.00 $3.00 $7,500 - $14,999 $15.00 $37.00 $26.00 $6.00 $15,000 - $22,499 $22.50 $55.50 $39.00 $9.00 $22,500 - $29,999 $30.00 $74.00 $52.00 $12.00 $30,000 - $37,499 $37.50 $92.50 $65.00 $15.00 $37,500 - $44,999 $45.00 $111.00 $78.00 $18.00 $45,000 - $52,499 $52.50 $129.50 $91.00 $21.00 $52,500 - $59,999 $60.00 $148.00 $104.00 $24.00 $60,000 - $67,499 $67.50 $166.50 $117.00 $27.00 $67,500 and over $75.00 $185.00 $130.00 $30.00
Source: Broward County Administrative Code
Exhibit 4: A significant majority of BARC’s Fiscal Year 2007 clients were male
Female 1,525 (30%)
Male 3,529 (70%)
Source: BARC
3 In this example, the client’s family size is one.
10
Exhibit 5: Most of BARC’s Fiscal Year 2007 clients were white4
White 3,172 (63%)
Black 1,625 (32%)
Multi-Racial219 (4%)
Other38 (1%)
Source: BARC
Exhibit 6: BARC served a range of client age groups in Fiscal Year 2007
65 and over (1%)
18-29 (31%)30-44 (41%)
45-64 (27%)
Source: BARC
4 493 of BARC’s 5,054 (9.8%) Fiscal Year 2007 clients were Hispanic.
11
Exhibit 7: BARC clients were most frequently admitted for alcohol dependence treatment in Fiscal Year 20075
Alcohol 2,862
Opioid 1,447
Cocaine 1,743
Cannabis 890
Source: BARC
Program Resources As shown in Exhibit 8 below, BARC’s Fiscal Year 2008 operating budget of $19 million is primarily funded by the general fund and a grant from the Florida Department of Children and Families (DCF). Exhibit 8: Most of BARC’s Fiscal Year 2008 funding comes from the general fund
Funding Source Revenue % of BudgetGeneral Fund $13,701,300 72.0%DCF Grant6 $3,745,180 19.7%Other State Grants $701,900 3.7%Federal Grant (HUD) $406,700 2.1%Kids in Distress Grant $262,930 1.4%Medicaid Revenue $150,000 <1%Revenue from Operations $55,000 <1%Miscellaneous Revenue $6,000 <1%Total $19,029,010 100%
Source: Office of Management & Budget 5 Per BARC, a client may be diagnosed with multiple addictions when entering treatment. Therefore, the number of admissions per addiction type is larger than the number of clients served in Fiscal Year 2007. Opioids are a class of drugs derived from the opium poppy plant or produced synthetically that are used to relieve pain, dull the senses or induce sleep. Examples of opioids include heroin, codeine and methadone. 6 DCF annually provides BARC with negotiated grant funding for substance abuse treatment services via a three-year contract approved by the Board of County Commissioners; BARC’s current contract with DCF expires after Fiscal Year 2008.
12
Division Organization BARC is organized into 3 sections: Administration/Support Services; Outpatient Services; and Residential Services. Exhibit 9 below summarizes BARC’s Fiscal Year 2008 operating budget, full-time equivalent (FTE) positions and major services, by section. Exhibit 9: In Fiscal Year 2008, BARC was appropriated $19 million and had 226 FTE positions
Section
Budget
FTE
Major Services
Administration/ Support Services
$2,124,060
25
--Overall direction, oversight, management and administrative support of BARC’s programs and operations
Outpatient Services
$6,137,800
69
Access Services --Comprehensive physical and mental health assessment Outpatient Services --Individual and group counseling --Basic life management and interpersonal skills education --Psychiatric evaluations --HIV testing, education and counseling Non-Residential Day Treatment --Intensive individual and group counseling --Support services Resources for Recovery --Case management and peer mentoring Specialized Services --Day treatment and outpatient services for previously mentioned target populations Other Services --Partner with local law enforcement and community agencies to provide drug screening
Residential Services
$10,767,150
132
Detoxification Services --Medical stabilization --Individualized assessments --Addiction and recovery education Residential Treatment --Individual and group counseling --Substance abuse and nutrition education --Family support services Specialized Services -- Residential treatment for previously mentioned target populations
TOTAL
$19,029,010
226
Source: Office of Management & Budget
13
Section 1: Assessment of BARC’s Accountability System This section presents our assessment of BARC’s accountability system. Establishing an effective accountability system is essential to ensure public funds are spent in a manner that achieves desired results and improves the provision of public services. An effective accountability system provides quality information to enable:
• Citizens to hold elected officials, managers, employees and private contractors accountable for the efficient and effective use of public funds;
• Elected officials and other policymakers to make informed budget and policy decisions; and
• Managers to detect and correct operational deficiencies and improve program results
As a framework for evaluating accountability, we assessed BARC’s performance measurement and reporting system relative to six best practices. Specifically, we have examined whether the Division has:
• Established a complete set of performance measures, as required by the Office of Management & Budget (OMB);
• Collected and analyzed unit cost data for each major service and activity; • Collected and analyzed data to assess operational efficiency and effectiveness; • Benchmarked its performance against other agencies that provide similar services
or generally accepted industry standards; • Established processes to ensure performance data reliability; and • Used performance data to modify current practices or change operational
processes in order to achieve better program results Appendix A on page 45 summarizes our Section 1 findings. Best Practice 1: Establish a complete set of performance measures According to performance measurement literature, a comprehensive set of performance measures provides information to enable policymakers and managers to assess each agency’s workload, operating efficiency, effectiveness, and societal impacts. The Office of Management and Budget Services (OMB) requires County agencies to collect and report information relative to four types of performance measures: work output, efficiency, client benefit/effectiveness, and strategic outcome.7
7 “Work output focuses on the quantity of service provided; efficiency measures the ratio of output per input; client benefit/effectiveness assesses the quality of the service from the clients’ perspective; and strategic outcome states the consequences of the program in a ‘big picture’ sense.”
14
BARC established a complete set of performance measures during Fiscal Year 2008 We found that BARC has not historically met OMB’s performance measure reporting requirement due to a lack of efficiency measures for its three sections. However, in January 2008, BARC worked with OMB officials to develop efficiency measures for each of its sections. Specifically, BARC’s Administration/Support Services Section will begin reporting the efficiency measure, “Number of purchasing forms and credit card purchases processed per certified agency buyer” during Fiscal Year 2008. Similarly, BARC’s Residential Services Section and its Outpatient Services Section will report the “Number of direct service and contact hours per clinical staff per week.” With these new efficiency measures, BARC meets OMB requirements. Appendix B on page 46 lists BARC’s Fiscal Year 2008 performance measures. Best Practice 2: Collect and analyze unit cost data for major services Unit cost refers to the cost of producing an output or outcome.8 Collecting and analyzing unit cost data is important because it can be used to:
• Compare an agency’s performance to similar service providers, • Predict how changes in the demand for services will affect an agency’s budget, • Assess how changes in operations could affect costs, • Identify wasteful processes, • Set appropriate service fees, and • Estimate the impact of budget decisions
BARC has established a goal of providing the most cost effective methods of treatment For Fiscal Year 2008, BARC established a goal of providing “the most cost effective method of treatment” to clients. According to substance abuse treatment literature, measuring cost effectiveness is increasingly important as demands to justify service costs have heightened due to pressures on public funding. Industry research indicates that “other things equal, the lower the treatment cost, the more cost effective a program or intervention will be.”9 To determine its treatment costs, BARC should collect and analyze unit cost data. For example, BARC should routinely monitor the “cost per client per day” of providing its primary services, such as detoxification and residential treatment to ensure these services are provided in a cost effective manner. Additionally, BARC should use unit cost data to make certain that client service fees (generally set on a “per day” or “per hour” basis) are
8 Source: Florida Legislature’s Office of Program Policy Analysis and Government Accountability (OPPAGA) 9 Source: “Economic Benefits of Drug Treatment: A Critical Review of the Evidence for Policy Makers” Treatment Research Institute at the University of Pennsylvania. February 2005.
15
appropriate and to target reasonable reimbursable rates during contract negotiations with Florida’s Department of Children and Families (DCF).10 BARC has not historically collected and analyzed unit cost data, but the Division began collecting this data in February 2008 We found that BARC has not historically collected unit cost data. Therefore, we could not evaluate the Division’s cost effectiveness across time. However, in February 2008 the Division developed unit cost data for the following services: • Detoxification, • Day treatment, • Residential treatment, • Access (intake and assessment), and • Outpatient treatment
BARC’s development of unit cost data in Fiscal Year 2008 is important for two reasons.11 First, it enables managers and program evaluators to compare BARC’s service costs with similar agencies and/or industry standards (in Section 2 of this report, beginning on page 28, we compare BARC’s service delivery costs for detoxification and residential service to national averages). This type of analysis generally provides an indication of whether treatment is provided in an efficient manner. Additionally, BARC’s unit cost data provides justification for client service fees. BARC’s February 2008 unit cost data raises questions about the appropriateness of Fiscal Year 2008 client service fees BARC’s unit cost data suggests that its client service fees bear no relationship to the actual cost of providing service. For example, BARC’s maximum fee for Access services (i.e. intake and assessment) is $75 per hour, while BARC’s actual cost to provide these services is reported to be $60 per hour. As BARC officials told us that many clients are indigent, it is critical that they are not overburdened by service fees. However, as of Fiscal Year 2008, BARC has not established criteria defining the relationship that should exist between service cost and client fees.12 10 DCF annually provides BARC with negotiated grant funding for substance abuse treatment services via a three-year contract approved by the Board of County Commissioners; BARC’s current contract with DCF provides $3.7 million of annual program funding and expires after Fiscal Year 2008. 11 While BARC’s unit cost methodology captures its base operating cost of providing services, it does not capture the County’s full cost of providing services because it excludes capital budgeted expenditures, such as building depreciation. 12 For example, the County could pass 100% of its actual cost of providing substance abuse treatment services to clients, or it could establish an appropriate level of subsidy for each service. Regardless of the fee criteria chosen, BARC’s existing sliding fee scale would provide a level of subsidy to low income clients.
16
We also noted that BARC has not developed separate unit cost data for its two types of outpatient service, (1) individual and (2) group counseling. Rather, the Division developed one aggregate unit cost for both services. As BARC has a different client service fee for individual outpatient counseling than it does for group outpatient counseling, it should develop unit cost data for both services to assess the appropriateness of its fees. Exhibit 10 below shows BARC’s cost per service and its Fiscal Year 2008 client service fees. Exhibit 10: BARC’s cost of providing services bears no apparent relationship to its Fiscal Year 2008 client service fees Service BARC Cost
Per Service BARC Client Service
Fee Range13 Detoxification $458 per day $18.50 - $185 per day Residential $261 per day $13 - $130 per day Day Treatment $248 per day $7 - $70 per day Access $60 per hour $7.50 - $75 per hour
$9 - $90 per hour (individual) Outpatient $60 per hour $3 - $30 per hour
(group) Source: BARC and Broward County Administrative Code, Chapter 36.1 Recommendation 1: We recommend that the Board of County Commissioners direct the County Administrator to take the following actions:
• Establish criteria defining the appropriate relationship between client service fees and BARC’s actual cost of providing services by September 30, 2008
• Develop unit cost data for both individual and group outpatient treatment to ensure client service fees for these services are appropriate; any proposed client service fee changes should be submitted to the Board of County Commissioners for approval by September 30, 2008
Best Practice 3: Collect and analyze internal operational data To effectively manage County programs and operations, managers need more data than what is reported in the annual budget. The data reported in the annual budget generally relates to an entity’s overall performance rather than specific functions and activities. While the budget data is useful for accountability purposes, managers commonly need additional information to effectively supervise daily operations. 13 As depicted in more detail on page 9, BARC has implemented a sliding fee scale based on each client’s annual income. Therefore, we provide the range of fees that can be charged for each BARC service.
17
BARC routinely collects internal operational data BARC officials provided several examples of collecting internal operational data to monitor program activities. For example, BARC provided two reports which are regularly used to monitor employee productivity. In one report, the Division monitors the “caseload per clinician” on a weekly basis to ensure that staff has been assigned the appropriate number of clients. Similarly, the Division uses another weekly report to monitor all activities performed by each clinician, including the number of counseling sessions and client drug screenings conducted. This data, while not reported to policy makers, is necessary to ensure the efficient delivery of treatment services. Client management information systems (CMIS) provide consistent and timely internal operational data To be optimally useful, internal operational data needs to be obtained in a timely manner. Data that is time-consuming to collect and analyze impedes clinical employees from focusing on their primary function of providing direct treatment services to clients. Additionally, supervisors need access to real-time data to monitor productivity and assess the outcomes of new initiatives. To help reduce the burden of data collection, many agencies use client management information systems (CMIS) as their primary source of data collection and management reports. According to CMIS literature, human services agencies often use these systems to collect client information and track the services they have received. As CMIS data is stored in a central database, management and other staff can easily access it to track progress toward desired outcomes. We found that effective CMIS provide two primary advantages over manual data collection:
• Improved data reliability. An effective CMIS requires staff to complete designated fields during client intake and subsequent counseling. This provides assurance that critical data is consistently captured for every client served. Also, by removing the human element from data computations, a CMIS decreases the opportunity for errors.
• Real-time data analysis. As CMIS data is stored in a central database, any
authorized user can access reports that provide up-to-the minute assessments of agency performance. This is superior to manually collected data, which is often collected by staff for a designated period (usually one week or month) and then tallied into a summary management report. Also, because CMIS data is in electronic form, managers can easily use it to forecast the effect of adding or removing resources from particular service areas.
18
Despite investment in a CMIS, BARC relies on manually collected internal operational data In September 2002, BARC purchased a client management information system (hereafter called “ECHO”) from The Echo Group, Inc. for $461,482. According to company literature, ECHO provides users with the ability to create electronic client records, treatment plans and management reports. However, we found that many internal operational data reports regularly used by BARC supervisors are generated through manual data collection processes. Specifically, clinical staff uses Excel spreadsheets or standardized paper forms to track basic client data, such as the number of clients seen each week. Additionally, more detailed information about the specific services provided by each clinician, such as the number of individual and group counseling sessions conducted, is also captured on a weekly basis through manual reports. Manually collected data is aggregated on a weekly or monthly basis for supervisory review. BARC officials told us that the Division continues to rely on manually collected data due to its slow implementation of ECHO and its lack of technical resources to develop desired reports within the system. For example, ECHO has not been configured to provide supervisors with key productivity reports, such as the amount of time each counselor has spent providing group counseling. While each clinician inputs this data into ECHO on a daily basis, clinicians have also been instructed to manually collect this data in Excel spreadsheets for supervisory review. Supervisors stressed the importance of monitoring this data because it is necessary to determine if BARC is meeting established goals for direct service hours.14 Better utilization of BARC’s CMIS should improve program efficiency BARC officials told us that they believe fully realizing ECHO’s capabilities would increase the number of direct service hours available to clients. This would increase the number of clients that each clinician could counsel on a weekly basis, or allow each clinician to provide more intensive counseling services to existing clients. We believe this is important because increasing the number of clients treated or the effectiveness of existing services within existing resources would improve BARC’s cost effectiveness. Recommendation 2: We recommend that the Board of County Commissioners direct the County Administrator to take the following action:
• Discontinue the use of time-consuming manual data collection, where possible, by developing reports within ECHO that will allow supervisors to efficiently and effectively monitor staff productivity and client outcomes by September 30, 2008
14 BARC defines direct service hours as the “actual time spent on activities directly associated with a single client, including case staffing. Time may include travel if the travel is integral to a service event or otherwise billable.”
19
Best Practice 4: Contact outside sources, such as industry and other government agencies, to benchmark performance The Government Finance Officers Association recommends benchmarking as a method of obtaining information that can be used to identify inefficient practices and strategies to achieve better results. Benchmarking refers to “performance comparisons of organizational business processes against an internal or external standard of recognized leaders.” County managers should systematically and routinely compare policies, practices, and performance measures against those of other local governments, private contractors that provide similar services and industry standards. BARC has historically conducted limited benchmarking In response to our request to provide examples of benchmarking, BARC cited its participation in a study conducted by the Joint Commission in 2001.15 In this study, the performance of BARC and two other South Florida substance abuse treatment providers was compared. Specifically, BARC was compared to Hanley-Hazelden of West Palm Beach, Florida and Transitions Recovery Program of North Miami Beach, Florida. The study evaluated several measures of performance, including the “average length of stay (per client)” and the “percent (of clients) with a satisfactory discharge rating,” which is a common measure of treatment completion. Several factors limit the usefulness of BARC’s previous benchmarking effort Our review disclosed several concerns with the 2001 benchmarking effort which severely limit its value in assessing BARC’s performance. These include:
• Peer comparability concerns. BARC officials expressed concern that Hanley-Hazelden and Transitions Recovery Program were not appropriate BARC peers. These officials advised that BARC typically serves lower income Broward County residents, while Hanley-Hazelden and Transitions Recovery Program generally market their services to a more affluent clientele. As an example of the difference in client demographics served by these agencies, BARC officials surveyed these agencies in Fiscal Year 2008 and found that Hanley-Hazelden charges $25,000 per month for its residential treatment program, while Transitions Recovery Program charges $12,000 per month. BARC’s highest monthly fee for residential treatment in Fiscal Year 2008 is $3,640.
• Lack of efficiency or outcome comparisons. Our review of BARC’s
benchmarking study revealed that it provided very limited insight about the efficiency and effectiveness of each agency. For example, the study did not include any comparisons of service costs, staff productivity (i.e. cases per clinician) or the percentage of clients that remained substance free after treatment. Without this data, cost effectiveness comparisons cannot be performed.
15 The Joint Commission is an independent, non-profit organization that has accredited and certified more than 15,000 health care agencies in the United States, including BARC.
20
• Age of study. As the Joint Commission study was conducted in 2001, its value has diminished over time. For example, the expected “average length of stay (per client)” may change through the years as new treatment modalities are researched and adopted in the substance abuse treatment industry. Accordingly, it is important for administrative and treatment personnel to make reasonably current comparisons of performance.
BARC took initial steps in Fiscal Year 2008 to identify appropriate peer agencies and benchmark agency performance; however, as of April 2008 BARC has not identified peers During our review, BARC officials expressed frustration in identifying appropriate peer agencies. These officials said that providing direct substance abuse treatment services makes BARC a unique entity with the State of Florida (in Section 3 of this report beginning on page 39, we examine the substance abuse treatment delivery methods used by other Florida counties). However, BARC took several steps in Fiscal Year 2008 to identify appropriate peers for benchmarking. For example, in November 2007, BARC officials attended a half-day benchmarking training sponsored by the OMB. Additionally, as of March 2008, the Division reports that it continues to research benchmarking opportunities and strategies with the Joint Commission and the U.S. Substance Abuse and Mental Health Services Administration (SAMHSA). Despite these efforts, as of April 2008 BARC had not identified peers for benchmarking. Recommendation 3: We recommend that the Board of County Commissioners direct the County Administrator to take the following actions:
• Identify appropriate peer agencies (or industry standards) that can be used as reasonable benchmarks of BARC’s performance in future evaluations
• Identify appropriate performance measures to benchmark, including service cost and staff productivity comparisons
• Report the results of Division’s initial benchmarking analysis to the Board of County Commissioners by September 30, 2008
Best Practice 5: Establish a process to ensure performance data reliability According to the U.S. Government Accountability Office (GAO), performance data should be reasonably timely, complete and accurate to be useful for budget allocation and policymaking purposes. To ensure reliable performance data, government agencies should implement internal controls, such as establishing clear and unambiguous performance measure definitions, having front line and middle managers independently review performance data, and testing a sample of performance data at least quarterly.
21
BARC has historically lacked internal controls to ensure reliable client counts are reported to the Office of Management & Budget BARC’s historical client counts, as reported to the OMB each fiscal year, show a significant decrease in the number of clients served beginning in Fiscal Year 2006. For example, in Fiscal Year 2005 BARC reported 6,231 outpatient client admissions. However, in Fiscal Year 2006, BARC reported only 1,295 outpatient client admissions, a 79.2% decrease. Exhibit 11 below shows the number of outpatient client admissions BARC has reported to the OMB since Fiscal Year 2004. Exhibit 11: The number of client admissions to BARC’s outpatient program, as reported to OMB, declined by 79.2% between Fiscal Year 2005 and 2006
Number of Outpatient Client Admissions
6,2317,005
1,295 1,696
0
2,000
4,000
6,000
8,000
FY04 FY05 FY06 FY07
Source: Office of Management & Budget We also observed an apparent decrease in clients served in BARC’s residential program during this same period. Specifically, in Fiscal Year 2005 BARC reported admitting 1,742 clients to its residential program. However, in Fiscal Year 2006, BARC reported admitting only 735 residential clients, a 57.8% decrease. Exhibit 12 below shows the number of residential client admissions BARC has reported to the OMB since Fiscal Year 2004. Exhibit 12: The number of client admissions to BARC’s residential program, as reported to OMB, declined by 57.8% between Fiscal Year 2005 and 2006
Number of Residential Client Admissions
1,512 1,742
735 815
0
500
1,000
1,500
2,000
FY04 FY05 FY06 FY07
Source: Office of Management & Budget
22
BARC officials told us that they believe the number of clients served between Fiscal Year 2005 and 2006 did not decline at the levels suggested by data historically reported to OMB. Rather, we were told that BARC has historically struggled to obtain accurate client counts due to its reliance on manually collected data, which is often “vulnerable to misinterpretation and of limited value.” In Fiscal Year 2006, Division officials made a concerted effort to ensure that reasonably accurate client counts were reported to the OMB from that point forward. This effort included having supervisory staff review data accuracy during monthly meetings. Division officials told us that data reported prior to Fiscal Year 2006 is inaccurate; however, data reported after this period presents a truer picture of the Division’s annual workload.16 Consequently, we could not use historical client count data to evaluate the Division’s past performance. We identified four primary concerns with BARC’s Fiscal Year 2007 data which suggest additional steps should be taken to ensure data reliability Despite actions taken in Fiscal Year 2006 to improve data reliability, we identified four primary data reliability issues with BARC’s Fiscal Year 2007 data. These issues suggest additional steps should be taken to ensure that accurate data is reported to the OMB, policymakers and other entities in Fiscal Year 2008. Discrepancies in Client Counts As described above, BARC has historically relied upon manually collected data to determine annual client counts. However, annual client count data is also collected within the Division’s CMIS, called ECHO. To validate the client count data reported to the OMB in Fiscal Year 2007, we compared it to client count data generated from the ECHO system that was provided to us by BARC officials. Our comparison suggests that client count data reported to OMB in Fiscal Year 2007 is inaccurate. For example, BARC reported to OMB that it admitted 703 clients to its Day Treatment program in Fiscal Year 2007. Conversely, BARC’s ECHO system indicates that the Division admitted 1,036 clients to the Day Treatment program. As shown in Exhibit 13 on the next page, other BARC services, such as residential, outpatient and detoxification showed smaller variances between these two reports.
16 Some of the decline in client admissions in Fiscal Year 2006 may have been due to the closing of one of BARC’s facilities after Hurricane Wilma. However, Fiscal Year 2007 client admissions, which were not affected by Hurricane Wilma, are fairly consistent with Fiscal Year 2006. This indicates that inaccurate data reporting was the primary reason for the apparent decline in admissions observed from Fiscal Year 2005 to Fiscal Year 2006.
23
Exhibit 13: Data provided by BARC indicates that client counts reported to OMB are incorrect
Service OMB
Report ECHO Report
% Difference
Day Treatment 703 1,036 -47.4% Access 3,956 3,430 13.3% Residential 815 870 -6.7% Outpatient 1,696 1,739 -2.5% Detoxification 1,559 1,594 -2.2%
Source: BARC & Office of Management & Budget Significant Understatement of Service Costs in “Results Team Proposal Form” During the development of the Fiscal Year 2008 budget, each Broward County agency was directed by the OMB to complete a “Results Team Proposal Form.” These forms required each agency to indicate if alternative providers outside Broward County Government could provide the same or similar services to residents. In its March 2007 response, BARC identified several private substance abuse treatment facilities that operate in Broward County, but indicated that privatization would lead to “greater expense to the community.” As evidence, BARC compared its unit cost of residential/outpatient service to an industry benchmark and reported that it “already operates in a highly efficient manner” and that further funding cuts were “likely to impact the quality of client care and its effectiveness.” However, as previously described on page 15, we found that BARC did not develop unit cost data until February 2008. Further analysis of data reported in the Fiscal Year 2008 Results Team Proposal Form revealed that BARC used a flawed unit cost methodology in its comparison, causing the Division to significantly understate its service costs. For example, BARC reported a cost of $10.49 per client per day for its residential treatment program. However, BARC’s February 2008 unit cost analysis indicates a cost of $261 per client per day for residential treatment, which we believe is a more realistic figure. Consequently, the cost comparison provided by BARC in its Fiscal Year 2008 Results Team Proposal Form should be considered invalid. As County officials use the information and data provided in the Results Team Proposal Forms to make funding allocation decisions, it is critical for this data to be reliable. Exhibit 14 below compares BARC’s per day service costs for residential and outpatient services to the costs it reported in its Fiscal Year 2008 Results Team Proposal Form. Exhibit 14: BARC significantly understated service costs in its Fiscal Year 2008 Results Team Proposal Form Service Type
FY08 Results Team Proposal Form
February 2008 BARC Unit Cost Analysis
Residential $10.49 per day $261 per day Outpatient $6.48 per day $60 per hour
Source: BARC & Office of Management & Budget (OMB)
24
Incomplete Data Submitted to the Florida Department of Children and Families (DCF) As part of its contractual requirements for grant funding, BARC must collect and submit monthly client treatment outcome data to DCF. Specifically, BARC’s contract with DCF contains two client outcome performance measures that are used to assess the effectiveness of BARC’s treatment. While additional client outcome measures exist in substance abuse treatment literature, we found that BARC relies on these two measures as the primary indicators of treatment success. Exhibit 15 below lists the two substance abuse treatment performance measures in BARC’s DCF contract, and the performance standards that BARC must meet to remain in contract compliance. Exhibit 15: BARC’s DCF contract contains performance measurement requirements DCF Performance Measure
Performance Target
Percent of discharged adults successfully completing treatment (no alcohol or other drug use during the month prior to discharge)
69%
Percent of adults employed upon treatment discharge 76% Source: Florida Department of Children and Families Performance Contract No. JD220 As we desired to use data reported by BARC to DCF to evaluate the Division’s performance, we met with BARC officials to discuss how this data is collected and reported. Our principle concern was the number of Fiscal Year 2007 client discharges reported by BARC to DCF. Specifically, BARC officials provided an internal report that indicated the Division had discharged 2,540 clients in Fiscal Year 2007. However, data provided by DCF showed BARC reported discharging only 546 clients in Fiscal Year 2007. BARC officials told us that data reported to DCF has historically been incomplete. Specifically, Division officials told us of continued technical difficulties uploading data to DCF’s database. As of the conclusion of our fieldwork in March 2008, BARC had not provided us with a corrective action plan to ensure that complete data is uploaded to DCF in future reporting periods. However, it is important for BARC to correct this data reporting problem because DCF officials indicated to us that future grant funding decisions may be more heavily influenced by agency performance. Incomparability of Outpatient Staff Productivity Data In each Program Performance Review, we evaluate employee productivity data to assess program efficiency. In response to our request, BARC provided Fiscal Year 2007 productivity data for its outpatient treatment units. Specifically, this data appeared to indicate the average caseload per clinician. We met with BARC’s four outpatient supervisors to review Fiscal Year 2007 productivity data. Our primary concern was the disparate level of productivity that we
25
observed across outpatient locations. For example, in March 2007, BARC’s outpatient unit at its Lauderhill location reported a caseload of 17 clients per clinician, while BARC’s outpatient unit at its Fort Lauderdale location reported a caseload of 94 clients per clinician. BARC’s outpatient supervisors told us that each unit collected productivity data through slightly different methodologies in Fiscal Year 2007; therefore productivity data is not comparable. For example, some supervisors reported the aggregate caseload of their unit, while others reported the average caseload per clinician. Consequently, we could not assess the efficiency of BARC’s outpatient program. Recommendation 4: To address the four data reliability issues we identified, we recommend that the Board of County Commissioners direct the County Administrator to take the following actions: • Develop data reliability controls, such as written measure definitions and
supervisory review, to ensure that accurate client count data is reported to the OMB as part of the County’s performance measurement and reporting system,
• Ensure the use of reliable unit cost data in future service cost comparisons submitted during the budget development process,
• Work with DCF to ensure that complete and accurate performance data is reported as contractually required,
• Standardize employee productivity reporting for outpatient services to ensure the accuracy and comparability of data across units
Best Practice 6: Use performance data to modify practices or change operational processes to achieve better results According to the U.S. Government Accountability Office (GAO), the benefit of collecting performance information is only fully realized when the information is actually used by managers to make decisions directed towards improving results. Managers can use performance information to identify operational problems and their causes, and to develop corrective actions; to effectively plan and prioritize workload and resources; and to identify more effective approaches to program implementation. BARC has established a structured, organization-wide process to focus agency resources BARC annually updates a document known as its Performance Improvement Plan. Similar to a strategic plan, BARC’s Performance Improvement Plan helps to focus resources on meeting Division objectives, such as reducing the risk of clients acquiring and transmitting infectious diseases. To meet its Performance Improvement Plan objectives, BARC has established inter-departmental committees to assess and improve specific objectives that have been
26
identified and prioritized by Division management. According to BARC’s Fiscal Year 2008 Performance Improvement Plan, each committee is comprised of “staff who are the most familiar with and knowledgeable about the process selected for assessment and improvement.” As of Fiscal Year 2008, ten committees meet on a monthly, bi-monthly or quarterly basis. These committees are: • Infection Control Committee • Environment of Care Committee • Utilization (Resource) Review Committee • Human Resources Committee • Information Management Committee • Clinical Care Committee • Medical Executive Committee • Communications Committee • Food and Nutrition Committee • Leadership Committee
BARC reports several successes due to the use of performance improvement committees During our review, we asked BARC officials to provide examples of program improvements that have resulted from the use of regular committee meetings. BARC officials provided the following: • Addition of medical staff to assist with complicated psychiatric conditions • Reassignment of clinical staff to address waitlists • Initiation of Co-occurring Disorders unit • Development of the Mature Adult Program • Introduction of a triage unit at detoxification • Development of improved infection control interventions at all units • Streamlining of medication formularies to reduce costs
BARC supervisors believe additional data could be collected to improve the performance improvement process We conducted a focus group of BARC supervisors to obtain their opinions about the quality and quantity of data available to performance improvement committee members. Supervisors generally expressed satisfaction with the performance improvement committee structure and process, but felt that they could benefit from additional performance data currently unavailable through the Division’s CMIS (ECHO). Specific examples of additional performance data include:
27
• “Average length of client stay per clinician” - supervisors told us that this
measure is useful for determining each clinician’s success at retaining clients for the duration of their prescribed treatment. Clinicians that demonstrate relatively poor client retention rates would be targeted for performance improvement.
• “Revenue per unit” – by reviewing payments made by clients at each service
unit, supervisors can better monitor whether clients are appropriately paying for services.
• “Percent of clients that move to next level of treatment” – clients often begin
treatment in a relatively intensive setting, such as BARC’s residential facility in Coral Springs. At completion of residential treatment, clients are generally prescribed to continue periodic counseling at one of BARC’s three outpatient locations. However, supervisors indicated that they currently lack data about the number of clients who continue to the “next level of treatment” as directed. By directing resources to the points where clients are most likely to discontinue treatment, BARC could improve client retention and treatment success.
BARC supervisors generally expressed concern that data captured by the Division’s CMIS (ECHO) is not easily accessible in report formats. However, supervisors were optimistic that access to additional performance reports would immediately improve the performance improvement committees’ ability to improve the quality and quantity of services provided to clients. Recommendation 5: To improve program accountability and effectiveness, we recommend that the Board of County Commissioners direct the County Administrator to take the following actions: • Develop a list of additional performance measures that are critical for monitoring
program accountability and effectiveness by September 30, 2008 • Develop reports within ECHO that will allow supervisors to access the new
performance measures by September 30, 2008
28
Section 2: Assessment of BARC’s Performance To evaluate BARC’s performance, we compared available BARC data to industry benchmarks as identified in U.S. Substance Abuse and Mental Health Services Administration (SAMHSA) publications and other substance abuse treatment literature. Specifically, our objective was to answer the following questions about BARC’s performance:
• How effective are BARC’s substance abuse treatment services? • Are BARC’s inpatient facilities utilized to capacity? • How do BARC’s inpatient service costs compare to a national benchmark?
Generally, we found that BARC has implemented a treatment model that contains many industry recommended practices. However, we could not evaluate the effectiveness of BARC’s services due to a lack of reliable client outcome data available to us by the conclusion of our fieldwork in March 2008. Additionally, we found that BARC’s inpatient facilities operate at a higher cost than the national average reported in a recent study, which appears to be caused by two factors: low bed utilization and high staffing levels. How effective are BARC’s substance abuse services? BARC has implemented many recommended practices of substance abuse treatment, but due to data reliability problems we could not evaluate client outcomes.
To assess the effectiveness of its services, we compared elements of BARC’s substance abuse treatment model with substance abuse treatment literature. We found that BARC has adopted many recommended practices that are likely to improve the efficacy of its treatment services. Specifically, we found that BARC offers:
• A continuum of treatment services. Substance abuse literature indicates that the treatment needs of substance abusers change as their treatment progresses. For example, heavy or habitual substance users often require detoxification services prior to beginning counseling. Subsequent to detoxification, substance abusers may need intensive residential treatment to ensure that they receive a supportive environment of counseling and educational services that promotes a continued substance free lifestyle. Upon completion of residential treatment, clients may continue to receive periodic counseling services on an outpatient basis. By offering the full continuum of detoxification, residential and outpatient services to clients, BARC increases the likelihood that clients continue treatment and achieve successful outcomes.
• A sufficient duration of treatment. The National Institute on Drug Abuse (NIDA) suggests that longer treatment durations positively impact client outcomes. Specifically, NIDA indicates that substance abuse treatment should
29
generally last for at least 90 days. While clients can voluntarily leave their treatment program at any time, BARC’s continuum of care does promote treatment durations greater than 90 days. For example, BARC’s outpatient treatment program generally has a six month duration (180 days).
• Culturally sensitive services. As cultural and social variables impact treatment outcomes, substance abuse literature advocates for treatment services that are sensitive to an individual’s values, such as religion and language. In response, BARC has created several treatment units dedicated to serving Broward County’s cultural diversity. For example, BARC provides outpatient counseling services in Spanish through its “Hispanic Track” program.
• Gender sensitive services. While a majority of its clients are male, BARC has dedicated resources to ensure that the special needs of female clients are considered during treatment. Specifically, BARC has residential and outpatient programs geared to women and expecting mothers. Research indicates that females generally begin abusing substances later in life than men and are more likely to enter treatment in the earlier stages of abuse. Consequently, treatment tailored to these gender differences increases the likelihood of client success.
• Prenatal Treatment Services. According to the American Pregnancy Association, consuming alcohol or illegal drugs during pregnancy increases the risk for miscarriage, low-birth weight, and fetal death. Additionally, it is estimated that the medical and social services cost of caring for a child seriously impaired by a mother’s substance abuse is $750,000 from birth to the 18th birthday. As such, BARC has created specialized residential and outpatient treatment tracks for pregnant women to increase the likelihood of substance free births.
• Transitional housing. As BARC reports that many individuals receiving its services are initially homeless, a transitional housing program has been implemented to provide a supportive environment that promotes sobriety. Specifically, BARC contracts with several community housing providers who provide temporary housing to clients while they are active in treatment. Transitional housing funding is funded via a grant from the U.S. Department of Housing and Urban Development.
• Joint Commission accredited services. The Joint Commission is an independent, non-profit organization that focuses on improving the quality and safety of care provided by health service providers. As such, the Joint Commission offers health care accreditation and related services to support the provision of high quality healthcare services. To attain Joint Commission accreditation, an organization must undergo a comprehensive evaluation demonstrating its compliance with established best practices and standards. To maintain accreditation, an agency must undergo an on-site reaccreditation survey at least every three years. BARC first attained Joint Commission accreditation in 1999 and successfully completed its last follow-up survey in October 2005.
30
Due to data reliability problems, we could not evaluate BARC’s client outcomes As previously described on page 24, BARC is required to report performance data to the Florida Department of Children and Families (DCF) on a monthly basis. Specifically, BARC is required to report client outcome data for two performance measures. These measures are: • Percent of discharged adults successfully completing treatment (no alcohol or
other drug use during the month prior to discharge) • Percent of adults employed upon treatment discharge
We intended to use the client outcome data reported by BARC to DCF to evaluate the effectiveness of BARC’s services. However, as described on page 24, BARC officials disclosed that data reported to DCF has been historically incomplete. Additionally, throughout our fieldwork BARC did not provide us with client outcome data from its CMIS (ECHO). Therefore, we could not evaluate BARC’s effectiveness in this review.17 Accurately measuring service effectiveness is a critical component of program evaluation. Without this data, BARC cannot establish performance baselines. In an era of budget cuts, it is imperative for BARC to consistently collect and report reliable effectiveness data to allow managers and policymakers to assess the impact of program modifications and budget cuts on service quality. As recommended on page 25, BARC should take immediate steps to collect and report accurate effectiveness data. Are BARC’s inpatient facilities utilized to capacity? BARC’s inpatient facilities are relatively underutilized; further analysis is necessary to ensure that a proposed $21.3 million facility expansion is financially prudent
According to healthcare services literature, maximizing bed utilization is critical to providing cost effective inpatient services.18 Typically, a facility’s “bed utilization rate” is measured by dividing the number of client occupied beds by the total number of beds available. For example, a 10 bed facility with nine beds occupied by clients has a 90% bed utilization rate. A relatively high bed utilization rate may indicate facility overcrowding, which could negatively impact the quality of treatment services provided. Conversely, an agency that maintains a relatively low bed utilization rate may be expending greater resources than necessary to meet community need. In both instances, cost effectiveness is negatively impacted.
17 While additional client outcome measures exist in substance abuse treatment literature, we found that BARC primarily relies on these two measures as the primary indicators of treatment effectiveness and thus has collected limited other data that could be used to evaluate program performance. 18 “Inpatient” services means the client is admitted to a facility for a period of at least 24 hours or longer. BARC provides two types of inpatient service, detoxification and residential treatment.
31
BARC provides inpatient services at two facilities. Specifically, detoxification is provided at “BARC Central” in Fort Lauderdale, while residential treatment is provided at the “Stephen R. Booher Building” in Coral Springs. According to Fiscal Year 2007 BARC data, the average client receiving detoxification spends eight days at BARC Central, while a client receiving residential treatment generally spends 28 days at the Booher Building.19 We used Fiscal Year 2007 data to analyze the bed utilization rates at BARC Central and the Booher Building.20 As a comparative bed utilization benchmark, we used data published in October 2007 by the U.S. Substance Abuse and Mental Health Services Administration (SAMHSA). Specifically, in its annual National Survey of Substance Abuse Treatment Services, SAMHSA reported an average bed utilization rate of 91% for inpatient substance abuse treatment facilities.21 BARC Central– 35 Beds for Detoxification In Fiscal Year 2007, BARC provided inpatient detoxification services to an average of 28 clients per day at BARC Central.22 This equates to a bed utilization rate of 80%, which is 12.1% below the national average. To achieve the national average of 91%, BARC needs to increase its average daily client count at BARC Central to 32. Exhibit 16 below compares the bed utilization rate of BARC Central to the national average. Exhibit 16: BARC’s detoxification bed utilization rate of 80% is below the national average (91%) for similar inpatient facilities
50%
60%
70%
80%
90%
100%
BARC Detoxification National Average
Bed
Util
izat
ion
Rat
e
Source: Office of the County Auditor Analysis of Fiscal Year 2007 BARC data
19 Pregnant women may stay in residential treatment until their babies are delivered and a 90-day residential treatment program is available to women with children 7 years of age and younger. 20 We used Fiscal Year 2007 data provided by BARC from its ECHO system because Division officials indicated that manually collected data has historically been prone to error. 21 The bed utilization benchmark of 91% is based upon the reported performance of 3,213 substance abuse treatment providers in the United States. 22 BARC managers report that client census counts at inpatient facilities are taken daily at 12AM, in accordance with guidelines established by the Florida Department of Children and Families.
32
Stephen R. Booher Building – 92 Beds for Residential Treatment In Fiscal Year 2007, BARC provided residential treatment services to an average of 62 clients per day at the Booher Building. This equates to a bed utilization rate of 67%, which is 26.4% below the national average. To achieve the national average of 91%, BARC needs to increase its average daily client count at the Booher Building to 84. Exhibit 17 below compares the bed utilization rate of BARC Central to the national average. Exhibit 17: BARC’s residential treatment bed utilization rate of 67% is below the national average (91%) for similar inpatient facilities
50%
60%
70%
80%
90%
100%
BARC Residential National Average
Bed
Util
izat
ion
Rat
e
Source: Office of the County Auditor Analysis of Fiscal Year 2007 BARC data BARC should analyze whether replacing BARC Central at an estimated cost of $21.3 million is financially prudent In Fiscal Year 2004, the Board of County Commissioners approved capital funding for the construction of a 50 bed detoxification facility to replace BARC Central. As proposed, this facility would be co-located with a new Sexual Assault Treatment Center operated by the Broward County Children Services Administration Division. In a January 2008 memo, the Human Services Department estimated construction costs of $30.5 million for the proposed facility.23 A majority of the construction cost ($21.3 million, or 70%) is budgeted for BARC’s portion of the facility. Additionally, it is estimated that expanding the BARC’s detoxification program from 35 to 50 beds will increase annual operating costs by $1.3 million. We believe BARC should analyze whether replacing BARC Central is financially prudent. As demonstrated above, BARC data shows that detoxification service demand is not exceeding the facility’s current capacity. Furthermore, capital funding for a new facility was approved in Fiscal Year 2004, presumably based on overstated client count data, as previously discussed on pages 21-22. Office of Management & Budget officials state that the analysis conducted in May 2002 to determine the need for facility expansion in “was not retained given the amount of time that has elapsed.” 23 Construction for the new facility has been significantly delayed due to an eminent domain lawsuit associated with the targeted property site for the new facility.
33
In an era of significant budget cuts, the cost of creating additional (and seemingly unnecessary) capacity through substantial capital investment requires further justification. Specifically, BARC should demonstrate why increased capacity is necessary and that the construction of the new facility will provide a long-term cost savings to the County (as opposed to continuing to maintain and repair BARC Central). This analysis should include the estimated increase in recurring operating costs ($1.3 million) that is associated with the expansion of the detoxification program. Recommendation 6: In light of BARC Central’s apparently low bed utilization, we recommend that the Board of County Commissioners direct the County Administrator to take the following action to ensure that an expansion of the detoxification program is financially prudent: • Provide detailed explanation of the need for a new facility by September 30,
2008; defer further action on the new facility until an evaluation of facility utilization and costs is made that justifies the capital investment and increased operating costs of the proposed new facility and expansion of the detoxification program.
How do BARC’s inpatient service costs compare to national benchmarks? BARC’s inpatient treatment programs operate at a significantly higher cost than comparable facilities in the United States
As discussed on pages 14-16 of this report, County agencies should collect unit cost data to assess the cost of services provided. We found that BARC has not historically collected unit cost data. However, in February 2008 the Division conducted a unit cost analysis of its primary services and provided us data for review. We reviewed the methodology used by BARC to determine its unit cost for several services and found it to be fairly reasonable.24 We used BARC’s unit cost data to compare its service costs to benchmarks found in a recent national study. Specifically, we compared the cost per client per day of BARC’s detoxification and residential treatment services to national averages published in the University of Pennsylvania’s “Economic Benefits of Drug Treatment: A Critical Review of the Evidence for Policymakers.” We chose this study as the source for our comparative benchmarks for two reasons. First, BARC cited the report’s conclusions multiple times in analysis provided to the OMB and the Board of County Commissioners during the development of the Fiscal Year 2008 budget. Additionally, the study’s conclusions are based on a comprehensive review of 126 economic evaluations of
24 While BARC’s unit cost methodology captures its base operating cost of providing services, it does not capture the County’s full cost of providing services because it excludes capital budgeted expenditures, such as facility depreciation, etc.
34
substance abuse treatment programs in the United States. Presumably, the large number of evaluations included in the study should limit the effect of regional cost differences in the calculation of a national average. BARC’s detoxification cost per client is 83.2% higher than the national average BARC’s detoxification services are provided at BARC Central, which is a 35 bed facility. As of February 2008, BARC has assigned 44 staff to its detoxification unit, including a medical doctor and nursing staff. In addition to personnel expenses, detoxification service cost is affected by the price of prescription drugs, which are provided to clients to lessen the uncomfortable symptoms associated with substance withdrawal. The “Economic Benefits of Drug Treatment” study indicates that the average cost for detoxification is $250 per client per day. Data provided by BARC shows that its detoxification service cost per client per day is $458, which is 83.2% more than the national study average. Improving detoxification bed utilization would reduce BARC’s cost per client by 10.2% We analyzed how increasing BARC’s bed utilization rate for detoxification services would affect its unit cost. We project that if BARC achieves the national average of a 91% bed utilization rate (it currently operates at 80%), its unit cost would decrease to $411 per client per day, which is a 10.2% reduction.25 However, even at this improved level of performance, BARC’s cost would remain 64.4% higher than the national average. Exhibit 18 below shows the results of our comparison of BARC’s detoxification cost to the national average, and the projected impact of increasing bed utilization. Exhibit 18: BARC’s detoxification service cost per client is significantly higher than the national average even if bed utilization improves from 80% to 91%
Detoxification
$411$458
$250
$100$200$300$400$500$600
National Average BARC (at 80%) BARC (at 91%)Cos
t Per
Clie
nt P
er D
ay
Source: Office of the County Auditor Analysis
25 This analysis assumes that BARC’s maintains its Fiscal Year 2007 level of expenditures while increasing the number of clients served.
35
BARC’s residential treatment cost is 153.4% higher than the national average BARC operates a 92-bed, short-term inpatient program called “residential treatment.” Clients are provided counseling, substance abuse and nutrition education and family support during their 28 days at the Stephen R. Booher Building in Coral Springs. As of February 2008, BARC has assigned 57 staff to its residential treatment unit, which includes nursing, counseling and kitchen staff. Our review of the “Economic Benefits of Drug Treatment” study indicates that the national average cost for residential treatment is $103 per client per day. Data provided by BARC shows that its residential treatment service cost per client per day is $261, which is 153.4% more than the national average. Improving residential treatment bed utilization would reduce BARC’s cost per client by 25.3% We analyzed how increasing BARC’s bed utilization rate for residential treatment services would affect its unit cost. We project that if BARC achieved the national average of a 91% bed utilization rate (it currently operates at 67%), its unit cost would decrease to $195 per client per day, which is a 25.3% reduction.26 However, even at this improved level of performance, BARC’s cost would remain 89.3% higher than the national average. Exhibit 19 below shows the results of our comparison of BARC’s residential treatment cost to the national average, and the projected impact of increasing bed utilization. Exhibit 19: BARC’s residential treatment cost per client is significantly higher than the national average even if bed utilization improves from 67% to 91%
Residential Treatment
$195$261
$103
$0
$100
$200
$300
$400
National Average BARC (at 67%) BARC (at 91%)Cos
t Per
Clie
nt P
er D
ay
Source: Office of the County Auditor Analysis
26 This analysis assumes that BARC maintains its Fiscal Year 2007 level of expenditures while increasing the number of clients served.
36
To meet the national average, BARC must reduce detoxification and residential treatment expenditures by $4.7 million in Fiscal Year 2009 Our analysis indicates that improving bed utilization rates at BARC Central and the Booher Building will help to reduce service costs per client. However, to meet the national average for detoxification and residential treatment costs, BARC must further reduce its expenditures for these services. As 69.3% ($13.2 million) of BARC’s operating budget is appropriated to employee salaries and benefits, it is unlikely that BARC can significantly reduce costs without reducing the number of FTE positions allocated to detoxification and residential treatment. Other alternatives that may help to reduce service costs include position reclassifications. For example, BARC may be able to employ Licensed Practical Nurses rather than Registered Nurses, who typically command higher salaries due to their advanced training. Exhibit 20 and Exhibit 21 below show the level of expenditure reductions that need to be made to achieve national cost averages for detoxification and residential treatment. Exhibit 20: BARC must reduce detoxification expenditures by $1.9 million (39.2%) to meet the national average Detox
(Actual ) Detox
(Projected) %
DifferenceExpenditures $4,780,791 $2,906,312 -39.2 Bed Utilization 80% 91% 13.8 $/Client/Day $458 $250 -45.4
Source: Office of the County Auditor Analysis Exhibit 21: BARC must reduce residential treatment expenditures by $2.8 million (47.2%) to meet the national average Residential
(Actual ) Residential(Projected)
% Difference
Expenditures $5,963,006 $3,147,453 -47.2 Bed Utilization 67% 91% 35.8 $/Client/Day $261 $103 -60.5
Source: Office of the County Auditor Analysis BARC should benchmark its residential treatment service cost with Miami-Dade County’s residential treatment program as part of further analysis After the conclusion of our fieldwork, BARC expressed concern that the national study average of $103 was not representative of local Level 2 residential treatment costs.27 While the national study reflects the average cost of residential treatment programs
27 Level 2 residential treatment is defined by the Florida Department of Children and Families as providing “a range of assessment, treatment, rehabilitation, and ancillary services in a less intensive therapeutic environment with an emphasis on rehabilitation, and may include formal school and adult educational programs.” Level 2 residential facilities are required to provide a minimum of 10 hours of counseling per client per week.
37
around the country, BARC felt that its Level 2 residential program provides a higher level of service and thus should be compared with a similar program. To address this concern, we believe BARC should benchmark its service costs with Miami-Dade County’s residential treatment program. Miami-Dade County’s residential program appears to be a relatively good benchmark to BARC’s residential program because (1) it is a County operated program within the same geographic region, (2) it provides comparable Level 2 residential treatment services and (3) it operates a similar number of beds as BARC (99 and 92 beds, respectively). We contacted Miami-Dade County officials to determine their actual cost per bed day of providing Level 2 residential treatment. Miami-Dade officials indicated that this data was not readily available, but offered to meet with BARC officials to ensure that an appropriate cost comparison could be made. However, a comparison of BARC’s staffing levels to Miami-Dade’s indicates BARC’s staffing levels could be high. Specifically, Miami-Dade’s 99 bed facility is staffed by 30 FTE, a ratio of 3.3 beds per FTE. In comparison, BARC’s 92 bed facility is staffed by 57 FTE, a ratio of 1.6 beds per FTE. While we did not extensively compare employee classifications between the two programs, we did find that BARC’s residential treatment program is staffed with 15.5 nurse FTEs, while Miami-Dade County reported that it does not have nurses on-site at its residential facility. Additionally, Miami-Dade’s reported residential bed utilization rate of 98% is significantly higher than BARC’s residential bed utilization rate of 67%. As demonstrated previously, lower bed utilization rates can negatively impact service costs. Exhibit 22 below compares salient aspects of BARC’s residential treatment program to Miami-Dade’s. Exhibit 22: BARC’s Level 2 residential program is staffed significantly higher than a comparable program operated by Miami-Dade County BARC
Residential Miami-Dade Residential
# Beds 92 99 # FTE 57 30 Beds per FTE 1.6 3.3 Bed Utilization 67% 98%
Source: Office of the County Auditor BARC should comprehensively review its detoxification and residential treatment programs to ensure the provision of cost effective services Our comparison indicates that BARC should take steps to reduce its detoxification and residential program costs. Primarily, BARC should conduct additional comparative analysis with similar programs and conduct a staffing review to ensure that substance abuse services are effectively provided at the lowest cost to the taxpayer. As previously described on page 30, collecting and analyzing reliable effectiveness data, such as client
38
outcomes, is an integral component of this analysis because BARC should strive to reduce costs without negatively impacting service quality. Recommendation 7: To reduce the cost of providing substance abuse treatment services, we recommend that the Board of County Commissioners direct the County Administrator to take the following actions: • Conduct further comparative analysis with similar detoxification and residential
programs, including Miami-Dade County’s residential program, to analyze service cost, bed utilization, staffing ratios/composition and develop a plan to reduce treatment costs in accordance with results; report the results of this analysis to the Board of County Commissioners by September 30, 2008
39
Section 3: Assessment of BARC Program Alternatives In this section, we present alternatives to BARC’s existing service delivery model that could potentially decrease annual operating expenses. We believe evaluating service delivery alternatives, such as the level of care currently provided by BARC or the potential impacts of contracting with private providers for the provision of some or all substance abuse services, is appropriate for three primary reasons: • BARC’s inpatient service costs per client are significantly higher than national
averages, and reducing these costs would allow the County to serve the same number of clients while decreasing general fund expenditures.
• BARC could potentially avoid the significant capital investment of $21.3 million for the construction of a new detoxification facility by contracting with a private provider for detoxification services.
• Unlike Broward County, most other large Florida counties contract with private
providers for the provision of substance abuse treatment services. However, we also explored ways that BARC could reduce its general fund dependence if policymakers and managers decide to retain BARC’s direct service delivery model. Specifically, we focused on two ways BARC could increase program revenues. We found that BARC may be able to: • Participate in Medicaid reimbursement programs. • Increase service fee revenues by improving collection rate.
Alternatives to BARC’s Direct Service Delivery Model According to the U.S. Substance Abuse and Mental Health Services Administration (SAMHSA), 93.4% of the 688 substance abuse treatment facilities in Florida are operated by private entities; only 3.4% (23 facilities) are operated by local governments. As of Fiscal Year 2008, Broward County (via BARC) operates four of these facilities. In 1999, a consultant hired to evaluate substance abuse treatment delivery in Broward County cited a “national trend to reduce the role of government in direct service.” The consultant stated that “a key (industry) concern is to eliminate the government as a competitor” to private providers. Based on national best practices, the consultant recommended that Broward County’s Human Services Department “initiate planning for a transition from a provider of substance abuse services to an administrative entity charged with the responsibility and authority to coordinate all substance abuse treatment activity in Broward County, including the letting of grants and contracts for services.”28
28 Bright, Preston. “Substance Abuse Service Delivery System Broward County,” 1999.
40
Most large Florida counties exclusively contract with private providers for the provision of substance abuse treatment services We surveyed the six most populous Florida counties (excluding Broward) to determine the services they fund, and their service delivery models. Five of the six counties surveyed do not provide any direct substance abuse treatment services to residents. Only Miami-Dade County’s residential treatment program, which was previously discussed on page 37, is directly operated by county staff. Our survey also indicated that some counties do not fund the full range of treatment programs that Broward County funds. Exhibits 23 and 24 show the results of our survey. Exhibit 23: Most large Florida counties do not directly provide detoxification treatment services County Method of service delivery
(Detoxification) BROWARD Direct service provider Jacksonville/Duval Contracts with vendor Hillsborough Does not fund Miami-Dade Contracts with hospital district Orange Does not fund Palm Beach Contracts with two vendors Pinellas Contracts with vendor
Source: Office of the County Auditor Survey Exhibit 24: Broward County and Miami Dade County are the only two large Florida counties that directly provide residential treatment services County Method of service delivery
(Residential Treatment) BROWARD Direct service provider Jacksonville/Duval Contracts with two vendors Hillsborough Contracts with three vendors Miami-Dade Direct service provider Orange Contracts with vendor Palm Beach Contracts with two vendors Pinellas Contracts with vendor
Source: Office of the County Auditor Survey Further evaluation of BARC’s service delivery model should consider elements of the various models used throughout Florida As shown above, most large Florida counties have contracted with private providers for the provision of substance abuse treatment services. This indicates that efficiencies may be available to the private sector that cannot be obtained in the public sector. However, as BARC explores opportunities to reduce service costs, it should also consider differences in these counties’ treatment models. For example, our survey of other large Florida counties indicates that:
41
• Hillsborough and Orange counties do not fund detoxification services, rather these counties rely on local public hospitals and other community providers to provide this service.
• Miami-Dade County contracts with a local hospital district for detoxification
services after County officials determined that the cost of directly operating a detoxification unit was prohibitive.29
• Miami-Dade and Orange counties do not fund outpatient services.30
• Orange County funds less intensive, less costly residential treatment than
Broward County. For example, in accordance with the DCF model for Level 2 treatment, BARC’s residential program requires a minimum of 10 hours of counseling per client per week. However, Orange County funds Level 4 treatment, which requires a minimum of two hours of counseling per client per week.
BARC’s lack of client outcome data impeded our ability to compare treatment effectiveness across the various service delivery models. Specifically, as Level 2 residential treatment services are more intensive (and thus more costly) than Level 3 or 4 services, we would expect to see better client outcomes. Therefore, once it has reliable client outcome data, BARC should analyze whether an alternative service delivery model, such as using contracted providers for some services or varying the level of residential treatment intensity, would improve cost effectiveness. Recommendation 8: In an era of budget cuts, it is important for County policymakers to consider opportunities for cost savings through service delivery alternatives. To assist the Board in making important policy decisions regarding the continuation BARC’s current service delivery model, we recommend that the Board of County Commissioners direct the County Administrator to take the following action: • Determine the potential effects of implementing various service delivery models
used in other large Florida counties, such as contracting with community providers for the provision of treatment services, providing less intensive residential treatment to clients and not funding the existing level of treatment service; this analysis should include, at a minimum, a comparison of service cost and quality (effectiveness); provide the results of this analysis to the Board of County Commissions within 60 days of the adoption of this recommendation
29 Miami-Dade County reported having a $900,000 lump-sum contract with Jackson Memorial Hospital for detoxification services. 30 This does not include outpatient services that may be provided within the corrections system through drug court programs.
42
Medicaid Reimbursement According to the U.S. Substance Abuse and Mental Health Services Administration (SAMHSA), Medicaid is commonly used by clients as a source of payment for substance abuse treatment services.31 Specifically, SAMHSA research indicates that 19.8% of females and 11.6% of males receiving treatment services in the United States pay at least a portion of their treatment costs via Medicaid. We found that BARC has historically collected a negligible amount of Medicaid revenues. While the Division did not have recent data indicating the percentage of its clients that were Medicaid eligible, we noted that BARC has historically projected $150,000 of annual Medicaid revenues in its operating budget. Thus, if its projections are reasonably accurate, BARC appears to be foregoing significant Medicaid revenues that could be used to decrease its dependence on the County’s general fund.32 Exhibit 25 below compares BARC’s annual budgeted Medicaid revenues to actual collected Medicaid revenues since Fiscal Year 2005. Exhibit 25: BARC has not historically met Medicaid revenue budget projections
Fiscal Year
Medicaid Revenues
(Budgeted)
Medicaid Revenues (Actual)
2005 $150,000 $1,182 2006 $150,000 $0 2007 $150,000 $0
Source: Office of Budget & Management We requested that BARC officials provide us with a summary of why Medicaid revenues have not met budget projections. In response, BARC officials indicated that technical issues with the State of Florida’s electronic Medicaid database “present obstacles which require assistance from outside the Division. Technical support has been limited.” We contacted the Florida Agency for Health Care Administration (AHCA) to determine ways that BARC could increase its Medicaid revenues. AHCA indicated that BARC could explore participation in the Federal Medicaid Local Match initiative. The Local Match initiative provides federal matching funds to Florida counties that allocate general fund monies toward certain Medicaid eligible treatment services. At the current match rate, if BARC provided $150,000 of eligible treatment services to Medicaid clients in a fiscal year (as its annual budget projection indicates it does), the County could potentially draw down $214,875 in federal matching funds. Thus, participating in the Local Match 31 Pursuant to Title XIX of the Social Security Act, Medicaid provides financial assistance for medical services to eligible persons and families with low incomes. The program is jointly funded and administered by the federal and state governments. The Florida Department of Children and Families (DCF) determines Medicaid client eligibility within the State of Florida. 32 Broward County’s general fund is the funding source for county agencies that are not self-supporting, such as libraries, parks, mass transit, and the Constitutional Officers. The primary source of funding for the general fund is property tax revenues (61%).
43
initiative could provide a significant increase in Medicaid revenues in Fiscal Year 2009, decreasing BARC’s general fund dependence. In April 2008, BARC officials indicated that they were exploring participation in the Medicaid Match program. Recommendation 9: To ensure that Medicaid revenues are maximized, we recommend that the Board of County Commissioners direct the County Administrator to take the following actions:
• Determine the additional revenue that could be generated by participating in Medicaid reimbursement programs, such as the Local Match initiative, and develop an action plan by September 30, 2008 to ensure Medicaid revenues are maximized each fiscal year
Service Fee Collections In accordance with Florida Department of Children and Families requirements, BARC has implemented a sliding fee scale to ensure that each client is not overly burdened by the cost of treatment. For example, BARC’s Fiscal Year 2008 sliding fee scale indicates that a client with an annual income of $35,000 and a family size of one would be charged 50% of the regular fee for any service received. Accordingly, this client’s fee for one day of detoxification treatment would be $92.50, which is half the regular daily fee of $185. While Section 397.431(2), Florida Statutes, requires clients receiving treatment to contribute as much to the cost of services as they can, it is BARC’s responsibility to determine the appropriate fee for each client and pursue fee collection. It is important for BARC to diligently collect service fees to reduce the Division’s dependence on general fund revenues. Additionally, BARC officials told us that a concerted effort to pursue fee collections is necessary to “eliminate the perception throughout the community and the court system that services are free.” We reviewed BARC’s client service fee collection rate for the past two years and found that the Division has maintained a low collection rate. Specifically, BARC achieved a 3.97% collection rate in Fiscal Year 2006 and a 5.73% collection rate in Fiscal Year 2007. 33 Exhibit 26 below shows the amount of client service fees assessed and collected during Fiscal Years 2006 and 2007. Exhibit 26: BARC collects a small percentage of client service fees assessed Fiscal Year
Fees Assessed
Fees Uncollected
Fees Collected
Collection Rate
2006 $949,228 $911,518 $37,710 3.97% 2007 $903,822 $852,057 $51,765 5.73%
Sources: BARC and OMB
33 BARC’s “client service fee collection rate” is calculated by dividing the dollar value of client service fees collected by the dollar value of client service fees levied during the fiscal year.
44
We requested that BARC officials provide us with a summary of steps that could be taken to improve service fee collections. In response, BARC proposed several strategies, including the establishment of client proof of income requirements, discussion of fees and payment as part of client counseling and providing clients with a service bill upon discharge. Additionally, Division officials believe that inspiring a “culture change within the Division and the community at large” is necessary because “some staff view BARC as a free service provided to indigent and homeless individuals.” Recommendation 10: To ensure that client service fee revenues are maximized, we recommend that the Board of County Commissioners direct the County Administrator to take the following actions:
• Implement a corrective action plan to increase the collection rate of client service fees by September 30, 2008
45
Appendix A Office of the County Auditor’s Assessment of the Broward Addiction Recovery Division’s Accountability System Best Practice County Auditor’s Assessment Recommendation Establish a complete set of performance measures
BARC established a complete set of performance measures during Fiscal Year 2008 (see pages 13-14)
None
Collect and analyze unit cost data for major services and activities
BARC began collecting unit cost data during Fiscal Year 2008; current fees bear no apparent relationship to the cost of providing services (see pages 14-16)
Establish criteria defining the appropriate relationship between client service fees and BARC’s actual cost of providing services by September 30, 2008
Collect and analyze internal operational data
BARC relies on manually collected data despite its investment in CMIS technology, which impedes supervisors’ ability to monitor the efficiency and effectiveness of operations (see pages 16-18)
Discontinue the use of time-consuming manual data collection, where possible, by developing standard reports within existing CMIS by September 30, 2008
Contact outside sources to benchmark performance
BARC has historically conducted limited benchmarking analysis (see pages 19-20)
Identify appropriate peers and performance measures to conduct a benchmarking analysis by September 30, 2008
Establish a process to ensure performance data reliability
BARC has historically reported unreliable data to the Office of Management & Budget and DCF; several steps should be taken to ensure the collection and reporting of reliable data (see pages 20-25)
Develop data reliability controls, such as written measure definitions and supervisory review, to ensure that accurate data is collected and reported to stakeholders by September 30, 2008
Use performance data to improve results
BARC has established a performance improvement process to help focus resources; however, additional data should be collected to improve accountability and program effectiveness (see pages 25-27)
Develop a list of performance measures that are critical for monitoring program accountability and effectiveness; develop standard reports within CMIS by September 30, 2008
Source: Office of the County Auditor
46
Appendix B BARC’s Fiscal Year 2008 Performance Measures
Administrative/Support Services Section Performance Measures Work Output • Number of purchasing forms and credit card purchases processed Efficiency • Number of purchasing forms and credit card purchases processed per certified
agency buyer Effectiveness (Client Benefit)
• Percent of personnel records in compliance with all requirements
Strategic Outcome
• Percent of services screened for Medicaid eligibility per quarter • Percent of performance improvement indicators that meet or exceed target levels
Source: Office of Management & Budget
Outpatient Services Section Performance Measures Work Output • Number of consumers receiving bio-psychosocial assessment for treatment services
other than detoxification • Number of consumers admitted to outpatient program • Number of consumers admitted to specialty track outpatient programs • Number of consumers admitted to day treatment programs
Efficiency • Number of direct service and contact hours per clinical staff per week Effectiveness (Client Benefit)
• Consumer satisfaction rating (%)
Strategic Outcome
• Percent of staff spending 50% of available time providing direct services per quarter • Percent of consumers employed upon discharge • Percent of pregnant women attending perinatal addiction program giving birth to
substance free newborns • Percent of consumers that successfully complete treatment (no alcohol and other
drug use during the month prior to discharge) Source: Office of Management & Budget
Residential Services Section Performance Measures Work Output • Number of consumers receiving detoxification triage services
• Number of consumers admitted to detoxification program • Number of consumers admitted to intensive residential treatment program • Number of consumers admitted to mature adult residential treatment program • Number of consumers admitted to perinatal addiction residential treatment program
Efficiency • Number of direct service and contact hours per clinical staff per week Effectiveness (Client Benefit)
• Client satisfaction rating (%)
Strategic Outcome
• Percent of staff spending 50% of available time providing direct services per quarter • Percent of consumers successfully completing detoxification services • Percent of consumers that successfully complete residential treatment services (no
alcohol or other drug use during the month prior to discharge) • Percent of pregnant women attending perinatal addiction program giving birth to
substance free newborns Source: Office of Management & Budget
