Performance Review
Small Business Development Program within the Office
of Economic and Small Business Development Office
May 9, 2011 Report No. 11-08
Office of the County Auditor Evan A. Lukic, CPA
County Auditor
Executive Summary
This report presents the results of our performance review of the Small Business Development
Program (Program) within the Office of Economic and Small Business Development (OESBD). The
purpose of this review is to
1) Assess the quality of the Program’s performance measurement and reporting system,
2) Evaluate the Program’s performance and identify options for improving services and reducing
operating costs, and
3) Describe the status of OESBD’s implementation of recommendations made in Office of the
County Auditor Report No. 08-16, dated July 10, 2008, Report on Tasks Performed at the
Request of the County Attorney’s Office Regarding Broward County’s Disadvantaged Business
Enterprise Program.
Quality of Program’s Accountability System
We identified deficiencies with the Program’s performance measurement and reporting system and
benchmarking efforts. Specifically, we found that the Program has
Established appropriate performance measures but needs to collect more meaningful data to
better evaluate program results and manage operations,
Reported inaccurate data prior to FY 2009 relating to staff workload and productivity, major
activity costs, and outcomes due to flawed collection methods and deficiencies with previous
databases,
Deficient internal controls that raise questions about the reliability of output, efficiency and
outcome data reported for fiscal years 2009 and 2010, Recently taken steps to obtain better quality information, and Taken initial steps to benchmark performance but additional work is needed
Evaluating Program Performance
Due to unreliable performance data and incomplete benchmarking mentioned above, we were unable
to evaluate the Program’s operational efficiency and effectiveness. For example, we do not know
whether the Program achieves its goal to increase the number of certified small businesses, if Program
staff works as productively as possible or if their efforts have the desired results. Consequently, at this
time, we cannot credibly determine whether the Program could take actions to enhance its impact or
decrease its operating costs.
Status of Implementation of Prior Audit Recommendations
OESBD provided us with sufficient documentation indicating it has fully implemented recommendations
made in County Auditor Report No. 08-16.
1
Recommendations
To correct deficiencies with the Program’s accountability system, we recommend the Board of County
Commissioners direct the County Administrator to take the following actions
1. Research and establish additional efficiency and impact measures, develop a data collection
mechanism, and report results to the Board by September 30, 2011
2. Implement appropriate internal controls including written measure definitions, supervisory
review, and periodic validation of reported data; correct existing data reliability problems with
ALTS and Small Business certification directory; and determine the feasibility of obtaining
actual, rather than estimated dollars awarded to certified vendors. A report of the Program’s
progress in obtaining reliable performance data should be provided to the Board by September
30, 2011.
3. Continue efforts to survey outreach event participants and gather staff productivity data, and
report results to the Board by September 30, 2011
4. Continue efforts to conduct benchmarking of identified peers; specifically the Program should
(1) identify valuable performance measures for benchmarking purposes, (2) collect and
analyze comparative data from peer agencies, and (3) use the analyses to identify potential
areas for improvement. A report of the Program’s progress in benchmarking its performance
should be provided to the Board by September 30, 2011.
Purpose and Scope
This report presents the results of our performance review of the Small Business Development
Program (Program) within the Office of Economic and Small Business Development (OESBD). The
purpose of this review is to (1) assess the quality of the Program’s performance measurement and
reporting system, (2) evaluate the Program’s performance and identify options for improving services
and reducing operating costs, and (3) describe the status of implementation of recommendations made
by OESBD in response to Office of the County Auditor Report No. 08-16. To accomplish our
objectives, we
Reviewed pertinent performance measurement, general management, and small business and
economic development literature;
Reviewed websites of 11 other local governments’ small business, women-owned, and minority-
owned business development agencies and programs;
Analyzed available performance data for Program operations;
Analyzed information provided by Program managers and staff relative to specific management
processes and controls;
Reviewed applicable Program records and documents, including relevant policies and
procedures;
Reviewed applicable Broward County Ordinances and Administrative Code pertaining to small
business and economic development; and
Interviewed OESBD managers and staff.
2
Background
The mission of the Office of Economic and Small Business Development (OESBD) is to “stimulate
economic development by attracting, retaining and expanding targeted industries, including a special
focus on small business growth, and thereby, enhance the quality of life for Broward County residents
and visitors.”
OESBD administers four distinct programs designed to encourage small business growth: Small
Business Enterprise (SBE); County Business Enterprise (CBE); the U.S. Department of Transportation
(USDOT) Disadvantaged Business Enterprise (DBE); and the USDOT Airport Concession
Disadvantaged Business Enterprise (ACDBE) program. OESBD is the County’s designated office for
overseeing compliance with federal regulations that concern DBE programs under Title 49 of the Code
of Federal Regulations.
Small Business Enterprise (SBE)
In the Business Opportunity Act of 2004 (Ordinance No. 2004-07), the Board of County Commissioners
directed projects estimated to cost less than $250,000 to be identified as sheltered market/reserved
contracts for certified small businesses.
Through this race and gender neutral program, certified small business enterprise (SBE) firms receive
first consideration for contracts under $250,000. The program’s goal is to increase the number of small
businesses acting as prime contractors or consultants for the County.
To be eligible for the program, the business must meet the following requirements:
For professional consultants, annual gross sales averaged over the previous three years should
not exceed $500,000
For firms in contractual services and commodities, annual gross sales averaged over the
previous three years should not exceed $1 million
For firms in construction, annual gross sales averaged over the previous three years should not
exceed $3 million
Each business must be independently owned and operated with 25 or fewer permanent full-time
employees
The vendor must have a Broward Business Tax Receipt (previously titled Occupational License)
and be located in and be doing business in Broward County
The business must be established for a period of one year prior to submitting its application, and
must serve a commercially useful purpose
County Business Enterprise (CBE)
The CBE program was created by the County Business Enterprise Act of 2009 (Ordinance No. 2009-
40), which repealed the earlier Community Disadvantaged Business Enterprise (CDBE) program. The
CBE program’s intent is to “increase participation by small businesses in County projects as both prime
contractors and subcontractors.”
3
To qualify as a CBE, businesses must meet the following criteria:
Each owner must have a personal net worth not exceeding $750,000 (excluding their primary
residence and business)
Has an office in Broward County, meaning that it has a current business tax receipt issued by
Broward County, a physical business address (not a post office box) located within Broward
County in an area zoned for the conduct of such business, and has sufficient full-time
employees in Broward County to perform the contracted work
Is an independent firm
The applicant may not exceed the average annual maximum gross receipts standard of $5
million, calculated over the previous three years
Disadvantaged Business Enterprise (DBE)
The Federal Disadvantaged Business Enterprise (DBE) program provides opportunities for small businesses that are “both socially and economically disadvantaged” to bid on contracts involving
federally funded U.S. Department of Transportation projects, including airport and mass transit projects.
According to federal guidelines (49 CFR Part 26), a DBE is a business that is at least 51% owned and
controlled by an individual who is socially and economically disadvantaged, and who has a personal net
worth, excluding the owner’s business and personal residence, of less than $750,000. Women and
members of certain race and ethnic groups are presumed to be socially and economically
disadvantaged.
Airport Concession Disadvantaged Business Enterprise Program (ACDBE)
OESBD also has certification authority and goal setting for the Airport Concession Disadvantaged
Business Enterprise Program (ACDBE) under 49 CFR 23. It is a separate program from Part 26.
Small Business Development Program Sections
OESBD has three major functions related to these four programs: Certification, Compliance, and
Business Assistance. Certification and Compliance work primarily with the Small Business Program,
certifying firms and ensuring compliance with local and federal regulations. Business Assistance is
intended to provide client-focused services to businesses including but not limited to opportunities
through the County’s procurement program, technical assistance with issues faced by entrepreneurial
and well-established businesses, and facilitation of workshops, seminars, and outreach.
Certification Section
This Section processes applications submitted by local small businesses in order to bid on Broward
County procurement contracts with SBE, DBE and CBE program requirements. Section staff reviews
applications to ensure SBE, DBE or CBE program eligibility requirements are met. Staff then verifies
each applicant through additional research and site visits.
Compliance Section
This Section ensures that certified firms and County agencies comply with pertinent federal and local
regulations, policies and procedures. Major Section activities include: conducting contract compliance
4
reviews, processing complaints made against certified firms and/or County agencies, and conducting
Sheltered Market concurrence reviews.
Business Assistance Section
This Section assists businesses with varying needs and at different stages of development to increase
state and local contracting opportunities. The Section was formed in January 2010, serves both the
Small Business and Economic Development units of the office, and includes staff with specialties in
Small Business and Economic Development. Major Section activities include:
Business assessments and corporate visitations Seminars, workshops and forums Technical assistance to individual businesses Goal Setting and Associated Reviews for CBE and DBE programs Supporting Major Office Studies (targeted industries, disparity study) Disaster preparedness and recovery planning
OESBD Resources
For FY 2011, OESBD had a total operating budget of $2,570,800 with 24 full-time equivalent (FTE)
positions.1 The FY 2011 operating budget for Small Business Development Program was $1,201,160.
For FY 2011, Business Assistance, Certification and Compliance sections each have 3 FTE positions.
Findings and Recommendations
This report presents the results of our performance review of the Small Business Development
Program (Program) within the Office of Economic and Small Business Development (OESBD). The
purpose of this review is to
1) Assess the quality of the Program’s performance measurement and reporting system,
2) Evaluate the Program’s performance and identify options for improving services and reducing
operating costs, and
3) Describe the status of OESBD’s implementation of recommendations made in Office of the County Auditor Report No. 08-16, dated July 10, 2008, Report on Tasks Performed at the
Request of the County Attorney’s Office Regarding Broward County’s Disadvantaged Business
Enterprise Program.
Part 1: Quality of OESBD’s !ccountability System
According to performance measurement literature, there are three primary purposes for collecting and
reporting performance data:
To allow citizens to hold elected officials, managers, employees and private contractors accountable for efficiently and effectively spending public funds,
1 In September 2010, a new OESBD Assistant Director will oversee Small Business Development activities
5
To help elected officials and other policymakers to make informed budget and policy decisions,
and
To enable managers to detect and correct operational deficiencies and improve program results
Our review concluded that the Program lacks sufficient meaningful and reliable information to
effectively evaluate program results and manage operations, but has taken steps to obtain better
quality information.
Finding 1
The Program has appropriate performance measures but needs to collect more
meaningful data to better evaluate results and manage operations
The Office of Management and Budget (OMB) requires County agencies to collect and report
information relative to four types of performance measures: work output, efficiency, client benefit/
effectiveness, and strategic outcome.2
The Small Business Development Program has established a set of performance measures that meets
OMB requirements. As shown in Exhibit 1 below, the Program’s set of performance measures includes
output, client benefit, efficiency, and strategic outcome measures, as required by OMB.
Exhibit 1
The Program’s FY 2010 performance measures meet OMB requirements
Type of Measure Performance Measure
Number of business service activities conducted
WORK OUTPUTS Number of compliance reviews conducted
Total number of certifications, all programs
EFFICIENCY Number of compliance reviews per employee
CLIENT BENEFIT/EFFECTIVENESS External customer satisfaction rating
STRATEGIC OUTCOME Dollars awarded to certified firms, all programs
Source: Office of Economic and Small Business Development
We also found that the Program’s set of performance measures are similar to those collected and
reported by other local governments. In our review of 11 other local governments’ performance
measurement systems, most of them collected and reported measures that are similar to the measures
the Program reports quarterly to OMB. For example, most local governments we reviewed collect
comparable work output, efficiency and outcome data, and most of them also conduct customer
satisfaction surveys.
However, some other governments have additional efficiency and impact measures that the Program
has not routinely collected and reported. Several other governments collect and report other efficiency
2 Work output focuses on the quantity of service provided; efficiency measures the ratio of output per input; client
benefit/effectiveness assesses the quality of the service from the clients’ perspective; and strategic outcome states the consequences of the program in a “big picture” sense.
6
and impact measures that provide more meaningful information for policymakers and managers to
evaluate program efficiency and effectiveness. For example, unit cost data3 can be used to assess
how changes in operations could affect costs, identify wasteful processes, and estimate the impact of
budget documents. Additionally, impact measures can be used to assess the direct or indirect effects
or consequences resulting from achieving program goals. Exhibit 2 below shows additional efficiency
and impact measures reported by other governments.
Exhibit 2
Additional measures collected and reported by other governments
Type of Measure Performance Measure
Average certification time, in days
Cost per certification
EFFICIENCY Cost per compliance review
Cost per dollar of CBE/DBE/SBE transaction
Cost per outreach event
IMPACT
New jobs created/retained as a result of contract awards
Cumulative economic impact
Source: Office of Economic and Small Business Development
We believe these measures provide meaningful information that would enable a more complete
assessment of the Program’s operational efficiency and effectiveness. Therefore, as part of this
review, we requested that the Program provide us with data for these measures for the past five fiscal
years. In response to our request, the Program provided five years worth of data for most of these
measures. However, our analysis of this data revealed data reliability problems that will be
subsequently discussed in this report.
Recommendation
To provide policymakers and Program managers with meaningful information to help ensure services
are provided as efficiently and effectively as possible, we recommend that the Board of County
Commissioners direct the County Administrator to take the following actions:
1. Research and establish additional efficiency and impact measures, develop a data collection
mechanism, and report results to the Board by September 30, 2011
Finding 2
The Program reported inaccurate data prior to FY 2009 due to flawed data collection and
recordkeeping methods, impeding valid trend analyses
3 Unit cost refers to the cost of producing an output or outcome.
7
Trend analysis is a commonly used technique in descriptive statistics. The term “trend analysis”
generally refers to the concept of collecting information over a period of time to discern patterns, which
are instructive of past performance and may be indicative of future performance. In program
evaluation, trend analysis is typically used to monitor rates, such as unemployment, infant mortality and
crime, over a relatively longer period of time (e.g., ten or more years). Trends in observed rates
provide information that can be helpful to policymakers and managers in conducting activities such as
needs assessment, program planning, and policy development.
To analyze Program trends, we reviewed data for the measures identified in Exhibits 1 and 2 as well as
other pertinent performance indicators for the five-year period from FY 2006 to FY 2010. This data
would enable us to determine trends in staff workload and productivity, major activity costs, and
program outcomes.
However, we could not perform valid trend analyses because Program managers acknowledged during
our fieldwork that data prior to FY 2009 for most of these measures was not reliable. Specifically,
Program managers disclosed the following data reliability problems:
For the measures dealing with Business Assistance Section activities (e.g., number of outreach
activities and cost per outreach activity), Program managers and staff stated they did not
consistently compile information on the number and types of outreach activities they conducted.
They said that beginning in FY 2009 they made a concerted effort to more uniformly gather this
information, thus they concluded the newer data “is deemed to be more accurate than the
information that was previously reported.”
For the measures dealing with Certification Section activities (e.g., number of applications
received, number of certifications, average certification time in days, and cost per certification),
Program managers and staff stated that data from the MS SharePoint database, which was
used prior to FY 2009, is not reliable due to data entry control issues. They said that FY 2009
and FY 2010 data is more accurate because they had fully transitioned to the new ALTS4
database by the start of FY 2009.
Finding 3
Deficient internal controls raise questions about the reliability of data reported for fiscal
years 2009 and 2010
According to the U.S. Government Accountability Office (GAO), performance data should be
reasonably timely, complete and accurate to be useful for budget allocation and policymaking
purposes. To ensure reliable performance data, government agencies should implement internal
controls, such as establishing clear and unambiguous performance measure definitions, having front
line and middle managers independently review performance data, and testing a sample of
performance data at least quarterly.
To ascertain the reliability of the Program’s newer reported data, we (1) analyzed source records
contained in two primary systems: ALTS and Small Business certification directory; (2) reviewed
4 Activity Log Tracking System (ALTS) is a SQL database
8
certified vendor award data found in Advantage and PMIS; and (3) interviewed Program managers and
staff to identify existing internal controls relating to data reliability.
We found data reliability problems with reported FY 2009 and FY 2010 performance data due primarily
to internal control weaknesses. Specifically, we found:
Inaccurate output and efficiency data from ALTS.
o Number of applications. To validate the accuracy and completeness of data contained
in ALTS, we reviewed all electronic records maintained in the system as of January 6,
2011. We identified missing and incorrect data for the measure “number of applications
received” for fiscal years 2009 and 2010. In analyzing 3,099 records, we found 766
(24.7%) had blank values in the “Date Received” and/or “Date Completed” fields;
another eight records had inaccurate dates (e.g., values that went beyond the date of
the database such as 2013 and 2019). These deficiencies make it probable that
reported FY 2009 and 2010 data is unreliable. A primary reason for this problem
appears to be a lack of automated controls to prevent data entry errors.
o Average certification time. Program managers and staff told us that because ALTS is
“flawed” they needed to do “work-arounds” to extract data for the measure “average
certification time, in days.” They said that the 30-day reported figure for each of those
fiscal years was a “reasonable estimate” of the time it takes for an applicant to submit
required information and for staff to complete the certification process rather than the
actual time involved. Using source data obtained from OESBD we tried to replicate
results; but found 898 records of 3,100 total certifications, or 29%, contained blank or
incorrect date values in either the “Date Received” or “Date Completed” fields. Thus, we
could not determine the actual “average certification time, in days” due to data reliability
problems, apparently caused by a lack of data entry controls.
Inaccurate certification data from the Small Business (SB) certification directory.
o Number of certified firms. We found similar reliability problems with data contained in
the SB certification directory for the measures “number of new firms certified each year”
and “total number of firms certified as of September 30 each year.” For example, in
reviewing 5,828 vendor records in the SB certification directory, we found 125 records
(2.1%) had blank or invalid certification start/end dates (e.g., certification start date in the
year 3009, certification end date in the year 2109, etc.) In addition, 3,573 vendors
(61.3%) had expired certifications. These apparent system errors impede the Program’s
ability to report reliable vendor certification data. These errors can be prevented and
detected through proper internal controls, including built-in mechanisms for identifying
data entry errors.
Invalid strategic outcome data.
o Dollars awarded to certified firms. Data reported quarterly to OMB for the measure
“dollars awarded to certified firms, all programs” is an estimated dollar value not an
actual amount. Program managers and staff stated that reported figures are a
“calculation” derived from two sources: (1) an Advantage report of contract awards to
certified prime vendors, and (2) a manual compilation of all goals committed to in
9
contracts awarded to non-certified prime vendors. The sum of the two amounts is used
to report dollars awarded to certified firms, based on the assumption that non-certified
vendors meet 100% of their goal commitments. This method of reporting is an
estimated not actual result because some prime vendors fail to attain goals while others
exceed goals.
Recommendation
To ensure the Program collects and reports reliable data that is essential for effective accountability
and program management purposes, the Board of County Commissioners should direct the County
Administrator to take the following actions:
2. Implement appropriate internal controls including written measure definitions, supervisory
review, and periodic validation of reported data; correct existing data reliability problems with
ALTS and SB certification directory; and determine the feasibility of obtaining actual, rather than
estimated dollars awarded to certified vendors. A report of the Program’s progress in obtaining
reliable performance data should be provided to the Board by September 30, 2011.
Finding 4
The Program has recently taken steps to obtain better quality information to enhance
outreach efforts and optimize staff productivity
In FY 2011, Program managers undertook two initiatives aimed at obtaining information that will help
them better manage operations:
Customer Satisfaction Survey. The Program has conducted a customer satisfaction survey to
meet OMB requirements for the past several years. However, Program managers thought that
tool was too general to help them determine the value of the Outreach program. They said they
needed a new survey tool that would help them identify the efficacy of specific outreach
activities and strategies. Consequently, in March 2011 OESBD tested a new survey instrument
on an actual group of Business Assistance event attendees. The new survey instrument, which
is composed of 10 basic questions, is designed to “inform, review, and direct future
enhancements of outreach efforts.” Program managers also plan to test an on-line survey that
could be “more efficiently administered” to participants.
Time and Motion Study. Also in FY 2011, Program managers began collecting and evaluating a
“formalized set of time and motion study data” that they hope will enable them to understand
how time and staff resources are being allocated and to “address areas in need of increased
efficiency and capacity.” Program managers plan to conduct individual coaching meetings with
staff to discuss areas of possible productivity improvement and to have follow-up sessions
within 30 days to reevaluate progress. According to Program managers, this effort should result
in staff being “optimally productive.”
10
Recommendation
To ensure Program managers have quality information to help them better manage operations, we
recommend the Board of County Commissioners direct the County Administrator to take the following
actions:
3. Continue efforts to survey Business Assistance event participants and gather staff productivity
data, and report results to the Board by September 30, 2011
Finding 5
The Program has taken initial steps to benchmark performance but more effort is
needed
The Government Finance Officers Association recommends benchmarking as a method of obtaining
information that can be used to identify inefficient practices and develop strategies to achieve better
results. Benchmarking refers to “performance comparisons of organizational business processes
against an internal or external standard of recognized leaders.” In order to ensure optimal performance
in its operations, Program managers should systematically and routinely compare its performance
against those of peer agencies that provide similar services.
To assess the quality of the Program’s benchmarking efforts, we requested that Program managers
provide us with benchmarking and any other comparative analyses of its performance against
comparable agencies conducted in the past several years.
Based on our review of the information provided, we conclude the Program has not yet performed
sufficient benchmarking or comparative analyses. However, in FY 2011, Program managers laid the
groundwork to facilitate benchmarking activities. Specifically, in November and December 2010, the
Program
Identified 11 local economic development entities as peer agencies.5 To determine its peers,
the Program considered the following factors: organizational size; scope and structure of
operations; median household income; median housing prices. In addition, the Program
identified southern U.S. counties that “consistently win projects recognized for their capital
investment, the impact on their local economy, and creation of jobs.”
Conducted initial data gathering, including (1) identifying primary contact information for each
peer agency, and (2) documenting each peer agency’s public/private funding and staffing levels.
These preliminary activities appear to be reasonable and appropriate. The next steps would involve
obtaining comparative performance data to assess relative efficiency and effectiveness. Appropriate
efficiency measures may include cost per outreach event and cost per certification; outcome measures
5 Peer agencies identified by OESBD are: Dallas County, TX; Wayne County, MI; King County, WA; Santa Clara County, CA;
Philadelphia County, PA; Sacramento County, CA; Cuyahoga County, OH; Hillsborough County, FL; Orange County, FL; Fairfax County, VA; and Mecklenburg County, NC
11
may include Return-on-Investment, percentage of all contract awards made to certified vendors, and
new jobs created/retained as a result of contract awards to certified vendors.
Recommendation
To obtain comparative information about peer agencies that can help Program managers assess
relative efficiency and effectiveness and make necessary changes to improve results, we recommend
that the Board of County Commissioners direct the County Administrator to take the following actions:
4. Continue efforts to conduct benchmarking of identified peers; specifically the Program should
(1) identify valuable performance measures for benchmarking purposes, (2) collect and
analyze comparative data from peer agencies, and (3) use the analyses to identify potential
areas for improvement. A report of the Program’s progress in benchmarking its performance
should be provided to the Board by September 30, 2011.
Part 2: Evaluating Program Performance
Finding 6
Due to unreliable data and incomplete benchmarking, we could not evaluate the
Program’s operational efficiency and effectiveness
Prior findings in this report disclosed that the Program has not collected reliable data or conducted
sufficient benchmarking to enable us to perform a meaningful evaluation of its operational efficiency
and effectiveness. Specifically, we found that the Program
Reported inaccurate output and efficiency data prior to FY 2009 due to flawed collection
methods and deficiencies with previous databases;
Reported inaccurate output, efficiency and outcome data reported for fiscal years 2009 and
2010 due to deficient internal controls; and
Conducted limited benchmarking and comparative analyses
Without good data, we do not know whether the Program meets its goal of increasing participation of
certified small businesses or if program staff is working as productively as possible or if their efforts
have an impact in certifying small businesses. In addition, without a good evaluation, we cannot
credibly determine whether there are actions the Program can take to achieve better results or
decrease its operating costs.
Part 3: OESBD’s Implementation of County Auditor Recommendations
Finding 7
OESBD has satisfactorily implemented prior County Auditor recommendations relative
to the DBE program
Office of the County Auditor Report No. 08-16, Report on Tasks Performed at the Request of the
County Attorney’s Office Regarding Broward County’s Disadvantaged Business Enterprise Program,
dated July 10, 2008, recommended that the Board of County Commissioners direct the County
Administrator to
12
Establish adequate controls to ensure the accurate and complete tracking and reporting of all
projects subject to DBE participation goals and attainment levels.
Take immediate action to submit the Uniform Report that was due to the FTA June 1, 2008.
Take steps to ensure all future Uniform Reports are submitted by required due dates.
Develop, with the assistance from the County Attorney’s Office, comprehensive operating procedures to ensure that the DBE program is consistently administered in compliance with
federal guidelines to include:
o Sufficiently detailed procedures to guide staff in all aspects of program operations (e.g.,
goal setting and program reporting),
o Mandatory training of staff in the application of federal program requirements,
o Adequate supervisory review and oversight of program activities and staff, and
o Minimum documentation requirements and retention standards
As part of this review, we requested that OESBD managers provide us with documentation showing the
status of implementation of recommendations made by OESBD in response to Office of the County
Auditor Report No. 08-16.
In reviewing OESBD’s response and supporting documentation, and interviewing OESBD managers,
we conclude that OESBD provided sufficient documentation indicating it has fully implemented
recommendations made in Report No. 08-16.
13