American Institute of Certified Public Accountants Award to Dr. Peter Hughes as 2010 Outstanding CPA of the Year for Local Government GRC (Government, Risk & Compliance) Group 2010 Award to IAD as MVP in Risk Management 2009 Association of Certified Fraud Examiners’ Hubbard Award to Dr. Peter Hughes for the Most Outstanding Article of the Year – Ethics Pays 2008 Association of Local Government Auditors’ Bronze Website Award 2005 Institute of Internal Auditors’ Award to IAD for Recognition of Commitment to Professional Excellence, Quality, and Outreach RISK BASED AUDITING GAO & IIA Peer Review Compliant – 2001, 2004, 2007, 2010, 2013 Internal Audit Department CONTROL SELF-ASSESSMENT & PROCESS IMPROVEMENT BY OC PUBLIC WORKS Four Workshops Facilitated by the Internal Audit Department between September and November 2013 REPORT NO: 1361 REPORT DATE: JUNE 18, 2014 Director: Dr. Peter Hughes, MBA, CPA, CIA Senior Audit Manager/Facilitator: Michael Goodwin, CPA, CIA Audit Manager/Facilitator: Michael Dean, CPA, CIA, CCSA Administrative Manager/Facilitator: Renee Aragon Staff Specialist/Facilitator: Maribel Garcia The Internal Audit Department facilitated four CSA-PI workshops in OC Public Works at the request of the former interim Department Director. The four workshops were conducted between September and November 2013. CSA-PI is not an audit; rather, it is a consulting service that assists County departments in conducting their own internal assessment of the effectiveness of their controls and business processes. OC Public Works’ new management team has been partnering with the work teams in taking action to address issues raised in the four workshops. OC Board of Supervisors ’ 1 st District – Janet Nguyen 2 nd District – John M.W. Moorlach 3 rd District – Todd Spitzer 4 th District – Shawn Nelson, Chairman 5 th District – Patricia C. Bates, Vice Chair O R A N G E C O U N T Y 6 t h Largest County in the USA Control Self-Assessment & Process Improvement (CSA-PI) is a management tool where work teams within a department or agency assess their own effectiveness in meeting select business and control objectives.
Transcript
American Institute of Certified Public Accountants Award to Dr. Peter Hughes as 2010 Outstanding CPA of the Year for Local Government
GRC (Government, Risk & Compliance) Group 2010 Award to IAD as MVP in Risk Management
2009 Association of Certified Fraud Examiners’ Hubbard Award to Dr. Peter Hughes for the Most Outstanding Article of the Year – Ethics Pays
2008 Association of Local Government Auditors’ Bronze Website Award
2005 Institute of Internal Auditors’ Award to IAD for Recognition of Commitment to Professional Excellence, Quality, and Outreach
The Internal Audit Department facilitated four CSA-PI workshops in OC Public Works at the request of the former interim Department Director. The four workshops were conducted between September and November 2013. CSA-PI is not an audit; rather, it is a consulting service that assists County departments in conducting their own internal assessment of the effectiveness of their controls and business processes. OC Public Works’ new management team has been partnering with the work teams in taking action to address issues raised in the four workshops.
O
C
Bo
ar
d
of
S
up
er
vi
so
rs
’
1st
Dis
tric
t – J
anet
Ngu
yen
2
nd D
istr
ict –
Joh
n M
.W.
Moo
rlach
3rd D
istr
ict –
Tod
d S
pitz
er
4th
Dis
tric
t – S
haw
n N
elso
n, C
hairm
an
5th D
istr
ict –
Pat
ricia
C. B
ates
, V
ice
Cha
ir
OR
AN
GE
C
OU
NT
Y
6t
h
La
rg
es
t
Co
un
ty
i
n
th
e
US
A
Control Self-Assessment & Process Improvement (CSA-PI) is a management tool where work teams within a department or agency assess their own effectiveness in meeting select business and control objectives.
12 Civic Center Plaza, Room 232 Santa Ana, CA 92701
Phone: (714) 834-5475 Fax: (714) 834-2880
To access and view audit reports or obtain additional information about the OC Internal Audit Department, visit our website: www.ocgov.com/audit
OC Fraud Hotline (714) 834-3608
Independence Objectivity Integrity
i
Control Self-Assessment & Process Improvement Report
Transmittal Letter
At the request of the former OC Public Works Interim Director, we have completed our facilitation of a Control Self-Assessment & Process Improvement (CSA-PI) by OC Public Works (OCPW). We conducted four CSA-PI workshops with OCPW employees, supervisors and managers between September and November 2013. The Control Self-Assessment & Process Improvement Report is attached. The CSA-PI process was approved by the Audit Oversight Committee on April 27, 2011, with the understanding that the impressions and perspectives captured in the workshop summaries would be retained exclusively by those elected officials who requested a CSA-PI. It was further understood that: 1) the Internal Audit Department would provide a draft workshop summary to the County Executive Office (CEO) for those departments and agencies reporting directly to the CEO; 2) the Internal Audit Department would follow up on any internal control weaknesses or issues identified in the workshop in the manner they deemed warranted; and 3) the Internal Audit Department would provide a summary notice of completion in the format and with the content contained in this Control Self-Assessment & Process Improvement Report. Each month I submit an Audit Status Report to the Board of Supervisors (BOS) where I detail any critical or significant audit findings released in reports during the prior month and the implementation status of audit recommendations as disclosed by our Follow-Up Audits. Although the CSA-PI process is not considered an audit, this report will be included in a future status report to the BOS. Attachments Other recipients of this report are listed on the Control Self-Assessment & Process Improvement Report on page 5.
Report No. 1361 June 18, 2014
TO: Shane Silsby, Director OC Public Works
FROM: Dr. Peter Hughes, CPA, Director Internal Audit Department
SUBJECT: Control Self-Assessment & Process Improvement by OC Public Works
Control Self-Assessment & Process Improvement Report Control Self-Assessment & Process Improvement by OC Public Works Report No. 1361 Four Workshops Conducted between September and November 2013
Transmittal Letter i
Control Self-Assessment & Process Improvement Report
Overview and Purpose of CSA-PI 1
Workshop Impressions 3
Results 3
CSA-PI Teams’ Evaluation 4
Some Team Comments 4
ATTACHMENT A: Control Self-Assessment/Process
Improvement Workshop Overview 6
Control Self Assessment & Process Improvement By OC Public Works Report No. 1361 Page 1
Control Self-Assessment & Process Improvement Report
CSA-PI Highlight The Internal Audit Department facilitated four Control Self-Assessment & Process Improvement workshops in OCPW. Each workshop was attended by managers, supervisors and staff from Operations & Maintenance, OC Engineering, OC Facilities and Administration. Each team assessed their department’s effectiveness in meeting its business and control objectives. CSA-PI is a consulting service to assist County departments assess the effectiveness of their business processes.
OVERVIEW AND PURPOSE OF CSA-PI As requested by the former OC Public Works Interim Director, we provided facilitation of four Control Self-Assessment & Process Improvement (CSA-PI – formerly referred to as CSA) workshops in OC Public Works (OCPW). The primary purpose of CSA-PI is to help teams assess their effectiveness in meeting select departmental business and control objectives. We conducted four workshops including employees from Operations & Maintenance, OC Engineering, OC Facilities and Administration. In order to ensure workshop participants are comfortable providing assessments that may be critical of existing processes and procedures, the Internal Audit Department uses anonymous voting technology to capture the participants’ perceptions regarding the effectiveness of resolving internal conflicts, promoting and supporting teamwork, establishing achievable goals, measuring performance, empowering employees, providing recognition, inviting employee comments and suggestions, communicating expectations, sharing financial and operational information, and training and developing staff. CSA-PI workshops have three fundamental purposes. First, the workshops provide the teams with a formal process for self-evaluation. This process addresses strengths and obstacles that impact the teams’ ability to achieve their objectives and to decide upon appropriate action. Second, the Internal Audit Department prepares a draft summary of results of the workshops that provides a basis for upper management to evaluate the control environment within their department. Last, the process provides a methodology for participants to apply these workshop principles and dynamics in the workplace by identifying problems and obstacles, and developing action plans to address concerns and build upon strengths. Participants are encouraged to communicate openly, identify concerns, and collaborate in deciding on appropriate actions. The Internal Audit Department serves primarily as workshop facilitators and not auditors. This process is based on the consensus of impressions, comments, opinions, and perspectives of the workshop teams and is not “audited or validated” by the Internal Audit Department. Nevertheless, we find that the workshop results provide useful insights regarding the status of OCPW’s control environment. Also, the Internal Audit Department will follow up on any internal control weaknesses or issues identified in the workshops in the manner they deem warranted.
Report No. 1361 June 18, 2014
TO: Shane Silsby, Director OC Public Works FROM: Dr. Peter Hughes, CPA, Director Internal Audit Department SUBJECT: Control Self-Assessment & Process Improvement By OC Public Works
Control Self Assessment & Process Improvement By OC Public Works Report No. 1361 Page 2
Control Self-Assessment & Process Improvement Report Control Self-Assessment & Process Improvement is a process where managers and employees of a unit led by trained facilitators (Internal Audit Department) come together to discuss the strengths and concerns impacting their ability to accomplish their objectives. Workshop participants, using anonymous voting technology, vote on a series of 51 statements in 11 categories of control elements (see Attachment A). The statements are used to assess business operations and help both the participants and management focus on common business objectives and controls. The categories presented to the group for voting and discussion are:
Values & Beliefs Information Team Environment & Responsibilities Skills & Knowledge Communication & Relations Rewards & Recognition Setting Team Objectives Control Activities Planning & Risk Assessment Monitoring, Learning & Managing Change Resources
The teams are asked to vote anonymously on whether they agree or disagree with the 51 statements. The voting is on a scale of 1 through 7, with 1 corresponding to “disagree strongly” and 7 corresponding to “agree strongly.” For example, one statement under the category “Skills & Knowledge” reads: “I am able to provide the quality of work that is necessary to meet our objectives.” After the participants voted on this statement, graphic results were instantly displayed showing the results similar to the sample graph below:
After viewing the results of their voting, comments are shared and recorded relating to strengths, concerns, and possible actions to be taken to address the issues raised in the statements. The impressions and perceptions reflected in the statements are used by management to stimulate further discussions with the objective of revising, enhancing or improving controls, processes and managerial issues identified.
Control Self Assessment & Process Improvement By OC Public Works Report No. 1361 Page 3
Control Self-Assessment & Process Improvement Report
WORKSHOP IMPRESSIONS CSA-PI workshop results are based on the consensus of impressions, comments, opinions, and perspectives of the workshop teams. The benefit of the workshop is to get issues out so management and the team participants can evaluate, research, clarify, and ultimately verify the observations expressed. Management responds by working with employees and developing concrete corrective actions that the workshop team and management determine have validity to pursue. CSA-PI is a multi-step process: First, the Internal Audit Department acts as facilitators to flush out strengths and concerns from the participants in meeting department objectives. Second, management and the teams evaluate the workshop results. Third, management and the team develop concrete corrective actions in an action plan. Management implements the action plan and ensures they are provided to workshop participants and to all employees. Last, the Internal Audit Department will pursue any control concerns that come out of a CSA-PI workshop, either through a separate audit, or discussions with management as considered warranted.
RESULTS After completion of the workshops, each team began working with OCPW management to prepare an internal Action Plan addressing the higher-priority issues brought forward during the workshops. Action Plans were received from Operations & Maintenance and OC Facilities. A new Department Director was appointed to OCPW as we were conducting the workshops. We met with the new Director and Executive Management team, discussed the progress made by OCPW, and recognized they had addressed the more significant issues. Due to the change in Executive Management, OCPW will complete Action Plans for OC Engineering and Administration next fiscal year. The following were the key strengths and concerns identified in the workshops:
Strengths & Successes: 1. Skills, Knowledge and Experience. OCPW staff and management possess the necessary
education, skills, and knowledge to accomplish their objectives. Several staff have more than 20 years of County and/or public works experience.
2. Strong Teamwork. OCPW has effective teamwork, collaboration and communication within the divisions (Operations & Maintenance, OC Facilities, OC Engineering and Administration). Employees help each other in their respective divisions to achieve their objectives.
3. Dedicated Staff Who Enjoy Their Work and Meet Objectives. OCPW staff are dedicated to their jobs, enjoy their work, and meet their objectives despite the many obstacles they face. Several staff expressed passion for providing high-quality customer service.
Concerns & Challenges:
1. Staffing Levels. OCPW staff expressed concerns regarding lack of adequate staffing levels. Employees were concerned that the large number of vacancies were impacting their service levels and also impacting the amount of preventative maintenance being performed throughout the County.
2. Leadership. Participants were concerned regarding frequent management changes in the department. Morale has been impacted by the events surrounding a former high-ranking employee. Some staff were concerned that leadership had not established a clear mission, vision, goals and objectives.
3. Resources. Staff expressed concern regarding other resources. Losing the ability to take home vehicles has resulted in a loss of productivity. In addition, participants cited the inability to make quick purchases when out in the field which negatively impacts efficiency. Team members also felt that field technology could be better utilized via mobile devices.
Control Self Assessment & Process Improvement By OC Public Works Report No. 1361 Page 4
Control Self-Assessment & Process Improvement Report
There were no issues identified during the four workshops that warranted the Internal Audit Department pursuing any further action regarding control concerns. We noted that OCPW Executive Management is taking the CSA-PI process seriously and is taking immediate action to address certain issues raised in the workshops.
The draft summaries of the raw and unverified comments, impressions and voting results were provided to OCPW. As such we limited distribution strictly to OCPW for use as a management tool in facilitating their internal assessment. However, because the OCPW Director is appointed and reports to the CEO, we discussed the OCPW workshops with the CEO/Chief Operating Officer (COO). The COO had read the four workshop summaries and was enthusiastic about the CSA process. CSA-PI TEAMS’ EVALUATION At the conclusion of the workshops, facilitators ask teams to evaluate the CSA-PI process by anonymously voting on five standard quality assurance statements. The participants provided high ratings for the CSA-PI process and their individual workshops as indicated by their voting. The average scores from the 59 participants evaluating the CSA-PI process are shown below. The participants vote by selecting a numerical rating from a scale of 1 through 7. A score of 7 indicates the participants strongly agree with the statement. A score of 4 indicates they neither agree nor disagree. A score of 1 indicates the participants strongly disagree with the statement. See the table below of Evaluation Statements for the averaged score results for the three workshops.
Evaluation Statement
Top Score Possible
Average Score
Percentage
I understand the purpose of CSA-PI. 7.00 6.76 96.6%
The workshop helped to highlight issues significant to this team. 7.00 6.64 94.9%
I was able to express my opinions freely. 7.00 6.71 95.9%
The Internal Audit Department facilitators did a good job. 7.00 6.88 98.3%
I would recommend this process to other teams. 7.00 6.86 98.0%
SOME TEAM COMMENTS “I believe this was a very instrumental and positive exercise and will help us in the future.” “I like that it promoted a lot of constructive discussion amongst the group.”
“The workshop identified many things the department could improve on.”
“It was interesting to see the common themes. It will help us create a good action plan.”
“I liked the voting system. It was very helpful to see our responses graphically.”
“The workshop was very effective…we had the freedom to speak freely about important
issues.”
I liked the ability to find and offer recommendations.”
Control Self Assessment & Process Improvement By OC Public Works Report No. 1361 Page 5
Control Self-Assessment & Process Improvement Report
ACKNOWLEDGMENT We appreciate the courtesy and cooperation extended to us by the personnel at OC Public Works. If you have any questions regarding the CSA-PI process, please contact me directly at 834-5475, or Mike Goodwin, Senior Audit Manager at 834-6066. Distribution Pursuant to Audit Oversight Committee Procedure No. 1:
Members, Board of Supervisors Members, Audit Oversight Committee Michael B. Giancola, County Executive Officer Mark Denny, Chief Operating Officer Robin Uptegraff, Assistant Director of OC Public Works Phillip Cook, Deputy Director, Facilities Design & Construction, OCPW Mary Fitzgerald, Deputy Director, Administrative Services, OCPW Kevin Onuma, Deputy Director, Infrastructure Programs, OCPW Khalid Bazmi, Deputy Director, Operations & Maintenance, OCPW
Foreperson, Grand Jury Susan Novak, Clerk of the Board of Supervisors Macias Gini & Co. LLP, County External Auditor
Control Self Assessment & Process Improvement By OC Public Works Report No. 1361 Page 6
Control Self-Assessment & Process Improvement Report ATTACHMENT A: Control Self-Assessment/Process Improvement Workshop Overview
Control Self Assessment & Process Improvement By OC Public Works Report No. 1361 Page 7
Control Self-Assessment & Process Improvement Report ATTACHMENT A: Control Self-Assessment/Process Improvement Workshop Overview (continued)
Control Self Assessment & Process Improvement By OC Public Works Report No. 1361 Page 8
Control Self-Assessment & Process Improvement Report ATTACHMENT A: Control Self-Assessment/Process Improvement Workshop Overview (continued)
Control Self Assessment & Process Improvement By OC Public Works Report No. 1361 Page 9
Control Self-Assessment & Process Improvement Report ATTACHMENT A: Control Self-Assessment/Process Improvement Workshop Overview (continued)
Control Self Assessment & Process Improvement By OC Public Works Report No. 1361 Page 10
Control Self-Assessment & Process Improvement Report ATTACHMENT A: Control Self-Assessment/Process Improvement Workshop Overview (continued)
Control Self Assessment & Process Improvement By OC Public Works Report No. 1361 Page 11
Control Self-Assessment & Process Improvement Report ATTACHMENT A: Control Self-Assessment/Process Improvement Workshop Overview (continued)
