City of Calgary QAR Report - Final June 2010

City of CalgaryInternal Audit Quality Assurance ReviewJune 2010 ReportStrictly Private and Confidential

This report is confidential and intended solely for the information and use of the Audit Committee of The City of Calgary and is not to be used,referred to or distributed to others for any purpose. PwC disclaims any contractual or other responsibility to others based on its use and, accordingly,this information may not be relied upon by anyone other than The City of Calgary.

Contents

Executive Summary 1

Underlying Themes 3

Conformance with the IIA Standards 8

CAO Current State Assessment 12

Appendices 37

PricewaterhouseCoopers LLP

PricewaterhouseCoopers LLP 1

Executive Summary

Project Scope & Objectives

• The City of Calgary (“City”) Audit Committee (“AC”) engagedPricewaterhouseCoopers LLP (“PwC”, “we”) to perform a Quality Assurance Review(“QAR”, “review”) of the City Auditor’s Office (“CAO”) with the following objectives:

- Assess compliance with the Institute of Internal Auditors (“IIA”) Standards;and

- Compare the CAO to leading internal audit (“IA”) practices in public andprivate sector organizations to highlight strengths and identify areas forimprovement in seven core areas (summarized in the “Core IA Focus Areas”diagram on the right).

Approach

• Our review took place during May 2010 and consisted of:

- Interviews with 19 of individuals including:

• Members of the AC (select aldermen and citizen members);

• Senior members of administration including recent auditees; and

• CAO leadership in addition to current and former staff members.

- Review of the CAO charter, current and prior year CAO audit plans, aselection of summary reports to the AC, a sample of full audit reports andsupporting working paper files.

Summary Findings

• We found compliance deficiencies with the majority of IIA standards (see table to theright – “CAO Compliance with IIA Standards”).

• There are significant opportunities for improvement to bring the CAO in line withleading IA practices in public and private sector organizations in the seven coreareas considered.

Core IA Focus Areas

Organization

HumanResources

WorkingPractices

InformationTechnology

Communication &Reporting

KnowledgeManagement

QualityAssurance

Description

Generally

Conforms

Partially

Conforms

Does not

conform

Purpose X

Independence X

Proficiency X

Quality X

Value-add X

Risk focus X

Planning X

Field w ork X

Reporting X

Follow -up X

Risk escalation X

2

Executive Summary - Continued

Underlying Themes

• Throughout the report, we have identified short and long term recommendations to improve CAO conformance with IIA standardsand adopt leading IA practices.

- In our view, these recommendations result from and are grounded in four themes that must be addressed before moretactical opportunities for improvement can be considered.

• Mandate interpretation and expectations of the CAO function must be aligned between the CAO, AC and seniormanagement;

• The annual audit plan should be a collaborative effort linked to the critical risk areas in the organization;

• Project execution and reporting timelines as well as report formats should be designed to meet stakeholder needs;and

• Processes and procedures are needed to attract, develop and retain the necessary skills to execute the risk-basedplan in a value-add manner.


Underlying Themes


4

Underlying Themes

• The following four themes underlie many of the concerns / opportunities for improvement noted in our report:

1. Mandate interpretation and expectations of the CAO function are not aligned between the CAO, AC and seniormanagement.

• The CAO views itself as an “Auditor General” function working directly in an oversight / “watchdog” capacity for citizensto uncover process exceptions and mismanagement.

• The AC and senior administration expect the CAO to provide value to the organization by assessing compliance withestablished processes and assisting administration in identifying and recommending timely fixes to gaps in known andemerging risk areas.

• This philosophical difference in perspectives results in significant friction between the CAO and its key stakeholders(the AC and administration).

- As a result, both administration and the AC are forced to engage third parties or find alternative methods tofocus on known and emerging risk areas leading to inefficiencies and higher than necessary all-in complianceand risk mitigation costs.

Recommendation

• We recommend a select group of AC members, with input from senior administration, work collaboratively with theCAO to establish the expected mandate of the CAO within the context of existing bylaws related to the function.


5

Underlying Themes - Continued2. The annual audit plan should be a collaborative effort linked to the critical risk areas in the organization.

• The City is a complex organization requiring a number of processes and controls to effectively mitigate a diverseuniverse of risks. As a result, administration has set up a risk management function to identify and monitor these risks.

• Currently, the CAO performs its own risk assessment and does not actively solicit input or use the administration’s riskassessment and management processes as a basis for its audit plan.

- This approach results in a “siloed” audit plan disconnected from the key risks of the organization, furtheraccentuating the perceived value gap between the CAO and its stakeholders.

• Furthermore, there is not an established framework in place to evaluate the impact of emerging orchanging risks on the plan throughout the year or assess new audit requests from stakeholders;

• This limits function effectiveness and increases overall risk to the organization as critical processes andcontrols may not be covered in the audit plan; and

• May also lead to inefficiencies (e.g. time spent by both CAO and administration resources) and waste iflow risk areas are audited.

Recommendations

• Develop a collaborative three year risk-based audit plan using the administration’s current risk management plan as astarting point. Year one should include a detailed plan with expected audit outcomes and preliminary timing. Year twoand three should include key focus areas.

- Solicit input from key stakeholders (AC members, senior management, and department heads in theadministration) throughout plan development; and

- You may need third party assistance to facilitate audit plan development given the existing relationship betweenthe CAO and the administration, and the previously discussed differences in mandate interpretation.


6

Underlying Themes - Continued3. Project execution and reporting timelines as well as report formats should be designed to meet stakeholder needs.

• Many projects span a number of months and in some cases years of elapsed time, significantly reducing the relevanceof recommendations as processes may have changed and issues may have been resolved between project initiationand reporting.

• In addition, reports presented to auditees often contain a large number of recommendations that are not prioritized orrisk ranked.

• In combination, these items cause the administration to dispute and / or dismiss many findings rather than focussing oncritical areas requiring attention / remediation.

- Leads to inefficiencies and wasted resources for both the CAO and the administration, and ultimately the ACwho may be required to mediate.

Recommendations

• Establish agreed-upon timelines for all projects prior to commencement.

- Timelines should be measured in weeks (e.g. 4 – 8 weeks from planning to initial reporting) consistent withmany IA functions rather than months or years; and

- Establish / re-design internal processes to enable the shortened timelines.

• Develop a value-add reporting template in consultation with stakeholders.

- We recommend the template contain a simple prioritization methodology (e.g. red, yellow, green, or high,medium, low) adopted by many IA functions to focus remediation efforts.

• Develop and execute a plan to prioritize and wrap-up all open audits; for those audits deemed to warrant reporting, usethe newly developed reporting template.


7

Underlying Themes - Continued

4. Processes and procedures are needed to attract, develop and retain the necessary skills to execute the risk-basedplan in a value-add manner.

• The CAO has experienced significant turnover at the staff-level in the recent past. The impact of this is accentuated bychallenges the department has experienced in recruiting qualified internal audit resources.

- As a result, existing CAO staff-level internal audit skills are limited and not conducive to adding value to theorganization; and

- In addition, we understand the current audit plan is temporarily on hold due to resource constraints.

• Current department practices relating to establishing career paths, objective setting, performance evaluation andfeedback, and training are not effective. Succession planning within the CAO is not formally considered.

- Staff do not view the CAO as a department in which to further their career aspirations or develop their skill sets.

• Even if a strong risk-based audit plan is developed as previously recommended, it would be very difficult to execute theplan without significant investment in improving department processes, training existing staff and recruiting new staff.

Recommendation

• Identify the skills needed to effectively execute the risk-based audit plan recommended previously and assess gapsbetween required skills and current staff expertise.

- Develop and execute a strategy to source required skills.

• May be a combination of internal resources (establish processes to attract, train and retain) and thirdparty subject matter specialists.


Conformance with theIIA Standards


PricewaterhouseCoopers LLP9

Conformance with the IIA StandardsOn the basis of our review, the City of Calgary’s CAO “Generally Conforms’ to two, “Partially Conforms” to four, and “Does Not Conform” tofive IIA Standards as summarized below:

StandardNumber

IIA StandardGenerallyConforms

PartiallyConforms

Does NotConform

1000The purpose, authority, and responsibility of the internal audit activity should be formally defined in a charter,consistent with the Standards, and approved by the board.

X

1100The internal audit activity should be independent, and internal auditors should be objective in performing theirwork.

X

1200 Engagements should be performed with proficiency and due professional care.X

1300

The chief audit executive should develop and maintain a quality assurance and improvement program thatcovers all aspects of the internal audit activity and continuously monitors its effectiveness. This programincludes periodic internal and external quality assessments and ongoing internal monitoring. Each part of theprogram should be designed to help the internal auditing activity add value and improve the organization’soperations and to provide assurance that the internal audit activity is in conformity with the Standards and theCode of Ethics.

X

2000The chief audit executive should effectively manage the internal audit activity to ensure it adds value to theorganization.

X

2100The internal audit activity should evaluate and contribute to the improvement of risk management, control, andgovernance processes using a systematic and disciplined approach.

X

2200Internal auditors should develop and record a plan for each engagement, including the scope, objectives,timing and resource allocations.

X

2300Internal auditors should identify, analyze, evaluate, and record sufficient information to achieve theengagement's objectives.

X

2400 Internal auditors should communicate the engagement results. X

2500The chief audit executive should establish and maintain a system to monitor the disposition of resultscommunicated to management.

X

2600

When the chief audit executive believes that senior management has accepted a level of residual risk that maybe unacceptable to the organization, the chief audit executive should discuss the matter with seniormanagement. If the decision regarding residual risk is not resolved, the chief audit executive and seniormanagement should report the matter to the board for resolution.

X


The IIA standards represent an expected level of performance from all internal audit departments. We believe there are severalchallenges that need to be addressed in order for the City Auditor’s Office to be in conformance with IIA Standards. The following are thekey points in relation to these challenges:

Requirements to be in Conformance with the IIA Standards

See pages 15-16and 18-20 for moreinformation on theseissues.

• Interviewees (AC and administration) indicated internal audit activities are not adding value to the organization based onthe focus of audit projects, the lack of timely project completion and reporting, and the nature / tone of communicationbetween administration and the CAO.

• The CAO’s interpretation of its mandate is not aligned with the Audit Committee’s or administration’s views.

• Staff development and human resource practices within the CAO are weak.

2000

(Does notconform)

See pages 35-36 formore information onthese issues.

• Quality assurance practices are inconsistent within the CAO. Individual audit files are typically reviewed as an auditclosing procedure (i.e. not during fieldwork), and in some cases performed following the issuance of reports.• Ongoing monitoring of compliance to IIA Standards and internal quality assessments are not performed.

• The CAO does not maintain effective mechanisms to drive continuous improvement.

• The CAO does not solicit feedback from auditees.

1300

(Does notconform)


• Several examples exist where audit completion has been significantly delayed (i.e. by months or even years).

• While staff turnover has impacted achieving timely completion of projects, inconsistencies in CAO processes andpractices have resulted in the inability to proficiently transition projects within the department.

• Significant project “rework” and duplication in execution efforts has transpired, which is not indicative of due professionalcare.

1200

(Does notconform)

IIAStandard

Key Challenges Reference

2100

(Does notconform)

• Audit plans are developed in isolation and do not link to critical risk areas identified by senior administration.

• Coordination is minimal to leverage risk-related information from the Integrated Risk Management group.

• CAO processes supporting a “systematic and disciplined approach” for the function have only recently been addressed,and have not been implemented or well communicated to CAO staff. This has resulted in inconsistent working practices.


2200

(Partiallyconforms)

The CAO generally performs planning procedures for each audit project. However, the CAO “partially conforms” to this IIAstandard for the following reasons:

• Project planning procedures are not consistently or effectively performed.

• Resource allocation is typically based on staff availability as opposed to project needs. This results in inadequateunderstanding and coverage of some audit areas.

• End-to-end audit project cycles typically extend well beyond planned timelines.

See pages 18-20and 22-24 for moreinformation on theseissues.


Requirements to be in Conformance with the IIA Standards -continued


• Reports are not issued to auditees in a timely manner, due to the elongated nature of audit projects. This diminishes thevalue and relevance of findings.

• The CAO does not employ a rating system to provide management with an indication of the severity / priority of findings.Reports typically contain a large number of non-prioritized findings.

• Management is given 30 days to respond to audit findings. This is generally viewed as being inappropriate by auditeesgiven the length of time taken to receive the report following project initiation and the number of items requiring attentionwithin the reports.

2400

(Does notconform)

See pages 22-24 and35-36 for moreinformation on theseissues.

CAO personnel typically analyze and evaluate information to support engagement objectives. However, the CAO “partiallyconforms” to this IIA standard for the following reasons:

• In the recent past, several examples exist whereby CAO personnel were required to circle back with clients to obtainadditional information following the completion of fieldwork, with audits being “re-opened”.

• This appears to be a by product of inconsistent practices with respect to obtaining, analyzing, and recording sufficientinformation to achieve engagement objectives, as well as non-timely quality assurance procedures (i.e. during an auditproject, as opposed to after the completion of fieldwork).

2300

(Partiallyconforms)


Spreadsheets and TeamMate are used to track whether audit recommendations have been addressed by management.However, the CAO “partially conforms” to this IIA standard for the following reasons:

• A formal process is not employed whereby individuals are assigned responsibility to regularly follow-up withmanagement regarding progress on implementing recommendations. Monitoring is undertaken on an ad-hoc basis whentime permits.

• The CAO’s current inventory of items for tracking totals over 500 dating back to 2000, bringing into question therelevance and timeliness of monitoring activities.

2500

(Partiallyconforms)

IIAStandard

Key Challenges Reference

2600

(Partiallyconforms)

CAO audit reports generally communicate to management a large number of risk-related observations. However, the CAO“partially conforms” to this IIA standard for the following reasons:

• Audit findings are not prioritized based on severity or risk. Thus, the appropriateness of residual risk acceptance bymanagement is not appropriately considered, as all audit findings are uniformly treated.

• The CAO’s “watchdog” approach results in an adversarial relationship with management, which is a barrier to engagingin constructive dialogue with respect to the nature of risks accepted.

See pages 15-16 and29-30 for moreinformation on theseissues.


CAO Current StateAssessment


Our Approach

For the current state assessment, we utilized the PricewaterhouseCoopers’ Internal Audit Framework (the “Framework”) and assessed thecurrent state of the CAO against this Framework. For this purpose, we undertook a comparative analysis of the strategy, processes, controlsand procedures at the CAO versus industry practices (considering other City Auditor functions across Canada and general internal auditdepartment trends and leading practices).

In reporting the results of our review, we provide analysis for each internal audit process category in our Framework (including Organization,Human Resources, Working Practices, Information Technology, Communication and Reporting, Knowledge Management, and QualityAssurance).

Our reporting on these internal audit processes includes the following:

• A description of the Framework component, i.e. objectives considered during our assessment;

• Observations and findings regarding the CAO and its practices derived through interviews and documentary review procedures; and

• Short-term and long-term recommendations for the CAO to improve/enhance its performance along with potential associatedbenefits.

Prior to conducting our analysis, it was important to ascertain the “value drivers” for the CAO’s key stakeholders (i.e. what stakeholders needfrom the CAO). On the basis of our overall observations and interviews, we found these to be:

• Foster an awareness of the City of Calgary’s risks and controls;

• Provide assurance on compliance within the organization, and evaluate the effectiveness of controls to mitigate key risks;

• Report to stakeholders in a transparent and concise manner, with key issues and topics being clearly highlighted;

• Balanced focus between the performance of independent reviews for the AC and administration, and providing consulting advice toadministration to improve the effectiveness of risk management, control and governance processes; and

• Operate with objectivity and serve as independent “eyes and ears” to the AC and administration on key risk areas.


Organization

OBJECTIVES CONSIDERED FOR OUR ASSESSMENT

In assessing the CAO’s organization and strategy, we considered the followinginternal audit objectives:

• Authority granted to the CAO by the Audit Committee is appropriate to its role andresponsibilities.

• Sponsorship provided by the Audit Committee and executive management supportsthe CAO in achieving its objectives.

• The CAO’s charter is aligned with stakeholder value drivers and corporate objectives.

• Organizational status is appropriate to assure the CAO’s independence.

• CAO leadership and staff maintain an appropriate level of objectivity in executingtheir responsibilities.

• The CAO’s structure reflects the needs and culture of the organization and isappropriate for accomplishing its objectives and stakeholder value drivers.

• Roles and responsibilities for all CAO positions are defined and understood within thedepartment and are aligned with stakeholder value drivers.

Organization

HumanResources

WorkingPractices



KnowledgeManagement

QualityAssurance

IA Process Categories


Organization

Current State

Sponsorship• The CAO aligns itself with City Council through the AC and is independent of City administration.• The CAO’s approach to engagement (i.e. distancing itself from City administration) results in low connectivity and visibility within the organization, and an

adversarial relationship with administration.• City administration questions the CAO’s ability to deliver timely and value-added internal audit services to their departments based on project experiences.

Mission / Authority• The CAO operates under a charter approved by the Audit Committee.• The charter outlines the CAO’s purpose, scope, authority, responsibility and standards of audit practice which are intended to align with IIA Standards. The charter

emphasizes the CAO’s independence, objectivity and the department’s role.• The responsibilities of the City Auditor are outlined in the charter. However, the charter does not clearly define the expectation of the CAO with respect to working

with City administration to develop the City’s annual audit plan or to identify potential risk areas.• The City Auditor views the CAO’s role as that of an “Auditor General” working directly in an oversight capacity to uncover process exceptions and

mismanagement. In contrast, administration requires a CAO function which provides value to the organization by assisting administration with identifying emergingrisk areas.

• Staff-level CAO personnel appear unclear of the department’s core mission and expectations from the organization.

Independence & Objectivity• The City Auditor reports functionally to the Audit Committee and administratively to City Council.• The City Auditor has in camera sessions with the Audit Committee.• The City Auditor has direct access to the Audit Committee Chair as needed.• There was a consensus amongst stakeholders interviewed that the CAO conducts its work objectively.

Organizational Structure• The CAO is comprised of a “two team” structure under each Deputy City Auditor. Occasionally, the two teams are cross functional. However, this ultimately results

in an internal divide within the CAO.• The Manager, Whistle-blower role lacks definition and resource support.• CAO Deputy City Auditors, Audit Associates and Auditors can view the Audit Committee proceedings on-line, and will occasionally attend the meetings.

15


Organization – continuedOpportunities for Improvement/Enhancement

PwC Observations Short-Term Recommendations Long-Term Recommendations Benefits and/or Industry Trend

The City Auditor’s interpretationof the CAO’s mandate is notaligned with the AC or the Cityadministration’s views.

The CAO views itself as an “AuditorGeneral” function working touncover process exceptions andmismanagement.

Alternatively, the AC and theadministration expect the CAO toprovide value to the organization byassessing compliance withestablished processes andassisting administration inidentifying and recommendingtimely fixes to gaps in known andemerging risk areas.

Clarify the CAO’s role.

We recommend a select group of ACmembers (along with input from seniorAdministration) meet with the CAO toclearly articulate the expected role ofthe CAO.

Define and clearly communicate theCAO’s mandate to staff.

Increase CAO staff’s clarity as to theCAO’s mandate and expectations fromthe organization through opendepartmental communication andeducation.

Educate / rebrand the CAO amongstadministration and management.

This could be achieved by revisitingthe CAO’s role and the manner inwhich the function can add value to theorganization.

Align core value drivers for thedepartment with ongoing changeswithin the organization andexpectations from stakeholders.

It will be important that the CAO staysabreast with the changing challengesand opportunities throughout theorganization. This can be achieved byrevisiting the value drivers for the CAOon an annual basis and engaging inopen dialogue with stakeholders.

Balance between independent assuranceand supporting administration will improvethe organizations risks and controlsapproach.

Clarity on the department’s core missionwill enable staff to execute with greaterperspective. Well communicatedexpectations will foster greatertransparency within the department.

Aligning value drivers with ongoingorganizational changes and stakeholders’expectations will help focus on the relevantareas of risks.

Current team structure results in“silos” within the CAO.

The current CAO structure dictatesthat each Deputy City Auditor has aseparate pool of staff-levelresources, which creates anunnecessary perception of divisionwithin the function. This structuredoes not appear to be appropriate,as neither “team” possessesunique skills or areas of specialty.

Establish a “one-team” structure within the CAO.

Realigning the CAO’s structure will increase connectivity and learning amongstteam members, and ultimately emphasize a “one CAO” approach to addressingthe function’s mandate. Under a “one-team” structure, Deputy City Auditors willbe able to draw upon a larger pool of resources, which also facilitates anincreased matching of skills to project requirements.

Revisit the Manager, Whistle-blower role.

Formally define the mandate and responsibilities for the Manager, Whistle-blower.Accordingly, determine the level of resource support and nature of skills requiredto support the achievement of this role’s mandate.

Most internal audit functions in both thepublic and private sector generally operateunder a pooled resource model for corestaff-level auditors.

Some larger, more sophisticated functionswith in-house specialty skills (e.g. IT, fraudprevention) may align based oncompetency areas; however, connectivityamongst team members within suchfunctions is typically maintained and cross-leveraged to increase audit value.


Human Resources


In assessing the CAO’s human resource practices, we considered the followinginternal audit objectives:

• A formal process is followed to ensure the resource plan, including the use of subjectmatter experts, is appropriate to deliver the value drivers for the CAO and theorganization as a whole.

• Professional development is designed to equip CAO staff to meet the stakeholders’value drivers, and it is linked to the resource planning process.

• Training is specific and appropriate for the audit needs of the staff; the CAO functionas a whole, and the staff’s individual career development.

• Project and individual performance objectives should be tied to the goals of thedepartment and the organization.

Organization

HumanResources

WorkingPractices



KnowledgeManagement

QualityAssurance



Human ResourcesCurrent State

CAO Skills & Recruiting• Over the past 3-4 years, the CAO has experienced significant staff turnover. Some hires during this period have left the CAO shortly after joining.• The department has encountered challenges recruiting qualified resources at the staff level (i.e. “Audit Associate” and “Auditor” levels); summer students have been

hired to augment audit resources. A number of vacancies continue to exist, and has recently increased.• The CAO does not have staff members with a Certified Internal Auditor (CIA) or Certified Information Systems Auditing (CISA) designation although there are some

members with professional designations such as CA, CGA, CMA and ACCA.• The CAO does not maintain a skills inventory or staff bios.• Subject matter specialists are not consistently engaged to provide support / value on projects.

Resource Planning• Resource planning to support delivery of the audit plan is based on existing capacity within the department, as opposed to obtaining the skills and experience

necessary to effectively conduct projects focusing on key risk areas which impact the achievement of the City’s strategic objectives.• Staff are assigned to audit engagements based on Deputy City Auditors’ judgement and staff availability.• A role description has not been created for the Manger, Whistle-blower position.

Professional Development and Training• Career development plans have not been created for CAO staff.• A succession planning process has not been established within the CAO.• During the past year, a process was implemented to track CPE requirements for staff and to maintain a record of training undertaken.• Per discussions, staff have little input into their training and development priorities. A departmental training budget is allocated amongst CAO personnel by the City

Auditor.• Staff CPE hours for the previous fiscal year were relatively low as compared to industry standards, as per the City Auditor’s 2009 Annual Report.• The CAO’s annual objective-setting and evaluation process is aligned to the City’s process (““Exempt Performance and Career Development Process Form”), but is

generic in nature and not tailored for CAO purposes.• Staff-level objectives and evaluations are generally facilitated by the Deputy City Auditors only.• Formal performance appraisal, feedback, and mentoring for Deputy City Auditors and the Manager, Whistle-blower is not provided.• Formal reviews of project performance are not conducted at the end of audit assignments.


Human Resources - continuedOpportunities for Improvement/Enhancement

PwC Observations Short-Term Recommendations Long-TermRecommendations

Benefits and/or Industry Trend

Existing CAO staff skill setsare limited at the staff-levels.

Due to high turnover during thepast few years at the staff-levelsand difficulties recruitingqualified personnel, there iscurrently a lack of core internalaudit knowledge and executionexperience in the CAO.

Staff input on areas fordevelopment and training is notactively sought or considered.

Develop staff-level internal audit expertise andestablish an interim staffing strategy.

Establish a staffing strategy and skill requirementsbased on the department’s audit plan, and assessgaps in required skills vs. current staff-level expertise.Consider both third-party solutions and the targetedhiring of experienced staff-level internal auditors.

Acquire financial and operational audit expertise(through training, certification programs such as theCIA designation) to build a team with stronger internalaudit knowledge at the staff-level.

CAO leadership and Deputy City Auditors should workwith staff-level personnel to identify specificdevelopment areas for each team member. Promotethe correlation of staff training to CAO objectives andareas of anticipated audit focus.

Consider staffingstrategy on an annualbasis.

Establish a policy toreview the CAO’s staffingstrategy every year at thetime of audit planningbased on skill, experienceand expertise needed torespond to organizationalchanges and initiatives,and effectively executeplanned audits.

The City of Calgary faces a number of newchallenges and opportunities within theshort- and medium-term. The CAO can playa significant monitoring role during thisperiod. However, having the right skill setsis a pre-requisite for adding value. Anassessment and realignment of skills wouldhelp the CAO add significantly more valueto the organization. Audit projects wouldhave more depth and the recommendationswould be more practical.

Increasing staff participation in theidentification of training requirementspromotes both taking personal ownership ofcareers and an increased sense ofbelonging and value.

Lack of clear career path forCAO personnel.

The CAO has not established acareer path for its staff. As aresult growth potential within thedepartment is not defined. Thishinders retention of employeeswithin the department anderodes institutional knowledge.

CAO leadership presence islimited during the hiring and on-boarding of staff-level personnel.

Identify career path for existing personnel andnew hires, and establish succession plans.

Develop / implement a structured career developmentprocess. Staff should have a clear sense as to whatthey need to do to grow within the organization.

Increased City Auditor involvement in the interviewand hiring process, as well as during on-boarding ofnew hires, is important for projecting a positive careerpath from the outset of staff employment.

Establish a succession plan for resources above thestaff-level in the CAO.

Create a rotationprogram.

Obtain senior executivesponsorship andmanagement support,and create a 2-3 yearrotation program in theCAO for fast track andhigh potential personnelwithin the City of Calgaryorganization.

A structured career development process isa powerful tool to motivate employees forbetter performance. It is also a tool forretention of good employees and strong skillsets. A rotation program promotes the flowof skill sets in and out of the CAO withretention within the organization.

Rotation programs are not pervasive withinthe public sector audit functions. However anumber of top performing audit functionsuse such programs as effective tools forskills management.


Human Resources - continued

20

Opportunities for Improvement/Enhancement

PwC Observations Short-Term Recommendations Long-Term Recommendations Benefits and/or IndustryTrend

Weak processes for performancefeedback.

While the CAO does have a performanceappraisal process aligned to the City’sprocess, the “Exempt Performance andCareer Development Process Form” is ageneric goal setting tool. CAO staff and theDeputy City Auditors overseeing thecompletion of these forms do not includeCAO specific goals and objectives, thereforereducing the relevance to ongoingimprovement of the CAO staff or thedepartment.

Specific weaknesses noted include:

• Staff-level annual objective setting andevaluations are generally facilitated solelyby the Deputy City Auditors.

• Performance appraisals, feedback, andmentoring for Deputy City Auditors and theManager, Whistle-blower is not provided.

• Formal reviews of performance are notconducted at the end of audit projects.

• Setting objectives and performanceappraisals are perceived to be pointless byCAO staff due to lack of leadershipemphasis and involvement.

Increase leadership emphasis on objective setting, feedback, andperformance appraisal processes.

The importance of setting personal objectives aligned to departmental goals andshould be enforced through the “tone at the top” from CAO leadership. Increasedinvolvement from the City Auditor in the goal-setting and performance appraisalprocesses for all personnel, including the messaging of performance ratings anddevelopment points.

Set formal guidelines for the completion of annual performance processesmilestones. Formally review staff progress towards achieving objectives on aregular basis (e.g. semi-annually).

Provide formal project-specific feedback to staff from supervisor(s). Examples ofareas for feedback include: effectiveness of audit planning, coaching of staff,execution of audit steps, managing auditee relationships, timeliness of auditcompletion and adherence to budget, quality of work papers and reports, andtimeliness of issue of audit reports.

Integrate the performance evaluation for each audit with mid-year and annualperformance appraisals to determine the overall performance.

Create a culture of coaching and performance recognition.

“Coaches” should be assigned to staff members at all levels to drive performancefeedback and evaluation processes, and provide mentoring, representation, andsupport to staff. Coaching objectives should be included in annual plans forsenior CAO resources, with the achievement of these objectives factoring intoperformance evaluations.

Effective objective setting,feedback, and appraisal processesare cornerstones for staffdevelopment in most internal auditdepartments.

Such processes representpowerful means for increasing staffengagement and motivation,accountability for personal anddepartmental performance, and away in which to drive a culture ofcoaching and continuousimprovement. Leadership byexample from the City Auditor iscritical for the success of theseprocesses.


Working Practices


In assessing the CAO’s working practices, we considered the following internalaudit objectives:

• Risk assessment process in place is effective in that all risks are captured andprioritized and the results evaluated by the CAO.

• Annual audit planning process ensures all appropriate information on risk and controlis used by the CAO when developing the annual audit plan. Also, the plansincorporate all areas of the charter and are aligned with stakeholder value drivers,strategic objectives, and the department’s strategic plan.

• Assignment planning process ensures staff with appropriate skills and experience areassigned to perform work, and the scope of procedures are based upon a fullunderstanding of the risks to be reviewed, the assurance needs of the Board andAudit Committee, and the expectations of administration.

• Methodology used to deliver CAO service is efficient and effective.

• Assignments are adequately controlled and delivered efficiently.

Organization

HumanResources

WorkingPractices



KnowledgeManagement

QualityAssurance



Working Practices

Current State

Risk Assessment and Annual Planning• The CAO does not consistently leverage the work of the Integrated Risk Management (IRM) group when creating the audit plan.• The CAO uses the 2005 risk assessment and the risk register as a basis for annual risk assessment and audit planning purposes; however, we could not confirm the

use of this risk register in the preparation of the 2010 risk assessment used to develop the 2010 – 2011 annual audit plan.• The audit plan is shared with select administration members for their input before being presented to the Audit Committee for approval. However, this appears to be

late in the process and administration does not see itself as having active input or participation.• The audit plan is a 18 month rolling audit plan that is updated annually, and does not align to stakeholder value drivers or critical risk areas identified by administration.• CAO staff members, other than the City Auditor, are not involved in the risk assessment or annual audit planning processes. Rather, they are “given” the plan.

Detailed Assignment Planning• The CAO performs planning procedures at the outset of each engagement in order to refine project objectives and scope; however planning procedures are

inconsistently performed.• The annual audit plan describes the objectives and risks very briefly.• Audit planning for projects is not integrated with the annual risk assessment.• The CAO utilizes templates made available within TeamMate to support the planning process.• Project resource allocation is often based on staff availability as opposed to project needs.

Methodology• CAO audit project policies and procedures have recently been updated in preparation for the QAR. Prior to the recent updates, the policies and procedures contained

in the manual were infrequently reviewed and updated.• Staff appear to possess little knowledge or familiarity as to the CAO audit manual’s contents, which has led to inconsistencies amongst engagement teams in terms

of the nature and extent of documentation.• The methodology used to deliver CAO service does not appear to be efficient or effective based on frequent and significant delays in completing audit projects, and

the level of “rework” required after initial fieldwork is complete.• The CAO utilizes the following frameworks and guidance: Committee of Sponsoring Organizations (COSO), Canadian Institute of Chartered Accountants (CICA) and

Control Objectives for Information and Technology (COBIT).


Working Practices – continued

An audit function’s manual containing thedepartment’s core processes and policiesserves as a cornerstone for consistency withinthe department. Beginning at the outset ofemployment, information contained withdepartmental manuals provides a usefulmeans for educating staff and increasing theirunderstanding of working practices. This canultimately result in increased efficiency andeffectiveness of the end-to-end audit process.

Review and update the auditmanual on a regular basis.

The audit manual should bereviewed for appropriateness andupdated on a regular basis (i.e.annually). Responsibilities forreviewing and updating the auditmanual should be formally assigned,with staff level involvement (andleadership oversight) to increaseownership and understanding ofCAO processes.

Raise staff-level awareness of CAOworking practices.

CAO leadership should conduct anassessment as to the completeness of theexisting audit manual, and theappropriateness of contents (i.e. considerwhether procedures support workingpractice efficiency). Thereafter, themanual should be formally rolled-out tostaff, with accompanying education.

For example, discuss within the monthlystaff meetings portions of the manual toraise departmental awareness andunderstanding of processes.

Establish a process for regularlyreviewing and updating the manual.

Lack of regular review andcommunication of the CAOaudit manual.

A significant number of updatesto the CAO audit manual werecompleted during April – May2010 in preparation for the QAR.Prior to these updates, themanual was infrequentlyreviewed or updated.

Staff appear to have littleknowledge of the audit manual’scontents, impacting theconsistency and effectiveness ofworking practices.

An enhanced risk assessment process willhelp the CAO to focus on the right risks (i.e.those which impact the achievement of theorganization’s strategic objectives).

Leading IA practices follow a comprehensiverisk assessment process with significantinteraction with senior and middle levelexecutives to ensure that audit’s efforts arefocused on the right risks. There is anincreased awareness and focus on ensuringthat efforts from risk management functionsand Internal Audit are coordinated to have aconsistent view of risks.

Expand the collaboration effortbeyond IRM to other monitoringfunctions within the organization.

The CAO should seek to furtherexpand collaborative efforts for theannual risk assessment beyond theIRM group and should activelyinteract with other monitoringfunctions within the organization toidentify areas for coordination.

Enhance the existing risk assessmentprocess and audit planning processes.

Develop a collaborative three year riskbased audit plan using the administrationscurrent risk management plan as astarting point. Year one should include adetailed plan with expected auditoutcomes and preliminary timing. Yeartwo and three should include key focusareas.

Input from key stakeholders should beactively sought throughout plandevelopment.

Establish increased integration andknowledge sharing with the IRM group tocross-leverage risk-related information.

Risk assessment andplanning processesperformed in isolation.

The CAO performs an isolatedrisk assessment and does notactively obtain input fromadministration.

Coordination is minimal inleveraging risk-relatedinformation from the IntegratedRisk Management (IRM) group.

It is unclear whether the 2010annual audit plan was preparedusing a formalized assessmentof risks, as little support wasavailable.




Working Practices – continued

24



Many projects span months /years of elapsed time.

It was noted that several auditprojects have taken months /years to complete and reportupon. Currently, projects remainin progress which werecommenced in prior annual auditcycles.

Non-timely completion of auditsand reporting significantlyreduces the relevance and valueof work performed, as processesand risks may have changedduring the elongated auditcycles. Time may be “wasted”by CAO personnel,administration, and the ACaddressing information that is nolonger pertinent.

Establish agreed upon timelines for all projects prior to commencement.

Project timelines should be measured in weeks (e.g. 4 – 8 weeks from planningto reporting). To achieve this, the CAO should look to establish and / or re-designits processes to enable the shortened project timelines.

Develop and execute a short-term plan to address and wrap-up all openaudits.

The CAO should assess the merits of performing further project fieldwork for allopen audits. Open audits should be assessed to determine whether theperformance of further work still has the potential to add timely value to theorganization. The amount of incremental time required to complete projectsshould also be considered. Thereafter, projects should be wrapped-upaccordingly.

Reducing project timelines will enable thepreparation of a final report which presentstimely information, thereby increasingrelevance and value.

Many internal audit functions measure theefficiency and duration of end-to-end auditcycles as a key performance indicator.


Information Technology


In assessing the CAO’s use of information technology, we considered the followinginternal audit objectives:

• Computer-assisted audit techniques (“CAATs”) are used to efficiently gather credibleaudit evidence.

• CAO resources are integrated to form audit teams comprised of complementary skillsets.

• CAO has appropriate involvement in technology projects for development,acquisition, installation, and maintenance of new systems.

Organization

HumanResources

WorkingPractices



KnowledgeManagement

QualityAssurance



Information TechnologyCurrent State

Technology Audit Organization• There are no IT Audit resources within the CAO. The CAO does not have any staff members with a Certified Information Systems Auditor (CISA) designation. The

CAO conducts minimal IT audit work; the most recent IT audit project was outsourced to an external provider.

IT Risk Assessment• IT risks are not adequately considered during annual risk assessment and audit planning processes. This appears to be driven by the lack of expertise within the

department to execute such projects as opposed to basing areas for IT risk coverage on potential impacts to the achievement of the City of Calgary’s objectives.• IT resources are not integrated onto audit teams.

Use of IT Tools• TeamMate software was implemented in the prior year. The implementation was led largely by the CAO’s Executive Assistant, who does not possess internal audit or

TeamMate-specific knowledge. Internal TeamMate experience (e.g. Deputy City Auditor) was not leveraged.• Coordination of TeamMate training was approached inconsistently. Some CAO personnel attended multi-day training; other team members possess TeamMate

experience from past work experience.• TeamMate is currently not being used in its full capacity. For example, while the Electronic Working Papers (“EWP”) functionality is being used, the Project Library

functionality is not being leveraged to it’s full extent.• Formalizing TeamMate processes and procedures remains a work in progress.• Data analysis skills (e.g. Audit Command Language “ACL” / Computer-assisted audit techniques “CAATs”) useful for gaining efficiencies are low in the CAO below

the City Auditor level.


Information Technology - continued

IT risks are a major contributor to the overallrisk of any organization. Thus it is importantthat IT risks be considered similar tooperational and financial risks during theannual risk assessment process.

Use of data analysis tools will help the CAO inperforming a more efficient and effective audit,and could support the review of 100% of anaudit population vs. just a sample.

Most top performing internal audit departmentshave a portion of their departments asdedicated IT audit staff. Generally, mostinternal audit functions also use data analysistools to provide achieve broader and moreefficient audit coverage.

Increased use of TeamMate can increase theconsistency and efficiency of end-to-end auditprocesses.

Leverage data analysis andIT tools.

Foster the use of dataanalysis tools for each auditby making it a mandatory partof the audit planning process.Train staff and / or hireresources with experienceusing such tools.

Further leverage TeamMateto increase projectefficiencies.

Libraries and TeamStorecould be leveraged to buildaudit procedures specific tothe audits regularlyperformed.

Developing and incorporatingtemplates within TeamStoreor within the Project Libraryfile will increase efficiency incompleting the requiredforms.

Incorporate a comprehensive IT riskassessment within annual auditplanning process.

Given the pervasive impacts IT has onthe City’s operations, it is important thatIT considerations are strongly integratedfrom the risk assessment processonwards. This can be achieved byleveraging IT audit skill sets (eitherthrough internal hire or via outsourcing),developing an IT risk matrix, identifyingIT risks that are most relevant for the City,assessing their impact on the operations,and incorporating the results within theannual audit plan.

Include IT related risks and controlsduring audit planning. Obtain IT resourceexpertise for execution of related testingprocedures.

IT risks-areas do not receivesignificant coverage in the annualrisk assessment, audit plan, andwhen executing projects.

The CAO does not have dedicated ITaudit staff, and IT audit skills within thedepartment are limited. As a result, ITaspects and risks are not adequatelycovered by the CAO. These couldinclude areas such as pre- and post-implementation of applications, reviewof IT effectiveness, business continuity,and data privacy.

There is a low use of data analysis andrelated tools during the audit planningand execution processes.


PwC Observations Short-Term Recommendations Long-TermRecommendations

Benefits and/or Industry Trend


Communication and Reporting


In assessing the CAO’s communication and reporting processes, we consideredthe following internal audit objectives:

• Appropriate communication takes place within the CAO department to ensure thatthe CAO’s mission, strategy, and tactics are clear and that the CAO’s services aredelivered effectively.

• Communication processes facilitate communication within the organization of thedepartment’s mission, strategy, and tactics.

• Reporting process to the audit committee, board, and line management is effective,and the process promotes desirable change.

Organization

HumanResources

WorkingPractices


Communication& Reporting

KnowledgeManagement

QualityAssurance



Communication and ReportingCurrent State

Communicating with the organization• CAO prepares an Initiation Letter giving advance notice to administration of the upcoming audit. The CAO also holds an opening meeting with administration to

discuss the challenges, risks and business performance prior to the issuance of a project “scoping” document.• City administration is generally given an opportunity to respond to CAO’s preliminary conclusions prior to drafting of the project report.

Communicating with the Audit Committee• The City Auditor presents quarterly reports to the Audit Committee detailing the progress made against the approved audit plan, results of work performed, and status

of action plans agreed by administration.• The City Auditor has access to the Audit Committee, and the Audit Committee Chair. The Audit Committee does not have any interaction with the Deputy City

Auditors.

Communicating within the CAO• The CAO holds regular internal meetings in efforts to promote communication. However, these meetings are deemed by CAO personnel to be ineffective in achieving

this objective due to the “top-down” nature of communication and directives (please refer to the “Knowledge Management” section).

Reporting• Several audit projects span a number of months or even years of elapsed time, which significantly reduces the relevance of recommendations as processes may

have changed between project initiation and reporting.• A rating system which provides an indication of the severity / priority of findings is not employed. This has led to a large number of findings presented in reports, with

a weak focus on priority issues for remediation.• Administration is typically given 30 days to respond to audit findings. This is viewed as inappropriate by auditees due to the length of time taken to receive the reports

following project initiation and the number of findings contained within reports.• Once reports are drafted by CAO personnel and provided to the City Auditor for review, their involvement in the reporting process concludes. Visibility as to the nature

and extent of edits and changes is not provided to CAO staff.

Follow-up and Tracking• The CAO is currently tracking over 500 management action plans that have not been addressed.• The management action plans for each audit, dating back to 2000, are set-up for tracking within an MS Excel tracking sheet and in TeamMate. However, a formal

process is not employed whereby individuals are assigned responsibility to regularly follow-up with administration as to their progress on implementingrecommendations. Monitoring is undertaken on an ad-hoc basis when time permits.

• CAO staff are assigned to monitor and test implemented recommendations only when time permits. The issues remain in TeamMate until closed.


Communication and Reporting – continued

The manner in which projectresults are reported directlyimpacts the value derived fromaudit work. Focusing attentionon the most significant findingswill assist administration inprioritizing items requiringattention. Most audit functionsprioritize findings based on risk/ severity within audit reports.

More timely release of theaudit reports will increase therelevance and applicability offindings.

Develop a value-add approach to audit reporting.

The CAO should develop a “new” reporting template in consultation withstakeholders which adds greater value. We recommend the template contain asimple prioritization methodology (e.g. red, yellow, green) to guideadministration’s attention to findings.

Please see “Working Practices” for recommendations relating to audit andreporting cycle time.

Current reporting timelines or formats do notmeet stakeholder needs.

With audit projects spanning a significant elapsedtime, the relevance of reports and associatedrecommendations is significantly reduced asprocesses and controls may have changedbetween project initiation and reporting.

Audit reports typically contain a large number offindings; because a rating system which providesan indication of the severity / priority of findingshas not been employed and there is a lack offocus on priority issues for remediation.

Auditees are of the view the 30 day timeline torespond to audit findings is inappropriate giventhe length of time taken to receive the reports andthe number of items put forth requiring attention.


PwC Observations Short-Term Recommendations Long-Term Recommendations Benefits and/or IndustryTrend

Weak monitoring of management’s actionplans for addressing audit findings.

The CAO is currently tracking over 500management action plans that have not beenaddressed, dating back to 2000. This calls intoquestion the relevance and timeliness ofmonitoring activities.

A spreadsheet and TeamMate are used to trackaudit findings and whether recommendationshave been addressed by administration.However, a formal process is not in place toassign responsibility for regular follow-up withadministration. Monitoring is undertaken on anad-hoc basis.

Establish a formal process for tracking management’s action plans.

This includes formalizing responsibilities for tracking and follow-up activities, aswell as timelines for doing so. This includes formally assigning staff to monitoradministration’s progress and test completed remediation efforts.

The CAO should assess the relevance of the current inventory of items beingtracked, and determine which management action plans warrant continuedmonitoring. This should be undertaken in a manner which includes dialogue withadministration regarding outstanding action plans, and current relevance thereof.

Formalizing a process formonitoring management’saction plans will promote thetimely implementation ofrecommendations, and driveboth CAO and organizationalaccountabilities as they relateto these recommendations.


Knowledge Management


In assessing the CAO’s knowledge management, we considered the followinginternal audit objectives:

• Knowledge sharing is integrated into working practices as part of the daily work.

• Knowledge creation, harvesting and transfer are an integral part of the CAO culture.

• The CAO has developed and maintains a repository to assist Internal Audit membersin locating relevant knowledge.

• The CAO transfers knowledge to the business unit (and not just vice versa).

Organization

HumanResources

WorkingPractices



KnowledgeManagement

QualityAssurance




Current State

The CAO does not have a formal knowledge management strategy.

• The CAO page on the City of Calgary website includes information regarding the CAO’s mission and authority, annual audit plans and reports, and audit projectreports.

• Regular CAO staff meetings are held, which are led by the City Auditor. Regular “leadership” meetings between the City Auditor and Deputy City Auditors / AuditManager, Whistleblower are also held. In both cases, the nature of communication is generally described by attendees as “top-down” and closed.

• Orientation for new hires in the department is informal and inconsistent. The head City Auditor does not consistently meet with hires upon commencement ofemployment.

• A consistent mechanism for storing information about business units and the issues identified that are not within the scope of current audit but may be useful forfuture audit planning has not been used in the past year.

• The City Auditor is involved in a number of external capacities to support the internal audit profession, including roles with the local Institute of Internal Auditors (“IIA”)chapter and the Association of Local Government Auditors (“ALGA”).


Knowledge Management - continuedOpportunities for Improvement/Enhancement


Department does not stronglyleverage or institutionalizeknowledge.

The CAO does not have aformalized knowledge managementstrategy for the department. As aresult staff learns largely “on thejob” instead of leveragingaccumulated organizationalinformation prior to engaging inaudit fieldwork.

An orientation program for theCAO’s new hires is not formalized.

Establish a knowledge managementstrategy.

Develop a formal knowledgemanagement strategy that identifiesknowledge management championswithin the department, key objectives,information needs and ways in whichknowledge will be stored and sharedwithin the department.

Promote knowledge sharing within thedepartment by linking this objective toannual goal setting.

Create an orientation package fornew hires.

Create a CAO-specific orientationpackage for new hires that includesthe CAO’s mission, goals,objectives, the IIA Standards,business process information,internal audit approach &methodology, tools, support, andfunctions.

An important complement toorientation materials is directinvolvement by CAO leadership inon-boarding activities.

Knowledge management plays a key role inthe growth and development of internal auditstaff especially if a large number of peoplewithin the department are new. A formalstrategy will enable efficient risk assessment,planning and execution.

The orientation package will help with moreefficient on-boarding of staff.

Knowledge management is a keyconsideration for high performing internal auditfunctions.

Inter-department knowledgesharing communication ispredominantly “top-down”.

While regular CAO team meetingshave been recently initiated,meeting agendas and discussiontopics are driven by the City Auditorwithout significant encouragementof dialogue. Knowledge sharingdiscussions are “top-down” anddon’t promote interactivediscussion.

Encourage open knowledge sharingduring CAO team meetings.

CAO team members should be madecomfortable to openly shareknowledge and promote dialogueduring team meetings.

Formalize regular presentation ofknowledge gained by CAO staffthrough project work and developmenttraining as a standing agenda topicduring team meetings. Encouragestaff-level auditors to presentknowledge topics to the CAO teamand disseminate information.

Presentation to and by City ofCalgary functions.

Strive to be a source of value-added information and knowledgeto the organization by presentingkey topics (e.g. emerging riskthemes and trends) to Citymanagement and staff.

Periodically invite City managementand staff to “present” keyorganizational developments andrisk considerations to CAO staff toincrease staff’s institutionalknowledge.

CAO personnel can serve as a valuablesource of organizational and technicalknowledge. By encouraging and activelysharing project, professional, anddevelopmental experiences with one another,aggregate CAO knowledge can increase in aneconomic manner.

Interdepartmental presentation on knowledgecan also promote the development ofpresentation and facilitation skills within a low-risk, internal environment. For this to beachieved, staff must feel comfortable engagingin open dialogue amongst the team; it is criticalthat this tone is set by CAO leadership.


Quality Assurance


In assessing the CAO’s quality assurance processes, we considered the followinginternal audit objectives:

• Quality assurance processes exist to ensure that the core processes of the CAOfunction continue to be aligned with the stakeholder value drivers and riskmanagement priorities of the organization.

• Appropriate performance metrics are in place to measure the effectiveness of thecore processes of internal audit and indicate whether stakeholder value drivers, CAOobjectives, and CAO mission have been achieved.

Organization

HumanResources

WorkingPractices



KnowledgeManagement

QualityAssurance



Quality AssuranceCurrent State

Quality

• The CAO has indicated the adoption of the IIA Standards as part of its policies and procedures for file quality assurance reviews, performance reviews, andtraining. However, the consideration of and compliance with IIA standards to support quality assurance policies and procedures is not readily apparent within theCAO.

• Internal CAO quality assurance processes and mechanisms to drive continuous improvement in the department are weak or nonexistent.

• Audit reports are often released prior to TeamMate files being quality-reviewed and signed-off by Deputy City Auditors.

• Written and/or oral auditee (customer) feedback is not gathered.

Performance Metrics

• The metrics reported to Audit Committee are inconsistent with the key performance indicators listed in the CAO audit manual.

• CAO productivity has been low, as indicated by recent 2009 performance during which only 42% of planned audits were completed.

• The CAO has recently developed a performance measurement process relating to monthly productivity, audit reporting cycle duration, and budget vs. actualhours for projects.


Quality Assurance – continuedOpportunities for Improvement/Enhancement


CAO Quality Assuranceprocesses do not supportcontinuous improvement.

Internal CAO quality assuranceprocesses are weak, with ad hocpractices being administered toaddress continuous departmentalimprovement.

Perform internal QualityAssurance reviews on a periodicbasis.

The CAO should develop andimplement quality assuranceprocesses with a view towardscontinuous improvement andincreasing compliance with IIAstandards.

Link internal and external qualityassurance metrics to performancegoals for all CAO staff.

Examples of performance metricssupporting continuous improvementinclude:

• Number of days betweencompletion of fieldwork andissuance of the draft or final report.

• Percentage of auditrecommendations implemented byadministration.

Internal quality assurance processes are a keyrequirement of the IIA standards. Theseprocesses help internal audit departmentspromote quality in project delivery, a culture ofcontinuous improvement, and better alignmentwith key stakeholder expectations.

Written and oral customerfeedback is not obtained.

The CAO does not actively seekcustomer feedback.

Feedback forms have not beenprovided to auditees in the recentpast.

By not obtaining regular customerfeedback, the CAO is forgoing theopportunity to receive valuableinput on areas for improvement.

Obtain regular feedback from customers.

The CAO should implement the use of customer feedback forms to solicitfeedback from customers regarding their end-to-end audit processexperiences. Feedback received should be reviewed by the City Auditor, andserve as a basis to improve CAO performance with respect to areas such ascommunication with auditees, reporting, and end-to end project execution.

On a semiannual basis, the City Auditor should meet with select organizationalmanagement and process owners, and obtain oral feedback on the CAO’sperformance.

Feedback received should be shared within the CAO to foster a culture ofcontinuous improvement.

By obtaining feedback directly from auditees,the CAO will obtain valuable perspectives froma key stakeholder group on areas forimprovement.

Obtaining and addressing customer feedbackis a common practice amongst internal auditgroups in support of continuous improvement.

AppendicesA. Interview List

B. Core IA Areas of Assessment



Appendix A: Interview List

CAO Management/Staff Interviewed

CAO Member Role

Tracy McTaggart City Auditor

Wally Markowski Deputy City Auditor

Trish McBeth Deputy City Auditor

Jackie DiSalvo Manager, Whistle-blower

Pam Lewis Audit Associate

Liz Ormsby Audit Associate

Carlos Salazar Audit Associate

CAO Stakeholders Interviewed

Stakeholders Title

Eric Sawyer Chief Financial Officer

Owen Tobert City Manager

George McLauchlin Director of Human Resources

Brad Stevens General Manager

Dale Hodges Alderman – City of Calgary

Gord Lowe Alderman – City of Calgary

Greg Draper Citizen Member – Audit Committee

John Carpenter Citizen Member – Audit Committee

Steve Patterson Security Advisor, City of Calgary

AuditorMichael Wilkison

Former CAO employeeJonn Robertson

Former CAO employeePaige Milner


Appendix B: Core IA Areas of Assessment

Working Practices

The processes and procedures in place that ensurethe efficient and effective completion of audit work,including the annual risk assessment and planningprocess, through to assignment management.

• Risk Assessment

• Annual and Operational Planning

• Detailed assignment Planning

• Internal Audit (IA) Methodology

• Assignment Management

Information Technology

The availability and use of technology tools,applications and automated auditing techniques thatenhance the efficiency and effectiveness of theauditing process.

• Automated work papers

• Data analysis tools

• Knowledge management repository

Organization

The way IA is structured and supported by theorganization to allow it to deliver its terms ofreference, including the independence, authority andsupport given to IA within the organization.

• Authority

• Sponsorship

• Mission

• Strategy

• Independence

• Structure

• Role and Responsibilities

Human Resources

The availability and management of audit resourcesto allow IA to deliver its remit; including how IAmanages its people requirements and thedevelopment needs of its staff.

• Resource Planning

• Professional Development

• Training

Communication and Reporting

The way that IA interacts with the organization andthird parties to ensure that the results of the auditwork are understood and acted upon.

• Internal

• External

• Departmental

• Reporting

• Issues Tracking and Follow-up

Quality Assurance

Whether performance metrics are in place tomeasure the effectiveness of IA againststakeholders’ objectives and that quality assuranceprocesses over IA work exists.

• Performance Metrics

• Quality Assurance


Whether knowledge is efficiently and effectivelyshared in the IA function and used to spread bestpractices throughout the organization.

© 2010 PricewaterhouseCoopers LLP. All rights reserved. “PricewaterhouseCoopers” refers to PricewaterhouseCoopers LLP, an Ontario limited liability partnership, or, as the context requires, thePricewaterhouseCoopers global network or other member firms of the network, each of which is a separate and independent legal entity.

Date post:	10-Apr-2015
Category:	Documents
Upload:	john-turner
View:	169 times
Download:	2 times

City of Calgary QAR Report - Final June 2010

Documents