38
IT(A)SA in Peer Review Mode Motivating and equipping SAIs to carry out peer reviews Bernhard Hamberger Head of Competence Center IT Audit Swiss Federal Audit Office
IT(A)SA in
Peer Review Mode
Motivating and equipping SAIs to carry out peer reviews
Bernhard Hamberger
Head of Competence Center IT Audit
Swiss Federal Audit Office
2
Agenda
IT(A)SA in ISSAI 5600
What is an IT(A)SA?
Why is an IT(A)SA important for a SAI?
How does an IT(A)SA work?
Conclusion
IT(A)SA in Peer Review Mode
3
IT(A)SA is part of ISSAI 5600
Peer Review and Self Assessment
3.18. Due to the nature of the peer review process and the likely publicity given to its findings, the
SAI could consider preceding the peer review with a self-assessment and initiate remedial actions
before the review takes place. There are different self-assessment tools available (see table). A
self-assessment can also be a useful means to help the SAI determine the focus of the proposed
peer review. The peer review can then include an assessment of the adequacy of the corrective
action being taken following the self-assessment.
3.19. A SAI can also refer to the results of recently completed internal assessments, inspections
and control measures it has undertaken to monitor progress and implementation, or for quality
control purposes. The results of these assessments can provide additional relevant input for use
when defining the focus and scope of the peer review.
IT(A)SA in Peer Review Mode
4
ITSA & ITASA in Peer Review Mode
Examples of tools that can be used for self-assessment and as a basis for peer
reviews by SAIs:
IT Self-Assessment (ITSA)
The IT self-assessment tool (developed by EUROSAI IT Working Group) aims to:
contribute to the work of SAIs by ensuring the quality and performance of the SAI’s own
information technology (IT) environment and by promoting awareness of IT governance;
develop the capacity of SAIs to meet their strategic goals through the use of IT (e.g. in relation to
internal management, through more effective audits and by developing the skills of staff).
Refer to www.eurosai-it.org
IT Audit Self-Assessment (ITASA)
The ITASA (also developed by EUROSAI IT Working Group) assesses the current and future
maturity of the IT audit function in the form of a workshop setting. ITASA is not a performance
evaluation though it provides an efficient evaluation of the current and desired status quo of IT
audit as perceived by participants.
Refer to www.eurosai-it.org
IT(A)SA in Peer Review Mode
5
Essence of IT(A)SA
Set of simple and standard actions to
identify improvements according to needs
of the SAI
Focused, pragmatic and impact-oriented
approach
Moderated Self-assessment
Developed by EUROSAI ITWG
IT(A)SA in Peer Review Mode
6
Why a Self-assessment?
IT(A)SA in Peer Review Mode
It allows for «proximity»
The evaluation is carried out by people:
who know the subject
who are interested in solving the problems
It is confidential
The organisation is in control of the results of the
evaluation and their distribution. A self-assessment is
not an audit or a peer review
The external moderation encourages participants
(members of staff) to express themselves freely
ITSA
IT Self-assessment - Approach
Bernhard Hamberger
Head of Competence Center IT Audit
Swiss Federal Audit Office
8
WHAT IS AN ITSA
IT(A)SA in Peer Review Mode
9
The objectives of an ITSA
Give preliminary answers to these questions:
How can the use of IT support the capacity of your SAI
to meet its strategic goals?
Does the SAI have the required level of IT to support
audit business?
How can we improve the level of IT audit support?
IT(A)SA in Peer Review Mode
10
Potential outcome
Provide Management with insight about the current state
of the IT support for their business processes
Helps positioning IT for the challenges ahead
Offers a platform for enabling close contact between
users and IT specialists of a SAI
Identify cultural and organisational obstacles to achieve
standardisation of business processes and applications
IT(A)SA in Peer Review Mode
11
WHY IS AN ITSA IMPORTANT
IT(A)SA in Peer Review Mode
12
An ITSA helps you to …
be an example for your auditees
work effectively and efficiently
not to waste money
IT(A)SA in Peer Review Mode
13
HOW DOES AN ITSA WORK
IT(A)SA in Peer Review Mode
14
Steps
Participants assess the maturity of the IT contribution
to achieve the SAI‘s strategic goals
Most important business processes and how they are
supported by IT
Most important IT processes and their maturity
«Gaps» are converted into actions
The suggested action plan is presented and submitted
to the Executive Management of the SAI
Re-performance of the ITSA after 3 years
IT(A)SA in Peer Review Mode
15
Two dimensions considered
IT(A)SA in Peer Review Mode
Business process 2
Business process 3
Business process 4
Business process 5
Business process 6
Business process 1
Business process 7
First
dim
en
sio
n =
Busin
ess
PO
1
AI1
AI2
PO
2
Planning and
Organisation
Acquisition and
Implementation
Etc
...
Etc
…
Etc. …
Second dimension = IT
16
Business areas covered by ITSA
IT(A)SA in Peer Review Mode
17
…or in terms of ISSAP
IT(A)SA in Peer Review Mode
18
Business dimension questionnaire
IT(A)SA in Peer Review Mode
no
ap
plic
atio
n s
oft
wa
re (
0)
low
(1
)
imp
ort
an
ce
le
ve
l (2
)
imp
ort
an
ce
le
ve
l (3
)
imp
ort
an
ce
le
ve
l (4
)
hig
h (
5)
no
ap
plic
atio
n s
oft
wa
re (
0)
low
(1
)
imp
ort
an
ce
le
ve
l (2
)
imp
ort
an
ce
le
ve
l (3
)
imp
ort
an
ce
le
ve
l (4
)
hig
h (
5)
ve
ry lo
w (
0)
qu
alit
y le
ve
l (1
)
qu
alit
y le
ve
l (2
)
qu
alit
y le
ve
l (3
)
qu
alit
y le
ve
l (4
)
ve
ry h
igh
(5
)
B1
Planifications annuelle (programme de travail annuel
ECA, Fiches A à E) et pluriannuelle des missions
(collecte et priorisation des idées) (Word et EXCEL
sauf fiche C de ASSYST)
B2
Organise the missions, Audit planing memorandum
(APM), programme de travail, validation de la mission,
ressources, notes de travail, projets de rapports
(ASSYST)
B3
CAATs (ACL, EXCEL, Access, SQL+XL, Visio, SAS,
IDEA, BO, etc.)
B4
Production du rapport (entre Draft ASSYST du GA et
rapport DEC; ECAP)
B5
Contradictory Procedure (y compris sauvegarde de
tous les documents issus dans le cadre de la
procédure); éventuellement avec CIRCA
B6
Production des rapports (jusqu'à publication dans le
Journal officiel)
B7
Suivi de la mise en œuvre des recommandations, track
the implementation of the recommandations
What is the
importance of the
current IT systems
for this business
process?
In which IT-process (see in Form
2) is the problem (especially if
quality level = 0 or 1)?
What is the quality
of the current IT
systems ?
What is the
importance of the
future IT systems
for this business
process?
Business added-value chain (BVC) Form 1.
Does the IT help to achieve the SAI's strategic
goals?
version 2.1
ECA 2006
19
IT dimension covered
IT(A)SA in Peer Review Mode
BUSINESS OBJECTIVES AND GOVERNANCE OBJECTIVES
Efficiency
Applications
Information
Infrastructure
PeopleDELIVER
AND
SUPPORT
MONITOR
AND
EVALUATE
ACQUIRE
AND
IMPLEMENT
INFORMATION
IT
RESOURCES
C O B I T
F R A M E W O R K
Effectiveness
Confidentiality
Integrity
Availability
Compliance
DS1 Define and manage service levels.
DS2 Manage third-party services.
DS3 Manage performance and capacity.
DS4 Ensure continuous service.
DS5 Ensure systems security.
DS6 Identify and allocate costs.
DS7 Educate and train users.
DS8 Manage service desk and incidents.
DS9 Manage the configuration.
DS10 Manage problems.
DS11 Manage data.
DS12 Manage the physical environment.
DS13 Manage operations.
ME1 Monitor and evaluate IT
performance.
ME2 Monitor and evaluate internal
control.
ME3 Ensure compliance with external
requirements.
ME4 Provide IT governance.
PO1 Define a strategic IT plan.
PO2 Define the information architecture.
PO3 Determine technological direction.
PO4 Define the IT processes,
organisation and relationships.
PO5 Manage the IT investment.
PO6 Communicate management aims
and direction.
PO7 Manage IT human resources.
PO8 Manage quality.
PO9 Assess and manage IT risks.
PO10 Manage projects.
AI1 Identify automated solutions.
AI2 Acquire and maintain application
software.
AI3 Acquire and maintain technology
infrastructure.
AI4 Enable operation and use.
AI5 Procure IT resources.
AI6 Manage changes.
AI7 Install and accredit solutions and
changes.
PLAN
AND
ORGANISE
Reliability
20
IT dimension questionnaire
IT(A)SA in Peer Review Mode
no
t su
re
no
t im
po
rta
nt
(1)
imp
ort
an
ce
le
ve
l (2
)
imp
ort
an
ce
le
ve
l (3
)
imp
ort
an
ce
le
ve
l (4
)
ve
ry im
po
rta
nt
(5)
COBIT's Domains and Processes no
n-e
xis
ten
t (0
)
initia
l /
ad
ho
c (
1)
rep
ea
tab
le b
ut
intu
itiv
e (
2)
de
fin
ed
pro
ce
ss (
3)
ma
na
ge
d a
nd
me
asu
rab
le (
4)
op
tim
ise
d (
5)
no
n-e
xis
ten
t (0
)
initia
l /
ad
ho
c (
1)
rep
ea
tab
le b
ut
intu
itiv
e (
2)
de
fin
ed
pro
ce
ss (
3)
ma
na
ge
d a
nd
me
asu
rab
le (
4)
op
tim
ise
d (
5)
Planning and Organisation
PO1 Define a Strategic IT Plan; IT strategy meeting
PO2 Define the information architecture
PO3 Determine the technological direction
PO4
Define the IT Organisation and Relationships; system
ownership; responsables d'application;
PO9 Assess risks
PO10 Manage projects
Acquisition and Implementation
AI2 Acquire and maintain application SW
AI4 Develop and maintain procedures
AI6 Manage changes
Delivery and Support
DS3
Manage performance and capacity; project Storage
Management
DS4
Ensure continuous service; disaster recovery site
(IT&T)
DS5
Ensure system security; project Corporate IT Security
policy; projet Remote Access
DS7 Educate and train users
DS8 Assist and advise customers
DS10 Manage problems and incidents
Which business processes (see
in Form 1) are affected by this
problem (especially if level = 0 or
1)?
current maturity
level of the
processImportance of the
process
CobiT Form 2: What is the maturity level of the
IT-processes? desired maturity
level of the
process?version 2.1
ECA 2006
21
Workshop Structure
Workshop with a group of up to 20 persons from different
disciplines within a SAI
IT and users represented
Different people from IT
(CIO, helpdesk, development, IT project manager)
Users from all business areas and from different levels of
hierarchy
Investment: 1 ½ - 2 days
(Co-)Moderators from other SAIs
IT(A)SA in Peer Review Mode
ITASA
IT Audit Self-assessment - Approach
Bernhard Hamberger
Head of Competence Center IT Audit
Swiss Federal Audit Office
23
WHAT IS AN ITASA
IT(A)SA in Peer Review Mode
24
The objectives of an ITASA
Preliminary answer to two questions Does the SAI have the required level of IT audit?
How can we improve the level of IT audit?
Additional objectives Define the appropriate level of IT audit according to the audit
strategy of the SAI and its mission
Increase the awareness of management and auditors of IT audit
Identify potential improvements and set up an action plan
IT(A)SA in Peer Review Mode
25
Potential outcome
Suggested volume, extent and alignment with
traditional audit activities
Advice on methodology, current and expected
competence of IT auditors and their training
Strategies for embedding IT Audit into existing
organisational structures of a SAI
IT(A)SA in Peer Review Mode
26
WHY IS AN ITASA IMPORTANT
IT(A)SA in Peer Review Mode
27
IT Audit is an element of all audit types
IT(A)SA in Peer Review Mode
Performance auditors Financial and
compliance auditors
IT auditors
Non IT auditors
IT audit
28
Financial Audit(according to ISA 315,330 and ISSAI 1315, 1330)
IT(A)SA in Peer Review Mode
29
Performance Audit(ISSAI 300)
Business case for IT projects
Evaluation of the need for an IT investment
Strategic alignment of an IT investment
Implementation of Open Source software
User satisfaction with application or services
IT procurement strategy
Costs and quality of service for different sourcing models
IT Architecture governance
IT Strategy development
…
IT(A)SA in Peer Review Mode
30
Compliance audits(ISSAI 400)
IT Governance, IT Operations
Compliance with project management guidelines
IT Procurement compliance
Cyber Security compliance
Business Continuity Management
Data Protection compliance
Archiving laws
Compliance with laws, policies, procedures,…
…
IT(A)SA in Peer Review Mode
31
HOW DOES AN ITASA WORK
IT(A)SA in Peer Review Mode
32
Steps
Participants assess current and future maturity
of the IT Audit function
«Gaps» are converted into actions
The suggested action plan is presented and
submitted to the Executive Management of the
SAI
Re-performance of the ITASA after 3 years
IT(A)SA in Peer Review Mode
33
Five areas to assess
IT(A)SA in Peer Review Mode
5. Quality management/monitoring/controlling
1. External requirements
2. Internal requirements
3. IT Audit process
Input
• Organisation &
approach
• Staff
• Tools
4. Output
• Reports
Steps of the process
• Risk analysis &
planning
• Audit work
• Reporting &
documentation ISS
AI re
fere
nce
Ris
kd
escri
ptio
n p
er
qu
estio
n
34
Workshop Structure
Workshop with a group of 15-20 staff
members from different disciplines within a
SAI
Investment: 1-1 ½ days
(Co-)Moderators from other SAIs
IT(A)SA in Peer Review Mode
35
CONCLUSION
IT(A)SA in Peer Review Mode
36
Conclusion
SAI could consider preceding a peer review with
a self-assessment and initiate remedial actions
before the review takes place
Through an IT(A)SA measurable improvements
can be achieved in IT Support for a SAI or in IT
Audit Capability
IT(A)SA in Peer Review Mode
37
And
… an IT(A)SA is only the beginning
– not the end
IT(A)SA in Peer Review Mode
38
Chair of EUROSAI IT Working Group (ITWG): Supreme Audit Office nik.gov.pl, Poland
Piotr Prokopczyk (Chairman), Director of the Department of Science, Education and National
Heritage
Secretariat: Beata Stephenson, [email protected], Senior Inspector at the Department
of Science, Education and National Heritage, phone +48 22 444 50 83
Website: www.eurosai-it.org
Lead of ITSA and ITASA Subgroups: Swiss Federal Audit Office (SFAO)
Bernhard Hamberger, Head of Competence Center IT Audit,
IT(A)SA Backoffice:
Emmanuel Hofmann, IT Auditor,
IT(A)SA in Peer Review Mode
Contacts
