2015 ISACA NACACS - Audit as Controls Factory

transcript

Audit As A Controls Factory

Nate Anderson, Internal Audit, SearsCliff Nuxoll, Internal Audit, Sears

PRESENTATION OBJECTIVES

• Overview of data analytics concepts– Summarize audit analytics concepts & tools– Reinforce concepts through examples & lessons– Analytics team best practices– Present practical tools & approaches to

analytics

• Challenge traditional view of Audit Analytics– Consider services Audit can provide while

remaining independent and objective

OUTLINE

• Audit analytics – Overview

• Key ingredients to audit analytics– Methodology & Approach– Building an analytics team– Overview of commonly used tools

• Analytics in action– Monitoring controls– Audit aids– Ad-hoc analysis

• Lessons learned• Maintaining Independence & Objectivity

AUDIT ANALYTICS OVERVIEW

• Definition

• Industry Insights

• Key Trends

• Key Ingredients

AD-HOC ANALYSIS

Auditor obtains useful data

Data is loaded for analysis

Results of analysis

Summary insights

Goals: Test general hypothesis (e.g., determine root cause for sample of negative margin sales)

AUDIT AUTOMATION

Auditor aid engaged

Automated routine

Results for auditor

Analytics Routine/Program

Goals: Improve efficiency, accuracy, or effectiveness of audit processes

CONTINUOUS AUDITING / MONITORING

Analytics Routine/Program

Data feed to audit

Automated routine

Output for action/decision

Goal: Enable risk monitoring, support risk decision, and/or facilitate control activity

STATISTICAL ANALYSIS / MODELINGData feed

to auditStats/modeling

routineOutput for

action/decision

Goal: Descriptive statistics procedure or modeling to test hypothesis, increase understanding, or make

prediction

INDUSTRY INSIGHTS

• PwC 2014 State of the IA Profession Survey

• Protiviti 2015 IA Capabilities & Needs Survey

PWC 2014 STATE OF PROFESSION SURVEYHow is Internal Audit doing?• 49% (senior mgmt) & 60% (board) believe IA is

delivering on expectations• 45% (senior mgmt) & 70% (board) believe IA adds

significant value• 29% (senior mgmt) & 51% (board) believe IA is

leveraging technology effectively in execution of audit services

Where are the opportunities for IA to improve?• #1 area respondents want greater IA involvement

in: – Increased reliance on big data & analytics (80%)

• “[IA] functions should always be looking to add value by expanding their capabilities in [data analytics].”

PROTIVITI 2015 IA SURVEY

• 5 of 7 areas (out of 36 total) where audit improvement is most urgently needed relate to analytics.

• Data analytics skills were the top area of desired growth in 2013 (4 of top 5) and 2014 (6 of top 9)

“Need to Improve” Rank1 Auditing IT Security

1 (tie) Computer-assisted audit tools (CAATs)

3 Data analysis tools – data manipulation

4 Marketing internal audit internally

5 Fraud – monitoring

6 Data analysis tools – statistical analysis

7 Continuous auditing

PROTIVITI 2015 IA SURVEY

• “There continues to be significant dialogue among internal audit functions about the need to leverage technology-enabled auditing tools, but they are not achieving progress.”

• “CAEs and internal audit leaders should consider whether this is becoming a never-ending journey”

• “Will [audit analytics] continue to be discussed but not implemented?”

KEY TRENDS

• Democratization of data

• Visualization growth

• On-demand computing power

KEY TRENDS: DEMOCRATIZATION OF DATA

Major growth in data

Unstructured Structured

80% 20%

Majority is unstructured & raises new opportunities & concerns

New methods to store, access & analyze unstructured data

KEY TRENDS: DATA VISUALIZATION GROWTH

Significant advances in visualization tools

KEY TRENDS: ON-DEMAND COMPUTING POWER

Leverage cloud for power & storage

KEY INGREDIENTS TO AUDIT ANALYTICS

Approach

Methodology

AUDIT ANALYTICS METHODOLOGY

Problem to analyze

Get/Process

Analyze results

Measure insights

Apply learnings

ELEMENTS OF AGILE PHILOSOPHY

Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.Just do it.

Just do it.

AGILE MANIFESTO

“We are uncovering ways of developing software by doing it and helping others do it. Through this work we have come to value:

That is, while there is value in the items on the right, we value the items on the left more.”

Individuals & interactions Over Processes & toolsWorking software Over Comprehensive documentationCustomer collaboration Over Contract negotiationResponding to change Over Following a plan

AGILE ELEMENTS WITHIN OUR APPROACH• Agile

– Obsess over problem to be solved– No “analysis paralysis”– Delivery early, often, and modestly (small

releases)– Improve incrementally– Learn from reality quickly and with little money

• Traditional– Dangerous set up: Design everything, code

everything, promise to deliver big later.– Rigid scope and plan– Over-reliant on consultants

ATTRIBUTES OF AGILE TEAMS

• Culture of transparency without penalties

• Reward early experimentation (and failure)

• Self-organizing and self-managing teams

• Cross-functional teams“I had never failed. I’ve just found 10,000 ways which do not work.”

- Thomas Edison

CHANGING WITH TECHNOLOGY

Leverage data warehouses

Leverage big data

Leverage open source

1970 2015Time

AUDIT ANALYTICS TEAM

Insights

Analyst

Business

Expert

SKILLSET: BUSINESS EXPERT

• Leverages personal insights and relationships• Focus on solving real world problems• Business unit experience• Prioritize risks

Problem to analyze

Get/Process

Analyze results

Measure insights

Apply learnings

SKILLSET: CODER

• Knows where and how to gather data• Able to code in multiple languages• Works well with key IT practitioners• Developer experience

Problem to analyze

Get/Process

Analyze results

Measure insights

Apply learnings

SKILLSET: ANALYST

• Evaluate key risks based on data• Drive solutions based on analysis• Excellent problem solver• Can visualize results

Problem to analyze

Get/Process

Analyze results

Measure insights

Apply learnings

ANALYTICS LEADERSHIP TEAM

AnalystsBusiness Experts

Coders

• Sponsor key to success• Must be open to any approach that gets results

• Strong practitioner• Great business knowledge

• Strong practitioner• Understands how to

manage IT resources and projects

Analysts

IT Audit Lead

Corporate Audit Lead

TYPICAL ANALYTICS PROCESS FLOW

Requirements

Business Expert

Coder Analyst

LESSONS LEARNED: RESOURCING

1. Diversity is critical.

2. Be ready to replace key personnel.

Auditors Coders

Coders Business Experts

AUDIT ANALYTICS TOOLS

Visualize

Analyze

OrganizeAcquire

MICROSOFT OFFICE SUITE

Acquire / ETL Organize Analyze Visualize Price Difficult

TOP AUDIT ANALYTICS SOFTWARE

GARTNER MAGIC QUADRANT – BI TOOLS

Top tier Open source

Completeness of Vision

TOP VISUALIZATION SOFTWARE

MICROSOFT BI TOOLSET

TOP BI OPEN SOURCE (FREE)

TECHNOLOGIST TOOLS

Acquire / ETL Organize Analyze Visualize Price

ANALYTICS SOLUTION EXAMPLES

• Monitoring Controls– Patriot Act Compliance – Pharmacy Compliance – Gift Card Compliance

• Audit Enhancement– Access Benchmark

• Ad-Hoc Risk Analytics– Gift card analytics– Employee Store Risks– Telecom spend

MONITORING CONTROLS

• Hosted web applications– Patriot act compliance– Pharmacy compliance– Gift Card compliance

• Collaboration between business & audit

• Aid business in mitigating significant risks

PATRIOT ACT COMPLIANCE

• Replaced pre-existing weekly Excel reports with continuous online tracking system – accuracy improvement of 500%

• Findings are generated nightly and appended to the current report

• Related transaction details are populated under each finding

PHARMACY POLICY COMPLIANCE

• Requested by Legal to protect against costly fines• LDAP-authenticated system requires Pharmacists

and Pharmacy Managers to agree/ disagree to policy on a weekly basis

• Users sign in and enter pharmacy location number

PHARMACY POLICY COMPLIANCE

• Once signed into the system with a user id and location number, users come to the policy page

• Upon agreement, user information and pharmacy location are logged

• In the case of a disagreement, Managers & Directors are notified via email to take appropriate action

GIFT CARD COMPLIANCE

Periodic review and action (sign-off) on potential risk events:

• Required sign-off

• Business unit management oversight of sign-off, participation, risk events

AUDIT ENHANCEMENT

• Hosted web application– Access benchmark

• Improves audit activities

• Typically enhances: – Efficiency– Effectiveness– Uniformity of approach

ACCESS BENCHMARK

Concept: - Access list repository for audit & IT compliance- Regular snapshots of access for critical IT assets- Enables self-service access reviews by control owners

ACCESS BENCHMARK – COVERAGE

Sarbanes-Oxley IT Components

Environments (LDAP, AD, etc.) 10+

Applications 50+

Databases 150+

Systems 200+

Datasets 50+

Production Directories 50+

Utilities 5+

• Implemented across LDAP, Active Directory, mainframe hosts, Sun, AIX, Linux, HP-UX, Windows, AS/400, MySQL, SQL Server, DB2, Oracle, Teradata, Informix, PeopleSoft, etc.

ACCESS BENCHMARK – WALK-THROUGH• Primary functions:

– Admin – Add IT assets, map reviewers, manage access

– Reviewer – Down/upload of mapped access reviews– Auditor – Download of completed reviews

ACCESS BENCHMARK – REVIEWER VIEW

# of accounts requiring review

All IT assets related to

Download current list

Relevant technology layer

Enabled drag and drop of completed access reviews

Upload occurs; data validation performed

ACCESS BENCHMARK – AUDITOR VIEW

Download List

Select technology layer

Select review “as of” date

ACCESS BENCHMARK – BENEFITS

• Effective access reviews and re-certifications

• Uniformity in approach & quality

• Enables 100% coverage (all IT assets & accounts)

• Solution is scalable (can leverage for SOX, PCI, etc.)

• Accurate “critical information asset” inventory

• Value of weekly access snapshots

AUDIT ENHANCEMENT “MUST HAVES”• Ready access to:

– employee & contractor data– Key transactional data access (e.g., point-of-sale)

• Statistical aides (assist with sample selection, etc.)

• Focus on repetitive activities in areas such as compliance

AD-HOC RISK ANALYTICS

• Conducted with desktop software– Gift card analytics (tableau)– Store employee risks (power bi)– Telecom spend (tableau)

• Enhances risk assessments, audits

• Requires savvy & assertive auditors

GIFT CARD ACTIVITY OVER TIME

Day Dt

Aug 11 Aug 26 Sep 10 Sep 25Day of Day Dt [2014]

Oct 1 Nov 1 Dec 1 Jan 1Day of Day Dt [2014]

Jan 14 Jan 29 Feb 13Day of Day Dt [2015]

Gift Cards Issued

Gift Card Trend by Date

Continuous control implemented Flawed program

launched; quickly addressed

Gift Card by State

SUSPICIOUS ACTIVITY BY STATE

States with significant activity

States where no activity is allowed

Dist Mgt Name

0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150Gift Cards Issued

Abraham

Bill Joe

Billy Bob

Billy Jack

Billy Jean

Bobbie Sue

Carol Sue

Jasper

John Boy

Johnny

Krista

Krueger

Richard

Ricky Bobby

Tim Bo

Virgil

Wilber

Gift Card by District Manager

SUSPICIOUS ACTIVITY BY DISTRICTDistricts with significant suspicious activity

STORE EMPLOYEE RISKS

Shifts < 3 hours

Qty of edits

Qty of self-corrects

STORE EMPLOYEE RISKSHigh qty of self-corrections to hours

High qty of manual hours edits

High qty of both concerns

TELECOM SPEND

• Where is biggest cost recovery opportunity?– Over allocation / overcharge– Obscure service charges– International call/text usage– Unneeded feature removal– Closed sites / lines not in use– Call/text/data plan optimization– General use overage

TELECOM SPEND: VENDOR 1Quickly highlight key cost recovery opportunities

~$350k savings proposed

TELECOM SPEND: VENDOR 2Quick overview of amount of recovery by reason

~$2.2m savings proposed

Top recovery reason: Unused lines/circuits

TELECOM SPEND: CLOSED SITE/ UNUSED LINES

SHMC-38445 and SHMC-99999 may be false positives; need more data

Abnormally large sites: - Store- Corporate

Significant number relate to corporate

TELECOM SPEND: BY SITE

Identify greatest opportunities for preventive controls

TELECOM SPEND: DRILL-DOWN ON CORPORATE

Visualization Summary: • Quick, big-picture view• Convey conclusions & approach to key stakeholders

LESSONS LEARNED

• Most valuable technical skill

• Toolbox approach

• Affordably sourcing team

MOST VALUABLE TECHNICAL SKILLS1. SQL. And then really advanced SQL.Learn it.Love it.Live it.Essential for finding, browsing, evaluating, analyzing, and filtering data

2. Excel – Lots can be done before limitations emerge

3. Tableau – Includes all essential ingredients

4. Depends on the need, familiarity, etc.

TOOLBOX APPROACH: BEST TOOL WINS• What step are you on in your data analytics

journey?

• How to move forward without:– Looking too far ahead– Spending unnecessary $$$

• Successful tools for Sears Holdings:– Everyone: Excel, Access– Front-end team: ACL, Tableau– Back-end team

• Linux servers (free, powerful server)• MySQL (free, powerful database)• Cassandra (free, powerful NoSQL database)

AFFORDABLY SOURCING TEAM

1. Coders as interns– Freedom and creativity of role should appeal to

them– Do not ask them to be auditors

2. Data analysts as interns – Subject matter is attractive (fraud, security, etc.)

3. Auditors with coding background– Increases likelihood of obtaining versatile data

analytics practitioners

ENTERPRISE RISK MANAGEMENT FAN

* Internal Audit acts as facilitator and host only

INDEPENDENCE & OBJECTIVITY

“Independence is the freedom from conditions that threaten the ability of the internal audit activity to carry out internal audit responsibilities in an unbiased manner.”

“Objectivity is an unbiased mental attitude that allows internal auditors to perform engagements in such a manner that they believe in their work product and that no quality compromises are made. Objectivity requires that internal auditors do not subordinate their judgment on audit matters to others.”

– Section 1100 – Independence and ObjectivityInternational Standards for the Professional

Practice of Internal Auditing

INDEPENDENCE IMPAIRMENT THOUGHTS• Are we “implementing risk responses on management’s

behalf”? • Are we “taking accountability for risk management”?• Are we remaining able to audit these controls without

1. We are remaining independent of the performance of the control, we are unbiased, while we are increasing our control oversight.

2. We do not make risk response decisions; we do not manage risk for management.

Most Importantly: If we never have to answer these questions, how much value are we adding?

THANK YOU

Contact Information

Nate Andersonnate.anderson@searshc.com

Cliff Nuxollcliff.nuxoll@searshc.com

2015 ISACA NACACS - Audit as Controls Factory

Data & Analytics