3/1/2017

Esther R. Sawyer Research Manuscript

Wesley Ladd LOUISIANA STATE UNIVERSITY

Contents I. Introduction ............................................................................................................................................... 1

II. Providing Independent Assurance ............................................................................................................ 1

IIa. Conflicting Goals ................................................................................................................................ 2

IIb. The Relationship Between Internal Audit and Upper Management................................................... 3

III. Scientific and Other Advanced Methodologies ...................................................................................... 5

IIIa. Empirical, Peer Reviewed Studies .................................................................................................... 7

IIIb. Cybersecurity .................................................................................................................................. 11

IIIc. Advanced Analytical Techniques .................................................................................................... 13

IV. Conclusion ............................................................................................................................................ 16

1

I. Introduction

It is a fundamental challenge of any organization to establish a raison d’etre, or reason

for being. In considering the rational ground of existence for Internal Audit, it is natural to start

with the Institute for Internal Audit’s (IIA) opinion. According to the UK chapter of IIA, “The

role of internal audit is to provide independent assurance that an organisation’s risk

management, governance and internal control processes are operating effectively” (emphasis

added).1 This quote states the terminal goal of Internal Audit as it relates to Governance, Risk,

and Compliance. This paper will examine how scientific methodologies can help Internal Audit

better fulfill the purpose set forth. From peer-reviewed empirical studies to sophisticated data

analysis techniques to understanding the latest cybersecurity tools and processes, it is the

responsibility of Internal Audit to employ the most rigorous methodologies available to ensure

its continued viability, improvement, and success. Creating an atmosphere that promotes

utilizing contemporary techniques will allow Internal Audit departments to maintain and increase

credibility relative to Governance, Risk, and Compliance.

II. Providing Independent Assurance

The Internal Audit Standards define independence as, “the freedom from conditions that

threaten the ability of the internal audit activity to carry out internal audit responsibilities in an

unbiased manner.”2 Establishing this independence is a difficult task under the best of

circumstances. The structure of an organization will dictate, in the most general sense, the

1 https://www.iia.org.uk/about-us/what-is-internal-audit/ 2 https://na.theiia.org/standards-guidance/Public%20Documents/IPPF-Standards-2017.pdf

2

ability to discharge the Internal Audit activity. Careful consideration must be given to the

reporting structure of the Internal Audit function, so that independence is not subverted.

Organizations must develop a reporting structure that furthers the department’s purpose. This

structure is often dictated or molded by the regulatory environment in which the organization is

situated. For the purpose of simplification, the focus of this paper will center on publicly owned

corporations in the United States. Publicly owned corporations typically establish a two-tiered

reporting structure.3 This structure consists of tier one: Board of Directors or Board of

Governors and tier two: senior management. Senior management reports to the Board of

Directors, who represent the final line of defense for shareholder value.

IIa. Conflicting Goals

The Board of Directors/Governors are elected by the shareholders of a company and have

a fiduciary responsibility to the shareholders.4 This means that they may be held liable for

breaches of care, loyalty, or obedience.5 In turn, the Board of Directors/Governors hires senior

management that is tasked with the day-to-day operations of the company. The Board of

Directors is concerned with governance and oversight. Senior management is concerned with

operations and strategy. On most occasions, senior management will have more information

about the business and processes of the company than the Board. This mismatch of information

leads to differing goals and expectations between the Board and senior management. Best case

scenario, this leaves a Board relying on the good faith efforts of its management to inform the

Board of potential issues. At worst, it provides a critical lever to mask fraud or malfeasance.

3 http://www.investopedia.com/articles/basics/03/022803.asp 4 http://www.oecd.org/daf/ca/corporategovernanceprinciples/1872746.pdf 5 http://agb.org/briefs/fiduciary-duties

3

In order to guard against this conflict, the Board can define an Internal Audit function.

Internal Audit thus provides assurance to the Board as well as stakeholders that the process

controls in place are sufficient to adequately mitigate risks posed both internally and externally.

To guard against undue influence by those who would typically be an employee’s superior,

namely senior management, a special reporting structure must be designated for the Internal

Audit function.

The reporting structure for Internal Audit is further complicated by the distinction

between functional and administrative reporting. According to IIA, “Administrative reporting is

distinguished from direct reporting in the sense that the administrative unit facilitates the day-to-

day operations of the internal audit activity, i.e., approving budgets and preparing performance

evaluations.”6 The appropriateness of administrative reporting to upper management is widely

accepted with the caveat that overall budgeting for the Internal Audit function be remanded to

the Board. Functional reporting must be maintained separately so that upper management does

not have the capability to undermine the independence of Internal Audit.

IIb. The Relationship Between Internal Audit and Upper Management

It should be clear from the description of Internal Audit’s reporting structure that the

relationship with upper management requires a delicate balance. Compounding the issue, the

pressure to perform that is placed on senior management exacerbates ethical issues that occur in

the course of business. “Nineteen percent of internal audit professionals and 32 percent of

Fortune 500 representatives said at least one senior officer at their organization was dismissed

6 https://na.theiia.org/iiarf/Public%20Documents/Internal%20Audit%20Reporting%20Relationships%20Serving%20Two%20Masters.pdf

4

for unethical conduct.”7 There is no holistic guide to navigating this issue, however, it is

important for the Chief Audit Executive to have a direct line of communication and mutually

positive relationship with the Board. To help foster such a relationship, Internal Audit must be

perceived as adding significant value to the organization, thereby discrediting the notion that

Internal Audit is merely a cost center.

Internal Audit Key Issues 8

Internal Audit departments at their best are thought leaders for a company in the realm of

risk. According to a 2013 report by PwC, many stakeholders are concerned with the

effectiveness of Internal Audit at addressing key issues such as quality improvement and

innovation, staffing appropriate talent, and the use of technology.8 While many Internal Audit

departments seem to compare favorably with stakeholder expectations, there is still opportunity

to proactively utilize technology and develop talent that drive innovation throughout the

7 https://www.proformative.com/articles/evolving-relationship-between-internal-audit-professionals-cfos 8 http://www.pwc.com/us/en/risk-assurance-services/publications/assets/pwc-2013-state-of-internal-audit-profession-study.pdf

5

company. Rather than being seen as an identifier of problems, Internal Audit departments can be

embraced for recommending powerful solutions.

The issues identified seem prescient in hindsight. As technology becomes an even more

integral part of the everyday world, Internal Audit departments may be slow to adapt to the new

techniques available. If an operational unit is utilizing a technique, there should be a

corresponding Internal Auditor with a fundamental understanding of the technique employed.

Underlying this issue, Internal Audit struggles to hire and retain talent capable of providing

assurance on highly sophisticated, technical projects. In order to provide true assurance in the

contemporary world, in addition to the philosophical framework, it is vital for the Institute to

develop and support scientific methods and tools.

A piecemeal application of scientific tools and methods leaves an individual or

organization at the mercy of unknown risks. Knowing just a little bit about scientific

methodologies can be more dangerous than knowing nothing about them due to improper

confidence in incomplete knowledge. An auditor that tells you a sample is statistically significant

but cannot tell you at what level of significance poses more of a threat than an inadequate or

missing control. Staffing is not the integral ingredient to a successful internal audit department,

rather, it is correct staffing. Internal Auditors who approach the task with a foundational

understanding of rigorous tools and methods can deliver the assurance that stakeholders

desperately need.

III. Scientific and Other Advanced Methodologies

Before explaining individual scientific tools and methods, it is important to articulate a

misconception that limits understanding in relation to science. The education many people

6

receive serves to over-simplify the methods of science, distilling science down to a model similar

to the graphic above.9 While this singular method may be a tremendous tool to excite and inspire

young minds, it does a disservice to those attempting to apply it as an absolute framework.

In a 2013 article for Wired, Rhett Allain, an associate professor of physics at

Southeastern Louisiana University, points to four examples of significant scientific discoveries

that did not come from this methodology.10 From the discovery of penicillin to the discovery of

general relativity, there is a wide range of examples where science does not occur in the manner

prescribed by the ‘scientific method’. This problem has been known to philosophers of science

since before the first recorded use of the “scientific method”. While the method in question was

first published in 194510, Karl Popper had already laid out a refutation of the method in his 1934

treatise, Logik der Forschung. Zur Erkenntnistheorie der modernen Naturwissenschaft or The

Logic of Research: On the Epistemology of Modern Natural Science. Unfortunately, this work

was not translated into English until 1959, by which time the “scientific method” had taken hold.

9 http://www.biology4kids.com/files/studies_scimethod.html 10 https://www.wired.com/2013/04/whats-wrong-with-the-scientific-method/

7

The reductionist model skews public perception of science to this day despite mounting evidence

indicating that science does not work consistently in that manner. Instead, science is a framework

of empirical methods and tools that provides a truly rigorous approach to problem-solving. There

are limits to each method and tool outlined below. However, utilizing these methods and tools is

a clear route to increasing Internal Audit’s value for organizational governance, risk mitigation,

and validating the effectiveness and efficiency of controls.

IIIa. Empirical, Peer Reviewed Studies

Consultancies and other organizations frequently publish papers on best practices in

Internal Audit. These papers attempt to establish benchmarks in Internal Audit as well as identify

shortcomings and issues with the Internal Audit function. However, many of these papers work

from the perspective of on-the-ground experiences. The pervasiveness of literature with this

viewpoint elevates the anecdotal experience over experimental analysis. Unfortunately, these

papers do not possess the rigor of empirical studies. An empirical study undertakes the job of

analyzing an experience using qualitative and, more often, quantitative methods in order to

determine the underlying structure and order of a process. Internal Audit as a function could

benefit greatly from use of such rigorous methodology. While subjective papers may provide

valuable insight, they are no substitute for empirical testing.

One of the largest issues with doing empirical examinations of Internal Audit is the

ambiguity of defining end goals for the Internal Audit function. As this paper has established, the

purpose of Internal Audit is to provide reasonable assurance in regards to the mitigation of risk

for an organization. This is a well-defined goal; however, it can take a multitude of forms

depending on corporate structure, business sector, regulatory and/or legal restrictions, etc. An

abundance of forms and functions presents a challenge to research in the Internal Audit space.

8

Additionally, many of these factors are subject to change that can drastically re-orient the

Internal Audit function. An acquisition or market pivot can overhaul corporate structure and

regulatory requirements overnight. How, then, can the Institute support empirical research? The

answer lies in a top-down approach to research.

Determining a reporting structure for Internal Audit is the most basic constraint when

establishing an Internal Audit function. In response to this issue, a basic question that could

spawn tremendously valuable insight is: What is the ideal reporting structure for an Internal

Audit department? Based on business sector? Based on organization size? Based on department

size? A meaningful examination could be done by weighting unfavorable outcomes (regulatory

fines, loss of business due to reputational damage, high employee turnover) and then clustering

companies according to structure and sector to determine which organizations have the most

effective structural frameworks, ceteras peribus, for discharging the duties of the Internal Audit

function. As the importance of reporting to the Board of Directors and distinction between

functional and administrative reporting is already well known, this study would focus on the

internal organizational structure of Internal Audit departments.

This issue is of increasing importance as the Internal Audit function is transformed by the

reliance of organizations on Information Technology. As IT Audit grows to rival the non-IT

Internal Audit staff in size, are current structures sufficient to discharge IT related

responsibilities? Are Chief Audit Executive’s the best advocate for IT Audit given the highly

complex and technical issues of IT Audit? If not, should new hiring criterion be established for

CAE’s? Should the IT Audit function sit underneath the overall Internal Audit function or run

parallel to it? These are just some of the questions to which empirical studies could provide

answers for Internal Audit. If a manufacturing company of less than 1000 employees desires to

9

institute a framework, would they be best served by parallelizing the IT Audit function or is the

integrated function more appropriate? The initial hypothesis might be that the traditional Internal

Audit structure is appropriate given the size of company and relative reliance on IT. However, a

Financial Services company with 15,000+ employees might need a dedicated, separate IT Audit

function to provide the assurance needed. In some cases, it may not always be sufficient to rely

on the Internal Audit function to discharge the IT Audit function. Empirical studies can provide

the groundwork going forward to establishing not just the reason for, but the value of being for

Internal Audit.

The parallelizing of IT Audit potentially serves as valuable opportunity to develop the

prestige of the IT Audit function. One of the greatest challenges of Internal Audit departments

today is an inability to hire talent with sufficient IT expertise. IT Audit typically finds itself

competing with System Administration, Database Administration, Software Development, and

Operational Cybersecurity roles for talent. Because of the difficulty of transfer from IT Audit to

some of these other positions, IT Audit may not be perceived as the optimal career track for the

technologist. Such perception serves as a ceiling for the talent available in IT Audit. While this

can be supplemented by staff augmentation, there is no substitute for in-house technical talent.

Most companies will not be using staff augmentation for intermediate or long range planning;

and therefore, they may be making crucial mistakes in planning that will reverberate through the

department. Mitigating risk is not possible if Internal Audit is improperly allocating resources.

How might Internal Audit leadership assess resource allocation? An empirical study may provide

answers.

The issue with research in Internal Audit is two-fold. The first is the sizing of the

research market. The Internal Audit Foundation advertises that they have published over 200

10

books and released more than 300 research papers. This amount of scholarly research is not

sufficient if Internal Audit departments are to be considered experts on risk mitigation. The

threats and opportunities for companies today are numerous, sophisticated, and technology-

driven. Executives and Board Members seeking true assurance will turn to trusted advisors. It is

the duty of the auditor to position themselves as subject matter experts on the risks posed

throughout a company: financial, operational, legal, reputational, and technological.

Additionally, many of the research papers supported by the IIA may not have sufficient

rigor for publication in scientific journals. There are issues of distinction between causal

relationships that disqualify many such papers. Namely, some of the IIA papers work from an

initial premise and use statistics to support the initial statement. The conventions of scientific

writing require that an author propose an initial premise and couch such a premise in terms of a

“hypothesis” that may then be further confirmed or disproven when benchmarked against a

control group. Such rigor is a hallmark of reputable scientific study and often missing from the

Internal Audit research landscape. If the Institute is interested in developing the standards of

rigor that scientific study is known for, it is necessary to fund the academic operations that such

research relies on. A strong step in that direction would be to endow Internal Audit Research

chairs at IIA partner institutions. With dedicated chairs, IIA can drive research in a direction that

not only creates new value for organizations with the Internal Audit function, but substantiates

and validates the already present value of Internal Audit. Validation of the value of Internal

Audit serves to develop a positive feedback loop that perpetuates continually increasing research

which further substantiates the value of Internal Audit.

11

IIIb. Cybersecurity

Depending on the exposure a company is perceived to have from cybersecurity, it will

allocate financial and human capital in response. Internal Audit can provide assessments as to

industry best practices, proper assurance of third-party cybersecurity solutions, and specific

security programs. In cooperation with third party vendors and Internal Audit data analytics

teams, companies can identify solutions that provide the most holistically cost-effective response

to the tremendous threat of bad actors in cyberspace.

Many companies have disjointed cybersecurity operations without clearly defined

organizational and functional roles. Internal Audit can and should provide expertise as to the

proper form for functional and organizational reporting to reduce a company’s cybersecurity

attack surface. Organizational misalignment can leave individuals with access to information

without due reason. Functional misalignment can lead to IT practices that unnecessarily expose a

company to external and internal threat actors. For instance, proper access provisioning cannot

occur without well-defined functional roles. Analyzing industry and global best practices in

relation to organizational and functional alignment will mitigate some of the most endemic and

fundamental risks a company faces from cybersecurity. This is the type of effort that can provide

senior management with the assurance they seek.

Employing third party solutions to protect data and IT infrastructure is increasingly

common. Whether these solutions are implemented locally or in the cloud, Internal Audit must

be aware of the total risk exposure to the company posed by these vendors. Regulatory

compliance certifications for third party vendors have been no panacea. Pre- and post-

implementation reviews by Internal Audit could provide a tremendous defense against utilizing

obsolete or ill-fitting tools that provide inadequate protection against intrusion and exfiltration.

12

Verifying appropriate Service Level Agreements (SLA) ensures that the company has recourse if

expectations are not met. This is but one example amongst a myriad of risk mitigation techniques

that can be employed for third party vendors.

Internally-owned security programs are of significant concern to Internal Audit. As most

companies view security programs as cost centers, these internally owned security programs are

often inadequately funded and sources of tremendous risk. A Network or Security Operations

Center (NOC or SOC) is intended to provide front line protection for the company’s IT assets.

These centers are operating 24 hours a day, seven days a week. They are frequently staffed by

entry level employees. Ensuring that incident escalation rules and chains of command are

properly implemented is crucial to their overall effectiveness of these programs. Frameworks

exist to be audited against by Internal Auditors that ensure proper, time-sensitive plans are in

place to mitigate the cyber incident chain. Minutes of response time can be the difference

between an inconsequential and a catastrophic breach. Providing independent assurance for

incident response tools and procedures can help close the gap between an organization and

would-be infiltrators.

For Internal Audit to provide proper assessment of risk vulnerabilities, it is crucial to

have access to subject matter experts, but not sufficient to add the value that Internal Audit must

provide if it will maintain its importance in the future. Even the entry-level, ‘blocking and

tackling’ auditors must have a proper frame of reference in relation to the totality of the risk that

IT poses for all organizations going forward. Businesses, from the smallest independently

owned to the largest transnational corporation, are all vulnerable to the threat of bad actors in the

realm of technology. Whether it’s the small business’s credit card machine being skimmed or

the transnational corporation’s data center being DDoS’ed, the threat posed by cybersecurity can

13

be existential. It must be recognized that though cybersecurity may be only a functional

department in organizations currently, it will increasingly be considered part of the core business

of most organizations. Internal Audit departments can position themselves now to be the primary

guide for a business navigating the cybersecurity landscape.

IIIc. Advanced Analytical Techniques

As companies adopt new technological innovations, the traditional role of the auditor

may change dramatically. Rather than tick marking financial statements, auditors will be

expected to audit technology and its implementations. “… traditional internal auditors simply are

not valued as the trusted advisor of senior management on matters of risk—analytics experts

with the business savvy to ask the right questions are. The traditional auditor is obsolete, and

they generally don’t yet realize it.”11 The intention for computers has always been to automate

human tasks. Many companies are developing tools that automate formerly complex human-

driven audit procedures. For Internal Audit to thrive in the future, many departments will require

individuals to diversify their skillsets to audit automated processes and the technology

surrounding them. The tools for automated auditing are increasingly available. “Over the course

of the now quickly evolving data revolution, data bots will systematically replace knowledge

workers in the audit, compliance, and risk management process… bad news for the traditional

internal auditor.”11 This disruption need not foreshadow the end of Internal Auditing. Rather,

Internal Audit can play a crucial role in verifying appropriate implementation of these tools.

Such disruption also provides an opportunity for Internal Auditors to pivot focus towards value

added services and relationship management, both crucial to developing stakeholder buy-in.

11 http://www.acl.com/2016/05/the-future-of-big-data-risk-analytics-and-obsolescence-of-the-traditional-internal-auditor/

14

Beyond preparing to audit automated technological processes, Internal Audit should, in

many cases, benefit from moving beyond risk analytics into risk data science. Machine Learning

models have proved in recent years to have greater predictive power than simple regression

techniques or naïve classification analytics. Auditors can create more value for executives by

embracing an understanding that the value of Internal Audit to executives is in the ability to

detect or predict future exposure. Clustering algorithms, shallow and deep neural nets, random

forests, support vector machines, association and feature extraction algorithms are all techniques

that a risk analytics team could benefit from utilizing. In fact, it should be concerning to Risk

Management and Internal Audit if companies that have a technological capacity aren’t using

these algorithms operationally. They represent a significant form of competitive advantage that is

almost universal in application. If an auditor is expected to provide assurance on the operational

aspect of said company, it would be a reasonable expectation to have some knowledge of these

algorithms as well. The power of Machine Learning algorithms can be orders of magnitude

greater than less sophisticated statistical techniques previously deployed. It cannot be overstated;

Machine Learning algorithms are a unique technological revolution. Some of the most

sophisticated algorithms deployed by high tech companies have predictive power that exceeds

human experts currently.12 Other technologies may work to make human tasks obsolete,

Machine Learning and data science seek to make human thought obsolete.

In another example of the potential impact, employing these algorithms in the Human

Resources function can expose a company to legal liability if the parameters of the algorithm

drive biases against protected classes. As these algorithms may utilize unsupervised machine

learning, the possibility is a credible and often overlooked threat. Laws do not currently

12 http://www.wired.co.uk/article/ibm-watson-medical-doctor

15

distinguish between whether a human is discriminating or an algorithm. A company can still be

held liable for the biases of recruitment tools. How can Internal Auditors help protect against this

potential threat?

It is not reasonable for all auditors to be expected to understand the inner workings of

such algorithms. However, it is vital for Internal Auditors to understand the limits of the tools in

function and implementation. Internal Auditors must define proper controls to protect companies

against their misuse. Such misuse already undoubtedly occurs and poses grave legal,

reputational, and financial risk to many companies. When an algorithm is more sophisticated in

action than most humans can understand, the algorithm’s use can have surprising and unintended

consequences. Just last year, Microsoft received bad press when a chat bot algorithm used on a

Microsoft twitter feed went rogue and started tweeting offensive and racist remarks.13 Clearly,

pre-implementation review was inadequate to mitigate the risk posed by the algorithm. Not only

did it tweet offensive remarks initially, the algorithm was adjusted and published a second time,

when it proceeded to continue producing offensive comments. This is not an isolated incident.

Other algorithms have been found to make decisions in manners that might pose reputational risk

to a company. The complexity of these algorithms only exacerbates this issue. On a recent

interview with Ari Shapiro, professor Christian Sandvig stated, “The systems are of a sufficient

complexity that it is possible to say the algorithm did it. And it's actually true. The algorithm is

sufficiently complicated, and it's changing in real time. It's writing its own rules on the basis of

data and input that it does do things, and we're often surprised by them.”14 Internal Auditors must

13 https://techcrunch.com/2016/03/24/microsoft-silences-its-new-a-i-bot-tay-after-twitter-users-teach-it-racism/ 14 http://www.npr.org/2016/03/14/470427605/can-computers-be-racist-the-human-like-bias-of-algorithms

16

understand the risks companies are now facing in order to fulfill the reason for which Internal

Audit exists.

IV. Conclusion

It is Internal Audit’s main purpose, its raison d’etre, that it provides reasonable assurance

as to the mitigation of risks posed to an organization. The Internal Audit charter may lay down

specific requirements on reporting and organizational structure, but it is most appropriate that

Internal Audit be functional reporting to the Audit Committee of the Board of Directors or direct

stakeholders of an organization. This ensures that Internal Audit maintains the independence

that is so important to providing true assurance.

The future of Internal Audit is predicated on its embrace of sophisticated tools and

methods espoused by the scientific community. The utilization of empirical, peer-reviewed

studies would improve the perceived rigor of the Internal Audit function. Beyond a philosophical

framework, Internal Audit can employee rigorous scientific frameworks to justify assessments

and increase trust as an advisor to executives. Internal Auditors can develop skills in assessing

cybersecurity that will provide value to companies awash in cybersecurity problems with no

clearly defined solutions. These skills can establish Internal Audit as a primary resource and

trusted advisor in a specific area where many companies are struggling. Advanced analytical

methods, specifically data science algorithms, provide a significant source of risk as well as a

significant tool for Internal Audit to deploy in the department workflow. Gaining knowledge of

and an ability to utilize these tools will allow Internal Audit to provide more meaningful

assurance to stakeholders.

17

Internal Audit has provided significant assurance and comfort to executives and

stakeholders over the course of IIA’s existence. The precarious position that Internal Audit holds

must balance the need for access to operational resources with the requirement of providing true

assurance. New methods and tools can allow Internal Audit to maintain positive relationships

with both stakeholders and employees. These same tools can ensure that Internal Audit

departments remains an organization’s most trusted advisor.