1
Measuring the Effectiveness of Your Ethics and Compliance Program
SCCE Compliance & Ethics Institute, Chicago, IllinoisSeptember 15, 2014
Michael WardAtlas Compliance Solutions
First things First . .
• What exactly are you trying to measure?
• What does “effectiveness” mean?
• Effectiveness at doing what?
• Who is your audience/stakeholders?
• What are their effectiveness criteria?
2
Why measure program effectiveness?
• Demonstrate achievement of minimum legal requirements
• Track attainment of internal program objectives
• Demonstrate efficient allocation of program resources and/or justify additional resources
• Provide feedback/motivate employees
• Diagnose opportunities for improvement
Key Stakeholders and Criteria
Enforcement Agencies– Priorities:
• Your success in preventing non‐compliant outcomes• Your best efforts in trying to prevent non‐compliance• Your awareness/response to previous non‐compliance
– Criteria:• US Federal Sentencing Guidelines or similar published stds• Adherence to company’s own procedures/policies
– Relevant Metrics:• Program assessment (alignment to FSG elements)• Previous incidents and response• More concerned with effort expended than efficiency
3
Key Stakeholders and Criteria
Company Board and Senior Management– Priorities:
• Success in preventing non‐compliant outcomes
• Prudent balancing of company resources against risk
• Reliable process to identify/escalate issues or risks
– Criteria:• US Federal Sentencing Guidelines or similar stds
• Efficient and prudent allocation of resources
– Relevant Metrics:• Program and risk assessment (alignment to FSG elements)
• Trend and benchmark data (within and outside company)
Measurement Challenges
• Availability of data
• Unintended consequences
• Inconsistent, incompatible and siloed information
• Measuring a negative
• “Black swan” infrequent events
• "Not everything that counts can be counted, and not everything that can be counted counts.”
4
Common Compliance Program Metrics
Hotline– No. of allegations– Types of allegations– Locations/business units– Substantiation rate– Percentage of anonymous reports– Case cycle time– Method/source of report?– Level of offender?– Repeat reporter?– Satisfied reporter?
Not everything that can be counted counts . . .
Program
– No. of FTEs assigned to compliance?
– Compliance budget dollars
– Average annual hours of compliance training/FTE
– Code of Conduct delivery frequency
– Average no. of days since last compliance certification
– Employee perceived value of training
5
9/11/2014 2:43 PM 9
Activity‐Based Indicators Results‐Based Indicators
• Compliance program investment• Total dollars• FTE
• Number of investigations• Number of training sessions conducted• Number of employees trained • Hours of employee training delivered• Mean number of days to resolve an allegation• Number of disciplinary actions
• % of employees who actually understand specific policies• % of employees who observed misconduct• % of employees who fear retaliation• Cost of non‐compliance
• Litigation costs• Claims paid
• % of high risk areas addressed by control improvements• Number of complaints received?• Percentage of complaints anonymous?
Are you measuring effort or results?
9/11/2014 2:43 PM 10
Trailing Indicators Leading Indicators
• Integrity Hotline calls received• Total allegations• Types of allegations• % of anonymous calls• Substantiated allegations• Case cycle time
• Employee disciplinary actions• Claims filed• Days lost to injury
• Risk Assessments• Inherently likely events• Objectively immature controls
• Employee surveys• Report issues w/out fear of retaliation• Awareness of resources• Ethical attitudes• Policy understanding
• Predictive Analytics• Real time flagging of potentially non‐compliant activity
Reporting the past or being proactive?
6
Proactive/Leading IndicatorsCompliance Risk Assessment Process
• Proactive effort to inventory and prioritize compliance risks to organization
• Consistent methodology across risks areas allows comparison to one another and prioritization
• Identification of risk areas with higher likelihood or weaker controls before a serious compliance breach
• Ensures limited compliance budget is consistently applied to reducing the most risk
• Enables board and senior management to fulfill their program oversight responsibilities and see the big picture
• Reduces the consequence of a compliance issue by demonstrating company best efforts to prevent
• Risk assessment is element of FSG Compliance Program Standards
EXAMPLE – Acme Co. Alignment to FSG Standards
Federal Sentencing Guideline Requirement Best Practices Current Acme Co. Action Items
Compliance Program Oversight
Board shall be knowledgeable and exercise reasonable oversight of compliance program. [8B2.1(b)(2)(A)]
A specific senior officer shall be assigned “overall responsibility for the compliance and ethics program.” [8B2.1(b)(2)(B)]
Assign Day‐to‐Day Responsibility for Compliance [8B2.1(b)(2)(C)] “Specific individuals shall be delegated day‐to‐day operational responsibility for the compliance and ethics program.”
Such individual(s) shall report periodically to senior officers and the Board of Directors on the effectiveness of the compliance and ethics program.
Such individual(s) shall be given adequate resources, appropriate authority and direct access to the Board of Directors.
To obtain full benefit of compliance program, individual with operational compliance responsibility shall have formal reporting relationship to Board.
• Audit Committee has direct oversight.• Gov’t consent decrees often separate CCO and General Counsel role. in highly regulated industries.
• Regular compliance content on agenda • Formal independent CCO reporting relationship to Board. • CCO is senior level executive evidencing importance of compliance program.
• Audit Committee has direct oversight.• General Counsel assigned overall responsibility.
• Senior level company executive coordinating ethics and compliance program. • Sufficient financial resources.• Added key personnel to bolster team capabilities.• Compliance program updates are regular agenda item of Audit Committee. • Regular 1:1 meetings of Audit Committee and Compliance Officer.
• No specific gaps to address.
• No specific gaps to address.
12PRIVILEGED & CONFIDENTIAL ATTORNEY‐CLIENT COMMUNICATIONS
7
Federal Sentencing Guideline Requirement Best Practices Current Acme Co. Action Items
Establish policies, standards and internal controls to prevent and detect misconduct and noncompliance.[8B2.1(b)(1)]
Continuing communication and training of compliance program components to employees and third party agents. [8B2.1(b)(4)(A)]
The organization shall take reasonable steps to periodically communicate its standards and procedures through training and other means.
Audience shall include Board members, senior executives, employees and agents/ partners.
• Paper and on‐line Code of Conduct with learning aides and FAQs.• Corporate policy portal with related compliance resources.
• Deliver both on‐line and live training.• Training content should be risk‐based according to role/responsibilities.• Periodic refresher messaging outside of training content.• Business partners and agents should be trained.
• Employee Code of Business Conduct.• Business Partner Code of Conduct distributed to third parties.• Corporate compliance and policy website.• Employee COI Questionnaire. • Gifts/Hospitality policy and reporting.
• Code of Conduct course and additional courses delivered on‐line and translated to local languages. • Live training delivered annually to sales force. • FCPA training made available to resellers. • Compliance standards provided to all resellers, agents and suppliers.
• Refresh of Code of Business Conduct.• Scheduledupdate of FCPA, COI and other policies.• Policy portal update to deliver key policies to mobile devices.
• Implement “just in time” and on demand training program for key compliance processes.
13PRIVILEGED & CONFIDENTIAL ATTORNEY‐CLIENT COMMUNICATIONS
EXAMPLE - Acme Co. Alignment to FSG Standards
Acme Software Co. Compliance Risk Dashboard
Key Legal RisksInherent Risk(FY11)
Controls Rating Residual Risk
FY11 FY12 FY11 FY12
Anti‐Corruption HIGH HIGH HIGH MEDIUM MEDIUM
Antitrust/ Competition MED‐HIGH MEDIUM MEDIUM MEDIUM MEDIUM
Online Services) HIGH MEDIUM MEDIUM MEDIUM MED‐HIGH
Conflicts of Interest HIGH MEDIUM HIGH MEDIUM MEDIUM
Employment HIGH HIGH HIGH MEDIUM MEDIUM
Export Controls HIGH MEDIUM MEDIUM MEDIUM MEDIUM
Government Contracting MED‐HIGH LOW MEDIUM HIGH MEDIUM
Government Relations MEDIUM MEDIUM MEDIUM MEDIUM MEDIUM
Information Privacy: Corporate HIGH MEDIUM MEDIUM MEDIUM MEDIUM
Information Privacy: Products HIGH MEDIUM MEDIUM HIGH MED‐HIGH
Intellectual Property HIGH MEDIUM MEDIUM MEDIUM MEDIUM
Marketing/ Trademarks MEDIUM HIGH HIGH LOW LOW
Records Retention & Information Management (RIM) MEDIUM MED‐LOW MED‐LOW MEDIUM MEDIUM
Revenue Recognition/ Side Letters HIGH HIGH HIGH MEDIUM MEDIUM
Securities/Insider Trading MEDIUM HIGH HIGH LOW LOW
14PRIVILEGED & CONFIDENTIAL ATTORNEY‐CLIENT COMMUNICATIONS
8
Expected Consequence/Impact of Event
Inheren
t Likelih
ood of Event
LOWER HIGHER
HIGHER
LOWER
HIGH
MEDIUM
MED
IUM
1. Anti‐Corruption (FCPA)
2. Antitrust/Competition
3. Online Services
4. Conflicts of Interest
5. Employment
6. Export Controls
7. Government Contracting
8. Government Relations
9. Information Privacy:
10. Intellectual Property
11. Marketing/Trademarks
12. Records Retention & Information
Management (RIM)
13. Revenue Recognition/Side Letters
14. Securities/Insider Trading
2
3
12
5
10
11
8
Compliance Risks
13
14
16
9a
Acme Software Co. Compliance Risk Heat Map
4
7
15PRIVILEGED & CONFIDENTIAL ATTORNEY‐CLIENT COMMUNICATIONS
Mature and optimized.
Key controls present , opportunity to improve.
Assessed Effectiveness of Controls
Controls are immature
Residual Risk:
Risk Assessment Criteria
Likelihood:This section describes the inherent likelihood (likelihood excluding the impact of any controls) of the particular compliance risk by these criteria: The number of opportunities for non‐compliance The degree of individual incentives for non‐compliance The complexity and number of dependencies in achieving compliance The rate of change in the environment (expansion or contraction of
people, processes and systems) Recent regulatory and enforcement trends Observed non‐compliance by similarly situated companies
Severity: The criteria for the expected severity or impact of non‐compliance are: Civil v. criminal enforcement Private v. government enforcement Potential termination or suspension of operations Potential for class action lawsuit Impact on reputation Impact on customers Employee recruiting or retention consequence Increased future cost of compliance
Controls:Standard controls (below) as well as risk specific controls are considered: • Have preventative controls been embedded in the business process? • Has a written compliance standard been provided to employees? • Is there periodic compliance training of the affected employees?• Is a specific person assigned accountability for achieving compliance? • Is there a defined process to respond to detected noncompliance?• Is there a periodic risk assessment by an SME of both the compliance
obligations and the existing controls?
The residual risk rating is the net assessment of the risk to the enterprise from the specific risk. It is the combination of both inherent risk (likelihood and severity) and the assessed state of controls.
Key Processes/Owners: • This section identifies the specific business processes that generate the risk under assessment. • Other primarily control processes that are central to managing the risk should also be identified.
Legal Owner:The Legal owner is typically the company’s legal subject matter expert for the specific risk.
Action Item(s) Owner Status
This section will identify recommended or ongoing risk remediation projects (process changes, policy updates, training initiatives) that are expected to improve the control state or reduce the likelihood of the specific risk.
Compliance Q4 2014
16PRIVILEGED & CONFIDENTIAL ATTORNEY‐CLIENT COMMUNICATIONS
9
Residual Risk:
Anti‐Corruption
Likelihood:The inherent likelihood (excluding the impact of any controls) of an anti‐corruption issue is assessed to be relatively HIGHER. This is due to the level of sales activities to foreign government customers (both traditional government entities as well as education, healthcare and state‐owned commercial entities), the involvement of multiple partners and tiers in channel transactions, and the inherent variability and complexity in software pricing and incentives. The new UK Bribery Act also increases exposure for commercial bribery. Finally, there is potential for criminal liability to Acme for the acts of third party agents.
Severity: The expected severity or impact on the enterprise from non‐compliance in the anti‐corruption area is relatively HIGH. Noncompliance is ordinarily criminally prosecuted both against the company and individual executives. The financial penalties are likely to be in the millions of dollars including disgorgement of any improperly obtained profits. Finally, even the investigation alone would generate great negative publicity and a conviction could result in the company’s suspension disbarment from federal government contracts.
Controls:It is assessed that control state for this risk is HIGH. Acme has a comprehensive third party due diligence process and channel on‐boarding process to mitigate the likelihood and severity of third party risks. Acme has also implemented a comprehensive country gift matrix providing employees with guidance on local legal standards and a gift and hospitality reporting and approval tool for exception management. We are also implementing a supply chain and business partner due diligence process.
Even with the significant improvement of controls, the residual anti‐corruption risk remains MEDUM. The new Dodd‐Frank financial bounties for whistleblowers increases the likelihood of a report and the new UK Bribery law expands attention on commercial non‐governmental corruption.
Key Processes/Owners: • Retention of channel partners and any professional service providers or agents who interact with government.• Any provision of gifts or hospitality to government employees.
Legal Owner:Mark Ericksen
Action Item(s) Owner Status
Continued expansion of on boarding/due diligence process beyond channel partners to others at risk intermediary relationships.
Compliance Ongoing
Revise/update FCPA policy to cover newly prohibited commercial bribery risks.
Compliance Q4 2012
Publish gift/hospitality standards matrix and launch reporting and approval tool.
Compliance Done
HIGHER 2011‐ H
HIGHER 2011‐ H
MEDIUM 2011‐M
HIGHER 2011 ‐ H
17PRIVILEGED & CONFIDENTIAL ATTORNEY‐CLIENT COMMUNICATIONS
Proactive/Leading IndicatorsDiagnostic Integrity Surveys of Employees
• Allows direct measurement of employee attitudes and perceptions
• “Pushed” to employees who might otherwise not report issues or provide feedback
• Anonymity used to encourage candor• Combination of data points allows correlation and other
analytical tools to expand insights• Can be standalone, integrated with general employee
survey, or both.• Allows detection and remediation of undisclosed issues and
attitudes before they become allegations• Supports tailored and targeted response to issues instead
of “one size fits all” uniform approach
10
9/11/2014 2:43 PM 19
Employee Integrity Surveys
– “I have observed misconduct at Company X in the past year.”
• Yes 18.92%
• No 81.08%
– “Did you report the misconduct or raise a concern?”
• Yes 79.27%
• No 20.73%
– “To whom or how did you report the misconduct or concern?”
• My supervisor/manager 34.88%
• Human Resources 27.91%
• Corporate Security 19.38%
• Ethics Line 2.33%
• All other 15.51%
9/11/2014 2:43 PM
20
Reason for Not Reporting (Percentage of Non‐Reporting Explanations)
Employee Integrity Surveys
Don’t know why
Assumed someone else would report
Did not want to get anyone fired
Person who committed it was senior
Not certain it was a violation
Did not think they had enough information
Fear of retaliation
Did not want to become involved
Did not think the company would do anything
Other
Knew the person
Did not think anyone would believe them
11
9/11/2014 2:43 PM
21
Employee Integrity Surveys
Observed Misconduct by TypePercentage of Survey Respondents
22
Employee Integrity Surveys
“I can report unethical behavior or practices without fear of retaliation at Company X”
Response Company Benchmark
– Strongly disagree 1.7% 2.3% – Somewhat disagree 0.9% 4.9%– Disagree 3.0% 3.1%– Neither 5.3% 6.2%– Agree 28.1% 41.7%– Somewhat agree 11.7% 12.5%– Strongly agree 49.3% 29.4%
12
9/11/2014 2:43 PM 23
Employee Integrity Surveys“I can report unethical behavior or practices without fear of retaliation at Company X” By Business Unit
Insufficient
Insufficient
89%
100%
91%
84%
100%
100%
83%
90%
89%
89%
88%
83%
85%
0% 20% 40% 60% 80% 100% 120%
Overall
BU A
BU B
BU C
BU D
Finance
Sales
Engineering
Facilities
APAC
LATAM
Government
EMEA
North America
IT
% Favorable
24
Employee Integrity Surveys
“Senior leaders at my company take appropriate action upon unethical or inappropriate behaviors and practices”
Response Company Benchmark
– Strongly disagree 0.9% 1.7%
– Somewhat disagree 1.2% 3.5%
– Disagree 3.9% 2.3%
– Neither 20.9% 18.8%
– Agree 28.1% 39.0%
– Somewhat agree 14.0% 12.8%
– Strongly agree 30.9% 21.9%
13
25
Employee Integrity Surveys“Senior leaders at my company take appropriate action upon unethical or inappropriate behaviors and practices”
Insufficient
Insufficient
73%
63%
82%
59%
85%
63%
66%
71%
81%
89%
60%
78%
31%
0% 20% 40% 60% 80% 100%
Total
Compliance
Distribution
Finance
HR
Legal
Marketing
R&D
Procurement
Sales Ops
Bus Dev.
Research
Facilities
IT
Security
% Favorable
26
Measuring Across Different Levels“When was the last time [your manager spoke with you] [you spoke with your direct reports] about the Company Code of Conduct or the importance of business ethics?”
Direct Report Response
W/in past week 2.78%
W/in past month 12.17%
W/in past 3 months 16.69%
w/in past 6 months 6.84%
W/in past year 28.62%
Never 32.91%
Manager Response
W/in past week 6.68%
W/in past month 17.87%
W/in past 3 months 24.73%
W/in past 6 months 29.60%
W/in past year 4.51%
Never 16.61%
14
27
“You have just learned from a co-worker that our company is about to acquire another company. There have been recent media accounts describing the target as an “up and coming” company and you were already thinking of buying their stock. Which of the following courses of action is appropriate?
• Personally purchasing stock in the vendor company 4.62%
• Telling family and friends so they may purchase stock
in the vendor company 0.46%
• Neither of the above 79.21%
• I don’t know 15.70%
Measuring Employee Understanding
Potential Key Effectiveness Indictors
Indicators for Likelihood of Non‐Compliance• Number/rate of allegations reported (Hotline/case mgmt system)• Number/rate of instances of misconduct observed (surveys/focus groups)• Fear of retaliation (surveys/focus groups/allegations)• Willingness to seek assistance (helpline/survey/focus groups)• Understanding of policies (surveys/direct testing/focus groups)• Effectiveness of process controls to prevent/detect (risk assessment)
Indicators of Commitment to Ethics and Compliance • Number/rate of deviations from disciplinary stds• Exceptions/non‐completion rate with key procedures/controls
– Third party on‐boarding process– Training completion– Gifts, Travel & Entertainment processes and satndards– Conflict of Interest disclosures & recusal process
• Manager achievement of E&C performance criteria (performance appraisals/employee surveys)
15
Summary• Think before you report
– What am I measuring and why?– Is the data/information reliable?– Who is the audience and what do they need to know?– What is the take away or implication? What should be done? – Keep the main thing the main thing.
• Use charts (but not pie charts!)– Benchmarks (internal and external)– Time series– Internal comparisons (reporting units and geographies)
• Activity‐based v. results‐based metrics• Trailing v. leading indicators• Don’t hoard information
– Use effectiveness indicators to motivate and engage stakeholders.– Business people are competitive.
Resources
• [email protected]• www.atlascompliancellc.com• Metrics Reporting and Display
– Stephen Few, Information Dashboard Design– Stephen Few, Show Me the Numbers– Gene Zelasny, Say It With Charts
• Compliance Metrics– OCEG, Measurement & Metrics Guide
• Compliance Risk Assessments– Jeff Kaplan, Compliance & Ethics Risk Assessment: Concepts, Methods and New Directions (e‐book)
• Employee Diagnostic Surveys– CELC, Risk Clarity (Leslie Altizer)– Ethics Resource Center
• Program and Hotline Benchmarking Data– NAVEX Global– The Network– LRN– CELC
