Butterflies, Black Swans, and Beautiful Security Metrics
Curtis ColemanVisiting Assistant Professor
Cybersecurity Program DirectorOffice: PEC-230B
Phone: 405.425.5472Cell: 405-990-1842
Week 1
DAY 1
- Who am I?
- The Butterfly Effect
- The Black Swan Metaphor
- 10 Lessons from Last Year’s Black Swans
- My Black Swan - 2015
- Beautiful Security Metrics
Who am I?
The Butterfly Effect
• Tiny changes within a complex system lead to results that are impossible to predict
• The flapping of a butterfly’s wings could create tiny changes in the atmosphere that lead to violent weather conditions elsewhere on the planet.
• In 1961 due to weather prediction modeling performed by meteorologist Edward Lorenz. He found that changes that should have been statistically insignificant led to completely different weather scenarios.
Image Source: http://www.intekworld.com/butterfly-effect/
Imagine . . .
• Sr. Network Engineer Connor gets an email with an attachment from a LinkedIn friend telling him about an exciting Network Engineering Conference in Las Vegas. He opens the attachment.
• The attachment is an Adobe PDF flyer of an upcoming Software Defined Network Forum offered at the conference. Connor is already planning to attend the forum. He closes the PDF flyer and goes back reading his emails.
• His computer is now running a Gh0st RAT application that connects to a site on the Internet that is used by the bad guys to control his computer.
APT . . . Stealthy, Data Focused, Targeted
Top 10 Personal Cyber Hygiene
1. Choose your password carefully2. Regularly update your software3. Make regular backups4. Secure your WiFi access (use VPN on public WiFi)5. Secure your mobile device (password/biometric,
encryption)6. Protect your Personal Information and Data (2FA, SSL,
AES-256 encryption)7. Be care when using email; know how to spot
phishing, don’t click on unknown links/files8. Download software from official sites9. Use a personal firewall10. Install anti-Malware protection
The Black Swan Metaphor
The Black Swan: The Impact of the Highly Improbable, by Nassim Taleb
• Prior to the discovery of the Australian
black swan, everyone in the world
assumed all swans were white
• The importance of the black swan -
something that falls outside the realm of
regular expectations -- is that a single such
sighting can invalidate a general statement
(i.e. "All swans are white.").
• What we don't know is frequently more
important than what we do know (or think
we know). There is compelling evidence
that people -- especially so-called experts -
- tend to overestimate what they know and
underestimate the uncertainty that is
derived from those things they don't know.
Thus the need for metrics!
Image Source: https://www.amazon.com/dp/B00139XTG4/ref=dp-kindle-redirect?_encoding=UTF8&btkr=1
My Black Swan - 2015
Image Source: https://krebsonsecurity.com/2016/03/seagate-phish-exposes-all-employee-w-2s/
Top 10 Lessons of 2017-2018 Cybersecurity Black Swans
1. System patches were not properly maintained2. Poor vetting of 3rd Party security3. Need limitations and alerts on bulk download of data4. Did not monitor databases for abnormal behavior and
violations of Segregation of Duties5. Sensitive data was poorly handled6. Poor change management control resulted in
accidental exposure7. Password policy was not enforced8. Did not close terminated accounts9. Poor management of service accounts and shared
password accounts10. Poor Cyber Situational Awareness Program
Cyber Situational Awareness Program
October is National Cyber Security
Awareness Month
1. Week 1: Oct. 1–5: Make Your Home a Haven for Online Safety
2. Week 2: Oct. 8–12: Millions of Rewarding Jobs: Educating for a Career in Cybersecurity
3. Week 3: Oct. 15–19: It’s Everyone’s Job to Ensure Online Safety at Work
4. Week 4: Oct. 22–26: Safeguarding the Nation’s Critical Infrastructure
How do we discover what we don’t know?
How do we discover the unknowns before they become Black Swans?
Image Source: https://lexfun4kids.com/iceburg
The Power of Measuring
How do we discover the unknowns before they become Black Swans?
When you can measure what you are speaking about, and
express it in numbers, you know something about it; but when
you cannot measure it, when you cannot express it in numbers,
your knowledge is a meager and unsatisfactory kind; it may be
the beginning of knowledge, but you have scarcely, in your
thoughts, advanced to the state of science.
—William Thomson, Lord Kelvin, 1883
Possible Operational Metrics
How do we discover the unknowns before they become Black Swans?
Summary and Trends
• Number of exposures fixed
• Number of Incidents prevented
• Systems down due to security incidents (and time to fix)
• Security operating efficiency
• Technical Compliance
• Successful attacks (not initially detected)
• Unsuccessful attacks (detected and blocked)
Incidents
• Total number of incidents per reporting period
• Total number of incidents per fiscal year
• Total time to detect and mobilize
• Impact of sever incidents (time and cost to repair, impact on business)
• Systems un-available due to security incidents
Risk and Compliance• Top 10 exposures• Top 10 emerging threats• Likelihood of impact• Projected cost of impact• Projected Remediation Costs• Time since last assessment• Time since last content update
Projects• Projects % complete & period target
Financial• Cost of all controls (capitol and
operational)• Cost of doing nothing• Efficiency / Modified ROI
What the CISO wants to know –
These are high-level metrics that
show security performance against
desired levels of service. Each
security manager will need to
formulate what they need to measure.
Conceptual Security Balance Scorecards
How do we discover the unknowns before they become Black Swans?
Conceptual Security Balance Scorecards
How do we discover the unknowns before they become Black Swans?
Conclusion
• You must look beyond conventional modes of defense to achieve a security posture that is dynamic, not static.
• This requires:
1. Cyber Situational Awareness
2. Measurements
3. Information Sharing with Management & Employees
• The audience take-away includes:
1. Use National Cyber Security Awareness Month to launch your own company Cyber Situational Awareness program
2. A Pragmatic Operational approach to Cybersecurity Metrics
3. 2 Model Security Balance Scorecards for information sharing
“Praise be to the LORD my rock, who trains my hands for war, and my fingers for battle.” Psalm 144:1