Date post: | 08-Feb-2017 |
Category: |
Technology |
Upload: | chrissanders88 |
View: | 222 times |
Download: | 3 times |
Investigation Theory A Cognitive Approach
Chris Sanders
Chris Sanders (@chrissanders88)
Analyst @ FireEye Founder @ Rural Tech Fund PhD Researcher GSE # 64 BBQ Pit Master Author:
Practical Packet Analysis Applied NSM Investigation Theory Course
Symptoms of a Cognitive Crisis1. Demand for expertise greatly
outweights supply2. Most information cannot be trusted or
validated3. Inability to mobilize and tackle big
systemic issues
Ethnography of the SOC
“An analyst’s job is highly dynamic and requires dealing with constantly evolving threats. Doing the job is more art than science. Ad hoc, on-the-job training for new analysts is the norm."
Sundaramurthy, S. C., McHugh, J., Ou, X., Rajagopalan, S. R., & Wesch, M. (2014). An anthropological approach to studying CSIRTs. Network, 100, 2.
Ethnography of the SOC
“The profession [security] is so nascent that the how-tos have not been fully realized even by the people who have the knowledge…the process required to connect the dots is unclear even to analysts.
Sundaramurthy, S. C., McHugh, J., Ou, X., Rajagopalan, S. R., & Wesch, M. (2014). An anthropological approach to studying CSIRTs. Network, 100, 2.
Symptoms of a Cognitive Crisis1. Demand for expertise greatly
outweights supply2. Most information cannot be trusted or
validated3. Inability to mobilize and tackle big
systemic issues
The Cognitive Revolution1. Understand the
processes used to draw conclusions
2. Develop repeatable methods and techniques
3. Build and advocate training that teaches practitioners how to think
What separates novice and
expert analysts?
Mapping the Investigation Sample:
Novice and expert analysts Methodology:
30+ case studies Stimulated recall interviews Focus on individual investigations of
varying types Perform key phrase analysis – analyze
results
Key Phrase Mapping Dual Process Theory
Intuition: Implicit, unconscious, fast Reflection: Explicit, controlled, slow
IntuitionExperimentation
RestructuringImaginationIncubation
MetacognitionEvaluation
Goal SettingMaking Plans
ReflectionAnalytically
Viewing DataRule-Based Reasoning
Considering Alternatives
Results
Expe
rimen
tation
Restruc
turing
Imag
inatio
n
Incub
ation
Evalu
ation
Goal S
etting
Making
Plan
s
Viewing
Data
Consid
ering
Alter
nativ
es
Novices Experts
Intuition Metacognition Reflection
Analyzing the Flow of
the Investigation
Investigations as Mental Labyrinths
The investigation is the core construct of information security.
How do we study them when everyone has a different toolset? Follow the Data!
Alert
OSINTReputation
File Hash
Sandbox Behaviors
AV Detections
(VT)
Imphash More File Hashes
Friendly Host
Network PCAP
Host
Windows Logs
Security Log
System Log
App LogRegistry
File SystemHostile
Host NetworkPCAP
Flow
Studying the Investigation Process
Studying the Investigation Process
What data did analysts look at first?
72%16%
12%
Observed
PCAP FlowOSINT
Data Suggests: Analysts prefer a higher context data set…
…even if other data sets are available …even if lower context data sets can lead to a resolution.
Did the first move affect analysis speed?
Data Suggests: While PCAP provides richer context, it may slow down the
investigation if that’s where you start Starting with a lower context data source can increase
speed when working with higher context data
PCAP Flow OSINT
16
10 9
Avg Time to Close
What happens when Bro data replaces PCAP?
46%
25%
29%
Observed (Bro)
Bro Flow OSINT
72%
16%
12%
Observed (PCAP)
PCAP Flow OSINT
What happens when Bro data replaces PCAP?
PCAP Flow OSINT
16
10 9
Avg Time to Close (PCAP)
Bro Flow OSINT
10 10 11
Avg Time to Close (Bro)
Data Suggests: Better organization of high context data
sources can yield improvements in analysts performance
What data sources were viewed most and least frequently?
Data Suggests: Network data is used more frequently than host data…
…even when host data can be used exclusively to resolve. …even when easy access is provided to host sources.
Revisting data is more prevalent on higher context data sources
PCAP Flo
wOSIN
T
Host FS
OS Log
s
Memory
Data Sources Viewed Data Sources Revisited
PCAP84%
Flow11%
OSINT5%
How many steps were taken to make a disposition judgement?
Data Suggests: At some point, the number of data sources you
investigate impacts the speed of the investigation Understanding where data exists and when to use it can
impact analysis speed
6-10 11-15 16-20 21-250
5
10
15
6
129
3
Number of Steps
6-10 11-15 16-20 21-2505
1015202530
9 12 14
24
Avg Time to Close
Did analysts investigate friendly or hostile systems first?
9%
91%
Observed
Friendly Hostile
Data Suggests: Analysts are more compelled to investigate unknown external
threats than internal systems Analysts don’t fully understand their own techniques
41%59%
Friendly
Friendly Hostile
Thank You!
Mail: [email protected]: @chrissanders88
Blog: chrissanders.orgTraining:
chrissanders.org/training