Date post: | 25-Dec-2015 |
Category: |
Documents |
Upload: | maximillian-cobb |
View: | 215 times |
Download: | 1 times |
EFFECTIVE AND EFFICIENT MALWARE DETECTION
AT THE END HOST
Presentation by Clark Wachsmuth
C. Kolbitsch, P. M. Comparetti, C. Kreugel, E. Kirda, X. Zhou and X. Wang
or: How I Learned to Stop Worrying and Love
the Malware-infested Internet
2
THE PROBLEM
Malware! Ineffective (and/or inefficient) detection
models Can be evaded by fairly simple means by
malware authors, such as using polymorphism, obfuscation or system call reordering
Resource-heavy detectors may be effective, but not efficient enough for average consumer computer
3
PAST IMPLEMENTATIONS
Network-based detection Pros:
Useful for detecting some network malware Modern malware is heavily network-bound
Cons: For network-based malware only Content sniffers thwarted by encrypted data Blending attacks make malicious data match
normal data signatures
4
PAST IMPLEMENTATIONS
Host-based detection Pros:
Has the resources to see complete work of malware programs – not limited to specific host resource
Some pre-emptive strategies Cons:
Code obfuscation and polymorphism can easily bypass methods such as byte signatures while keeping the same functionality
System call sequence based detection, again, easily bypassed by reordering calls or making unused calls
5
PAST IMPLEMENTATIONS
Static analysis-based detection Pros:
More effective due to focus on malware behavior, thus less stymied by obfuscation and polymorphism
Cons: Method itself is difficult to employ Has its own vulnerabilities such as detecting
metamorphic code and runtime packaging Takes a heavy toll on system resources making
it unreasonable for home computer systems
6
PAST IMPLEMENTATIONS
Dynamic analysis-based detection Pros:
Less rigid focus on malware behavior allowing for a more general and broad way of detecting malware
Cons: Can require special hardware for detection
(data tainting) Large associated overhead making it unusable
in the home computer realm
7
THE SOLUTION
Effective and Efficient malware detection, duh! But how?
Effective: Can’t be duped by simple order-changing,
rearranging schemes Doesn’t rely only on known quantities; can detect
unknown running programs No false positives
Efficient: Not incurring a very significant chunk of system
resource overhead
8
THE PLAN
In a sandboxed environment, observe different malware and develop fine-grained models
Efficiently match these models up with the run-time behavior of an unknown program
If a match is found, terminate and eliminate
9
BUT HOW?!
By creating a behavior graph where each node is an “interesting” system call
The nodes store a symbolic expression (simple node) or a program “slice” (complex node) that can calculate the output of the system call
These expressions/slices used to detect if output is the argument of another interesting system call during runtime If found, an edge is created between the
two nodes
10
THE CONTROLLED ENVIRONMENT Uses Anubis (Analyzing Unknown Binaries)
Disassembles instructions (including system calls) and keeps an instruction log
Keeps memory log for instructions that read from memory, where (in memory) the instruction reads and writes
Each bite tainted to detect data dependencies between system calls
Any labels within a branch operation are labeled with the taint of the controlling instruction for control dependencies
11
THE INITIAL BEHAVIOR GRAPH
With all the instructions labeled, an initial graph is creating placing on it all system calls (as nodes)
Edges are created when a dependency is found Using the logs, a recursive backwards trace of
system call arguments is made to determine how the argument’s bits were created
These instructions are gathered into a program slice until either an instruction that can’t be traced further (from the outside) or a value produced by an immediate operand from an instruction or coming from the initialized data segment
12
PROGRAM SLICES FUNCTIONS With the slice, we know how and who
created the argument of the sys call It’s not necessarily the direct program
code, though (unrolled loops won’t match with different sizes)
Each line in binary that appears at least once in slice is marked and appropriate code copied to function. Non-marked lines become nops.
Stack needs fixing because stack creating code often not part of slice (uses instruction log)
13
SIMPLIFYING FUNCTIONS
Yay! We have a function that gives an expected output for a given input
Some functions can be quite long and fairly basic We can optimize it to a smaller symbolic
expression This optimization can have huge overhead
reduction at the end host Other functions aren’t so basic, so we
retain the program code of the function rather than have a symbolic reduction
14
SCANNING END HOST
Scanner monitors running program for sys calls Has admin privileges running is user-mode Assume programs can’t get to kernel
All nodes inactive in initial behavior graph When a system call is made, the scanner checks
graph for inactive nodes of the same type and sees if parent nodes are active If found, checks all arguments from sys calls for simple
functions; defers complex functions for later but allows complex function to hold
If all simple function arguments hold, node becomes active
15
SCANNING END HOST
When do we check the complex functions? When we reach an interesting node
Interesting if it is a security-relevant system call (writing to file system, network or registry, starting new processes)
Also interesting if node has no outgoing edges If complex function holds, the interesting
node is confirmed Otherwise, the node with the complex function
becomes inactive and any subgraph rooted under it becomes inactive as well as the edge being formed
16
MATCHING MALWARE
If an interesting node is confirmed, then the program is matched as malware However, if there is no complex function
dependency, then the graph created is not used to help detect future malware programs
The subgraph created with the interesting node is also a behavior graph that denotes a trait of the particular malware running
17
DETECTION EFFECTIVENESS
Generated behavior graphs for six popular malware families (Table 1)
100 samples of each family wereSelected from the database and the non-
interesting samples were tossed out
50 random samples chosen from remaining bunch to create behavior graphs and train dataset Not all samples could be detected due to
non-interesting behavior and complex function crashes
18
TESTING DATA
19
IS IT EFFECTIVE?
Some were effective and some weren’t so much AV software notoriously bad at classifying malware Confirmed by manual inspection, especially for Agent Restricting samples to 155 known variants yielded
92% effectiveness Also restricted data samples to 108 unknown variants
and still achieved 23% effectiveness, indicating that this method can even detect some unknown variants
This behavior-based method is more general than an AV scanner, therefore requires less graphs than signatures
20
WHAT ABOUT FALSE POSITIVES? Tested on WinXP using IE, Firefox,
Thunderbird, putty and Notepad Yielded no false positives When complex functions were unchecked
and allowed to hold, all of the above yielded false positives
Therefore, system call dependencies are at the root of this method’s success
21
OK, BUT IS IT EFFICIENT?
System setup for testing: WinXP, single-core 1.8Ghz P4 with 1GB RAM Tested using 7-Zip, IE, Visual Studio
22
UMM, DID THAT SAY 40%?
CPU / I/O-Bound tests showed low overhead
Compiling seems quite high at 40% System calls in compiling 5000/sec
compared to 7-zip’s 700/sec Compilation is worst-case scenario Improved symbolic execution engine could
possibly reduce high complex function evaluation of 16.7%
Still performed well for common tasks
23
LIMITATIONS
Authors could use time-triggered behavior or command and control mechanisms to prevent malware behavior during test
A reactive method that only works on running malware But, new graphs can be employed quickly and it can
detect some unknown variants Authors could change algorithms rendering
program slices unusable Changing algorithms is a lot of work and this
method still raises the bar considerably higher for malware authors
24
TECHNICAL CONTRIBUTIONS Developed effective models with
detailed semantic information about the malware family
Created a scanner that efficiently matches the behavior of an unknown, running program against the models by tracking system call dependencies
Experimental evidence that approach is feasible and usable in practice
25
CONCLUSION
Effective? Check With correctly labeled, known variants, a
92% effectiveness was obtained with no false positives
Efficient? Check While compiling was a worst-case scenario,
tasks common to the average end user incurred only a low overhead