The Semantic Gap Challenge

Stealthy Malware Detection Through VMM-Based “Out-of-the-Box” Semantic View Reconstruction

November 2007ACM: Association for Computing Machinery

Authors: Xuxian Jiang-North Carolina State UniversityXinyuan Wang-George Mason Univeristy &

Dongyan Xu-Purdue University

Definition

Semantic: of, pertaining to, or arising from the different meanings

of words or other symbols

Semantics: the study of meanings: the language used to achieve a desired effect on an audience especially through the use of words with novel or

dual meanings

Essential Data/Main Idea

There is a recent trend in malware to equip the software with stealthy techniques to detect, evade and avoid malware detection attempts The fundamental limitation of current host-based anti malware systems is they run inside the host they are protecting. This is called "in-the-box" which makes them vulnerable to counter detection and avoidance by certain malwares.

To fix this limitation, many solutions are using Virtual Machine technologies and placing the malware detection facilities outside of the protected VM bubble. This is called "out-of-the-box". Yet, they gain breaking into to at the cost of loosing the internal semantic view of the host which is enjoyed by the "in-the-box" approach. This causes a technical challenge called the "semantic gap".

Abstract

The paper about the design, implementation and evaluation of VM Watcher and "out of the box" approach that overcomes the semantic gap challenge.

New technique called "guest view casting" Developed to reconstruct internal semantic

views (files, ps and kernel modules) of VM from the outside, rather than typical inside approach.

Abstract

New technique casts semantic definitions of guest OS Data Structures and functions

Puts on the Virtual Machine Monitor (VMM) Level VM state

Semantic view reconstructed from multiple perspectives

Reconstruct these details for system call events (ps, call #, parameters, & return value) in the VM & increases the semantic view.

Abstract

With semantic gap bridged we identify two unique malware detection capabilities:

View comparison-based malware detection: and it's demonstration in rootkit detection

Out of the box deployment of host based anti malware software with improved detection accuracy & tamper resistance

Introduction

Internet malware-rootkits and bots are getting very sneaky and elusive. They hide their presence from detection factilities & anti malware software

Host based anti malwared systems are installed and executed inside the hosts they are monitoring and protecting: “in the box”

This makes the anti malware system visible, tangible, and unavoidable to the malware inside the host

Introduction

Now with Virtual Machine technologies we can use this to our advantage. Use the strong isolation and confines ps inside VM so that even if it's compromised by malware, it will be hard to compromise systems outside the VM

“semantic gap” between the VM view from inside the box vs outside the box

Inside views: ps, files, kernel modules Outside views: memory pgs, registers, disk

blocks

“In the Box” vs “Out of the Box”

VM Watcher

Advantages to both views. VM Watcher-a VMM based “out of the box”

approach overcomes the semantic gap challenge

It starts the Virtual Machine view in a non intrusive manner so it can inspect low level VM states without influencing the VM's execution

“guest view casting” a new technique

Guest View Casting

This new approach reconstructs the VMs internal view: files, dir, ps, and kernel level modules for “out of the box” malware detection

Based on the observation that the guest Operating System of a VM provides all the necessary definitions of guest data structures & functions to construct the VM sematic view & cast them on the VMM level observation

Also externally remake the sematic view of the target Virtual Machine

Design Goals

VM Watcher should not disturb the system state of the VM being monitored

VM Watcher should narrow the sematic gap so that malware detection systems run inside the VM can also run outside the VM

VM Watcher should be generic and applicable to a wide range of existing VMMs.

2 approaches: full virtualization (VMWare, QEMU) & para virtualization (Xen, User Mode Linux)

Enabling Techniques

Non Intrusive VM Introspection: provide low level VM states externally. Non intrusive technique to gain full VM state including registers, memory & disk

Guest View Casting: external reconstruction of the sematic level view of VM thus bridging the semantic gap

Implementation

VM Watcher w/ 4 existing VM's: VMWare, QEMU, Xen & UML. The implemenation details:

Open source VMM: QEMU, Xen & UML. Close source: VMWare only exposes raw disk blocks & raw memory pgs. Open source allows full access to low level VM states and events

Narrowing Semantic Gap

3 unique detection & monitoring capabilities: (i) view comparison: based malware

detection and its demonstration in rootkit detection

(ii) “out-of-the-box” deployment of off-the-shelf anti malware software with improved detection accuracy and tamper-resistance

(iii) nonintrusive: system call monitoring for malware and intrusion behavior observation

Experiments

Evaluation experiments with real-world malware

Includes elusive kernel-level rootkits Demonstrates VMwatcher's practicality and

effectiveness #1: Viewed comparison on volatile states #2: Viewed comparison on persistent states #3: Viewed comparison on both volatile &

persistent states #4: Cross platform malware detection

#1 View comparison on volatile states

Involves Windows kernel FU rootkit. It runs and hides in process w/ PID 336. VMWare running w/ host OS is Scientific Linux 4.4 & guest OS is Windows XP SP2.

Windows cmd shell PID 1080 is c reated and invokes the FU rootkit to hide ps 336. The hidden ps is running SSH. The Windows Task Mgr does not list the SSH client ps indicating that this ps has been hidden

Exposed by VM Watcher external view.

Experiment #1

The small box w/ solid lines indicates the SSHClient.exe ps which is not shown by Win Task Mgr.

VM Watcher can be readily adopted by real world honey pots to detect in the wild rootkit attacks.

Also recent incidents show the same FU rootkit has been actively used to hide the presence of advanced bots

Experiment #1

Ex #3-Adore-ng Rootkit

Advanced Linux kernel rootkit that replaces kernel level function pointers to hide files & ps

Adore-ng infection on a Xen Fedora Core 4 Virtual Machine

Four xterm windows 0: inside the VM where adore-ng kernel mod is loaded w/

backdoor ps PID 1490 1: external view of VM: mounted devices 2: files under the dir /root/demo in the VM 3: current running ps inside VM

Experiment #3- Volatile & Persistant States

Out of the box Malware Detection

Software VMM Guest OS Host OS

Symmanec Anti Virus 10.1.0396 VMWare Server 1.0.1 Windows XP/Red Hat 7.2 Windows XP(SP2)

VMWare Server 1.0.1/Xen 3.0.2-2 Windows XP/Red Hat 7.2 Windows XP(SP2)

VMWare Server 1.0.1/Xen 3.0.2-2 Scientific Linux 4.4

Kaspersky Anti-Virus 5.5 VMWare Server 1.0.1/Xen 3.0.2-2 Scientific Linux 4.4

F-Secure Anti—Virus 5.20 VMWare Server 1.0.1/Xen 3.0.2-2 Scientific Linux 4.4

Frisk F-PROT Antivirus for Linux Xen 3.0.2-2/QEMU 0.8.2 Red Hat 7.2, 8.0, 9.00 Scientific Linux 4.4

McAfee Virus Scan 4.24.0 UML 2.4.24 Red Hat 7.2, 8.0, 9.00 Red Hat

Sophos Anti Virus 4.05.0 QEMU 0.8.2 Red Hat 7.2, 8.0, 9.00 Red Hat

Tripwire 4.05.0 (Open Source) UML 2.4.24 Red Hat 7.2, 8.0, 9.00 Red Hat

ClamAV 0.88.5 (Open Source) UML 2.4.24 Red Hat 7.2, 8.0, 9.00 Red Hat

Windows Defender/Malicious Software Removal Tool

Trend Micros Server Protect for Linux 2.5

Red Hat FC4/Windows XP (SP2)/Red Hat 7.2, 8.0, 9.0

Red Hat FC4/Windows XP (SP2)/Red Hat 7.2, 8.0, 9.0

Red Hat FC4/Windows XP (SP2)/Red Hat 7.2, 8.0, 9.0

Anti-Virus Scanning Time

Summary

VM Watcher is a VMM approach that enables out of the box malware detection

Addresses the semantic gap challenge VM Watcher has stronger tamper resistance by

moving anti malware facilities out of the monitored VM while maintaining a current semantic view of the VM “inside the box” via external semantic view reconstruction

Summary

VM Watcher prototype on Linux and Windows platforms shows it's practical nature and effectivness

The experiments with real world self hiding rootkits demonstrates the power of new malware detection capabilites introducted by VM Watcher

Good/Bad Points

Good points: very concrete experiments shown towards end of the paper that brought it all together

Used a variety of open source & proprietary Operating Systems and current anti virus softwares in experimentations

Bad points:Was not able to discuss Experiments 2 and 4 due to time constraints (me)

Guest view casting Figures were confusing

Good/Bad Points

Vocabulary used was very extensive and advanced

With the technical nature of the paper, the vocabulary used should have been more basic in nature to facilitate better understanding

Had to reread the paper a few times to understand the jist of the paper

Improvements & Future Work

Great experiments were done in relation to malware/rootkit detection

Virtual Machine experimentation was great. Liked the use of open source VM's such as Xen, QEMU, and UML.

Talked about different VM states: full vs para virtualization. Future work with this would be great.

Further discussion of honey pots and “in the wild” rootkit attacks would improve the paper