CIS 5373 Systems Securityehoque/teaching/cis5373-fall...Slide Acknowledgment •Contents are based...

transcript

CIS 5373 – Systems SecurityTopic 3.1: OS Security – Basics of secure design

Endadul Hoque

Slide Acknowledgment

• Contents are based on slides from Ninghui Li (Purdue), John Mitchell (Stanford), Dan Boneh(Stanford)

What security goals does an operating system provide?

• Originally: time-sharing computers: enabling multiple users to securely share a computer

– Separation and sharing of processes, memory, files, devices, etc.

• What is the threat model?

– Users may be malicious, users have terminal access to computers, software may be malicious/buggy, and so on

• Security mechanisms

– Memory protection

– Processor modes

– User authentication

– File access control

• More recent past and present: Networked desktop computers: ensure secure operation in networked environment

• New threat?

– Attackers coming from the network. Network-facing programs on computers may be buggy. Users may be hurt via online communication.

• Security mechanisms

– Authentication; Access Control

– Secure Communication (using cryptography)

– Logging & Auditing

– Intrusion Prevention and Detection

– Recovery 4

• Present and near future: mobile computing devices

• New threat?

– Apps (programs) may be malicious or questionable.

– More tightly connected with personal life of the owner.

• Security mechanisms?

– Isolation of each app.

– Help users assess risks of apps.

– Risk communication. 5

Principles of Secure Design

• Compartmentalization– Isolation

– Principle of least privilege

• Defense in depth– Use more than one security mechanism

– Secure the weakest link

– Fail securely

• Keep it simple

Principles of Least Privilege

• Principle of Least Privilege

– A system module should only have the minimal privileges needed for its intended purposes

• What’s a privilege?

– Ability to access or modify a resource

• Assumes compartmentalization and isolation

– Separate the system into isolated compartments

– Limit interaction between compartments

Monolithic design

System

Network

User input

File system

Network

User device

File system

Monolithic design

System

Network

User input

File system

Network

User device

File system

Monolithic design

System

Network

User input

File system

Network

User device

File system

Component design

Network

User input

File system

Network

User display

File system

Component design

Network

User input

File system

Network

User display

File system

Component design

Network

User input

File system

Network

User display

File system

Example: Mail Agent

• Requirements

– Receive and send email over external network

– Place incoming email into local user inbox files

• Sendmail

– Traditional Unix

– Monolithic design

– Historical source of many vulnerabilities

• Qmail

– Compartmentalized design

Memory Protection: Access Control to Memory

• Ensures that one user’s process cannot access other’s memory– fence

– relocation

– base/bounds register

– segmentation

– paging

– …

• Operating system and user processes need to have different privileges

15Reading: http://flylib.com/books/en/4.270.1.46/1/

CPU modes (aka Processor modes or privilege)

• System mode (privileged mode, master mode, supervisor mode, kernel mode)

– Can execute any instruction

– Can access any memory locations, e.g., accessing hardware devices,

– Can enable and disable interrupts,

– Can change privileged processor state,

– Can access memory management units,

– Can modify registers for various descriptor tables

Reading: http://en.wikipedia.org/wiki/CPU_modes

CPU modes (aka Processor modes or privilege)

• User mode

– Access to memory is limited,

– Cannot execute some instructions

– Cannot disable interrupts,

– Cannot change arbitrary processor state,

– Cannot access memory management units

• Transition from user mode to system mode can only happen via well defined entry points, i.e., through system calls

Reading: http://en.wikipedia.org/wiki/CPU_modes

System Calls

• Guarded gates from user mode (space, land) into kernel mode (space, land)– use a special CPU instruction (often an

interruption) to transfer control to predefined entry point in more privileged code

– allows the more privileged code to specify where it will be entered as well as important processor state at the time of entry

– the higher privileged code, by examining processor state set by the less privileged code and/or its stack, determines what is being requested and whether to allow it

Reading: http://en.wikipedia.org/wiki/System_call

Kernel space vs User space

• Part of the OS runs in the kernel model

– known as the OS kernel

• Other parts of the OS run in the user mode, including service programs (daemon programs), user applications, etc.

– they run as processes

– they form the user space (or the user land)

• Why is root user more powerful than normal users?

Privilege Levels

• Security is often achieved by running control/protection code at a higher privilege level

• Components running at the same level can be isolated by a higher-privilege component

• If attack and defense are at the same level, then it is an arms’ race and there can be no guarantee

ISOLATION – THE CONFINEMENT PRINCIPLE

OS Security

Running untrusted code

• We often need to run buggy/unstrusted code:

– programs from untrusted Internet sites:• apps, extensions, plug-ins, codecs for media player

– exposed applications: pdf viewers, outlook

– legacy daemons: sendmail, bind

– Honeypots

• Goal: if application “misbehaves” ⇒ kill it

Approach: Confinement

Confinement: ensure misbehaving app cannot harm rest of system

Can be implemented at many levels:

– Hardware: run application on isolated hw (air gap)

⇒ difficult to manage23

air gap network 1Network 2

app 1 app 2

– Virtual machines: isolate OS’s on a single machine

Virtual Machine Monitor (VMM)

OS1 OS2

app1 app2

– Process: System Call Interposition

Isolate a process in a single operating system

Operating System

process 2

process 1

– Threads: Software Fault Isolation (SFI)

• Isolating threads sharing same address space

– Application: e.g. browser-based confinement

Implementing Confinement

• Key component: reference monitor

– Mediates requests from applications• Implements protection policy

• Enforces isolation and confinement

– Must always be invoked:• Every application request must be mediated

– Tamperproof:• Reference monitor cannot be killed

• … or if killed, then monitored process is killed too

– Small enough to be analyzed and validated

System call interposition

• Observation: to damage host system (e.g. persistent changes) app must make system calls:

– To delete/overwrite files: unlink, open, write

– To do network attacks: socket, bind, connect, send

• Idea: monitor app’s system calls and block unauthorized calls

• Implementation options:

– Completely kernel space (e.g. GSWTK)

– Completely user space (e.g. program shepherding)

– Hybrid (e.g. Systrace)

Linux ptrace: process tracing

process calls: ptrace (… , pid_t pid , …)

and wakes up when pid makes sys call.

Monitor kills application if request is disallowed29

OS Kernel

monitoredapplication

(browser)monitor

user space

open(“/etc/passwd”, “r”)

• If app forks, monitor must fork

– Forked-monitor monitors forked-app

• If monitor crashes, app must be killed

• Monitor must maintain all OS state associated with app

– current-working-dir (CWD), UID, EUID, GID

– When app does “cd path” • monitor must update its CWD

• otherwise: relative path requests interpreted incorrectly

cd(“/tmp”)open(“passwd”, “r”)

cd(“/etc”)open(“passwd”, “r”)

Complications:

Virtual Machines

Virtual Machine Monitor (VMM)

Guest OS 2

Guest OS 1

Hardware

Host OS

VM2 VM1

VMM security assumption

VMM Security assumption:

– Malware can infect guest OS and guest apps

– But malware cannot escape from the infected VM

• Cannot infect host OS

• Cannot infect other VMs on the same hardware

Requires that VMM protect itself and is not buggy

– VMM is much simpler than full OS

… but device drivers run in Host OS32

Problem: Covert channels

• Covert channel: unintended communication channel between isolated components

– Can be used to leak classified data from secure component to public component

Classified VM Public VM

secretdoc

listenercovert

channel

An example covert channel

• Both VMs use the same underlying hardware

• To send a bit b ∈ {0,1} malware does:

– b = 1: at 1:00am do CPU intensive calculation

– b = 0: at 1:00am do nothing

• At 1:00am listener does CPU intensive calc. and measures completion time

– b = 1 ⇒ completion-time > threshold

• Many covert channels exist in running system:

– File lock status, cache contents, interrupts, …

– Difficult to eliminate all34

VMM introspection – protecting IDS/AV systems

Intrusion detection / Anti-virus

• Runs as part of OS kernel and user space process

– Kernel root kit can shutdown protection system

– Common practice for modern malware

• Standard solution: run IDS system in the network

– Problem: insufficient visibility into user’s machine

VMM introspection – protecting IDS/AV systems

Better: run IDS as part of VMM (protected from malware)

– VMM can monitor virtual hardware for anomalies

– VMI: Virtual Machine Introspection

• Allows VMM to check Guest OS internals

Infected VMma

Guest OS

Hardware

Subversion of VMI is possible

Coming up

• Access control

CIS 5373 Systems Securityehoque/teaching/cis5373-fall...Slide Acknowledgment •Contents are based...

Documents