MANAGING CYBER THREATS

Issues, Approaches, and Challenges

MANAGING CYBER THREATS

Issues, Approaches, and Challenges

Edited by

VIPIN KUMAR University of Minnesota, U.S.A.

JAIDEEP SRIVASTAVA University of Minnesota, U.S.A.

ALEKSANDAR LAZAREVIC University of Minnesota, U.S.A.

Springer

Library of Congress Cataloging-in-Publication Data

Managing cyber threats : issues, approaches, and chalienges / edited by Vipin Kumar, jaideep Srivastava, Aleksandar Lazarevic.

p. cm. — (Massive computing) Includes bibliographical references and index. ISBN 0-387-24226-0 (alk. paper) 1. Computer networks—Security measures. 2. Computer security. 3. Data

mining. 4. Computer crimes—Investigation. I. Kumar, Vipin, 1956- II. Srivastava, jaideep. III. Lazarevic, Aleksandar. IV. Series.

TK5105.59.M368 2005 305.8-dc22 2005041303

ISBN-10: 0-387-24226-0 ISBN-13: 978-0387-24226-2 e-ISBN-10: 0-387-24230-9 e-ISBN-13: 978-0387-24230-9

Printed on acid-free paper.

© 2005 Springer Science+Business Media, Inc.

All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, Inc., 233 Spring Street New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now know or hereaftei developed is forbidden.

The use in this publication of trade names, trademarks, service marks and similar terms, even ii the are not identified as such, is not to be taken as an expression of opinion as to whether or noi they are subject to proprietary rights.

Printed in the United States of America.

9 8 7 6 5 4 3 2 1 SPIN 11367147

springeronline.com

TABLE OF CONTENTS

Contributing Authors

Preface

IX

Xl l l

PART I OVERVIEW

1 MANAGING THREATS TO WEB DATABASES AND CYBER SYSTEMS Bhavani Thuraisingham 3

2 INTRUSION DETECTION: A SURVEY Aleksandar Lazarevic, Vipin Kumar, and Jaideep Srivastava 19

PART II DATA MINING BASED ANALYSIS OF COMPUTER ATTACKS

3 LEARNING RULES AND CLUSTERS FOR ANOMALY DETECTION IN NETWORK TRAFFIC Philip Chan, Matthew Mahoney, and Muhammad Arshad 81

vi Table of Contents

4 STATISTICAL CAUSALITY ANALYSIS OF INFOSEC ALERT DATA Wenke Lee andXinzhou Qin 101

5 UNDERSTANDING NETWORK SECURITY DATA: USING AGGREGATION, ANOMALY DETECTION, AND CLUSTER ANALYSIS FOR SUMMARIZATION DaveDeBarr 129

PART III TECHNIQUES FOR MANAGING CYBER VULNERABILITIES AND ALERTS

6 EARLY DETECTION OF ACTIVE INTERNET WORMS Vincent H. Berk, George Cybenko, and Robert S. Gray 147

7 SENSOR FAMILIES FOR INTRUSION DETECTION INFRASTRUCTURES Richard Kemmerer and Giovanni Vigna 181

8 ENCAPSULATION OF USER'S INTENT: A NEW PROACTIVE INTRUSION ASSESSMENT PARADIGM Shambhu Upadhyaya, Ramkumar Chinchani, Kiran Mantha, and Kevin Kwiat 221

9 TOPOLOGICAL ANALYSIS OF NETWORK ATTACK VULNERABILITY Sushil Jajodia, Steven Noel, and Brian O'Berry 247

10 ANALYZING SURVIVABLE COMPUTATION IN CRITICAL INFRASTRUCTURES Yvo Desmedt 267

11 ALERT MANAGEMENT SYSTEMS: A QUICK INTRODUCTION Robert Grossman 281

Table of Contents vii

PART IV CYBER FORENSICS

12 CYBER FORENSICS: MANAGING, MODELING, AND MINING DATA FOR INVESTIGATION Erin Kenneally, and Tony Fountain 295

13 CYBER FORENSICS: ISSUES AND APPROACHES Jau-Hwang Wang 313

CONTRIBUTING AUTHORS

Name Affiliation E-mail address

Muhammad Arshad Department of Computer Sciences

Florida Institute of Technology

Melbourne, FL 32901

marshad@cs.fit.edu

Vincent H. Berk Institute for Security Technology

Studies, Dartmouth College,

Hanover, NH 03755

Vincent.Berkf^dartmouth.edu

Philip Chan Department of Computer Sciences

Florida Institute of Technology

Melbourne, FL 32901

Laboratory for Computer Science

Massachusetts Institute of

Technology, Cambridge, MA 02139

pkc@cs.fit.edu

Ramkumar Chinchani Department of Computer Science and rc27@cse.buffalo.edu

Engineering, University at Buffalo,

Buffalo, NY 14260

George Cybenko Institute for Security Technology

Studies, Dartmouth College,

Hanover, NH 03755

gvc@dartmouth.edu

Dave DeBarr The MITRE Corporation

Bedford, MA 01730

Contributing Authors

debarr@mitre.org

Yvo Desmedt Computer Science Department,

Florida State University,

Tallahassee, Florida FL 32306-4530

desmedt@cs.fsu.edu

Tony Fountain San Diego Supercomputer Center,

University of California San Diego

La JoUa, CA 92093-0505

fountain@sdsc.edu

Robert S. Gray Institute for Security Technology

Studies, Dartmouth College,

Hanover, NH 03755

Robert, Gray @dartmouth. edu

Robert Grossman Laboratory for Advanced Computing, grossman@uic.edu

University of Illinois at Chicago

Chicago, IL 60607

Open Data Partners

Sushil Jajodia Center for Secure Information

Systems, George Mason University

Fairfax, VA 22030-4444

jajodia@gmu.edu

Richard Kemmerer Department of Computer Science,

University of California, Santa

Barbara, Santa Barbara, CA 93106

kemm@cs .ucsb. edu

Erin Kenneally San Diego Supercomputer Center,

University of California San Diego

La Jolla, CA 92093-0505

erinf^sdsc.edu

Vipin Kumar Department of Computer Science and kumar@cs.umn.edu

Engineering, Army High Performance

Computing Research Center,

University of Minnesota

Minneapolis, MN 55415

Kevin Kwiat Air Force Research Laboratory

525 Brooks Road, Rome, NY 13441

kwiatk@afrl.mil

Contributing Authors XI

Aleksandar Lazarevic Army High Performance Computing aleks@cs.umn.edu

Research Center, University of

Minnesota, Minneapolis, MN 55415

Wenke Lee College of Computing

Georgia Institute of Technology

Atlanta, GA 30332

wenke@cc.gatech.edu

Matthew V. Mahoney Department of Computer Sciences

Florida Institute of Technology

Melbourne, FL 32901

mmahoney @cs.fit.edu

Kiran Mantha Department of Computer Science and kmantha@cse.buffalo.edu

Engineering, University at Buffalo

Buffalo, NY 14260

Steven Noel Center for Secure Information

Systems, George Mason University

Fairfax, VA 22030-4444

snoel@gmu.edu

Brian O'Berry Center for Secure Information

Systems, George Mason University

Fairfax, VA 22030-4444

boberry@gmu.edu

Xinzhou Qin College of Computing

Georgia Institute of Technology

Atlanta, GA 30332

xinzhou@cc.gatech.edu

Jaideep Srivastava Department of Computer Science and srivasta@cs.umn.edu

Engineering, Army High Performance

Computing Research Center,

University of Minnesota

Minneapolis, MN 55415

Bhavani

Thuraisingham

National Science Foundation

Arlington, Virginia 22230

The MITRE Corporation

Bedford, MA 01730

bthurais@nsfgov

Giovanni Vigna Department of Computer Science,

University of Califomia, Santa

Barbara, Santa Barbara, CA 93106

vigna@cs.ucsb.edu

xii Contributing Authors

Shambhu Upadhyaya Department of Computer Science and shambhu@cse.buffalo.edu

Engineering, University at Buffalo,

Buffalo, NY 14260

Jau-Hwang Wang Dept. of Information Management jwang@sun4.cpu.edu.tw

Central Police University

Tao-Yuan, Taiwan, ROC 333

Preface

Information technology (IT) has become the engine that drives our modem enterprises within the public and private sectors. Government agencies and businesses have become increasingly reliant on IT systems to carry out important missions and functions and to increase their productivity. However, the very same information infrastructure that has brought a high degree of agility to our society has also created a degree of fragility — which if not remedied can cause serious damage to societal and economic well-being. For example, there have been several incidents (e.g., Code-Red I & II, Nimda, and more recently the SQL Slammer and Blaster worm attacks) of large-scale, distributed denial-of-service attacks in just the last two or three years. The intention of these attacks was not simply to infect a few machines, but to affect large portions of the Internet by shutting down millions of servers and clogging the information "superhighways."

The brunt of these attacks has been borne by those responsible for computer security, and the security research and development community has come to their aid — developing a number of techniques to make it harder to launch attacks. However, this battle is becoming increasingly difficuh as a number of factors are aiding the attackers as well. First, the wide adoption of the Internet by the society at large has increased the number of organizations that can be accessed through a network, making them vulnerable to attacks from anywhere in the world. Second, information systems have become significantly more powerful and more complex during the past decade with an exponential growth in features and associated capabilities. The more complex systems are, the more difficult it is to thoroughly review all of their components and ensure the absence of security holes in them. Finally, since September 11*, 2001, we have discovered that

xiv Preface

there are well-organized groups — backed by the resources of certain govemments — whose express purpose is to cripple the society's information infrastructure.

Against the backdrop described above, there is a need to have a systematic and comprehensive approach to securing the society's information infrastructure, also called the ''cyber infrastructure". Thus, we define cyber threat management (CTM) as the collection of tools, techniques, policies, processes, and practices that are aimed at protecting the cyber infrastructure, and thwarting — both retro- and proactively — attacks against it,

There are a number of challenges to existing tools and techniques for cyber threat management. First, the amount of data being generated from various network-monitoring devices is at a scale that makes human analysis essentially impossible. This requires some form of automated analysis to extract higher-level information from the monitored system, in a form and scale comprehensible to a human analyst. Second, escalating importance of cyber security in our society creates the need for new techniques for managing cyber vulnerabilities and cyber alerts that will help to improve general computer security. Finally, by integrating these new techniques with other security disciplines such as cyber forensics, more complete and comprehensive systems for cyber threat management can be achieved.

The research community must address these and various other issues, to develop tools, techniques, policies, processes, and practices, that will contain the threat against the society's cyber infrastructure, and ensure its smooth functioning. Towards this, there is a need for in-depth analyses and surveys of existing literature — a significant fraction of it carried out by universities and national laboratories, and sponsored by the defense and intelligence communities — which will help refine the societal research agenda in the area of cyber threat management. This book is one such effort towards this goal.

The contributed chapters have been organized into four parts that focus on: (i) overviews of specific sub-areas, (ii) application of data mining to cyber threat management, (iii) techniques for managing cyber vulnerabilities and alerts, and (iv) cyber forensics techniques.

The first part provides two overview articles covering the topics of cyber threats and intrusion detection systems. In Chapter 1, Thuraisingham provides an overview of various cyber threats to information systems as well as to data management systems. These threats include access control violations, unauthorized intrusions, and inference and aggregation. In addition, the chapter also discusses potential solutions and challenges in detecting such cyber threats, which include role-based access control, data mining techniques, and security constraint processing. In Chapter 2,

Preface xv

Lazarevic, Kumar, and Srivastava provide a detailed survey of contemporary intrusion detection techniques. They first provide a taxonomy of computer attacks and describe basic characteristics of specified attack categories. Then, they present a general architecture of intrusion detection systems and give their taxonomy, together with a short description of significant approaches belonging to different intrusion detection categories.

The second part of the book focuses on the applications of data mining techniques for handling cyber attacks. In Chapter 3, Chan, Mahoney, and Arshad propose two anomaly detection techniques that use machine learning models for characterizing normal network behavior. The first method, called LERAD (Learning Rules for Anomaly Detection) is based on a rule learning algorithm that characterizes normal behavior in the absence of labeled attack data. The second method, named CLAD (Clustering for Anomaly Detection), uses a clustering algorithm to identify outliers in network traffic data. In Chapter 4, Lee and Qin describe a novel method for security alert correlation that is based on clustering algorithm followed by causal analysis. This method is used to discover new relationships among attacks. High volume of raw alerts is first reduced by combining low level alerts based on alert attributes, and then clustering techniques are used to group these low-level alert data into high-level alerts. The method is validated on several data sets including DARPA's Grand Challenge Problem (GCP) datasets, the 2000 DARPA Intrusion Detection Scenario datasets, and the DBF CON 9 datasets. DeBarr, in Chapter 5, focuses on the use of data mining/analysis techniques for effective summarization and prioritization of network security data. Event records are aggregated by source address and period of activity in order to reduce the number of records that must be reviewed. Anomaly detection is used to identify obvious host, port, and vulnerability scans, association discovery is used to recognize common sets of events, and cluster analysis is employed to provide a synopsis of distinctive behaviors within a group of interest.

The third part provides different practical and theoretical issues of managing cyber vulnerabilities and alerts. In Chapter 6, Berk et al. present an automated system for early detection of active scanning Internet worms, soon after they begin to spread. The implemented system collects ICMP-T3 (Destination Unreachable) messages from instrumented routers, identifies message patterns that indicate malicious scanning activities, and then identifies scan patterns that indicate a propagating worm. The chapter also examines an epidemic model for worm propagation and presents simulation results that illustrate detection capabilities. In Chapter 7, Kemmerer and Vigna present STAT framework for the development of new intrusion detection functionality in a modular fashion. In the STAT framework, intrusion detection sensors are built by dynamically composing domain-

xvi Preface

specific components with a domain-independent runtime. Each sensor has the ability to reconfigure its behavior dynamically. Dynamic reconfiguration and development of deployed STAT sensors is supported by a component model, called MetaSTAT sensor control infrastructure. The final product of the STAT framework is a highly-configurable, well-integrated intrusion detection infrastructure. Upadhyaya et al. in Chapter 8, propose a novel intrusion detection system that encapsulates the user's intent by querying her or him in a proactive manner. The encapsulated intent serves the purpose of a certificate based on which more accurate intrusion detection decision can be made. The authors present the working system implemented in a university environment. In Chapter 9, Jajodia, Noel, and O'Berry describe a Topological Vulnerability Analysis (TVA) prototype tool that implements an integrated, topological approach to network vulnerability analysis. This tool automates the labor-intensive analysis that is usually performed by penetration-testing experts. The TVA prototype includes modeling of network security conditions and attack techniques (exploits). It also generates a graph of dependencies among exploits, which represents all possible attack paths without having to explicitly enumerate them. In Chapter 10, Desmedt describes a novel methodology to model computer networks as well as information infrastructures. The chapter further proposes techniques that may be used to determine which infrastructures are critical and most vulnerable. The employed methodology is based on the PERT directed graphs. Grossman, in Chapter 11, provides a short overview of alert management systems (AMSs), which are designed to screen events, build profiles associated with the events, and send alerts based upon the profiles and events. This chapter provides a brief overview of the basic AMS architecture, as well as a few examples of such systems.

The last part of the book discusses both legal and technical aspects of employing cyber forensics in real life applications. In Chapter 12, Kenneally and Fountain describe the ongoing project P^ELE (Public-Private-Partnership Enabling Law Enforcement) at the San Diego Supercomputer Center. This project represents a research infrastructure for the management, analysis, and visualization of public and private multidimensional data. In addition, it also covers general legal (federal, law, govemmental) aspects of law enforcement process. Finally, in Chapter 13, Wang introduces the basic terms of cyber forensics to the reader. First, this chapter provides an introduction and motivation for development of this field, and then it introduces the computer forensics process as well as the digital evidence in the computer systems and computer networks.

Threats to the society's cyber infrastructure, and thus to the society as a whole, have never been clearer than they are today. Equally clear are the gaps that exist in the society's ability to protect against them. However, there

Preface xvii

is a need to take stock of what our current level of understanding of the issues is. Specifically, what issues have been addressed, and to what degree have they been successful and unsuccessful?

A book such as this would certainly not be possible without the efforts of a number of people. First, we would like to thank the authors of the chapters for accepting our invitations to present their recent research work in cyber threat management and for adhering to a tight publication schedule. We would also like to thank Angela Burke and Deborah Doherty of Springer for their continuous support throughout this project. Finally, we would like to thank the National Science Foundation, the Army Research Laboratory, and the Rome Labs for supporting the research on cyber security for the editors of this book.

PARTI

OVERVIEW