Towards Automatic Security Management: A Model-BasedApproach

Qian ChenElectrical and Computer

EngineeringMississippi State University

qc34@msstate.edu

Sherif AbdelwahedElectrical and Computer

EngineeringMississippi State University

sherif@ece.msstate.edu

Weston MonceauxUS Army Engineer Research

and DevelopmentCenter,Vicksburg, MS, USA

Weston.P.Monceaux@erdc.dren.mil

ABSTRACTThis paper introduces a security management approach thatintegrates system monitoring, intrusion detection, and auto-matic control in order to detect, classify and protect againstsecurity attacks automatically. The model-based controllercomputes the most effective control action to protect thesystem based on the system measurements, the current andthe expect level of future system utilization. In this paper,we implemented the security management framework on amulti-tier enterprise system comprising a set of routers, frontvirtual machines and hosts and test it with respect to vari-ous forms of denial of service attacks (DoS). Throughout theexperiment, the security management approach correctly de-tects and protects the system from these attacks. The paperpresents the simulation results and discusses possible exten-sions of the proposed structure for other forms of DoS andnetwork attacks.

Categories and Subject DescriptorsC.2.0 [Security and Protection]: Denial of Service Attack

General TermsManagement, Security

KeywordsDoS attack, Detection, Optimization, Protection

1. INTRODUCTIONIn this paper, an autonomic security management struc-

ture that aims to enhance the availability and robustnessof enterprise systems is introduced. These proposed struc-ture include an online monitor system, a data pre-processingmodule, intrusion detection systems, and a model-based con-troller. Features related to system and network utilizationare collected by various sensors in the online monitor sys-tem. Monitored data are then passed to the pre-processingmodule to extract relevant features for security analysis.The processed data are forwarded to the intrusion detection

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.CSIIRW ’12, Oct30 - Nov 01 2012, Oak Ridge, TN, USACopyright 2012 ACM 978-1-4503-1687-3/12/10 ...$15.00.

system (IDS) which then classifies the type of the currentnetwork flow. The model-based controller uses the systemmodel, the current measurements, the expected system uti-lization levels, and the IDS classification results to calculatethe appropriate action to defend the system against poten-tial attacks and maintain the system’s normal functionality.

The proposed framework can be extended to deal withsecurity problems at the host. We apply the data miningtechnology to build a (offline) training model to classify theattacks in real-time using system measurements. The pro-posed approach prevents known and unknown security at-tacks by analyzing traffic packets at run-time. Fuzzy controltechniques are used to select the best (in terms of cost and ef-fectiveness) protection method based on the current virtualmachine (VM) and host state. The security managementstructure redirects the attack flood to the front VM. At thefront VM, data is processed by the relevant data processingmodule, the intrusion detection system (IDS), and attackclassification module. The separation of these modules fromthe host is intentional to assure that these modules workproperly even when host is compromised. The front VM isindependent of the host which simplifies the setup and de-ployment of the management systems for various platforms.

To verify the applicability of the proposed structure, weimplemented the security management framework on a multi-tier enterprise system comprising a set of routers, front vir-tual machines and hosts. We then applied a set of commondenial of service attacks to the system. Throughout the ex-periment, the security management approach correctly de-tects and protects the system from denial of service attacks(DoS) [16, 19]. We presents the simulation results and dis-cusses possible extensions of the proposed structure for otherforms of DoS and network attacks.

2. RELATED WORKThe current state of the art of router-based IDS algo-

rithms allows fast detection and identification of DoS at-tacks. For example, flowing based DoS detection algorithm[14] detects spoofed IP packets in real time. However, system-based IDS is more powerful to handle deep packet inspec-tion tasks and is widely used to detect and classify networkattacks. The anomaly detection technique is able to dis-tinguish ‘Normal’ flow from ‘Abnormal’ flow. The misusedetection technique classifies ‘Abnormal’ traffic into differ-ent types based on attacks’ signatures. The authors in [7]adopt both the anomaly detection technique and the misusedetection technique. The combination technique increases

the accuracy rate of attack detection three times more thanthe accuracy rate by only implements misuse detection tech-nique. IDS based on neural network mechanism is appliedto detect SCADA injection attacks and DoS attacks [10].Authors in [12] propose a combination of data mining meth-ods and expert systems implemented by WEKA [5] that cangive better IDS performance. Paper [15] focuses on detectingDoS attack in an automated environment based on the dan-ger theory. In this work, the authors advocate a self-learningand self-adapting IDS by combining statistical anomaly de-tection algorithms to detect SYN arrival rate and monitorCPU utilization at run-time.

To protect web servers against DoS attacks, paper [8]lists various prevention methods for UDP, ICMP Flood andTCP SYN attacks. Trusted platform module is executedas an efficient protection method when the server is com-promised [8]. An intrusion prevention system (IPS) whichcombines firewall and IDS is introduced in [9] to detect andrespond to intrusions. The inline mode of Snort providesreal time and active prevention ability. The malicious com-munication packets match the underlying Snort inline ruleswill be dropped [20]. Control theoretic concepts have beenintroduced to detect and protect host system against variousnetwork attacks as well. For instance, Distributed Denial ofService (DDoS) attacks can be protected by adjusting in-coming traffic flow. The traffic flow rate must be lower thangiven thresholds which is altered by the PID controller [18].Paper [17] mentions the controller of host system reduceslow-priority flows’ traffic rate; hence the high-priority flowis able to access limited resources of the host server. Thepriority of the flow is defined by the fuzzy PID controllerbased on sets of rules provided by experts.

Most of the current research on security attacks focuseson detection and classification. A few studies discuss how togather attack detection, attack classification, optimized pro-tection response and automatic system recovery together.Our work takes into account a single organized structurethat contains all modules. This work introduces the securitysystem framework which uses a feedback loop to evaluate thesystem’s state by considering the system’s inputs. The com-promised host server can be protected by various protectionmethods, and the most successful one will be suggested bya fuzzy logic controller.

3. THE MODEL BASED SECURITY MAN-AGEMENT FRAMEWORK

The proposed security management framework is shownin Figure 1. In this framework, real time data are collectedby sensors in online monitor system and passed to the on-line pre-processing filter where irrelevant and outlying dataare eliminated. The IDS module adopts both anomaly andmisuse detection techniques to classify types of suspiciousflows. The protection method selected by the model-basedcontroller to protect the host depends upon the current sys-tem state, system specifications and constraints as well asthe suspicious flow types as classified by the IDS. Addition-ally, the suggested appropriate method must be the opti-mized one for the cost and efficiency among other candidatemethods.

An implementation of the security management frame-work is shown in Figure 2. The developed managementstructure consists of the router, the front VM and the hostserver. In this structure, suspicious flows are forwarded from

the router to the front VM instead of being forwarded to thehost directly. The IDS at the VM detects suspicious flowsand classifies types of them. The model-based controller atthe front VM calculates the optimal method to protect thesystem based on types of attacks, system utilization levels,and host states. The main components of the frameworkand the structure are discussed hereafter.

Figure 1: The main components of the model-basedsecurity management framework

System Monitors: Sensors in the online monitor systemmodule collect real-time data from various parts of the sys-tem. This data includes relevant performance, utilizationand security parameters. Monitors are divided into fourmain classes. The first class is resource monitors that col-lect system resources measurements like CPU and memoryutilization. The second class is the performance monitorsthat collect QoS parameters, such as web service responsetime. Network utilization sensors compromise the third classof monitors called network monitors. The fourth class is aset of security monitors checking the level of system compli-ance in regards to the rules provided by the system admin-istrators, for example, Security Technical ImplementationGuides (STIG) [4] and by the security certificates receivedfrom clients.

In our structure (Figure 2), real-time and fault-tolerantdistributed system monitor (RFDMon) [11] located in thenetwork, the front VM and the host server, monitors pa-rameters of network and system. These parameters includeavailable memory, total bytes received per second, total bytessent per second, total number of packets received per sec-ond, total number of packets sent per second, total numberof read requests per second issued to physical devices, to-tal number of write requests per second issued to physicaldevices, CPU idle time, CPU system time, and CPU usertime.Online Pre-processing Data Filter: In this module, rel-evant system and network parameters are pre-processed todiscretize the data, change its a type, and filtering noisefrom the dataset, etc. Measurements collected by sensorsmay have missing data due to the impact of DoS attackson CPU, memory and network performance. The missingdata are eliminated to provide the appropriate input file forWEKA which is used for classifying types of attacks in theIDS module (described later). In our experiment, we pre-process data by calculating the differences between currentand average values (no attack) of monitored parameters.Intrusion Detection: This module collects and analyzesanomaly and misuse information to detect and classify suspi-

cious flows. There are two IDSs in Figure 2. One is installedon the front VM and the other is on the host. The IDSat the front VM contains an anomaly detector and a mis-use identifier. Naive Bayesian Classifier is implemented tocompute the training model using the open source WEKA.The classifier analyzes the conditional probability for therelationship between both independent and dependent vari-ables [13]. The classification accuracy rate is about 99%since our experiment uses hybrid detection techniques andadopts machine learning techniques that reduce the falsealarm rate. Anomaly IDS at the host compares current datavalues to thresholds of training features set by experts. Inour experiment, when four or more of the ten features’ values(mentioned in Section 3) are higher than their thresholds,the host is considered as ‘Abnormal’. Threshold values andhow the system determines what is ‘Abnormal’ depend uponthe desired web server’s security guidance documents.Model-based Controller: The controller chooses the bestactions to maintain system availability and data integrityunder potential attacks. The system model in the model-based controller uses environment inputs and the mathemat-ical model of the enterprise system. The specifications andconstraints block reflects requirements and restrictions forevaluating protection methods. The performance optimizeraims to achieve the desired specifications by continuouslymonitoring the current system state, as well as selecting therecovery action that satisfies their constraints.

In the developed security management structure (Figure 2),a fuzzy logic controller is used at the front VM to computethe best action that can protect against attacks for the hostserver. The appropriate action is evaluated by consideringthe VM state, the host state and efficiency of the protec-tion process. The recovery unit, one part of the controller,is responsible for installing and configuring proposed pro-tection processes. In this implementation, we discuss sixprotection methods, namely, IPS, port disablement, legalflow filtering, network disconnection, firewall and host shut-down. Protection methods are evaluated by using the fuzzycontrol technique in which the method with the lowest scoreis considered the most efficient action.

Figure 2: The security management structure

4. EXPERIMENT RESULTSTo validate the proposed structure, several DoS attacks

are simulated. The simulation testbed is a web server systembuilt with IBM Web Sphere Application Server Community

Edition with Daytrader [2] to receive client requests sent byHttperf [3]. The experiment setting is outlined as follows:

1. Victim Host (Web Server): Windows XP (VM) onVMWare ESX server, IP address: 1.0.0.2, 1 core pro-cessor, 1GB RAM.

2. Front VM: CentOS 5 (VM), IP address:1.0.0.9, 2 coresprocessor, 4GB RAM.

3. Two Routers: CentOS 5 (VM), one IP address is 1.0.0.1,the second one is 3.0.0.1, 1 core processor, 1GB RAM.

4. Attackers: CentOS 5 (VM), Windows 7 (physical ma-chine), and Windows Server 2003 (VM). IP addresses:3.0.0.X.

5. Client: CentOS 5 (VM), IP address: 3.0.0.7, 1 coreprocessor, 1GB RAM.

Requests sent from clients to the web server vary from 300to 2000 requests per section and are based on user requestworkloads of 1998 World Cup Soccer (WCS-98) [6]. Wesimulated four types of DoS attack which are described inmore details the technical report [1]. In this paper, UDPflood attack is discussed in the following section. Note thatthe sample time of host is 1 second and the sample time ofthe front VM is 2 seconds.

4.1 Known UDP Flood AttackThe variance of the ten system features before (No At-

tack), during (Attack) and after (No Attack) UDP floodattacks of host is shown in Figure 3. Figure 4 shows the fea-tures’ variance of the front VM. Before the attack occurs, thehost server processes client requests accordingly, and the pa-rameters of VM behaves normally as well. The figure of thefront VM demonstrates that the attack is first detected atsample time 40 and ends after sample time 160. In Figure 3,the attack duration is from sample 80 to 320. As the IDSmodule at the front VM correctly classifies suspicious flowsas UDP flood attacks; the IDS at the host machine confirmsthat the state of the host is ‘Normal’. This ‘Normal’ confir-mation means the attack signatures are in training sets. Thesignatures of this type of attack has already learned offline.Attack flows have been redirected to the front VM beforebeing forwarded to the web server by the router.

The controller calculates the rank of the six candidate pro-tection methods, and the best recovery option is the intru-sion prevention system. In this experiment, the Snort_inlineis implemented to filter malicious packets. The IPS dropsthe illegitimate packets while legitimate packets are sent toport 5009 of the host server without any influence, thus theweb server is protected against UDP flood attacks. Tracesin Figure 3 describe the random client requests sent to thehost. CPU utilization, memory utilization and I/O requestsare increased and decreased correspondingly based on in-coming client requests which are represented by sub-figuresByte and Packets Number.

5. CONCLUSION AND FUTURE WORKIn this paper we introduced a novel model-based auto-

nomic security management framework for attack detectionand classification using data mining techniques. Optimalprotection is computed and enabled at the front VM to pro-tect the host server against network attacks. Our approachsupports a wide variety of hardware and operating systemsand provides system protection even under a compromisedweb server. The experiment results demonstrate that the

Figure 3: Features of host for known UDP floodattack with random client requests

Figure 4: Features of front VM for known UDPflood attack with random client requests

proposed approach is able to detect and prevent known UDPflood attacks as well as novel attacks. The novel attacks in-clude unknown DoS attacks such as UDP, TCP SYN, ICMPand Ping of Death attacks. In the future we plan to extendthe structure to support detection, classification and pro-tection against other varieties of network attacks includingworms, viruses, buffer overflow, and SQL injection attack.

6. ACKNOWLEDGEMENTThis work is supported in part by the NSF I/UCRC CGI

Program grant number IIP-1034897 and The Engineer Re-search and Development Center (ERDC) at Vicksburg, MS.

7. REFERENCES[1] http://www.ece.msstate.edu/~qc34/report.pdf.

[2] Apache geronimo v2.0 daytrader. https://cwiki.apache.org/GMOxDOC20/daytrader.html.

[3] httperf documentation. http://www.hpl.hp.com/research/linux/httperf/docs.php.

[4] Stigs home.http://iase.disa.mil/stigs/index.html.

[5] Weka. http://www.cs.waikato.ac.nz/ml/weka/.

[6] M. Arlitt and T. Jin. A workload characterizationstudy of the 1998 world cup web site. Network, IEEE,14(3):30 –37, may/jun 2000.

[7] M. A. Aydin, A. H. Zaim, and K. G. Ceylan. A hybridintrusion detection system design for computernetwork security. Computers & ElectricalEngineering, 35(3):517 – 526, 2009.

[8] Z. Chao-yang. Dos attack analysis and study of newmeasures to prevent. In Intelligence Science andInformation Engineering (ISIE), 2011 InternationalConference on, pages 426 –429, aug. 2011.

[9] S. Chebrolu, A. Abraham, and J. Thomas. Featurededuction and ensemble design of intrusion detectionsystems. Computers Security, 24(4):295–307, 2005.

[10] W. Gao, T. Morris, B. Reaves, and D. Richey. Onscada control system command and response injectionand intrusion detection. In eCrime ResearchersSummit (eCrime), 2010, pages 1 –9, oct. 2010.

[11] R. Mehrotra, A. Dubey, S. Abdelwahed, and K. W.Rowland. Rfdmon: A real-time and fault-tolerantdistributed system monitoring approach. In TheEighth International Conference on Autonomic andAutonomous Systems, St. Maarten, NetherlandsAntilles, 03/2012 2012.

[12] M. N. Mohammad, N. Sulaiman, and O. A. Muhsin. Anovel intrusion detection system by using intelligentdata mining in weka environment. Procedia ComputerScience, 3(0):1237 – 1242, 2011. World Conference onInformation Technology.

[13] W. Y. M. S. Muda, Z. and N. Udzir. A k-means andnaive bayes learning approach for better intrusiondetection. Inform. Technol. J., pages 648–655, 2010.

[14] P. Park, H. Yi, S. Hong, and J. Ryu. An effectivedefense mechanism against dos/ddos attacks inflow-based routers. In Proceedings of the 8thInternational Conference on Advances in MobileComputing and Multimedia, MoMM ’10, pages442–446, New York, NY, USA, 2010. ACM.

[15] S. Rawat and A. Saxena. Danger theory based synflood attack detection in autonomic network. InProceedings of the 2nd international conference onSecurity of information and networks, SIN ’09, pages213–218, New York, NY, USA, 2009. ACM.

[16] R. Richardson. Computer crime and security survey.Director, 15(1):1–42, 2011.

[17] S. Song and C. Manikopoulos. A control theoreticalapproach for flow control to mitigate bandwidthattacks. In Information Assurance Workshop, 2006IEEE, pages 348 –360, june 2006.

[18] M. Tylutki and K. N. Levitt. Mitigating distributeddenial of service attacks using aproportional-integral-derivative controller. InInternational Symposium on Recent Advances inIntrusion Detection, pages 1–16, 2003.

[19] Y. Uchiyama, Y. Waizumi, N. Kato, and Y. Nemoto.Detecting and tracing ddos attacks in the trafficanalysis using auto regressive model. IEICETransactions, 87-D(12):2635–2643, 2004.

[20] J. Xi. A design and implement of ips based on snort.In Computational Intelligence and Security (CIS),2011 Seventh International Conference on, pages 771–773, dec. 2011.