IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 20, NO. 5, SEPTEMBER 2012 1343

Clinical Decision Support and Closed-Loop Controlfor Cardiopulmonary Management and IntensiveCare Unit Sedation Using Expert Systems

Behnood Gholami, Member, IEEE, James M. Bailey, Wassim M. Haddad, Fellow, IEEE, andAllen R. Tannenbaum, Fellow, IEEE

Abstract—Patients in the intensive care unit (ICU) who requiremechanical ventilation due to acute respiratory failure also fre-quently require the administration of sedative agents. The need forsedation arises both from patient anxiety due to the loss of personalcontrol and the unfamiliar and intrusive environment of the ICU,and also due to pain or other variants of noxious stimuli. Whilephysicians select the agent(s) used for sedation and cardiovascularfunction, the actual administration of these agents is the responsi-bility of the nursing staff. If clinical decision support systems andclosed-loop control systems could be developed for critical caremonitoring and lifesaving interventions as well as the administra-tion of sedation and cardiopulmonary management, the ICU nursecould be released from the intense monitoring of sedation, allowingher/him to focus on other critical tasks. One particularly attractivestrategy is to utilize the knowledge and experience of skilled clini-cians, capturing explicitly the rules expert clinicians use to decideon how to titrate drug doses depending on the level of sedation. Inthis paper, we extend the deterministic rule-based expert systemfor cardiopulmonary management and ICU sedation frameworkpresented in [1] to a stochastic setting by using probability theoryto quantify uncertainty and hence deal with more realistic clinicalsituations.

Index Terms—Bayesian networks, cardiopulmonary manage-ment, decision support, expert system, intensive care unit (ICU)sedation, respiratory management, rule-based expert system.

I. INTRODUCTION

M ODERN control technology is having a revolutionaryimpact in modern medicine through medical robotics

(stereotactical brain surgery, implant fitting, and coronary pro-cedures), electrophysiological systems (pacemakers and auto-matic implantable defibrillators), life support (ventilators and

Manuscript received June 08, 2011; accepted July 12, 2011. Manuscript re-ceived in final form July 12, 2011. Date of publication August 18, 2011; date ofcurrent version June 28, 2012. Recommended by Associate Editor D. E. Rivera.This work was supported in part by the U.S. Army Medical Research and Ma-terial Command under Grant 08108002 and by NIH under Grant NAC P41RR-13218 and Grant NAMIC U54 EB005149.B. Gholami and A. R. Tannenbaum are with the Schools of Electrical

and Computer and Biomedical Engineering, Georgia Institute of Tech-nology, Atlanta, GA 30332-0150 USA (e-mail: behnood@gatech.edu;tannenba@ece.gatech.edu).J. M. Bailey is with the Department of Anesthesiology, Northeast Georgia

Medical Center, Gainesville, GA 30503 USA (e-mail: james.bailey@nghs.com).W.M. Haddad is with the School of Aerospace Engineering, Georgia Institute

of Technology, Atlanta, GA 30332-0150 USA (e-mail: wm.haddad@aerospace.gatech.edu).Digital Object Identifier 10.1109/TCST.2011.2162412

artificial hearts), and medical imaging (image-guided surgeryand therapy). An additional area of medicine that can benefitenormously from systems and control oriented ideas is clin-ical pharmacology, in which mathematical modeling plays aprominent role [2]–[8]. This is particularly true when dealingwith critically ill patients in the intensive care unit (ICU) orin the operating room. These patients often require administra-tion of drugs to regulate key physiological variables, such aslevel of consciousness, heart rate, blood pressure, ventilatorydrive, etc., within desired targets. The rate of administration ofthese drugs is critical, requiring constant monitoring and fre-quent adjustments. Open-loop control by clinical personnel canbe tedious, imprecise, time-consuming, and sometimes of poorquality. Hence, the need for closed-loop control (active control)in medical drug delivery systems is significant, with the poten-tial for improving the quality of medical care as well as cur-tailing the increasing cost of health care.One of the main drawbacks in developing active control-

based drug delivery systems is the lack of accurate mathemat-ical models for characterizing the dynamic behavior of drugson physiological variables. System nonlinearities, model pa-rameter variations from patient to patient, as well as parametervariations within the same patient under different conditionsmake it very challenging to develop models and effective con-trol law architectures for active drug delivery systems. Standarddata-driven system identification techniques may not be appli-cable to complex biological system modeling involving in situdiagnostics. This challenge is even more pronounced in ICU se-dation modeling, where sensors may not give reliable measuresof sedation levels. As an example, there have been a number ofinvestigations of processed electroencephalogram (EEG) mon-itoring (all using the bispectral index (BIS) monitor) of ICU pa-tients and the results have been inconsistent [7], [9], [10].Patients in the ICU who require mechanical ventilation due

to acute respiratory failure also frequently require the admin-istration of sedative agents. The need for sedation arises frompatient anxiety due to the loss of personal control and the unfa-miliar and intrusive environment of the ICU. In addition, painor other variants of noxious stimuli frequently require adminis-tration of anxiolytic and analgesic drugs for patient comfort. Inparticular, the interface between the patient and the ventilator istypically an endotracheal tube passing through the oropharynxand into the trachea. Due to the powerful gag reflex, this tubeis very noxious. Without sedation patients can become danger-ously agitated, risking dislodgement of life support devices in

1063-6536/$26.00 © 2011 IEEE

1344 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 20, NO. 5, SEPTEMBER 2012

the worst case and, in any case, resulting in stress that is eth-ically unacceptable and also physiologically unacceptable dueto deleterious increases in heart rate, blood pressure, and workof breathing.Sedation of mechanically ventilated patients in the ICU is an

important and challenging problem with ethical, clinical, andfinancial implications. At the ethical level, we have a self-evi-dent moral imperative to provide adequate anxiolysis and anal-gesia for patients in the ICU. From the clinical perspective, it isimportant that this be done without either overdosage or under-dosage as either may have undesirable clinical effects. At thefinancial level, sedation of patients in the ICU requires large in-vestments of health care provider time, with a commensurate fi-nancial cost, while inefficient titration of sedation and analgesiamay prolong ICU length of stay.While physicians select the agent(s) used for sedation, the ac-

tual administration of these agents is the responsibility of thenursing staff. The ICU nurse has one of the most task-laden jobsin medicine and titration of the sedative drug dose to achieve theoptimal levels of sedation can be a difficult and time consumingtask. If clinical decision support systems and closed-loop con-trol systems could be developed for critical care monitoring andthe administration of sedation, the ICU nurse could be releasedfrom the intense monitoring of sedation, allowing her/him tofocus on other critical tasks.In clinical practice the dose of sedative agent is varied, or

titrated, to achieve the desired level of sedation. The level ofsedation is currently based on clinical scoring systems. One ex-ample is the Motor Activity Assessment Score (MAAS) [11] inwhich patients are given an integer score of 0–6 as follows: 0)unresponsive; 1) responsive only to noxious stimuli; 2) respon-sive to touch or name; 3) calm and cooperative; 4) restless andcooperative; 5) agitated; and 6) dangerously agitated.To implement closed-loop control in an acute environment,

control of cardiovascular function also needs to be addressedalong with sedation since hemodynamic management and con-trol of consciousness are interrelated. For example, a major sideeffect of cardiac surgery is that patients can become hyperten-sive [12], requiring treatment to prevent cardiac dysfunction,pulmonary edema, myocardial ischemia, stroke, and bleedingfrom fragile sutures. Although drugs are available for treatingpostoperative hypertension, titration of these drugs to regulateblood pressure is often difficult. Underdosing leaves the patienthypertensive, whereas overdosing can reduce the blood pressureto levels associated with shock.Although blood pressure control is important, cardiovascular

function involves several other important variables, all of whichare interrelated [12]. The ICU clinician must ensure not onlythat blood pressure is within appropriate limits but also that car-diac output (i.e., the amount of blood pumped by the heart perminute) is acceptable and that the heart rate is within reason-able limits. Closed-loop control of blood pressure has been in-vestigated in clinical studies [13]. However, the fact that thisframework has not been widely adopted by clinicians is notsurprising when one considers the complex interrelationshipsamong hemodynamic variables [6].Since cardiovascular and central nervous system functions

are critical in the acute care environment, technologies have

Fig. 1. Closed-loop sedation control architecture.

evolved for their measurements. The challenge for extendingfeedback control technology to the problem of sedation of crit-ically ill patients, however, is finding the appropriate perfor-mance variable for control. Hence, the first step in the devel-opment of closed-loop control of sedation is the discovery ofan objective, continuously-measurable parameter that correlateswith clinician assessment of the level of sedation. Once sucha parameter is discovered and validated, it then becomes nec-essary to use the measure of sedation for the titration of drugdose. One particularly attractive strategy is to utilize the knowl-edge and experience of skilled clinicians, capturing explicitlythe rules expert clinicians use to decide on how to titrate drugdoses depending on the level of sedation [1]. In this paper, weextend the rule-based expert system approach for cardiopul-monary management and ICU sedation presented in [1] to a sto-chastic setting by using probability theory to quantify systemuncertainty and hence deal with more realistic clinical situa-tions.

II. CLOSED-LOOP SEDATION CONTROL ARCHITECTURE

In this section, we present a closed-loop feedback expertsystem architecture for ICU sedation control. The closed-loopsystem is composed of the controller, the plant (patient), andthe plant output measurement block (i.e., sedation assessmentblock). Within our sedation control framework, the plant (pa-tient) is a dynamical system with unknown dynamics, where theinput is the sedative drug dose and the output is the patient be-havior. Patient behavior refers to patient’s level of sedation andanalgesia, manifested through facial expression, gross motormovement, pain, agitation, blood pressure, and heart rate. Thegoal of the sedation assessment feedback block is to monitorthe patient’s behavior, and objectively assess the sedation levelbased on one of the clinical scoring systems (e.g., MAAS).The input to the controller is the desired level of sedation, andthe objective assessment of sedation provided by the sedationassessment block. The closed-loop system is shown in Fig. 1.The current clinical practice in the ICU involves human expertassessment of patient’s level of sedation (corresponding to thesedation assessment block), and titration of the correct dose ofsedatives (corresponding to the controller).Closed-loop control of ICU sedation is virtually nonexistent

in the literature. However, control algorithms have been de-veloped, simulated, and implemented for the related problemof closed-loop control of general anesthesia. The first of thesehave focused on the control of inhalation anesthesia and severaladaptive control algorithms have been proposed; see [14]–[21]and the references therein. These algorithms have been shownto provide superior control of general inhalation anesthesia in

GHOLAMI et al.: CLINICAL DECISION SUPPORT AND CLOSED-LOOP CONTROL FOR CARDIOPULMONARY MANAGEMENT AND ICU SEDATION 1345

simulations and animal studies. However, they are not directlyrelevant to the specific problem of ICU sedation since the con-trolled variable is the end-tidal anesthetic concentration. It isnot possible with current technology to rapidly measure theplasma concentration of the intravenously administered drugscommonly used for ICU sedation. Thus, drug concentration isnot a viable control variable. Furthermore, drug concentration,even if it could be measured rapidly, is not the best control vari-able. We are far more interested in drug effect than concentra-tion.One approach to closed-loop control of sedation is to design

a system that processes the information currently used by themedical staff and mimics the human process of decision makingfor ICU sedation. Such a system can be equipped with varioussensors, including the bispectral index (a derivative of the EEGsignal) monitor [22], [23], actigraph (accelerometer for mea-suring hand and leg movement) [24], [25], and digital imaging(for measuring facial expression and gross motor movement)[7], [26], [27]. In a recent study, machine learning methodshave been used to assess the level of pain in patients using fa-cial expressions and analyze the correlation between computerand human expert pain intensity assessments [28], [29]. Withmeasurements provided by different viable sensors, an expertsystem can be designed which mimics expert human actions andfollows a similar decision making process.Finally, it is important to note that temporal variability in the

pharmacological response of individual patients and limitationsin the reliable and reproducible effect of the medications usedpose a significant challenge for the problem of ICU sedationcontrol. In point of fact, this is exactly why “fixed” sedationdosing schemes do not work, and ongoing assessment of thedepth of sedation and, then, titration to desired effect is neces-sary. We postulate that clinical outcomes will be improved if asystematic probabilistic expert system framework for sedationmanagement is developed.

III. INSTRUMENTATION FOR CLINICAL PHARMACOLOGY

The sensors used in the ICU to monitor patient status includethose that measure hemodynamic status, respiratory status, renalfunction, and central nervous function. Hemodynamic status ismost typically assessed by continuous monitoring of heart rateand electrocardiograph (ECG). The ECGmeasures the electricalpotential difference between skin electrodes placed at varioussites on the torso and limbs, and can be analyzed to providecontinuous heart rate measurement as well as identify signs ofcardiac dysfunction. Hemodynamic function is also assessedusing blood pressure measurements. While this may be doneusing noninvasive methods, it is most typically done by placinga small plastic catheter directly into an artery (most often theradial artery as it passes through the underside of the wrist) andthen using a pressure transducer to convert the pulse pressurewave into an electrical signal. In a similar fashion, catheters arealso often placed into large central veins (such as the internaljugular vein) so that their tips are situated close to the entry ofthe main veins (superior vena cava or inferior vena cava) re-turning blood to the heart. Pressure waves in these veins are thentransduced into electrical signals to provide the central venous

pressure. This gives an indirect measure of the volume of bloodin the heart which is a major determinant of cardiac output, thevolume of blood pumped by the heart per minute.In some situations in which there is more profound cardiac

dysfunction, a pulmonary artery catheter is placed. This is acatheter that runs through the heart into the pulmonary artery(i.e., the artery going from the heart to the lungs) and can mea-sure pressures in the pulmonary artery (another indirect mea-sure of volume in the heart) as well as directly measure car-diac output. Finally, it is important to monitor the adequacy ofblood flow to the various tissues of the body. One common tech-nique is to measure the amount of oxygen in venous blood. Ifthe delivery of oxygen to tissue decreases, then there will be agreater relative extraction of oxygen from the delivered bloodby the tissue, and hence, the venous blood returning to the heartwill have less oxygen in it. This is most typically measured asthe percentage of hemoglobin molecules (the primary carrier ofoxygen in the blood) that are bound to oxygen (referred to asthe venous saturation).The purpose of respiration is to eliminate carbon dioxide from

and deliver oxygen to the blood. Hence, the most importantmonitors of respiratory function are measures of carbon dioxideand oxygen in the blood. With the most commonly used sensortechnologies these are not directly measurable; however, it ispossible to continuously measure hemoglobin oxygen satura-tion, the percentage of hemoglobin in arterial blood that is boundto oxygen, using absorbance spectroscopy and light emittingdiode technology. In addition, many ICUs use continuous anal-ysis of gas exhaled from the lungs to measure end-tidal carbondioxide concentration, an indirect and approximate measure ofblood carbon dioxide concentrations. Furthermore, modern me-chanical ventilators are equipped to measure the pressure usedto expand the lungs when the patient is undergoing mechanicalventilation, as well as respiratory rate.Assessment of renal function is not as sophisticated as ei-

ther hemodynamic or respiratory monitoring. Currently renalfunction is most typically assessed by the continuous measure-ment of urine output. Sensors for assessment of central nervoussystem function are currently in their infancy, at least as far asroutine clinical use is concerned.

IV. PROBABILISTIC EXPERT SYSTEM FOR CARDIOPULMONARYMANAGEMENT AND ICU SEDATION CONTROL

In this section, we introduce a probabilistic expert systemfor cardiopulmonary management and ICU sedation control. Aknowledge-based system, and, in particular, an expert system, isa computer program that is capable of making deductions basedon the information provided by the user and the informationstored in its knowledge base. In other words, a knowledge-basedsystem is a system which applies a “rules of thumb” approachto a symbolic representation of knowledge [30]. The main char-acteristic that distinguishes a knowledge-based system from aconventional computer program is its structure [31]. Expert sys-tems are a subclass of knowledge-based systems, where theirobjective is to emulate the human expert behavior [31], [32].Expert systems in general deal with two different types of

problems: deterministic versus stochastic. As a result, expert

1346 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 20, NO. 5, SEPTEMBER 2012

systems belong to one of the two general classes of: 1) deter-ministic expert systems and 2) stochastic expert systems. De-terministic expert systems are also referred to as rule-based ex-pert systems due to the fact that in such systems the deductionprocess is based on a series of rules [32]. A more challengingset of problems is that involving uncertainty in knowledge andin the problem variables. Stochastic expert systems specificallydeal with such problems and different frameworks exist to ad-dress uncertainty including certainty factors [33], fuzzy logic[34], theory of evidence [35], and, more recently, probabilitytheory [32]. In the probabilistic approach, a joint probabilitydistribution function over the set of variables is defined and theinference is based on probability rules. Such expert systems arereferred to as probabilistic expert systems.In [1], a rule-based expert system is introduced for cardiopul-

monary management and ICU sedation. One of the limitationsof the rule-based expert system proposed in [1] is its inabilityto deal with uncertainty. More specifically, it assumes perfectaccuracy in the measurement of present and previous MAASscores, blood pressure, and heart rate. While current technologyallows for high accuracy measurements of blood pressure andheart rate, the MAAS score, which quantifies the level of seda-tion and agitation of the patient, is subjective and can result in in-consistencies and variability in sedation administration. More-over, in a rule-based expert system there is no uncertainty as-sociated with the rules. A more general approach would allowfor rules with multiple conclusions, where a different level ofuncertainty is associated with each conclusion. In this section,we use probability theory to quantify uncertainty to extend therule-based expert system given in [1] to deal with more realisticsituations.In the Bayesian interpretation of probability, as opposed to the

classical interpretation, the probability of an event is an indica-tion of the uncertainty associated with the event rather than itsfrequency [36]. In the probabilistic approach to expert systems,the system variables are regarded as random variables and, incontrast to rule-based expert systems, probabilistic expert sys-tems do not possess “if-then” rules but rather the relationshipbetween the variables is defined using a joint probability dis-tribution [32]. If the joint probability distribution of a proba-bilistic model is known, probabilities associated with differentsituations can be computed using marginalization and proba-bility conditioning [37].A drawback of the probabilistic approach to expert systems

is computational complexity. The computational complexity in-creases with the increase in the number of random variables andthe number of possible values they can take. This increase is ex-ponential in the number of random variables.Bayesian networks[38] (also known as belief networks) is a graphical framework inmachine learning which exploits the conditional independencebetween variables to reduce the computational complexity ofthe probabilistic model.Before stating the main results of this section, we need the

following definitions.Definition 4.1 [39], [40]: A directed graph is a pair ,

where is the set of vertices andis the set of edges. Every edge

, corresponds to an ordered pair of vertices

, where and are the initial and terminalvertices of the edge . In this case, is incident into andincident out of is the parent of and is the child of. Moreover, is the set of all

parents of . A directed path from to is a set ofdistinct vertices such that

. A directed path is closed if . A cycleis a nontrivial closed path where all the vertices (except for thefirst and last) are distinct. A directed acyclic graph is a directedgraph containing no cycles.For the next definition, and denote the probability

density function and the conditional probability density functionoperators, respectively.Definition 4.2 [38]: Let , where

, is a random variable and takes on values from aset . A Bayesian network is an orderedpair , where is a directed acyclic graph,

is the set of edges, and is the set characterizingthe probabilistic relationship between the vertices (random vari-ables) and is defined by

(1)

where is the parent index set of, and is the cardinality operator.

Note that a Bayesian network defines a uniquejoint probability distribution over given by

(2)

where , and where, for simplicity of expo-sition, we denote the conditional probability density function

by .Moreover, a Bayesian network represents the causal relation-ships between different random variables. More specifically, if

, then directly influences(causes) . This interpretation of an edge between two ver-tices is crucial in the construction of the Bayesian network. Eachrandom variable is either observed (i.e., its value is known), orhidden (i.e., its value is unknown). In the graphical represen-tation of a Bayesian network, the vertices corresponding to theobserved random variables are shaded. Given a Bayesian net-work and the set of observed random variables, the inferenceinvolves finding the posterior probability distribution of any setof random variables given the observed random variables bymarginalizing the joint probability distribution. An advantage ofBayesian networks is that they reduce the computational com-plexity of the inference stage. For a more comprehensive dis-cussion on Bayesian networks, see [36]–[38].In this section, we use a Bayesian network framework

to design a probabilistic expert system for cardiopulmonarymanagement and ICU sedation control. We first start by con-structing a Bayesian network for ICU sedation control. Next,we extend the Bayesian network to control patient hemody-namics. Let represent the Bayesian network andlet the patient’s current MAAS score, previous MAAS score,blood pressure, heart rate, and required drug dose for sedation

GHOLAMI et al.: CLINICAL DECISION SUPPORT AND CLOSED-LOOP CONTROL FOR CARDIOPULMONARY MANAGEMENT AND ICU SEDATION 1347

Fig. 2. Graph of a Bayesian network capturing the relationships between thecurrent MAAS score , previous MAAS score , blood pressure ,heart rate , and required drug dose .

be given by the random variables , and ,respectively, where

,where denotes the set of positive scalars, and, for a givenfunction . Note that there are 12distinct actions (primary action) given in the first part of theconclusion of each rule in Table I in [1], and hence, we haveassigned a unique number to each distinct action. The graphfor this Bayesian network is given in Fig. 2. The current andprevious MAAS scores, blood pressure, and heart rate, whichconstitute the inputs to the expert system and directly influencethe required drug dose, are observed and their correspondingvertices are shaded in Fig. 2.A potential problem associated with the Bayesian network

given in Fig. 2 is its inability to capture the uncertainty associ-ated with the measurement of the MAAS scores. In particular,in order to perform a meaningful inference, the exact values ofthe current and previous MAAS scores should be known (ob-served). However, as discussed earlier, the assessment processis highly subjective, and the assessed scores can involve a highdegree of uncertainty. A closer examination of the current andprevious MAAS scores reveals that these random variables areessentially hidden variables, that is, they are “driven” by otherfactors. The MAAS score reflects the patient’s agitation and se-dation level, which can be observed through facial expressions,gross motor movement, guarding (i.e., a response in which thepatient withdraws from a potentially noxious stimulus), heartrate and blood pressure stability, noncardiac sympathetic sta-bility, and nonverbal pain scale. These observed factors can beregarded as random variables taking on values from appropriatesets. For example, machine learning techniques can be used toclassify photographs based on the patient’s facial expressionsinto pain and non-pain classes, which in turn can be used to as-sess pain intensity on a scale from 0 to 100 [28], [29]. Hence, amore complete model for the probabilistic expert system shouldinclude these observed random variables as well.Let the random variables represent the cur-

rent objective assessment of the facial expression, grossmotor movement, guarding, heart rate and blood pres-sure stability, noncardiac sympathetic stability, and non-verbal pain scale, respectively, and let repre-sent the previous objective assessment of these variables.Moreover, let ,

Fig. 3. More general graph of a Bayesian network capturing the relationshipbetween the MAAS score and other observable factors; namely, current andprevious objective assessments of facial expression , gross motor move-ment , guarding , heart rate and blood pressure stability ,non-cardiac sympathetic stability , and non-verbal pain scale .

where 0, 1, and 2 denote, respectively, a relaxed face, gri-macing and moaning face, and grimacing and crying face;

, where 0, 1, and 2 denote,respectively, lying quietly, cautious movement, and restlesswithdrawal; , where 0, 1,and 2 denote, respectively, lying quietly, splinting and tense, andrigid and stiff; , where 0, 1,and 2 denote, respectively, stable, moderate change, and markedchange; , where 0, 1, and 2denote, respectively, warm and dry skin, flushed and sweaty, andpale and sweaty; and ,where 0 and 10 denote, respectively, no pain and extreme pain.The graph of the Bayesian network which includes these new

random variables is given in Fig. 3. Note that the current andprevious MAAS scores are no longer observed, and hence, arenot shaded. It is worth noting here that the graph representedin Fig. 3 corresponds to a Bayesian network of a probabilisticexpert system and it is not aimed at modeling the interactionsbetween the variables involved in ICU sedation. The Bayesiannetwork capturing the actual interaction of these variables hasa different dependency structure, and hence, its correspondinggraph would be different from the graph given in Fig. 3.The Bayesian network corresponding to the graph given

in Fig. 3 can be used to determine the proper drug dosefor ICU sedation. Specifically, the joint probability dis-tribution , where

, and , canbe computed using the relationship given in (2); namely

(3)

1348 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 20, NO. 5, SEPTEMBER 2012

The probability distribution of the drug dose suggested by theBayesian network is given by

(4)

where we use marginalization to eliminate and , and, and from the probability density functions in the

numerator and denominator of (4), respectively. Note that, which captures the drug dosing pattern

of the medical staff, can be determined through statisticaltechniques (e.g., maximum likelihood estimates [37]) andclinical data collection. In addition, the probability distribu-tions and capturethe relationship between the facial expression, gross motormovement, guarding, heart rate and blood pressure stability,noncardiac sympathetic stability, and nonverbal pain scale andthe MAAS score, which also requires clinical data collection.The prior probability distributions over , anddenoted by , and , respectively,as well as the conditional probability distributions

, and canalso be determined by statistical techniques.Given the probability distribution of the drug dose suggested

by the Bayesian network , dif-ferent strategies for choosing the drug dose can be used. Onesuch strategy involves selecting the drug dose corresponding tothe mode of the distribution given by

(5)

where denotes the drug dose suggested by theBayesian network.Finally, note that the Bayesian network can also be used

to compute the probability distribution of the drug dose (andhence, the suggested drug dose) when only partial observationsare available. In particular, the posterior probability distributionon the drug dose can be computed when the observed variablesare a subset of the observed variables in Fig. 3. Partial observa-tion can result from sensor failure, where a particular state of thepatient is unavailable at the time of a decision. For example, ifonly the blood pressure, heart rate, facial expression, and grossmotor movement data is available,gives the probability distribution for the drug dose based onthese partial observations.The same probabilistic methodology can be used to account

for a secondary action (see Table I in [1]) in the Bayesian net-work. In this case, appropriate random variables have to be de-fined and the graph given in Fig. 3 should be modified accord-ingly. Alternatively, a hybrid probabilistic-deterministic expertsystem can be defined, where the primary actions given by thefirst part of the conclusions in Table I in [1] are described bya Bayesian network and the secondary action is given by Table

Fig. 4. Gsraph of a hybrid probabilistic-deterministic Bayesian network wherethe random variable controls the activation of the hemodynamic control expertsystem.

II in [1]. Specifically, define the switching random variable ,where . The random variable acts as aswitch, where denotes that the hemodynamic controlexpert system (HDCES) (see Tables I and II of [1]) is off-lineand denotes that the HDCES is activated. The acti-vation could be probability-based where the HDCES is acti-vated if , where

is the conditional probability operator, andis a threshold value. Note that based on Table I in [1], the

activation of the HDCES depends on the current and previousMAAS scores, blood pressure, heart rate, and the required drugdose for ICU sedation. The graph of the hybrid Bayesian net-work is shown in Fig. 4.

V. PROBABILISTIC ALARM ALGORITHM FOR CRITICALCARE MONITORING

A potentially key application of the probabilistic expertsystem developed in Section IV is its applicability to clinicaldecision support, critical care monitoring, and lifesaving in-terventions. A clinical decision support system is a computerprogram that can directly provide the medical staff with assess-ments and recommendations in the clinical decision makingprocess [41]. A clinical decision support system can be coupledto a closed-loop control system to provide a hierarchical hybridcontrol architecture characterized by continuous-time controlalgorithms at the lower-level units and logical decision-makingunits at the higher-level of the hierarchy. In particular, ahybrid controller would involve the clinician evaluating thepatient through a decision support system and an autonomousclosed-loop controller adjusting the desired regimen to maintainsedation at a desired level. This controller architecture allowsfor the expert system to directly aid in clinical decision makingas well as critical care monitoring and lifesaving interventions.In this section, we use the framework presented in Section IV

to design an alarm algorithm for agitation detection in ICU pa-tients. An alarm system refers to an automatic warning systemthat constantly monitors a specific state of the patient and no-tifies the medical staff in case of an abnormality [42]. An ag-itation detection alarm system can reduce the medical staff’s

GHOLAMI et al.: CLINICAL DECISION SUPPORT AND CLOSED-LOOP CONTROL FOR CARDIOPULMONARY MANAGEMENT AND ICU SEDATION 1349

Fig. 5. Graph of a Bayesian network of an alarm algorithm for critical caremonitoring.

workload as well as safeguard against life-threatening situationsin the ICU.To design an alarm algorithm for agitation detection, let

the patients’s facial expression, gross motor movement,guarding, heart rate and blood pressure stability, noncardiacsympathetic stability, nonverbal pain scale, and patient agi-tation be given by the random variables , and ,respectively, where

, and . Here,and denote, respectively, absence of agitation

corresponding to an MAAS score of 0–3 and presence of rest-lessness and agitation corresponding to an MAAS score of 4–6.The graph of the Bayesian network for the alarm algorithm isgiven by Fig. 5. Note that the random variables areconsidered as observed variables, and hence, are shaded.The joint probability distribution for this network is given by

(6)

where , and. In addition, the posterior probability of the patient’s

agitated state is given by

(7)

The patient’s agitation state is based on the observation of fa-cial expression, gross motor movement, guarding, heart rate andblood pressure stability, noncardiac sympathetic stability, andnonverbal pain scale and is given by

(8)

where we select the agitation state with the highest probability.To elucidate the efficacy of our proposed approach we apply

our framework to a retrospective study involving recorded se-dation and agitation data for 366 patients admitted to the ICU inNortheast Georgia Medical Center, Gainesville, GA, over theperiod of May 6, 2009 to April 27, 2010. The patient’s ageranged from 18 to 90 years. The length of stay in the ICU rangedfrom 1 to 93 days. In addition, the available data set included

TABLE ICOMPARISON OF THE HUMAN-ASSESSED PATIENT AGITATION WITH THEPREDICTED AGITATION STATE GIVEN BY THE ALARM ALGORITHM

15 052 measurements of facial expressions, gross motor move-ment, guarding, heart rate and blood pressure stability, noncar-diac sympathetic stability, nonverbal pain scale, and agitationstate. We used the hold-out method [36] for validation of thealarm algorithm, where 12 000 measurements of the randomvariables and were used to train the Bayesian net-work. The algorithmwas tested on the remaining 3052measure-ments of .We used the MATLAB version R2008a and the Bayesian

Network MATLAB Toolbox [43] to compute the posteriorprobability distributions of the patient’s agitation state. In thetraining stage, we used the maximum likelihood estimates ap-proach to estimate the probability distributions in (6) and useda uniform Dirichlet prior to avoid zero conditional probabilitiesfor cases not present in the training data set [37]. In addition, inthe testing stage, we used the junction-tree inference algorithm[36]. Table I gives the predicted agitation state givenby (8) as compared to the human-assessed patient agitation.Based on the results, when the algorithm predicted the presenceof patient agitation, in 18.9% of the test cases the patient wasnot agitated (false positive rate of 18.9% with a 95% confidenceinterval of 14.4% to 24.4%). In addition, when no agitation wasreported by the algorithm in 4.5% of the test cases the patientwas experiencing agitation (false negative rate of 4.5% witha 95% confidence interval of 3.8% to 5.4%). The confidenceinterval is calculated based on the framework presented in[44]. In 45 cases out of a total of 3 052 cases the algorithmwas undecided; that is, the posterior probability of the patient’sagitation state was uniform.

VI. CONCLUSION

In this paper, we presented an approach for designing clinicaldecision support and closed-loop control systems for cardiopul-monary management and sedation control in an ICU using ex-pert systems. It is important to note that expert systems are al-ready in widespread use in other branches of medicine, moreprominently in disease diagnosis, where the system inputs arethe patient’s details and symptoms, and the system outputs areprobable diagnoses, recommended treatments or drugs whichmay be prescribed. Such systems are typically open-loop andmay be regarded as rule-based search engines to help the clini-cian in his/her mapping of a given set of symptoms to a possiblecause (disease).Here, we are proposing to close the loop in a very specific

sedation and cardiovascular function scenario using a set ofheuristics in combination with Bayesian networks. A majorchallenge is the system identification aspect of the problem,that is, identifying a reasonable system model in case the plantis the patient. In contrast to more conventional identificationtechniques (e.g., sine sweeps), here the result must be moresubjective but still very useful. Nevertheless, putting problems

1350 IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, VOL. 20, NO. 5, SEPTEMBER 2012

in drug administration in a closed-loop control framework hasthe strong potential for making the therapies more robust andthus much less amenable to human error.The framework proposed here is just the first step in such a

program. Future work will involve the development of objectiveassessments for ICU sedation using gross motor movement, fa-cial expression, and responsiveness to stimuli. In particular, wewill explore the relationship between gross motor movement,facial expression, and responsiveness to stimuli, and the MAASscore in the form of probability distributions which are best de-termined by clinical data collection. In addition, the drug dosagesuggested by the expert system will be compared to the drugdose prescribed by various experienced clinicians.

REFERENCES

[1] B. Gholami, J. M. Bailey, W. M. Haddad, and A. R. Tannenbaum,“Closed-loop control for cardiopulmonary management and intensivecare unit sedation using expert systems,” in Proc. IEEE Conf. DecisionControl, 2010, pp. 3742–3747.

[2] J. A. Jacquez, Compartmental Analysis in Biology and Medicine.Ann Arbor, MI: Univ. Michigan Press, 1985.

[3] Y. Cherruault, Mathematical Modelling in Biomedicine. Reidel: Dor-drecht, 1986.

[4] F. S. Grodins, Control Theory and Biological Systems. New York:Columbia Univ. Press, 1963.

[5] D. S. Riggs, Control Theory and Physiological Feedback Mecha-nisms. Baltimore, MD: Williams and Wilkins, 1970.

[6] J. M. Bailey andW.M. Haddad, “Drug dosing control in clinical phara-macology: Paradigms, benefits, and challenges,” Control Syst. Mag.,vol. 25, pp. 35–51, 2005.

[7] W. M. Haddad and J. M. Bailey, “Closed-loop control for intensivecare unit sedation,”Best Pract. Res. Clin. Anaesth., vol. 23, pp. 95–114,2009.

[8] W. M. Haddad, V. Chellaboina, and Q. Hui, Nonnegative and Com-partmental Dynamical Systems. Princeton, NJ: Princeton Univ.Press, 2010.

[9] S. A. Nasraway, E. C. Wu, R. M. Kelleher, C. M. Yasuda, and A. M.Donnelly, “How reliable is the bispectral index in critically ill patients?A prospective, comparative, single-blinded observer study,” Crit. CareMed., vol. 30, no. 7, pp. 1483–1487, 2002.

[10] L. E. Simmons, R. R. Riker, S. Prato, and G. L. Fraser, “Assessingsedation during intensive care unit mechanical ventilation with the bis-pectral index and the sedation-agitation scale,” Crit. Care Med., vol.27, pp. 1499–1504, 1999.

[11] J. Devlin, G. Boleski, M. Mlynarek, D. Nerenz, E. Peterson, M.Jankowski, H. Horst, and B. Zarowitz, “Motor activity assessmentscale: A valid and reliable sedation scale for use with mechanicallyventilated patients in an adult surgical intensive care unit,” Crit. CareMed., vol. 1, pp. 1271–1275, 1999.

[12] J. H. Levy, L. G. Michelsen, J. S. Shanewise, J. M. Bailey, and J.G. Ramsay, “Postoperative cardiovascular management,” in CardiacAnesthesia, J. Kaplan, Ed., 4th ed. Philadelphia, PA: WB Saunders,1999.

[13] J. B. Slate and L. C. Sheppard, “Automatic control of blood pressure bydrug infusion,” IEE Proc. A: Sci. Meas. Tech., vol. 129, pp. 639–645,1982.

[14] D. W. Hawes, J. A. Ross, D. C. White, and R. T. Wloch, “Servocontrolof closed circuit anesthesia,” Brit. J. Anaesth., vol. 54, pp. 229–230,1982.

[15] D. R. Westenkow and W. S. Jordan, “The Utah system: Computer con-trolled anesthesia delivery,” Future Anesthesia Delivery Syst., vol. 8,pp. 221–233, 1984.

[16] R. G. Ritchie, E. A. Ernst, B. L. Late, J. D. Pearson, and L. C. Sheppard,“Closed-loop control of an anesthesia delivery system: Developmentand animal testing,” IEEE Trans. Biomed. Eng., vol. 34, pp. 437–443,1987.

[17] J. A. Spain, T. C. Janett, and E. A. Ernst, “The Alabama automatedclosed-circuit anesthesia project,” Future Anesthesia Delivery Syst.,vol. 8, pp. 177–183, 1984.

[18] R. Vishnoi and R. J. Roy, “Adaptive control of closed-circuit anes-thesia,” IEEE Trans. Biomed. Eng., vol. 38, no. 1, pp. 39–47, 1991.

[19] G. I. Jee and R. J. Roy, “Adaptive control of multiplexed closed-circuitanesthesia,” IEEE Trans. Biomed. Eng., vol. 39, pp. 1071–1080, 1992.

[20] C. H. Ting, R. H. Arnott, D. A. Linkens, and A. Angel, “Migratingfrom target-controlled infusion to closed-loop control in general anaes-thesia,” Comput. Meth. Prog. Biomed., vol. 75, pp. 127–139, 2004.

[21] M. Mahfouf, “Constrained closed-loop control of depth of anaesthesiain the operating theatre during surgery,” Int. J. Adapt. Control SignalProcess., vol. 19, pp. 339–364, 2005.

[22] P. S. Sebel, E. Lang, I. J. Rampil, P. White, R. C. M. Jopling, N. T.Smith, P. S. Glass, and P. Manberg, “A multicenter study of bispec-tral electroencephalogram analysis for monitoring anesthetic effect,”Anesth. Analg., vol. 84, no. 4, pp. 891–899, 1997.

[23] P. S. Glass, M. Bloom, L. Kearse, C. Rosow, P. Sebel, and P. Man-berg, “Bispectral analysis measures sedation and memory effects ofpropofol, midazolam, isoflurane, and alfentanil in normal volunteers,”Anesthesiol., vol. 86, no. 4, pp. 836–847, 1997.

[24] M. J. Grap, T. Borchers, C. L. Munro, R. K. Elswick, and C. N. Sessler,“Actigraphy in the critically ill: Correlation with acitivity, agition, andsedation,” Amer. J. Crit. Care, vol. 14, no. 1, pp. 52–60, 2005.

[25] A. A. Weinbroum, R. B. Abraham, T. Ezri, and J. Zomer, “Wristactigraphy in anesthesia,” J. Clinical Anesthesia, vol. 13, pp. 455–460,2001.

[26] J. G. Chase, F. Agogue, C. Starfinger, Z. Lam, G. M. Shaw, A. D.Rudge, and H. Sirisena, “Quantifying agitation in sedated ICU patientsusing digital imaging,” Comput. Meth. Prog. Biomed., vol. 76, no. 2,pp. 131–141, 2004.

[27] P. Becouze, C. Hann, J. Chase, and G. Shaw, “Measuring facial gri-macing for quantifying patient agitation in critical care,” Comp. Meth.Programs Biomed., vol. 87, pp. 138–147, 2007.

[28] B. Gholami, W. M. Haddad, and A. R. Tannenbaum, “Agitation andpain assessment using digital imaging,” in Proc. IEEE Eng. Med. Bi-olog. Conf., 2009, pp. 2176–2179.

[29] B. Gholami,W.M. Haddad, and A. R. Tannenbaum, “Relevance vectormachine learning for neonate pain intensity assessment using digitalimaging,” IEEE Trans. Biomed. Eng., vol. 57, no. 6, pp. 1457–1466,Jun. 2010.

[30] P. Jackson, Introduction to Expert Systems. Harlow, U.K.: Addison-Wesley, 1999.

[31] A. A. Hopgood, Intelligent Systems for Engineers and Scientists.Boca Raton, FL: CRC Press, 2001.

[32] E. Castillo, J. M. Gutierrez, and A. S. Hadi, Expert Systems and Prob-abilistic Network Models. New York: Springer, 1997.

[33] B. G. Buchanan and E. H. Shortliffe, The MYCIN Experiments of theStanford Heuristic Programming Project. Reading, MA: Addison-Wesley, 1984.

[34] L. A. Zadeh, “The role of fuzzy logic in management of uncertainty inexpert systems,” Fuzzy Sets Syst., vol. 11, pp. 199–227, 1983.

[35] G. Shafer, A Mathematical Theory of Evidence. Princeton, NJ:Princeton Univ. Press, 1976.

[36] C. M. Bishop, Pattern Recognition and Machine Learning. NewYork: Springer, 2006.

[37] K. Murphy, An Introduction to Graphical Models, 2001 [Online].Available: http://people.cs.ubc.ca/~murphyk/Papers/intro_ gm.pdf

[38] N. Friedman, D. Geiger, and M. Goldszmidt, “Bayesian network clas-sifiers,” Mach. Learn., vol. 29, pp. 131–163, 1997.

[39] K. Thulasiraman and M. N. S. Swamy, Graphs: Theory and Algo-rithms. Singapore: Wiley-Interscience, 1992.

[40] F. Harary, Graph Theory. Reading, MA: Addison-Wesley, 1969.[41] D. L. Hunt, R. B. Haynes, S. E. Hanna, and K. Smith, “Effects of com-

puter-based clinical decision support systems of physician performanceand patient outcomes,” J. Amer. Med. Assoc., vol. 280, pp. 1339–1346,1998.

[42] M. Imhoff and S. Kuhls, “Alarm algorithms in critical care moni-toring,” Anesth. Analg., vol. 102, pp. 1525–1537, 2006.

[43] K. Murphy, “Bayesian Network Toolbox for MATLAB®,” 2002. [On-line]. Available: http://code.google.com/p/bnt/

[44] R. G. Newcombe, “Two-sided confidence intervals for the singleproportion: Comparison of seven methods,” Stat. Med., vol. 17, pp.857–872, 1998.