Experiences with a Mobile Robotic Guide for the Elderly Michael Montemerlo, Joelle Pineau, Nicholas Roy, Sebastian Thrun and Vandi Verma Robotics Institute, Carnegie Mellon University 5000 Forbes Ave Pittsburgh, PA 15213 mmde,jpineau,nickr,thrun,vandi @cs.cmu.edu Abstract This paper describes an implemented robot system, which relies heavily on probabilistic AI techniques for acting under uncertainty. The robot Pearl and its prede- cessor Flo have been developed by a multi-disciplinary team of researchers over the past three years. The goal of this research is to investigate the feasibility of assist- ing elderly people with cognitive and physical activity limitations through interactive robotic devices, thereby improving their quality of life. The robot’s task in- volves escorting people in an assisted living facility—a time-consuming task currently carried out by nurses. Its software architecture employs probabilistic techniques at virtually all levels of perception and decision making. During the course of experiments conducted in an as- sisted living facility, the robot successfully demonstrated that it could autonomously provide guidance for elderly residents. While previous experiments with fielded robot systems have provided evidence that probabilistic tech- niques work well in the context of navigation, we found the same to be true of human robot interaction with el- derly people. Introduction The US population is aging at an alarming rate. At present, 12.5% of the US population is of age 65 or older. The Ad- ministration of Aging predicts a 100% increase of this ratio by the year 2050 [26]. By 2040, the number of people of age of 65 or older per 100 working-age people will have in- creased from 19 to 39. At the same time, the nation faces a significant shortage of nursing professionals. The Federation of Nurses and Health Care Professionals has projected a need for 450,000 additional nurses by the year 2008. It is widely recognized that the situation will worsen as the baby-boomer generation moves into retirement age, with no clear solution in sight. These developments provide significant opportuni- ties for researchers in AI, to develop assistive technology that can improve the quality of life of our aging population, while helping nurses to become more effective in their everyday ac- tivities. To respond to these challenges, the Nursebot Project was conceived in 1998 by a multi-disciplinary team of investi- gators from four universities, consisting of four health-care faculty, one HCI/psychology expert, and four AI researchers. The goal of this project is to develop mobile robotic assis- tants for nurses and elderly people in various settings. Over the course of 36 months, the team has developed two proto- type autonomous mobile robots, shown in Figure 1. From the many services such a robot could provide (see [11, 16]), the work reported here has focused on the task Copyright c 2002, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. of reminding people of events (e.g., appointments) and guid- ing them through their environments. At present, nursing staff in assisted living facilities spends significant amounts of time escorting elderly people walking from one location to another. The number of activities requiring navigation is large, ranging from regular daily events (e.g., meals), ap- pointments (e.g., doctor appointments, physiotherapy, hair cuts), social events (e.g., visiting friends, cinema), to simply walking for the purpose of exercising. Many elderly people move at extremely slow speeds (e.g., 5 cm/sec), making the task of helping people around one of the most labor-intensive in assisted living facilities. Furthermore, the help provided is often not of a physical nature, as elderly people usually select walking aids over physical assistance by nurses, thus preserving some independence. Instead, nurses often provide important cognitive help, in the form of reminders, guidance and motivation, in addition to valuable social interaction. In two day-long experiments, our robot has demonstrated the ability to guide elderly people, without the assistance of a nurse. This involves moving to a person’s room, alerting them, informing them of an upcoming event or appointment, and inquiring about their willingness to be assisted. It then involves a lengthy phase where the robot guides a person, carefully monitoring the person’s progress and adjusting the robot’s velocity and path accordingly. Finally, the robot also serves the secondary purpose of providing information to the person upon request, such as information about upcoming community events, weather reports, TV schedules, etc. From an AI point of view, several factors make this task a challenging one. In addition to the well-developed topic of robot navigation [15], the task involves significant interac- tion with people. Our present robot Pearl interacts through speech and visual displays. When it comes to speech, many elderly have difficulty understanding even simple sentences, and more importantly, articulating an appropriate response in a computer-understandable way. Those difficulties arise from perceptual and cognitive deficiencies, often involving a multitude of factors such as articulation, comprehension, and mental agility. In addition, people’s walking abilities vary drastically from person to person. People with walking aids are usually an order of magnitude slower than people with- out, and people often stop to chat or catch breath along the way. It is therefore imperative that the robot adapts to indi- vidual people—an aspect of people interaction that has been poorly explored in AI and robotics. Finally, safety concerns are much higher when dealing with the elderly population, especially in crowded situations (e.g., dining areas). The software system presented here seeks to address these challenges. All software components use probabilistic tech- niques to accommodate various sorts of uncertainty. The robot’s navigation system is mostly adopted from [5], and therefore will not be described in this paper. On top of
Transcript
Experienceswith a Mobile Robotic Guide for the ElderlyMichael Montemerlo, JoellePineau,NicholasRoy, SebastianThrun and Vandi Verma
This paper describesan implementedrobot system,which reliesheavily on probabilisticAI techniquesforactingunderuncertainty. TherobotPearl andits prede-cessorFlo have beendevelopedby a multi-disciplinaryteamof researchersover the pastthreeyears. The goalof this researchis to investigatethe feasibility of assist-ing elderly peoplewith cognitive and physical activitylimitations throughinteractive robotic devices, therebyimproving their quality of life. The robot’s task in-volvesescortingpeoplein an assistedliving facility—atime-consumingtaskcurrentlycarriedout by nurses.Itssoftwarearchitectureemploysprobabilistictechniquesatvirtually all levels of perceptionand decisionmaking.During the courseof experimentsconductedin an as-sistedliving facility, therobotsuccessfullydemonstratedthat it couldautonomouslyprovide guidancefor elderlyresidents.While previousexperimentswith fieldedrobotsystemshave provided evidencethat probabilistictech-niqueswork well in thecontext of navigation,we foundthe sameto be true of humanrobot interactionwith el-derlypeople.
Intr oductionTheUS populationis agingat analarmingrate. At present,12.5%of the US populationis of age65 or older. The Ad-ministrationof Aging predictsa 100%increaseof this ratioby the year 2050 [26]. By 2040, the numberof peopleofageof 65 or olderper100working-agepeoplewill have in-creasedfrom 19 to 39. At thesametime, thenationfacesasignificantshortageof nursingprofessionals.TheFederationof NursesandHealthCareProfessionalshasprojectedaneedfor 450,000additionalnursesby theyear2008. It is widelyrecognizedthatthesituationwill worsenasthebaby-boomergenerationmovesinto retirementage,with no clearsolutionin sight. Thesedevelopmentsprovide significantopportuni-tiesfor researchersin AI, to developassistivetechnologythatcanimprovethequalityof life of ouragingpopulation,whilehelpingnursesto becomemoreeffectivein theireverydayac-tivities.
To respondto thesechallenges,the NursebotProject wasconceived in 1998 by a multi-disciplinary teamof investi-gatorsfrom four universities,consistingof four health-carefaculty, oneHCI/psychologyexpert,andfour AI researchers.The goal of this project is to develop mobile robotic assis-tantsfor nursesandelderlypeoplein varioussettings.Overthecourseof 36 months,the teamhasdevelopedtwo proto-typeautonomousmobilerobots,shown in Figure1.
From the many services such a robot could provide(see[11, 16]), thework reportedherehasfocusedonthetask
Copyright c�
2002,AmericanAssociationfor Artificial Intelligence(www.aaai.org). All rightsreserved.
of remindingpeopleof events(e.g.,appointments)andguid-ing them through their environments. At present,nursingstaff in assistedliving facilities spendssignificantamountsof time escortingelderly peoplewalking from one locationto another. The numberof activities requiringnavigation islarge, ranging from regular daily events (e.g., meals),ap-pointments(e.g., doctor appointments,physiotherapy, haircuts),socialevents(e.g.,visiting friends,cinema),to simplywalking for thepurposeof exercising. Many elderlypeoplemove at extremelyslow speeds(e.g.,5 cm/sec),makingthetaskof helpingpeoplearoundoneof themostlabor-intensivein assistedliving facilities. Furthermore,the help providedis often not of a physical nature,as elderly peopleusuallyselectwalking aidsover physical assistanceby nurses,thuspreservingsomeindependence.Instead,nursesoftenprovideimportantcognitive help,in theform of reminders,guidanceandmotivation,in additionto valuablesocialinteraction.
In two day-longexperiments,our robothasdemonstratedtheability to guideelderlypeople,without theassistanceofa nurse. This involvesmoving to a person’s room, alertingthem,informing themof anupcomingeventor appointment,andinquiring abouttheir willingnessto be assisted.It theninvolves a lengthy phasewherethe robot guidesa person,carefullymonitoringtheperson’s progressandadjustingtherobot’s velocity andpathaccordingly. Finally, therobotalsoservesthesecondarypurposeof providing informationto thepersonupon request,suchas information aboutupcomingcommunityevents,weatherreports,TV schedules,etc.
From an AI point of view, several factorsmake this taska challengingone. In addition to the well-developedtopicof robotnavigation[15], thetaskinvolvessignificantinterac-tion with people. Our presentrobot Pearlinteractsthroughspeechandvisualdisplays.Whenit comesto speech,manyelderlyhave difficulty understandingevensimplesentences,and more importantly, articulatingan appropriateresponsein a computer-understandableway. Thosedifficulties arisefrom perceptualandcognitive deficiencies,ofteninvolving amultitudeof factorssuchasarticulation,comprehension,andmentalagility. In addition, people’s walking abilities varydrasticallyfrom personto person.Peoplewith walking aidsareusuallyan orderof magnitudeslower thanpeoplewith-out, andpeopleoften stopto chator catchbreathalongtheway. It is thereforeimperative that the robot adaptsto indi-vidual people—anaspectof peopleinteractionthathasbeenpoorly exploredin AI androbotics.Finally, safetyconcernsaremuchhigherwhendealingwith the elderly population,especiallyin crowdedsituations(e.g.,diningareas).
Thesoftwaresystempresentedhereseeksto addressthesechallenges.All softwarecomponentsuseprobabilistictech-niquesto accommodatevarious sorts of uncertainty. Therobot’s navigation systemis mostly adoptedfrom [5], andthereforewill not be describedin this paper. On top of
Figure 1: NursebotsFlo (left) andPearl(centerandright) interact-ing with elderlypeopleduringoneof ourfield trips.
this,oursoftwarepossessesacollectionof probabilisticmod-ules concernedwith peoplesensing,interaction,and con-trol. In particular, Pearl usesefficient particle filter tech-niquesto detectandtrackpeople.A POMDPalgorithmper-forms high-level control, arbitrating information gatheringandperformance-relatedactions.And finally, safetyconsid-erationsare incorporatedeven into simpleperceptualmod-ulesthrougha risk-sensitive robot localizationalgorithm. Insystematicexperiments,we found the combinationof tech-niquesto be highly effective in dealingwith the elderly testsubjects.
Hardware,Software,And Envir onmentFigure1 shows imagesof therobotsFlo (first prototype,nowretired)andPearl(thepresentrobot).Bothrobotspossessdif-ferentialdrivesystems.They areequippedwith two on-boardPentiumPCs,wirelessEthernet,SICK laser rangefinders,sonarsensors,microphonesfor speechrecognition,speak-ers for speechsynthesis,touch-sensitive graphicaldisplays,actuatedheadunits, and stereocamerasystems.Pearldif-fers from its predecessorFlo in many respects,including itsvisual appearance,two sturdyhandle-barsaddedto providesupportfor elderly people,a more compactdesignthat al-lows for cargo spaceanda removabletray, doubledbatterycapacity, asecondlaserrangefinder, andasignificantlymoresophisticatedheadunit. Many of thosechangeswerethere-sult of feedbackfrom nursesandmedicalexpertsfollowingdeploymentof thefirst robot,Flo. Pearlwaslargelydesignedandbuilt by theStandardRobotCompany in Pittsburgh,PA.
On thesoftwareside,bothrobotsfeatureoff-the-shelfau-tonomousmobile robot navigation system[5, 24], speechrecognitionsoftware[20], speechsynthesissoftware[3], fastimage captureand compressionsoftware for online videostreaming,facedetectiontrackingsoftware[21], andvariousnew softwaremodulesdescribedin this paper. A final soft-warecomponentis a prototypeof a flexible remindersystemusingadvancedplanningandschedulingtechniques[18].
The robot’s environmentis a retirementresortlocatedinOakmont,PA. Likemostretirementhomesin thenation,thisfacility suffers from immensestaffing shortages.All exper-imentsso far primarily involved peoplewith relatively mildcognitive, perceptual,or physical inabilities, thoughin needof professionalassistance.In addition,groupsof elderly insimilarconditionswerebroughtinto researchlaboratoriesfortestinginteractionpatterns.
Navigating with PeoplePearl’s navigation systembuilds on the onedescribedin [5,24]. In thissection,wedescribethreemajornew modules,all
concernedwith peopleinteractionandcontrol. Thesemod-ulesovercomeanimportantdeficiency of thework describedby [5, 24], which hada rudimentaryability to interactwithpeople.
Locating PeopleTheproblemof locatingpeopleis theproblemof determiningtheir � -� -locationrelative to therobot. Previousapproachesto peopletrackingin roboticswerefeature-based:they ana-lyzesensormeasurements(images,rangescans)for thepres-enceof features[13, 22] as the basisof tracking. In ourcase,the diversity of the environmentmandateda differentapproach.Pearldetectspeopleusingmapdifferencing: therobot learnsa map, and peopleare detectedby significantdeviationsfrom the map. Figure3a shows an examplemapacquiredusingpreexisting software[24].
Mathematically, theproblemof peopletrackingis a com-binedposteriorestimationproblemandmodelselectionprob-lem. Let � be the numberof peoplenearthe robot. Theposteriorover thepeople’s positionsis givenby��� ��� ���������������� ���� ��� ����� (1)
where ��� � with !#"%$&"'� is the locationof a personattime ( , � thesequenceof all sensormeasurements,� these-quenceof all robotcontrols,and � is theenvironmentmap.However, to usemapdifferencing,therobothasto know itsown location. The locationandtotal numberof nearbypeo-ple detectedby therobot is clearlydependenton therobot’sestimateof its own locationandheadingdirection. Hence,Pearlestimatesaposteriorof thetype:��� ��� ���������������� ���� ��� ��� ����� (2)
where� denotesthesequenceof robotposes(thepath)uptotime ( . If � wasknown, estimatingthisposteriorwouldbeahigh-dimensionalestimationproblem,with complexity cubicin � for Kalmanfilters [2], or exponentialin � with particlefilters [9]. Neitherof theseapproachesis, thus,applicable:Kalmanfilterscannotglobally localizetherobot,andparticlefilterswouldbecomputationallyprohibitive.
This factorizationopensthe door for a particle filter thatscaleslinearly in � . Our approachis similar (but not identi-cal) to theRao-Blackwellizedparticlefilter describedin [10].First, therobotpath � is estimatedusinga particlefilter, asin theMonteCarlolocalization(MCL) algorithm[7] for mo-bile robotlocalization.However, eachparticlein this filter isassociatedwith a setof � particlefilters, eachrepresentingoneof thepeoplepositionestimates��� ��� � ���� ��� ����� . Theseconditionalparticlefiltersrepresentpeoplepositionestimatesconditionedonrobotpathestimates—hencecapturingthein-herentdependenceof peopleand robot location estimates.The dataassociationbetweenmeasurementsand peopleisdoneusingmaximumlikelihood,asin [2]. Underthe(false)assumptionthatthismaximumlikelihoodestimatoris alwayscorrect,ourapproachcanbeshown to convergeto thecorrectposterior, andit doessowith updatetimelinearin � . In prac-tice, we found that thedataassociationis correctin thevastmajority of situations.Thenestedparticlefilter formulation
(a) (b) (c) (d)
Figure 2: (a)-(d)Evolution of theconditionalparticlefilter from globaluncertaintyto successfullocalizationandtracking. (d) Thetrackercontinuesto trackapersonevenasthatpersonis occludedrepeatedlyby asecondindividual.
hasa secondaryadvantagethat thenumberof people0 canbe madedependenton individual robot pathparticles. Ourapproachfor estimating 0 usesthe classicalAIC criterionfor modelselection,with a prior that imposesa complexitypenaltyexponentialin 0 .
Figure2 shows resultsof thefilter in action. In Figure2a,the robot is globally uncertain,andthenumberandlocationof thecorrespondingpeopleestimatesvariesdrastically. Astherobotreducesits uncertainty, thenumberof modesin therobot poseposteriorquickly becomesfinite, andeachsuchmodehasa distinctsetof peopleestimates,asshown in Fig-ure 2b. Finally, as the robot is localized,so is the person(Figure2c). Figure2d illustratesthe robustnessof the filterto interferingpeople.Hereanotherpersonstepsbetweentherobot andits target subject.The filter obtainsits robustnessto occlusionfrom a carefully craftedprobabilisticmodelofpeople’s motion 1�243 � � 65798 3 � � ;: . This enablestheconditionalparticlefilters to maintaintight estimateswhile theocclusiontakesplace,asshown in Figure2d. In a systematicanaly-sis involving 31 trackinginstanceswith up to five peopleata time, the error in determiningthe numberof peoplewas9.6%. The error in the robot position was <>= ?A@B?>=4C cm,and the peopleposition error was as low as DE= ?F@HG>= < cm,whencomparedto measurementsobtainedwith a carefullycalibratedstaticsensorwith @ID cmerror.
When guiding people,the estimateof the personthat isbeingguidedis usedto determinethe velocity of the robot,sothattherobotmaintainsroughlya constantdistanceto theperson. In our experimentsin the target facility, we foundtheadaptivevelocitycontrolto beabsolutelyessentialfor therobot’s ability to copewith thehugerangeof walking pacesfoundin theelderlypopulation.Initial experimentswith fixedvelocity ledalmostalwaysto frustrationonthepeople’sside,in thattherobotwaseithertooslow or too fast.
SaferNavigationWhennavigating in thepresenceof elderlypeople,therisksof harmingthemthroughunintendedphysicalcontactis enor-mous.As notedin [5], the robot’s sensorsareinadequatetodetectpeoplereliably. In particular, the laserrangesystemmeasuresobstacles18 cm above ground,but is unableto de-tect any obstaclesbelow or above this level. In the assistedliving facilities,wefoundthatpeopleareeasyto detectwhenstandingorwalking,buthardwhenonchairs(e.g.,they mightbestretchingtheir legs).Thus,therisk of accidentallyhittingaperson’s foot dueto poorlocalizationis particularlyhigh indenselypopulatedregionssuchasthediningareas.
Following anideain [5], werestrictedtherobot’soperationareato avoid denselypopulatedregions, using a manuallyaugmentedmapof theenvironment(blacklinesin Figure3a
JJJ
diningareas
(a)
(b)
Figure 3: (a) Map of the dining areain the facility, with diningareasmarked by arrows. (b) Samplesat the beginning of globallocalization,weightedexpectedcumulative risk function.
– thewhite spacecorrespondsto unrestrictedfreespace).Tostaywithin its operatingarea,therobotneedsaccuratelocal-ization,especiallyat the boundariesof this area.While ourapproachyieldssufficiently accurateresultson average,it isimportantto realizethat probabilistictechniquesnever pro-vide hardguaranteesthattherobotobeys a safetyconstraint.To addressthisconcern,weaugmentedtherobotlocalizationparticlefilter by a samplingstrategy that is sensitive to theincreasedrisk in thediningareas(seealso[19, 25]). By gen-eratingsamplesin high-risk regions,we minimize the like-lihood of beingmislocalizedin suchregions,or worse,thelikelihood of enteringprohibitedregions undetected.Con-ventionalparticlefiltersgeneratesamplesin proportionto theposteriorlikelihood 1�24K 8�L �M�N/ �M�O : . Our new particlefiltergeneratesrobotposesamplesin proportiontoP 24K ;: 1/26K 8�L M�N M�O : � 1�243 � � �8�L M-N M�O : (4)
whereP
is a risk functionthatspecifieshow desirableit is tosamplerobotposeK . Therisk functionis calculatedby con-sideringan immediatecost function Q�24K M�N : , which assignscoststo actionsR androbotstatesK (in our case:high costs
Assist
Act
Remind
Contact
RemindPhysioPublishStatus
RingBellGotoRoom
VerifyRelease
Inform
SayTimeSayWeatherVerifyRequest
RestRechargeGotoHome
Move
BringtoPhysioCheckUserPresentDeliverUser
VerifyBring
Figure4: DialogProblemAction Hierarchy
for violatinganareaconstraints,low costselsewhere).To an-alyzetheeffectof poorlocalizationon thiscostfunction,ourapproachutilizes an augmentedmodelthat incorporatesthelocalizeritself asa statevariable.In particular, thestatecon-sistsof therobotposeK , andthestateof thelocalizer, S . Thelatter is definedasaccurate( S UT D ) or inaccurate( S UTWV ).Thestatetransitionfunctionis composedof theconventionalrobotmotionmodel1�24K ;8 N �XY M K -XZ�: , andasimplisticmodelthat assumeswith probability [ , that the tracker remainsinthesamestate(goodor bad).Putmathematically:
1�26K M S 8 N �XY M K �XZ M S �XZ :\T1�24K �8 N -XZ M K �XZ�:�] [>^`_�a�bY_�adc�egfh2�D`iU[ : ^E_�a�jbY_�adc�e (5)
Our approach calculates an MDP-style value function,k 24K M S : , under the assumptionthat good tracking assumesgoodcontrolwhereaspoor trackingimpliesrandomcontrol.This is achievedby thefollowing valueiterationapproach:k 24K M S :gl imon�prq Q�26K M�N : fts uwv _ v�1�24Kx M S�x 8 K M S M�N : k 24Kx M S-x :
if S T D (goodlocalization)
q Q�26K M-N : fys uwv _ v�1�24Kx M S-x 8 K M S M�N : k 24Kx M S-x :if S T.V (poorlocalization)
(6)
wheres is thediscountfactor. Thisgivesawell-definedMDPthat canbe solved via value iteration. The risk function isthemsimply the differencebetweengoodandbadtracking:P 26K :zT k 24K M D : i k 26K M V : . Whenappliedto the Nursebotnavigationproblem,this approachleadsto a localizational-gorithm that preferentiallygeneratessamplesin the vicinityof thedining areas.A samplesetrepresentinga uniform un-certaintyis shown in Figure3b—noticetheincreasedsampledensitynearthe dining area. Extensive testsinvolving real-world datacollectedduring robot operationshow not onlythattherobotwaswell-localizedin high-riskregions,but thatour approachalso reducedcostsafter (artificially induced)catastrophiclocalizationfailure by 40.1%,when comparedto theplainparticlefilter localizationalgorithm.
High Level Robot Control and Dialog ManagementThe mostcentralnew modulein Pearl’s softwareis a prob-abilistic algorithmfor high-level controlanddialogmanage-ment. High-level robot control hasbeena populartopic inAI, anddecadesof researchhasled to a reputablecollectionof architectures(e.g.,[1, 4, 12]). However, existing architec-turesrarelytakeuncertaintyinto accountduringplanning.
Pearl’s high-level control architectureis a hierarchicalvariant of a partially observable Markov decisionprocess
Observation TrueState Action Rewardpearlhello requestbegun sayhello 100pearlwhatis like startmeds ask repeat -100pearlwhattime is it
for will the want time say time 100pearlwasonabc want tv askwhich station -1pearlwasonabc want abc sayabc 100pearlwhatis onnbc want nbc confirm channel nbc -1pearlyes want nbc saynbc 100pearlgo to thethat
prettygoodwhat sendrobot ask robot where -1pearlthatthathellobe sendrobot bedroomconfirm robot place -1pearlthebedroomany i sendrobot bedroomgo to bedroom 100pearlgo it eightahello sendrobot ask robot where -1pearlthekitchenhello sendrobot kitchen go to kitchen 100
Table1: An exampledialogwith anelderlyperson.Actionsin boldfont areclarificationactions,generatedby thePOMDPbecauseofhighuncertaintyin thespeechsignal.
(POMDP)[14]. POMDPsaretechniquesfor calculatingop-timal controlactionsunderuncertainty. Thecontroldecisionis basedon the full probabilitydistribution generatedby thestateestimator, suchasin Equation(2). In Pearl’s case,thisdistributionincludesamultitudeof multi-valuedprobabilisticstateandgoalvariables:{ robotlocation(discreteapproximation){ person’s location(discreteapproximation){ person’s status(asinferredfrom speechrecognizer){ motiongoal(whereto move){ remindergoal(whatto inform theuserof){ userinitiatedgoal(e.g.,aninformationrequest)Overall, there are 288 plausiblestates. The input to thePOMDP is a factoredprobability distribution over thesestates,with uncertaintyarising predominantlyfrom the lo-calizationmodulesand the speechrecognitionsystem. Weconjecturethat theconsiderationof uncertaintyis importantin thisdomain,asthecostsof mistakinga replycanbelarge.
Unfortunately, POMDPsof the sizeencounteredhereareanorderof magnitudelargerthantoday’sbestexactPOMDPalgorithmscan tackle [14]. However, Pearl’s POMDP is ahighly structuredPOMDP, where certain actionsare onlyapplicablein certain situations. To exploit this structure,we developed a hierarchical version of POMDPs, whichbreaksdown thedecisionmakingprobleminto acollectionofsmallerproblemsthatcanbesolvedmoreefficiently. Ourap-proachissimilarto theMAX-Q decompositionfor MDPs[8],but definedoverPOMDPs(wherestatesareunobserved).
The basicideaof the hierarchicalPOMDPis to partitionthe actionspace—notthe statespace,sincethe stateis notfully observable—intosmallerchunks.For Pearl’s guidancetasktheactionhierarchy is shown in Figure4,whereabstractactions(shown in circles)areintroducedto subsumelogicalsubgroupsof lower-level actions. This actionhierarchy in-ducesadecompositionof thecontrolproblem,whereateachnodeall lower-level actions,if any, areconsideredin thecon-text of a local sub-controller. At thelowestlevel, thecontrolproblemis a regular POMDP, with a reducedactionspace.At higher levels, the control problemis alsoa POMDP, yetinvolves a mixture of physical and abstractactions(whereabstractactionscorrespondto lower level POMDPs.)
Let |N be suchan abstractaction,and }�~q the control pol-icy associatedwith the respective POMDP. The “abstract”POMDPis thenparameterized(in termsof statesK , obser-vations L ) by assumingthatwhenever |N is chosen,Pearluses
0
0.5
1
1.5
2
2.5
3
User 1 User 2 User 3
Ave
rage
# o
f act
ions
to s
atis
fy r
eque
st
�
User Data -- Time to Satisfy RequestPOMDP Policy
ConventionalMDP Policy
2.0
2.5
1.631.9 1.86
1.42
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
User 1 User 2 User 3
Ave
rage
Err
ors
per
Act
ion
�
User Data -- Error Performance
ConventionalMDP Policy
POMDP Policy
0.1 0.10.18
0.55
0.825
0.36
0
10
20
30
40
50
60
User 1 User 2 User 3
Ave
rage
Rew
ard
per
Act
ion
�
User Data -- Reward AccumulationPOMDP Policy
Conventional MDP Policy52.2
36.95
49.72
24.8
6.19
44.95
(a) (b) (c)
Figure 5: EmpiricalcomparisonbetweenPOMDPs(with uncertainty, shown in gray)andMDPs(no uncertainty, shown in black) for high-level robotcontrol,evaluatedon datacollectedin theassistedliving facility. Shown aretheaveragetime to taskcompletion(a), theaveragenumberof errors(b), andtheaverageuser-assigned(not modelassigned)reward(c), for theMDP andPOMDP. Thedatais shown for threeusers,with good,averageandpoorspeechrecognition.
denotestherewardfunction. It is importantto noticethatsucha decompositionmayonly bevalid if reward is re-ceivedat theleafnodesof thehierarchy, andis especiallyap-propriatewhenthe optimal control transgressesdown alonga singlepath in the hierarchy to receive its reward. This isapproximatelythecasein thePearldomain,whererewardisreceived uponsuccessfullydelivering a person,or success-fully gatheringinformationthroughcommunication.
Using the hierarchicalPOMDP, the high-level decisionmakingproblemin Pearlis tractable,andanear-optimalcon-trol policy can be computedoff-line. Thus, during execu-tion time thecontrollersimply monitorsthestate(calculatesthe posterior)andlooks up the appropriatecontrol. Table1shows an exampledialog betweenthe robot anda testsub-ject. Becauseof the uncertaintymanagementin POMDPs,the robot choosesto aska clarificationquestionat threeoc-casions.Thenumberof suchquestionsdependsontheclarityof aperson’sspeech,asdetectedby theSphinxspeechrecog-nition system.
An importantquestionin our researchconcernstheimpor-tanceof handlinguncertaintyin high-level control.To inves-tigate this, we ran a seriesof comparative experiments,allinvolving real datacollectedin our lab. In oneseriesof ex-periments,we investigatedtheimportanceof consideringtheuncertaintyarisingfrom the speechinterface. In particular,we comparedPearl’s performanceto a systemthat ignoresthatuncertainty, but is otherwiseidentical.Theresultingap-proachis anMDP, similar to theonedescribedin [23]. Fig-ure5 showsresultsfor threedifferentperformancemeasures,andthreedifferentusers(in decreasingorderof speechrecog-nition performance).For poor speakers, the MDP requireslesstime to “satisfy” a requestdueto thelackof clarificationquestions(Figure5a). However, its errorrateis muchhigher(Figure5b), which negatively affects the overall reward re-ceivedby therobot(Figure5c). Theseresultsclearlydemon-stratetheimportanceof consideringuncertaintyatthehighestrobotcontrollevel, specificallywith poorspeechrecognition.
In a secondseriesof experiments,we investigatedtheim-portanceof uncertaintymanagementin thecontext of highlyimbalancedcostsand rewards. In Pearl’s case,suchcostsareindeedhighly imbalanced:askingaclarificationquestionis much cheaperthan accidentallydelivering a personto awronglocation,or guidinga personwho doesnot wantto bewalked. In this experimentwe comparedperformanceusing
twoPOMDPmodelswhichdifferedonly in theircostmodels.One model assumeduniform costsfor all actions,whereasthesecondmodelassumeda morediscriminative costmodelin which thecostof verbalquestionswaslower thanthecostof performingthewrongmotionactions.A POMDPpolicywaslearnedfor eachof thesemodels,andthentestedexper-imentally in our laboratory. Theresultspresentedin figure6show that thenon-uniformmodelmakesmorejudicioususeof confirmationactions,thusleadingto a significantlylowererrorrate,especiallyfor userswith low recognitionaccuracy.
Following this, we performedtwo daysof formal experi-mentsduringwhich therobotautonomouslyled12 full guid-ances,involving 6 differentelderly people. Figure7 showsanexampleguidanceexperiment,involving anelderlypersonwho usesa walking aid. The sequenceof imagesillustratesthe major stagesof a successfuldelivery: from contactingtheperson,explainingto herthereasonfor thevisit, walkingher throughthe facility, andproviding informationafter thesuccessfuldelivery—in this caseon theweather.
In all guidanceexperiments,the task was performedtocompletion. Post-experimentaldebriefingsillustrateda uni-form highlevel of excitementonthesideof theelderly. Over-all, only a few problemsweredetectedduringtheoperation.Noneof the test subjectsshowed difficulties understandingthemajorfunctionsof therobot.They all wereabletooperatethe robot after lessthanfive minutesof introduction. How-ever, initial flaws with a poorly adjustedspeechrecognition
Figure7: Exampleof asuccessfulguidanceexperiment.Pearlpicksup thepatientoutsideherroom,remindsherof a physiotherapy ap-pointment,walks the personto the department,andrespondsto arequestof the weatherreport. In this interaction,the interactiontookplacethroughspeechandthetouch-sensitivedisplay.
systemled to occasionalconfusion,which wasfixed duringthecourseof this project.An additionalproblemarosefromthe robot’s initial inability to adaptits velocity to people’swalking pace,which wasfound to becrucial for the robot’seffectiveness.
DiscussionThispaperdescribedamobileroboticassistantfor nursesandelderly in assistedliving facilities. Building on a robotnav-igation systemdescribedin [5, 24], new software modulesspecificallyaimedat interactionwith elderlypeoplewerede-veloped. The systemhasbeentestedsuccessfullyin exper-imentsin an assistedliving facility. Our experimentsweresuccessfulin two maindimensions.First, they demonstratedthe robustnessof the various probabilistic techniquesin achallengingreal-world task.Second,they providedsomeev-idencetowardsthe feasibility of using autonomousmobilerobots as assistantsto nursesand institutionalizedelderly.Oneof thekey lessonslearnedwhile developingthis robotisthat theelderlypopulationrequirestechniquesthatcancopewith their degradation(e.g.,speakingabilities)andalsopaysspecialattentionto safetyissues.We view theareaof assis-tive technologyasa prime sourcefor greatAI problemsinthefuture.
Possiblythemostsignificantcontribution of this researchto AI is the fact that the robot’s high-level control systemisentirely realizedby a partially observableMarkov decisionprocess(POMDP) [14]. This demonstratesthat POMDPshave maturedto a level that makesthemapplicableto real-world robotcontroltasks.Furthermore,our experimentalre-sultssuggestthat uncertaintymattersin high-level decision
making.Thesefindingschallengea long termview in main-streamAI thatuncertaintyis irrelevant,or atbestcanbehan-dleduniformlyatthehigherlevelsof robotcontrol[6, 17]. Weconjectureinsteadthatwhenrobotsinteractwith people,un-certaintyis pervasiveandhasto beconsideredatall levelsofdecisionmaking,not solelyin low-level perceptualroutines.
References[1] R. Arkin. Behavior-BasedRobotics. MIT Press,1998.[2] Y. Bar-ShalomandT. E. Fortmann.Tracking andData Asso-
ciation. AcademicPress,1998.[3] A.W. Black, P. Taylor, and R. Caley. The Festival Speech
SynthesisSystem. Universityof Edinburgh,1999.[4] R.A. Brooks. A robust layeredcontrol systemfor a mobile
robot. TR AI memo864,MIT, 1985.[5] W. Burgard, A.B., Cremers,D. Fox, D. Hahnel, G. Lake-
meyer, D. Schulz,W. Steiner, andS. Thrun. The interactivemuseumtour-guiderobot. AAAI-98
[6] G. De Giacomo,editor. NotesAAAI Fall SymposiumonCog-nitiveRobotics, 1998.
[7] F. Dellaert,D. Fox, W. Burgard,andS. Thrun. MonteCarlolocalizationfor mobilerobots.ICRA-99
[8] T. Dietterich. TheMAXQ methodfor hierarchicalreinforce-mentlearning.ICML-98.
[9] A. Doucet,N. deFreitas,andN.J.Gordon,editors.SequentialMonteCarlo MethodsIn Practice. Springer, 2001.
[10] A Doucet,N. de Freitas,K. Murphy, and S. Russell. Rao-Blackwellised particle filtering for dynamic bayesiannet-works. UAI-2000.
[22] D. Schulz,W. Burgard, D. Fox, andA. Cremers. Trackingmultiple moving targetswith a mobile robot using particlesfiltersandstatisticaldataassociation.ICRA-2001.
[23] S. Singh,M. Kearns,D. Litman, andM. Walker. Reinforce-mentlearningfor spokendialoguesystems.NIPS-2000.
[24] S. Thrun, M. Beetz,M. Bennewitz, W. Burgard, A.B. Cre-mers,F. Dellaert,D. Fox, D. Hahnel,C. Rosenberg, N. Roy,J. Schulte,and D. Schulz. Probabilisticalgorithmsand theinteractive museumtour-guiderobot Minerva. InternationalJournalof RoboticsResearch, 19(11),2000.
[25] S. Thrun,Langford.J.,andV. Verma. Risk sensitive particlefilters. NIPS-2002.
[26] US Departmentof Health and Human Services. Health,Unitedstates,1999.Healthandagingchartbook,1999.
![Humanoid Robots - CCC Event Blog · Humanoid Robots Sven Behnke ... •Japan: Atom-Project, Time: 30 years •USA: Cog, Kismet, Leo, Nursebot ... [the personal robot industry]](https://static.documents.pub/doc/80x56/5b7c12eb7f8b9a70138beaf9/humanoid-robots-ccc-event-blog-humanoid-robots-sven-behnke-japan-atom-project.jpg)
