USARsim for Robocup
Jijun Wang & Michael Lewis
Background..
USARsim was developed as a research tool for an NSF project to study Robot, Agent, Person Teams in Urban Search & Rescue
Katia Sycara CMU- Multi agent systemsIllah Nourbakhsh CMU/NASA- RobotsMike Lewis Pitt- PeopleJijun Wang- primary developer
We wanted it to:
Study Teams of Robots controlled by People
The Idea for USARsim in Robocup
High fidelity simulation environment that provides:
Detailed models of USAR environmentsDetailed models of robots & sensors
The user bringsIndividual robot & team control codeUser interface
Strategy vs. Dropped Leads:Why we need something between Robocup Rescue & the NIST arenas
The Devil is in the details.. USAR missions fail because the operator can’t figure out:Why the robot is stuck (all of us)That the robot has flipped (Robin Murphy)That the camera never pointed at the victimThat the camera is pointing at a victim but we didn’t see ….Etc!
We can be strategically perfect & still failThere needs to be a level of planning & execution
that mixes strategy & sensing & control issues
NIST’s Existing Transportable ArenasFORMING A CONTINUUM OF CHALLENGES(from Jacoff et al. 2003)
Yellow RegionSimple to traverse, no agility requirementsPlanar (2-D) mazeIsolates sensors with obstacles/targetsReconfigurable in real time to test mapping
Orange RegionMore difficult to traverse, variable flooringsSpatial (3-D) maze, stairs, ramp, holesSimilarly reconfigurable
Red RegionDifficult to traverse, unstructured environmentSimulated rubble piles, shifting floorsProblematic junk (rebar, plastic bags, pipes…)
YELLOW COURSE
ORANGE COURSERED COURSE
60'
4'
4'
4'
4'20' 28'
52'
24'
24'
60'
[START/FINISH]
STAIRS (5'/8')
SHALLOW RAMP (5'/15')
Human Factors ChallengesWorld through a straw (restricted FOV)Camera control for search/navigation Survey knowledge (mapping environment) from restricted FOV & impeded movementVisual “smearing” from close surfacesunfamiliar ground level perspectiveDifficult distance judgments from degraded 2D imageDifficult orientation judgments from visual cues in disorderly environmentDifficult locomotion due to out-of-view & negative obstacles
Multi-Robot Problems
Common mapping & awarenessPerceptual cooperation (you see what I’m stuck on)Exploiting heterogeneous capabilitiesDaisy chaining & communication modelingTeam planning in failure-prone environmentsHuman interaction/control of teamsEtc.
Simulation Desiderata
Expense and availability of simulation hardware and software to USAR robotics communityEase of programming to reflect targeted aspects of designFidelity of simulation w.r.t. aspects of design to be tested
Simulation Requirements
Video feed for teleoperation and visual search and identification
Sensor simulation- for autonomous control and fused displays
Simulated robot dynamics- for teleoperation and autonomous control
Multiple entity simulation- to allow interaction and cooperation among teams of robots
Why a Game Engine?(Lewis & Jacobson, CACM 2002)
HardwareBest available graphics now on Nvidia/ATI gpu’s for commodity priced pc’s
SoftwareCheapTakes advantage of current gpu featuresSophisticated IDE’s for both behaviors (mods) and environments (levels)Client-server architecture to support multiple robotsRanging commands used by ‘bots’ provide hooks needed for sensor simulationSophisticated physics engines such as the Karma engine allow high fidelity kinematical modeling
Why the Unreal Engine?
Lewis, M. & Jacobson, J. (2002) Game Engines in Research Communications of the Association for Computing Machinery, NY: ACM 45(1)
Good graphics, good physics, object orientation, good tools, & good documentation
Unreal is the defacto game engine used in research:Pitt & PARC- multiscreen (CAVE-like) displaysJohn Laird/ U Mich- AI test environmentJ. Anderson & C. Leibiere- Act-R & charactersK. Sycara, J. Hodgins, & G. Sukanthar- synthetic charactersArchitectural modeling (Notre Dame), etc.Mike Zyda & NPS- America’s Army, etc.ITC center at USC, Alterne project University of Teeside & others (using
our cave set-up)(crystal space, garage games, etc.)
Alternatives
Excellent 3D graphics & drivers from VR community like UCF toolsGood physics engines like ODE (being used for 3D soccer simulation)
But hard to put them together..With ODE you have to work with OpenGL in Xwindows &
lose the standard formats that let you use modeling tools like 3D studio max or Maya
With VR tools you have to program physical behaviors yourself
USARsim Architecture
Unreal Engine
Map Models (Robots model, Sensor model, victim model etc.)
Gamebots
Network
Control Interface
Unreal Client (Attached spectator)
Middle Level Control
High Level Control
Controller
Video Feedback
Controller
Unreal Client (Attached spectator)
Video Feedback
……
Team Cooperation
Unreal Data
Control Data
Control Interface
Middle Level Control
High Level Control
USARsim Architecture
Unreal Engine
Map Models (Robots model, Sensor model, victim model etc.)
Gamebots
Network
Control Interface
Unreal Client (Attached spectator)
Middle Level Control
High Level Control
Controller
Video Feedback
Controller
Unreal Client (Attached spectator)
Video Feedback
……
Team Cooperation
Unreal Data
Control Data
Control Interface
Middle Level Control
High Level Control
Gamebots Shot
Robot control architecture (simulation)
AttachedUnrealSpectator
Control software
GAMEBOTS
UnrealServer
Karma Physics enginerigid multi- body simulation
Vehicle class lets us characterize robot kinematics precisely..
Sensor hierarchyThe trace command gives ground truth for range informationSound is attenuated according to distanceHuman motion (pyroelectric) uses line of sight & magnitude of movement
Sensor
Range Sensor Sound Sensor HumanMotion Sensor
RangeScanner Sensor
Sensor Class
HiddenSensor This Boolean value is used to indicate whether the sensor will be visually showed in the simulator.
MaxRange It is the maximum distance that can be detected.
Noise It is the relative random noise amplitude. With the noise, the data will be data = data + random(noise)*data
OutputCurve It’s the distortion curve. It is constructed by a series of points that describe the curve.
Sensor visualizations (courtesy of Player)
Can be controlled through:
Native Gamebots interfacePyro middleware (including a hack for Windows)Player.. (USARsim plays the role of Stage)
The Arenas
Arena Simulations
ProEngineer solid model converted to Unreal formatDigital photographs used to create textures to be applied to the modelGlass, mirrors, orange safety fencing, and other “special effects” addedRubble, debris, and victim models added to simulationRobot characteristics adapted from Karma vehicle class
Data Collection at NIST ArenasGaithersburg, MD
Illumination levels in Lux
Yellow Arena
Yellow Arena Simulation
Fisheye view of Orange Arena(from Jacoff et al. 2003)
Simulation of Orange Arena
Red Arena Simulation
Orange Arena Platform: photo & simulation
Parts & materials to “build your own”
The Robots
Activmedia Pioneer P2AT
P2AT has:•Four wheels•Skid-steer•Size: 50cm x 49cm x 26cm•Wheel diam: 21.5cm
equipped with:•PTZ camera•Front sonar ring•Rear sonar ring
Pioneer P2DX
iRobot ATVJr
CMU experimental: Personal Explorer Rover (PER)
CMU experimental: Corky
First generation interface(runs with both Corky & simulation)
Observations
Both Corky and the simulation are very difficult to control with same problems:
Nonvisible obstaclesSurface blindingDisorientationCamera controlDifficulty with stairs or rubble
Validation
We are currently planning validation studies comparing real & simulated PER & Pioneer robots in Orange Arena
Hardware: Sensors, KinematicsBehavior: Sensors x KinematicsAutomation: scripted behaviorsHRI: human x automation
Stefano Carpin has a student doing a validation for laser rangefinder model
SimpleUI sample interface
Pyro Controls
Jijun’s USARsim Practicals
Today (Saturday) after lunchBasic- architecture & installation
Late afternoon- ControllersPyroPlayerVideo feedback
Sunday afternoon- Programming in Unrealbuild a sensor, robot
Hardware Issues
Requires current (2000 +) pentium 4 pcFor Linux requires Nvidia video cardServer uses few resourcesClients (attached spectators) are resource hogsWithout modification server handles 32 spectator clients & unlimited robots without spectatorsTo add robot
Create robot in simulationConnect socket to control robotAdd ‘spectator’ to provide camera view
Near term Mods
Rewrite vehicle classes to use downloadable Unreal Runtime (warfare) EngineExtend manual instructions for
Assembling new arenas from partsMore pointers to available Unreal tutorials & FAQs
Extend video control demo’d in SimpleUI to other releases of DirectXAdd Aibo (courtesy of Sheila Tejada)
Send us questions so we can write a better manual & FAQ
USARsim & Robocup
Who is the community?Our (Mike, Katia, Illah, & Jijun) interests are in:
Cooperating teamsRobots with adjustable autonomyHuman-Robot Interaction
So we would like to see USARsim in Robocupevolve from a simulation analgue of the USAR league into something in between the USAR physical robot league & the Robocup Rescue simulation league
USARsim & Robocup
So we would favor rules that:Encourage multiple robotsEncourage heterogeneous robotsEncourage Human-Robot Interaction
BUTFor now the IMPORTANT thing is to AGREE ON RULES HERE AT THIS CAMP
Modest Proposal
Accept USAR rules(but maybe with just a nod to the previous slide?)
Issues
Fixing or scoring platforms & sensorsa) Everybody runs the same robot & sensor
suiteb) Use weighting factor to reward performance
using low capability platform or sensorsc) As robots are added allow higher capability
platforms
Issues
Arenas & Nike Silo orNovel Environments
Who is going to make themWhat should be modeled
Issues USAR/USARsim Differences
Size of venueCould add dynamic events such as collapsesHazards such as fire, water, etc.Radio drop outSmoke, etc.
Not this year but 2006?
USARsim contol & maintenance
Maintain informally within USAR communityTransfer to NIST to administer like physical arenasMove to Sourceforge or similar opensourcerepositoryIdeas?
Download simulator & docs athttp://usl.sis.pitt.edu/ulab