Intelligent Autonomy Update
Marc SteinbergOffice of Naval Research (703) 696 – 5115, [email protected]
Naval Air Systems Command (301) 342 – 8567, [email protected]
SAE Control & Guidance Committee Meeting, 2 March, 2005
2
Share Mission Relevant InfoReport Status
Multiple Types of Vehicle Interfaces/Comms
Intelligent Autonomy Future Vision
Mixed-Initiative Human InterfaceHigh Level Mission Requirements/ROE
Common Operational Picture
Common Information Management
Multi-Vehicle Mission Planning
UUV Local Planning& SA
USV Local Planning& SA
UGV Locally Controlled,SharesInfo
UAV Local Planning& SA
Local Task Negotiation, Information Sharing
Task Scheduling, Routing,Constraints, Priorities
Management of 5-10 UxV’s
External C4I, Data Filtering
3
Summary • Team planning mechanisms geared towards
maximizing communications capabilities in adverse conditions while on the move.
• Integrated mission specification capabilities for parsing the tasks of overall mission objectives and mapping them onto heterogeneous UxV’s
• V&V of autonomous algorithms• Joint with DARPA
Accomplishments• Integration of large number of heterogeneous UxV’s
with different lower-level autonomy software• Framework integrating communications, perception,
and execution for UxV’s• New algorithm development & implementation for
communication sensitive behaviors, heterogeneous UxV tasking, distributed SA, and V&V of autonomous systems
MOUT Site Demonstration Univ. of Penn., Georgia Tech., USC, BBN, Mobile Intelligence
Future Work• Case-Based Reasoning & Multi-UxV Task
Allocation to support mission specification• Extend V&V approach to test IA systems
Completed Demo at MOUT Site•Operator tasks multiple types of UAV’s and UGV’s with high-level commands•Mission execution while maintaining communication constraints
62
64
62
64
62
59
5556
56
56
56
4
Summary• Single-operator mission management of multiple
heterogeneous unmanned vehicles• Extend UUV software w/ increased autonomy on-
board UUV & increased dynamic retasking capability for missions w/ comms constraints
• Integration with on-board vehicle sensors & external systems
Accomplishments• Initial software design & implementation• Integration of initial versions of SA, situation
assessment, mixed-initiative interface, control station planning, UUV on-board planning/SA.
• Initial integration with existing UUV lower-level autonomous control software
• Usability analysis by NAVAIR & NSWC to recommend improvements
Risk-Aware Mixed-Initiative Dynamic Replanning DemoDraper Laboratory/CRA
Future Work• Extend to increase UAV/UUV cooperation,
complexity of tasking, integration with other systems, & realism of simulation
• In-water demonstration
Completed Dynamic Replanning Demo w/ high-fidelity UUV simulation•Single UUV, UAV as comm link•Operator provides high-level tasking/constraints•Autonomously generates plan of activities for UUV to start the mission, transit, search shoreline for particular target classes, end mission•Monitors execution
NRegion 3
Region 4
Region 2
Keep-outNo CommsTarget region
Region 1
5
Summary• Allocates tasks based on operator high-level team
tasking and constraints• Determines team & individual vehicle tactics to
achieve objectives• Schedules activities for heterogeneous resources
• Inputs to lower-level vehicle planners• Cooperation w/ realistic comms limitations
Accomplishments• Extended Mission Control System (MCS) to provide
tasking and routing for naval autonomous systems. • Allocates tasks to vehicles including cooperative
search and data collection• Constraints include time windows, precedence
constraints, and no activity zones• Tasks prioritized as mandatory, high, med, or
low. Drops lower priority tasks if not feasible.
Mission Control for Multiple UxV’s DemonstrationBAE Systems, Aptima, MIT, BU, Univ. of Minnesota
Future Work• Simulation Demonstration in communication limited
environment (joint with Air Force)•Local planning under constraints on vehicle when outside of communication
Multi-UxV Simulation Demonstration•Firescout, J-UCAS, BAMS, USV•Operator provides high-level team tasking, ROE’s, & constraints•Replans following changes in environment or new tasking
6
4 boats found
100 200 300 400 500 600 700 800 900 1000
50
100
150
200
Summary• Maritime Image Understanding for autonomous
sensor-directed dynamic replanning• Supports low elevation UUV mast with no
mechanical stabilization of image• Detects, classifies, and tracks targets• Joint with DARPA
Accomplishments• Reliable surface object detection in clear and hazy
conditions• Image stabilization using software only• Low false alarm rate for shoreline man-made object
detection• Interesting object detection to direct data collection• Real-Time maritime scene segmentation/high speed
video array & limited classification capability
Maritime Image UnderstandingNorthrop/Carnegie Mellon University
Future Work• Integration with on-board dynamic replanning
software to enable replanning with sensed data• Additional in-water experimentation• Improved robustness
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.5
1
Shoreline Man-Made Object Detection
Shoreline Detection
Surface Object Detection & Tracking
CompletedIn-Water Demo on Spartan USV
7
Summary• Integrated suite of components to enable rapid
highly automated and fully autonomous mission planning/replanning by high-level objectives
• Determines optimized route that meets all constraints & mission objectives while also optimizing secondary priorities
Accomplishments• Integrated Multi-UV Task Allocation, Replanning,
Replan Assessment, Information/Alert Management, Operator Interface, & Lower level GNC components
• Objectives incorporated include search for stationary & mobile targets, EO/IR, SAR, Loiter, Steer-Point, Communication
• Constraints incorporated included no-fly zones and LOS comms requirements (moving & stationary)
Intelligent Control & Autonomous Replanning of Unmanned Systems (ICARUS) - Lockheed
Future Work• Integration with naval control station• Simulation Demonstration in warfare environment at
NAVAIR• Increased robustness/integration
Completed Simulation Demonstration & Evaluation by 2 Navy & 2 Marine Corps Operators with 7 UAV’s (Firescout, BAMS, Future ISR)•Replanning for Dynamic Mission Events•New/Dropped Mission Tasks•Change in order of tasks•Failures/Weather/changes in environment
comSAR1EOeng
hellfi
