Intelligent Systems Division (ICTC TI)
Collaborative and Assistant Systems (CAS)
Research
ROSES Partnership OpportunitiesRich Keller
[email protected]/17/05
Collaborative and Assistant Systems: Major Themes
• Human-centered systems– Work-systems design, modeling, simulation, and evaluation
– Procedure design, modeling, validation, training, and execution support
– Integrated field tests
– Multi-modal interfaces and human / automation / robot interaction
• Situation awareness and decision support systems
– Integrated simulations supporting what-if analysis
– Contingency planning and assistance
• Complex information and knowledge management – Heterogeneous data storage, search, integration, and analysis
– Semantic data organization, metadata management, and ontologies
– Lessons learned and expertise capture
• Software Composition and Integration – Middleware, components, services, frameworks and portals
Major CAS Projects
• Human-centered systems– Clarissa (ISS DTO) and Advanced Astronaut Assistants
– Regulus Open-source spoken dialog toolkit
– Mobile Agents and Advanced EVA technologies
– Optimal human interface synthesis
• Situation awareness and decision support systems
– Simulation-based Acquisition
– Intelligent Launch and Range
– UAV Collaboration and Coordinated Mission Mgmt
• Complex information and knowledge management – Simstation and Digital Shuttle
– Netmark and PMT (major rollout and EVM integration)
– SemanticOrganizer for Science and Accident Investigation
– Aviation data integration, mining, and trending
• Software Composition and Integration – CIP for MER Extended Mission– MCT Fabric demonstration for MSL
Human-Centered Systems
• Some technology pitfalls:– “Build it and they will come.”– “… but it’s a great technology.”– “Lets just ask them what they want.”
Technology
People & Culture
Physical Environment
• Human-Centered Systems: A systematic methodology for designing systems that optimize the teaming of humans and machines.
• Methodology:
• Work-systems design, modeling, simulation, and evaluation
• Procedure design, modeling, validation, training, and execution support
• User-centered design
• Integrated field tests
• Crew and team organizational design and modeling
• Flight deck and mission operations facilities design and evaluation
HCC & Fatigue Countermeasures: Improved data understanding and
Enhanced situational awareness
CIP: Customizable data navigation, search, and information management
Viz: High fidelity terrain modeling and analysis
MAPGEN: Activity plan development and analysis
MERBoard: Collaborative information analysis and sharing
MER science and uplink team members MER science and uplink team members have estimated thathave estimated that overall science overall science return increased by more than 20%.return increased by more than 20%.
Ames Capabilities for Mars Exploration Rover (MER) Mission Operations
Collaborative Information Portal
The technology developed for the CIP project has application to the broader Information Infrastructures technology (InfoCore) research goals. The long term objective of this research is to provide a seamless information infrastructure where data from large numbers of heterogeneous information systems are easily accessible and presented in a “smart” human-centered user interface. This will enable improved decision making, reduce risk, and maximize leverage of the vast information resource and corporate knowledge developed by NASA and industry.
A Web-based information management system being developed to support the surface operations phase of the Mars Exploration Rover (MER) Mission in 2004. This system will provide Mission Operations and Science Teams with rapid and intuitive access to a broad range of Mission status and planning information to assist in the daily decision making process.
Integrated MER Mission Engineering & Science Data Management
MERBoard/Xboard Collaborative Workspace
Collaborative Content Creationfor Mission Planning
Robot Mule tracks Astronauts & takes photos when commandedRobot in “follow me” mode
Voice annotation is recorded and transmitted to database in habitat & to RST on earth
Astros can work fully in parallel, talking to personal agents
Mobile Agents: Coordinating Human-Robot Interactions
Utah Field Tests 2003 and 2004
• 50 Participants over 17 days
• 3 NASA centers & 2 universities
• Diverse scenarios, rough terrain
• 2 geologists; authentic science
Analog Simulation of Mars Surface Exploration
Cart and instrument
package
Cameras: • environmental pan/zoom Webcam• zooming closeup sensor videocam
• wide-field mat still camera
ScienceOrganizerInformation Repository
Experiment management
tools
Microbial mats
Greenhouse server w/ intelligent agent
Science team
• experiment design• experiment scheduling• automated instrument control• automated monitoring• automated data acquisition
Remote control XYZ positioning table
Greenhouse Collaboratory: A remote control scientific experimentation facility
Greenhouse
Rotor Wing
Mishap Has site:
Found at:
Thrust bearingThrust bearing
System under investigation:
Has subpart:
Operated by:
Has hypothesis:
Improper Bolt
Preload
Has design record:
Supported by:
Design schematic
Rotor assembly
Wind tunnel
Rotor pitch link
John Smith
Semantic Network overlays the SemanticOrganizer Information Repository
SemanticOrganizer Interface
A customizable knowledge management system supporting distributed teams of science and engineering investigators
Digital information repository stores heterogeneous investigation documents, information, and metadata; Repository features:
• semantic cross-linkage for rapid access to interrelated info (e.g., evidence & hypotheses)
• intuitive, domain-specific terminology• automated knowledge acquisition via inference
Serves as an archival record of investigation
Major SemanticOrganizer Applications
• Early Microbial Ecosystems/ NASA Astrobiology Institute
• Astrobiology Center for the Study of Biosignatures
• Tulane/NIH Malaria Study
• Mars Analog Drilling Mission
• Simulated Human-Robot Mars Surface Exploration
• Mars Exploration Rovers
• Columbia Accident Investigation
• CONTOUR Spacecraft Mishap Investigation
• Helios UAV Crash
• Moffett Airshow Investigation
• Canard Rotor Wing Investigation
ScienceOrganizer
InvestigationOrganizer
Right side panel displays metadata for the repository node being inspected
Left side panel uses semantic links to display all information related to the repository node shown on the right
Scientific KnowledgeManagement
Decision Support: Intelligent Launch & Range Operations
Spoken Dialog Assistant Systems
• Problem:Problem:– Astronauts perform numerous tasks where they need to both
obtain information about activities and use their hands.– Use of a keyboard in zero-gravity is cumbersome.
• Solution:Solution:– Ames has been developing spoken dialogue systems since
1999 to assist astronauts and allow them to interact with the computer using natural language.
– Systems are able to track the dialogue to maintain context
• CLARISSA Flight Experiment: Intelligent Procedures and CLARISSA Flight Experiment: Intelligent Procedures and ChecklistsChecklists – Allows astronaut to interact using natural language with an
intelligent procedure/checklist system.– Eliminates the need for an astronaut to read the procedure
while one does the procedure.– Strong support from the astronaut office.– Path-finding flight experiment opening the door to a wide range
of intelligent astronaut assistant technology.