Post on 13-Jan-2015
description
transcript
U.S. Army Research, Development and Engineering Command
A perspective on the role
of AI and interactive
digital media
in military training
Presentation to the AI and Interactive Digital Entertainment Conference (AIIDE)October 2010Stanford University
Robert A. Sottilare, Ph.D.Chief Technology OfficerSFC Paul Ray Smith Simulation & Training Technology Centerrobert.sottilare@us.army.mil
SFC Paul Ray Smith Simulation & Training Technology Center
Disclaimer
• I will be talking about commercial products in addition
to government-developed technology
• What I say should not be construed as an endorsement
of any commercial products
• Much of what will be covered in this talk has application
outside of a military context
• Challenge is the word of the day…
SFC Paul Ray Smith Simulation & Training Technology Center
Agenda
• A little about my organization and my background…
• Motivation to exploit AI and IDM for military training
• AI and IDM in serious games and military training
• Opportunities to exploit AI and IDM
• Challenges and Next Steps
SFC Paul Ray Smith Simulation & Training Technology Center
My Organization
SFC Paul Ray Smith
Simulation & Training Technology Center
part of the
Army Research Laboratory
SFC Paul Ray Smith Simulation & Training Technology Center
A little about me…
• you can find me on
• 25+ years in military training & simulation
research, development and acquisition
• My research interests are in adaptive
tutoring
• my research is conducted in the Learning in
Intelligent Tutoring Environments (LITE) Lab
• and yes… I like the Simpsons
LITE Lab
SFC Paul Ray Smith Simulation & Training Technology Center
Motivation to exploit AI and Interactive
Media for Military Training
SFC Paul Ray Smith Simulation & Training Technology Center
Motivation to exploit AI and
Interactive Media for Military Training
• U.S. Army is deployed worldwide on a consistent basis
so training technology needs to be:
– accessible wherever the Soldier is located
– interactive regardless of infrastructure available
– intuitive so it can be used in the absence of instructors
• The time allotted for training is limited so training
needs to be:
– efficient as well as effective
– engaging, challenging and relevant to the Soldier’s
mission
– adaptive to the trainee’s needs and capabilities
Today, Soldiers do more than their traditional missions…
SFC Paul Ray Smith Simulation & Training Technology Center
Learning & Entertainment
[1] Gagne, R. M. (1965). The conditions of learning. New York: Holt, Rinehart and Winston.
Learning Objectives1
• Gain attention
• Inform
• Stimulate recall
• Present stimulus
material
• Provide learner
guidance
• Elicit performance
• Provide feedback
• Assess performance
• Enhance retention
transfer
LEARNING FUN
Gain/maintain attention
Inform
Stimulate interaction
Stimulate immersion
Motivate
Guide
SFC Paul Ray Smith Simulation & Training Technology Center
Training
the process of bringing a person,
team or organization to an
agreed standard of proficiency
by practice and instruction
• relevant task and conditions
• standards
• measures of performance
• constructive and timely feedback
SFC Paul Ray Smith Simulation & Training Technology Center
Exploit AI and Interactive Digital
Media to improve Soldier…
• learning: acquisition of knowledge and/or skill
• performance: accomplishment of a given task
measured against some standard
• retention: ability to perform a learned behavior (task)
over time without the reinforcement of practice
• competence: demonstrated level of ability through
performance over time; potential for future performance
SFC Paul Ray Smith Simulation & Training Technology Center
Available and emerging
technologies
• intelligent technologies for training
– adaptive tutoring systems
– virtual humans
• serious games
• virtual worlds
• mobile learning
SFC Paul Ray Smith Simulation & Training Technology Center
PURPOSE: Design, develop, apply and
evaluate intelligent agent technologies
(tools and methods) to enhance training,
leader development and education; and
to reduce associated support costs
TECHNICAL CHALLENGES: Application
of intelligent agents in ill-defined domains;
understanding natural language in multi-
sided conversations; interrupted
conversations team tutoring; rapid
authoring; and virtual human cognition
Intelligent Technologies for Training
OBJECTIVE: Use intelligent agents to
supplement one-to-one and one-to-many
TLE experiences where human support is
limited, impractical or completely
unavailable
Virtual Humans
Adaptive Tutors
SFC Paul Ray Smith Simulation & Training Technology Center
Intelligent Technologies for Training
Virtual HumansAdaptive Tutors
use real behaviors and
physiological
measures to model and
predict trainee
cognitive states
use simulated
cognitive states to
drive virtual human
behaviors
SFC Paul Ray Smith Simulation & Training Technology Center
TECHNICAL CHALLENGES: Determining
optimal sensor suite to accurately/passively
identify cognitive state; development of machine
learning models to adapt instruction
Adaptive Tutoring Research
PURPOSE: Enable computer-based intelligent
tutors to adapt to trainee “readiness-to-learn”
by evaluating the influence of performance
variables in the trainee model
OBJECTIVE: Experimentally evaluate trainee
state variables (self-reported, observed,
physiological and behavioral) to accurately/
passively assess the trainee’s cognitive state
& adapt content/interaction to meet trainee
needs
sensors predicted trainee state instructional strategy selection
SFC Paul Ray Smith Simulation & Training Technology Center
Individual Tutoring Systems
Sottilare, R. (2010). Toward the Development of an Intelligent Tutoring System for Distributed Team Training through Passive Sensing. In
Proceedings of the 10th Intelligent Tutoring Systems Conference, Pittsburgh, June 2010.
SFC Paul Ray Smith Simulation & Training Technology Center
Sottilare, R. (2010). Toward the Development of an Intelligent Tutoring System for Distributed Team Training through Passive Sensing. In
Proceedings of the 10th Intelligent Tutoring Systems Conference, Pittsburgh, June 2010.
Team Tutoring Systems
SFC Paul Ray Smith Simulation & Training Technology Center
Adaptive Tutoring Systems
• Few military training simulations today implement intelligent tutoring…
• In most of those cases, the trainee model is mainly focused on performance goals and know little or nothing about:
– trainee cognition (including affect)
– trainee past performance (competence level)
• Many of today’s tutors focus in well-defined domains (e.g. physics, math) and the world is messy…
• The difficult work is ahead of us…
SFC Paul Ray Smith Simulation & Training Technology Center
Virtual Humans
• What parts of real people are represented in virtual
humans?
– physical appearance, behaviors and interaction
– limited cognition (including affect)
– culture, values?
– goals, beliefs
– social interaction
– trust
– persistence…
Arnold, Lazarus, Frijda, Scherer, Ortony et al.
SFC Paul Ray Smith Simulation & Training Technology Center
Appraisal Theory
Arnold, Lazarus, Frijda, Scherer, Ortony et al.
courtesy of Stacy Marsella, Institute for Creative Technologies
SFC Paul Ray Smith Simulation & Training Technology Center
Virtual Humans
• Many current and potential applications:
– training partners, adversaries or neutral characters
– tutoring, mentoring, coaching
– marketing
– social support
– entertainment
SFC Paul Ray Smith Simulation & Training Technology Center
Virtual Humans
courtesy of Jon Gratch, Institute for Creative Technologies
SFC Paul Ray Smith Simulation & Training Technology Center
• Create medically accurate virtual patients
with advanced interaction capabilities
• Train Combat Medics on the proper
procedures for sick call
TECHNICAL CHALLENGES:
• Identifying/creating appropriate and meaningful medical dialog/content
• Developing adaptive, realistic human behaviors
Virtual Patients/Virtual Sick Call
SFC Paul Ray Smith Simulation & Training Technology Center
PURPOSE:
Develop a capability for sustainment
training that included interrogations of
detainees, civilians and persons of
interest
TECHNICAL CHALLENGES:
• Speech and natural language processing
• Question and Answer type of conversations
• Authoring capability to create new characters
Tactical Questioning System
OBJECTIVE:
• Develop 3D Interactive environment
• Soldier interacts with virtual humans
to sustain their interrogation skills in
realistic manner
• Utilize translator and foreign
languages
SFC Paul Ray Smith Simulation & Training Technology Center
Multi-national Training Experiment
using Virtual Battlespace 2
• uses commercial game
• primarily kinetic operations
• distributed shared environment
SFC Paul Ray Smith Simulation & Training Technology Center
UrbanSim
PC-based game that allows Battalion and
Brigade Commanders to practice situational
understanding and operations in complex,
contemporary environments.
Key Learning Objectives:
– Develop and maintain situational
understanding
– Develop Commander’s intent
– Develop Commander’s critical information
requirements
– Direct operations
– Anticipate 2nd and 3rd order effects
– Command outside the chain of command
SFC Paul Ray Smith Simulation & Training Technology Center
Bi-lateral Negotiation
Simulation (BiLAT)
Challenge – combine pedagogy, game play design, interactiveresearch technologies, and software development to address key objectives
PC, game engine-based, cognitive training tool used
for developing skills in how to plan for and conduct
bi-lateral meetings and/or negotiations
in different cultural settings
Key Learning Objectives:
Negotiation Strategy: Planning and executing
tactics that help achieve mission objectives
Cultural Awareness: Applying culturally-
appropriate interpersonal skills
Trust-Building Strategy: Developing/applying
tactics to gain negotiating partners’ trust
SFC Paul Ray Smith Simulation & Training Technology Center
Social and Cultural Modeling
Bi-lateral Negotiation
Simulation (BiLAT)
SFC Paul Ray Smith Simulation & Training Technology Center
Virtual Worlds
• Next Generation VW Platforms must be:
– scalable – work across operating systems and devices
– easily deployed
– secure
– open
– interoperable
– future proof - scale dynamically
– economically sound – good business model
David Smith, Chief Innovation Officer, Lockheed Martin Global Training and Logistics
A Virtual Worlds Architecture Framework: A Shorter Path to Mastery for All DOD Personnel
SFC Paul Ray Smith Simulation & Training Technology Center
OnLine Interactive
Virtual Environment (OLIVE)
SFC Paul Ray Smith Simulation & Training Technology Center
Multinational Training Experiment
using OLIVE
SFC Paul Ray Smith Simulation & Training Technology Center
Enhanced Dynamic Geo-social Environment (EDGE)
3
5
Blends traits from MMO, serious games and virtual world technology to replicate the operational environment as accurately as possible.
SFC Paul Ray Smith Simulation & Training Technology Center
Federal Virtual Worlds Challenge
• What is the Federal Virtual Worlds Challenge?
• It is an open, global Challenge to the world for the best of
the best implementations within virtual environments.
• The event is conducted to explore innovative and interactive
solutions in virtual environments. The criteria are
intentionally unbounded to encourage creative results.
• This year the focus is on artificial intelligence!
• http://www.fvwc.army.mil/index.php
SFC Paul Ray Smith Simulation & Training Technology Center
Challenges and Next Steps
• Economic challenges
• AI for training challenges
• IDM for training challenges
• Interaction design challenges
• Grand challenges for educational technology
• Next steps for educational technology
SFC Paul Ray Smith Simulation & Training Technology Center
Economic Challenges
• The U.S. Army is a large population distributed
worldwide and covering a variety of occupational
specialties
– Maintaining the proficiency of this population is challenging
and expensive
– New missions = new training content
– New personnel and equipment = continuous training
• AI technology (tools and methods) are needed to:
– reduce training costs
– while maintaining training effectiveness
– and improving availability and adaptability
SFC Paul Ray Smith Simulation & Training Technology Center
AI for Training Challenges
• Adaptable tutoring systems
– improve perception/prediction of trainee’s state
– improve assessment of learning, performance, competence and retention
– improve selection of appropriate instructional strategies based on the trainee’s state (e.g. competency level)
• content, tempo, challenge, flow and feedback
– automate authoring of content and expert models
SFC Paul Ray Smith Simulation & Training Technology Center
IDM for Training Challenges
• Interactive mixed reality
training environments
– everyone wants the holodeck,
but its not economically or
technically feasible now
– explore alternative evolutionary
steps to get holodeck-like
capabilities now
– natural locomotion
– no adverse effects (e.g.
simulator sickness)
– real-time, distributed effects
SFC Paul Ray Smith Simulation & Training Technology Center
Interaction Design
Challenges for AI and IDM
• Trainee situational awareness
comparable to “live”
environments
• Passive machine perception
of trainee and “live”
environment
• Natural Language
Understanding & Generation
• Natural locomotion in virtual
environments
SFC Paul Ray Smith Simulation & Training Technology Center
Grand Challenges for
Educational Technology
• Personalize Education
• Assess Student Learning
• Support Social Learning
• Diminish Boundaries
• Develop Alternative Teaching Methods
• Enhance the Role of Stakeholders
• Address Policy Changes
Woolf, B. P. (2010). A Roadmap for Education Technology. National Science Foundation # 0637190
SFC Paul Ray Smith Simulation & Training Technology Center
Next Steps for
Educational Technology
• User Modeling
• Mobile Training
– Sensing trainee state during individual training
– Interactive team training and performance assessment
• Networking Tools
• Serious Games
• Intelligent Environments
• Educational Data Mining (authoring tools)
– Content
– Models (student models, expert models…)
• Rich Interfaces
Adapted from: Woolf, B. P. (2010). A Roadmap for Education Technology. National Science Foundation # 0637190