Dr. Dana Ulery, Program ManagerDr. Dana Ulery, Program Manager

ARL Knowledge Fusion COE Program 2

Outline

BORG Challenge & Overall Goal Situation Semantics Knowledge as a Situated Process BORG Approach & Project Objectives Conclusion

ARL Knowledge Fusion COE Program 3

BORG: Battlefield Organic Robotic Grid

Technology centerpiece for the ARL Knowledge Fusion COE Program

BORG R&D environment implemented

at KIMCOE at MSU

Technology partners include Clark Atlanta,

U of MD, Monmouth, PNNL, & U of VA.

ARL Knowledge Fusion COE Program 4

Lifting the Fog of Terrorism & War

Situational Understanding Adversarial Reasoning Battle Management Action Aids

Quick Access to Actionable Information

Multiple, MultimediaInformation Sources

ENVIRONMENT Mobile High Stress Dynamic Uncertain Real-time

ARL Knowledge Fusion COE Program 5

Real-Time Attack Questions

Am I under attack? What is the nature of the attack? (who, where, what) What is the impact? (damage, response) What are they trying to do? What is their next action? What action should I take now? What action should I take longer term?

Currently, answering these questions is a majorchallenge at all levels of command.

ARL Knowledge Fusion COE Program 6

Real-Time Attack Answers

Sharp and focused Quickly understood Critical information available “at a glance” Fused from multiple, global & local key sources No information overload can be tolerated In-depth data on demand only

The BORG program strives to prove that Attack Answer tools can be automated using fusionmethodologies & techniques.

ARL Knowledge Fusion COE Program 7

Linguistic Situation Semantics

Early Situation Semantics (Barwise & Perry, 1983) & other mathematically-based semantic theories:

• Assume words, phrases, sentences have unique or canonical meanings• Relate situations to objects: for a given situation s and an object a, a is a constituent of s or it is not.

Situation Semantics: a branch of linguistic semantics • Concerned with how we understand each other in natural conversation• Concerned with the way we use social knowledge to communicate

A. Is there any water in the refrigerator? B1. Yes – the refrigerator coils have a lot of condensation built up. B2. No – but there’s some lemonade. B3. Yes – there’s a bottle of water with lemon in it to cover up the taste of rust from the pipes.

Every speech act occurs in a context, with a background shared by speakerand hearer (Winograd, 1985; Winograd & Flores, 1986)

Deficiencies of the Barwise-Perry Model

ARL Knowledge Fusion COE Program 8

Modern Theory of Meaning

Modern Situation Theory (Keith Devlin, 1990): 2 Notions of Meaning

abstract meaning: Answers the question “What does this sentence s mean ingeneral” where s is taken out of context.

meaning-in-use: Answers the question “What does this sentence s mean as itis being used in this instance?” where s is in a particular context.

Situations are not “set of objects” or “regions of space-time”, but fundamentalentities in their own right.

This is a significant shift from the linguists’ classical structural approachto language to the social scientists’ and psychologists’ procedural approach. The sociolinquist observes that language users rely on common cultural experiences, shared knowledge of language, and various interpretive strategiesto “understand” how to interpret what is said.

ARL Knowledge Fusion COE Program 9

Knowledge as a Situated Process

Situated Process: recent extension of Knowledge Based Systems technologies in KM, AI, CS, organizational behavior, psychology, cognitive and social science.

• Knowing is viewed as a community process that people collectively do, rather than a product an individual possesses.

• Emphasizes the role of interaction and context in intelligent behavior.

• Reinterprets the technologies of symbolic representation (Suchman, 1987; Lave, 1988, 1993):

– denies classical AI assumption of pre-existing internal representation– representation is moved to the cultural level, coordinated and constrained by interacting neural and social structures.

• Organizational behavior cannot be reduced to a function of autonomous decisions (Gammack & Stephens, 1997).

ARL Knowledge Fusion COE Program 10

Implications for Knowledge Fusion

Current State:

Information Fusion primarily combines research from: the engineering sciences dealing with sensors & complex dynamical systems the computer sciences, especially AI

Future State:

Knowledge Fusion must have stronger links to: linguistics cognitive science psychology social sciences

ARL Knowledge Fusion COE Program 11

BORG Approach to Knowledge Fusion

Situation Semantics and Situated Process are considered in project & system design.

Projects are linked by Army context and through systems integration.

An early BORG project defined and developed two Army contexts for BORG technology projects.

– significant tool for defining and studying interactions and relationships– strong aid for mapping research to problem

Considers Knowledge Fusion as a Situated Army Process and considers the prospects for developing Situational Understanding based on this view andSituation Semantics theory.

ARL Knowledge Fusion COE Program 12

BORG Project Objectives

Create an automated environment for researching Situational Understanding in an Army context.

Show that real-time Attack Answer tools can be automated using fusion technologies.

Include modern Situation Theory and Situated Process concepts to offer new perspectives in fusion research for Army applications.

Successfully transition Attack Answer tools to Army demonstrations.

ARL Knowledge Fusion COE Program 13

BORG Research Areas of Special Interest

Internet Technologies – a socially based shared memory Ontologies – advances to accommodate situation semantics

& processes Multimedia Interfaces – diagram interpretation & generation Information Understanding – evolved and understood in a

context Stories – narratives that structure histories to fit current

situations

ARL Knowledge Fusion COE Program 14

BORG Army Context Project

Wargaming methodology was adapted to focus on fusion research technology issues.

2 detailed scenarios – developed by US intelligence experts– battlefield– terrorist

Project is complete & documented.

ARL Knowledge Fusion COE Program 15

Scenario: Civil War and Peninsular Conflict

Kim Jong-il dies suddenly.

Over next several months:

– Small group of more moderate leaders announce their assumption of control of the central government.

Moderate’s View: Detrimental to survival of the country to continue hardened stance against the West and South.

Moderate’s Intentions: Open a more active dialogue. Status: Unable to gain complete control or vows of loyalty from

all Party members loyal to Kim Jong-il’s legacy, and ominously, from some of the more hard-line elements within the military.

– Hardliners oppose moderates. See their positions of relative privilege and power being placed

in jeopardy. Supporters: Include the Navy Chief of Staff, who has gained

the backing of several senior officers in Pyongyang military headquarters, and two corps commanders, one near the DMZ and one north of Pyongyang.

ARL Knowledge Fusion COE Program 16

3 days ago: FBI issued chemical, biological, or radiological (CBR) attack alert.

Attack is imminent.

Threat information obtained from Guantanamo Bay detainee.

Cousin is local “middleman” for catastrophic event planning

Incident in Northeast US and fairly soon.

NATIONWIDE SECURITY ALERT

““MIDDLEMAN”MIDDLEMAN”

ARL Knowledge Fusion COE Program 17

Conclusion

The BORG program strives to prove that Attack Answer tools can be automated using fusion methodologies & techniques.

BORG considers Knowledge Fusion as a Situated Army Process and considers the prospects for developing Situational Understanding based on this view and Situation Semantics theory.

Multiple BORG Projects are in progress.

ARL Knowledge Fusion COE Program 18

References

Barwise, J. and Perry, J. Situations and Attitudes, Bradford Books, MIT Press (1983).

Devlin, K. Extending Barwise and Perry’s Relational Theory of Meaning, presentedat the Jon Barwise Memorial Conference, Stanford University, 2003.http://www.cs.tcd.ie/Tim.Fernando/B/DevlinPaper.pdf

Gammack, J. G. and Stephens, R. A. Knowledge Acquisition as a SituatedProcess: Implications for Information Systems Design, in Proceeding 3rd

Australian Knowledge AcquisitionWorkshop at 10th AJCAI, 1997.http://www.cse.unsw.EDU.AU/~timm/pub/aka97/papers/

Winograd, T. Moving the Semantic Fulcrum, Linguistics and Philosophy 8(1), 1985.

Winograd, T. and Flores, F. Understanding Computers and Cognition, AblexPublishing, 1986.