GILA Explanation Component(Phase 2)

Oct 5, 2008

Outline

• Overview • Background• Conceptual Model• Implementation

– Browser2: GILA log data browser– TW OIE: OWL Instance Data Evaluation– GilaExplainer: GILA log analyzer and explainer

• Future Work

Overview

Overview

• GILA is a general purposed integrated multi-agent platform for learning and problem-solving

• GILA aims at solving problems by learning from several examples

• Currently GILA is evaluated by a conflict resolution scenario in battle field air space control domain– Each run of GILA produces an OWL log recording how

agents are learning and solving problem– Explanation component is needed to show how the

background knowledge and examples are used in learning and solving problems

Research Problems

• GILA log validation– Overall log data connectivity validation (e.g. CC analysis

on instance data)– Individual log data structure validation (e.g. check GILA

log using integrity constrains from GILA ontologies )

• Explain data flow– Associate final solution with initial conflicts– Attribute the contribution of ILRs– Associate solutions with the provided knowledge

Background

GILA Data Driven Computing• [User] asks [MRE-DM] to initiate a task

– To learn from experts’ knowledge • one expert trace • several expert exercises

– To solve ACO Problem• [MRE-DM] queues the tasks, i.e. learning, CPL demonstration,

CPL-training, performance, and run one task a time– [MRE-DM] informs [ALL] task begins– [ALL] work with each other

• [LearnerX] Ask a Problem on BB• [LearnerY] Reply with the corresponding Solution on BB• [LearnerY] Reply with No-More-Solution on BB

– [MRE-DM] informs [ALL] task ends (succeed or failed)

Provided Knowledgeruntime inputruntime input

prior input prior input

Learned Knowledge

Final Output

Abstract GILA Data Flow

Experts’ execution trace

background knowledge

constraint-violation

final solution ExecutionTrace

1: learning

2. performance

Q/A from users

facts embedded inthe input problem

problem/solution

constraints

conflict priority

multi-phase iteration

Prior Knowledge from Exports• learning mode

– One expert trace• execution trace• initial state• final state

• CPL (demonstration; practice) mode– Several exercises

• initial state• final state

• performance mode – One problem, i.e.

the initial state

Expert’s example(ExecutionTrace)

• pstep_1• pstep_2• ….• pstep_n

Initial State

• ACO• ACMReq

Final State

• ACO• ACMReq

Initial State

• ACO• ACMReq

Initial State

• ACO• ACMReq

Final State

• ACO• ACMReq

GILA KnowledgeKnowledge Category Producer Consumer OWL class

Execution trace Provided MREs, ILRs gilcore:ExecutionTrace

Input ACO problem Provided MREs, ILRs gilaco:ProblemACO ?

Other Background knowledge Provided Who? ACO ontology?

Constraint Learned CL ILRs (who?) gilcore:Constraint

Constraint Violation Learned SC MRE, ILRs gilaco:ViolationStatement

ACO problem state (context) Learned MRE ILRs gilaco:ProblemACO

Cost of an ACO problem state Learned DTL MRE gilaco:CostStatement

Conflicts in an ACO problem state Learned 4D MRE gilcore:Conflict

Order of conflicts Learned ILRs MRE TBD

Intersection of conflict Learned 4D MRE gkst:IntersectionDetails

Learning Goal Learned MRE-ML ILRs

Choice of partial solution Learned MRE-DM ILRs

Final solution Learned MRE-DM PstepManager gilcore:PstepList

legend

Domain Knowledge

World Knowledge

GILA Ontology Dependency (imports) Graph

Airspace Control Order (aco)

GILA Inter-component Language(gilcore)

Data Structure(ds)

Abstract Steps(Asteps)

PML-Provenance(pmlp)

PML-Justification(pmlj)

Constraint(cons)

Partial-plan Steps(Psteps)

GIL - ACO domain(gilaco)

Sensing Steps(Psteps)

General Knowledge Spatial Temporal

(gkst)

changed

unchanged

new

Conceptual Model

Event based Provenance Modelfor one-step in the log

Event

Input Data

Output Data

Agent

Operation

State i

State i+1

about

about

Time

Location

Input Data

Output Data

…

…

Can be referred as PML Lite Ontology

ACO State i

ACO State i+1

Example

ACO

WorkingACMReq1

ACO

WorkingACMReq2

Implementation

Browser2: GILA log data browser

• GILA log consists of OWL instances, and they are interconnected

• This tool let users – navigate instances by their connections – look into detailed description of instances

• Note– Some links may fail because not all GILA

ontologies are available on the Web.

List all OWL Instances by Type

Navigate One Instance and its related Instances

type

outlink

inlinks

Details

Show the details of an Instance and its embedded isntances

TW OIE: OWL Instance Data Evaluation

• Motivation– Log entries are encoded as OWL instance data– As log entries are generated by ILRs and MREs, they may

miss some required fields

• OWL instance Data checks integrity constraints – e.g. missing property value, unspecific instance type– Currently implemented using SPARQL– http://onto.rpi.edu/demo/oie/

Load an instance file

Evaluation Result

GilaExplainer: Explaining GILA log

• Extract generic structure from log – Generate PML Lite relation from GILA log– Convert RDF graph to Instance graph

• SPARQL based Explanation Template

RDF graph V.S. Instance Graph• Focus on subject of Resource

(which is described)• Skip classes and properties

Conflict1

ACM1 ACM2

Has Identifier

ACM1_ACM2_CONFLIT

“ACM F4” “ACM FUEL 1”

Has IdentifierHas Identifier

Shape2Oval

hasConflictingACMshasConflictingACMs

Connectivity Analysis (Initial Results)

• Goal: to check if GILA-log is well-connected

• Input data– OWL file, 22M– No blank node.

• Approach– Create instance graph from RDF graph

• Initial results:– Many Islands, e.g. instance of

constraint, not linking to any other instances

– One Big connected component (2M)– Some small components (about 5K)

RDF graph Instance graph

triples 187,881 47,138

subjects 57,208 20,314

objects 47807 40,660

No-inlink 16548 323

No-outlink 7,147 20,669

Multi-Step Explanation for End Users

ACMReq’(final state)

ACM’

Psteps

ACMReq(Initial state)

1. Which ACMs have been changed?• via which psteps by whom?

2. How each conflict is resolved• by which psteps with what constraint blame

ConflictIntersectionDetails

<owl:Class rdf:about="http://www.mindswap.org/2006/GILA/GK/gkst.owl#IntersectionDetails">

ACM

Constraint

Sparql based Explanation Template“List ID of all ACMs involved in conflict”

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX aco: <http://www.atl.lmco.com/aco.owl#> SELECT ?conflictid ?acmidFROM <http://tw.rpi.edu/2008/gila/log1/g_con.rdf>WHERE { ?acm rdf:type aco:ACMDescriptor . ?acm aco:hasIdentifier ?acmid . ?conflict rdf:type aco:Conflict . ?conflict aco:hasIdentifier ?conflictid . ?conflict aco:hasConflictingACMs ?acm . ?conflict aco:hasConflictingACMs ?acm .}

Future Work

• Scalable GILA log storage and reasoning– Deal with log dumps generated by different system executions– Scalable Reasoning and Query support

• Enriching Explanation – More log entries from ILRs and MREs – Finer reference to provided knowledge– User-defined explanation

• Knowledge discovery– Duplicated entries– Log summary– Identify patterns of ILRs’ solutions, uncover interesting/strange

behaviors – A frequent set of behavior patterns, which are explainable to end-

users, shared by GILA components

Backup

Log Entity Duplication Detection• Goal: detect individual duplication using CWA.• Observations:

– There could be some duplicated OWL individuals in log that can be detected by

• IFP (one identical property-value pair)• Identical KEY ( multiple identical PV pairs)• Identical content ( all identical PV pairs)

– We may need to ignore temporal aspect, and adhere to Close World Assumption for now

– GILA ontology has not IFP defined• Directions

– Efficient duplication detection using hash function?– Simple delta computation?

Log summary

• Goal: Provide human operator summary (at different granularity?)

• Hypothesized summary entries– Overall size in terms of bytes, triples, resources,

literals…– Topology analysis, e.g. connected components– Term frequency, e.g. list of # of class instances– …