Ontology DevelopmentOntology DevelopmentTo SupportTo Support

AQUAAQUA Question-Answering Question-Answering

Richard FikesRichard Fikes

Jessica Jenkins Bill MacCartney Rob McCool Deborah McGuinnessJessica Jenkins Bill MacCartney Rob McCool Deborah McGuinness

Knowledge Systems LaboratoryKnowledge Systems LaboratoryStanford UniversityStanford University

www.ksl.stanford.eduwww.ksl.stanford.edu

Knowledge Systems Laboratory, Stanford University

2

KSL and the WMD CoalitionKSL and the WMD Coalition Tools for ontology creation, evolution, and maintenanceTools for ontology creation, evolution, and maintenance

Coalition teams have adopted KSL’s Ontolingua and Chimaera as Coalition teams have adopted KSL’s Ontolingua and Chimaera as a standard for ontology development, maintenance, and analysis a standard for ontology development, maintenance, and analysis

(Additionally, we use other internal tools – JTP, DQL, IW)(Additionally, we use other internal tools – JTP, DQL, IW)

Initial evaluation and ongoing supportInitial evaluation and ongoing support KSL evaluated initial stage of WMD ontology using our Chimaera KSL evaluated initial stage of WMD ontology using our Chimaera

tools, reviewed findings with other teams, and taught others how tools, reviewed findings with other teams, and taught others how to use the tools themselvesto use the tools themselves

Knowledge Systems Laboratory, Stanford University

3

KSL and the WMD CoalitionKSL and the WMD Coalition

Tools for ontology creation, evolution, and maintenanceTools for ontology creation, evolution, and maintenance

Initial evaluation and ongoing supportInitial evaluation and ongoing support

Knowledge representation and consulting workKnowledge representation and consulting work KSL providing some core new KB development for CNS core – KSL providing some core new KB development for CNS core –

Russian naval facilities and Newly Independent States facilities – Russian naval facilities and Newly Independent States facilities – and is augmenting this with extraction resultsand is augmenting this with extraction results

Providing consultation on sources / merging opportunities -Providing consultation on sources / merging opportunities -Counter-terrorism KBs built for DARPA (HPKB, ISX-Cyladian- Counter-terrorism KBs built for DARPA (HPKB, ISX-Cyladian- HORUS, SAIC-Cycorp, … ) semantic web for the military (SWMU HORUS, SAIC-Cycorp, … ) semantic web for the military (SWMU ontology tutorial), ontologies for information fusion, etc.ontology tutorial), ontologies for information fusion, etc.

Knowledge Systems Laboratory, Stanford University

4

KSL and the WMD CoalitionKSL and the WMD Coalition Tools for ontology creation, evolution, and maintenanceTools for ontology creation, evolution, and maintenance

Initial evaluation and ongoing supportInitial evaluation and ongoing support

Knowledge representation and consulting workKnowledge representation and consulting work

Knowledge extractionKnowledge extraction KSL knowledge extraction tools for RNF and NISKSL knowledge extraction tools for RNF and NIS

New focus on utilizing other important useful KB sources New focus on utilizing other important useful KB sources SUMO is a core ontology for ontology sharing: ~3,900 axioms; SUMO is a core ontology for ontology sharing: ~3,900 axioms;

relations and sets; processes and objects; temporal, spatial, and relations and sets; processes and objects; temporal, spatial, and mereological relations; agents, etc.mereological relations; agents, etc.

Domain ontologies: WMDs, terrorism, biological viruses, … Domain ontologies: WMDs, terrorism, biological viruses, … Written in SUO-KIF, a proprietary dialect of KIFWritten in SUO-KIF, a proprietary dialect of KIF Published by Teknowledge under GNU public license as part of Published by Teknowledge under GNU public license as part of

IEEE SUO working group (ontology.teknowledge.com/)IEEE SUO working group (ontology.teknowledge.com/)

Knowledge Systems Laboratory, Stanford University

5

SUMOSUMO

SUMO requires translation to be used with any reasonerSUMO requires translation to be used with any reasoner

KSL has successfully translated SUMO to plain-vanilla KIFKSL has successfully translated SUMO to plain-vanilla KIF Full translation: complete semantic content retainedFull translation: complete semantic content retained Highly portable: should be fully compatible with most FOL reasonersHighly portable: should be fully compatible with most FOL reasoners Accurate: most test queries demonstrably answerable from translated Accurate: most test queries demonstrably answerable from translated

axiomsaxioms

However, the result is not yet fully usableHowever, the result is not yet fully usable Most test queries not answered from full axiom set in reasonable timeMost test queries not answered from full axiom set in reasonable time SUMO was not designed for efficient automated reasoningSUMO was not designed for efficient automated reasoning Solution: a smarter translator, and some SUMO “brain surgery”Solution: a smarter translator, and some SUMO “brain surgery”

Translation to DAML may provide another path…Translation to DAML may provide another path… The existing translation is quite lossy, but enables some query-answeringThe existing translation is quite lossy, but enables some query-answering Further work will enable a more complete and accurate translationFurther work will enable a more complete and accurate translation

Knowledge Systems Laboratory, Stanford University

6

DAML Versions of SUMO, WMD, and TerrorismDAML Versions of SUMO, WMD, and Terrorism

DAML translations of SUMO, WMD, and 5 terrorism-related ontologies DAML translations of SUMO, WMD, and 5 terrorism-related ontologies and knowledge bases (KBs) provided by Teknowledgeand knowledge bases (KBs) provided by Teknowledge

ClassesClasses PropertiesProperties InstancesInstances TriplesTriplesDroppedDropped

AxiomsAxioms

SUMOSUMO 577577 188188 6868 35243524 ~800~800

WMDWMD 186186 99 6969 10211021 5959

TerrorismTerrorism

OntologyOntology8484 00 00 200200 4444

TerrorismTerrorism

KBsKBs22 11 28922892 96689668 29192919

TotalTotal 849849 198198 30293029 1441314413 38223822

Knowledge Systems Laboratory, Stanford University

7

DAML Versions of SUMO, WMD, and TerrorismDAML Versions of SUMO, WMD, and Terrorism

A few simple translations were used by Teknowledge to translate A few simple translations were used by Teknowledge to translate original KIF content to DAMLoriginal KIF content to DAML

Example: subrelationExample: subrelation

(subrelation father parent)(subrelation father parent)<daml:ObjectProperty rdf:ID=“father”><daml:ObjectProperty rdf:ID=“father”> <rdfs:subPropertyOf rdf:resource=“#parent”/><rdfs:subPropertyOf rdf:resource=“#parent”/></daml:ObjectProperty></daml:ObjectProperty>

Example: KIF triples to RDF triplesExample: KIF triples to RDF triples

(part MadridSpain Spain)(part MadridSpain Spain)<rdf:Descriptioin rdf:ID=“MadridSpain”><rdf:Descriptioin rdf:ID=“MadridSpain”> <part rdf:resource=“#Spain”/><part rdf:resource=“#Spain”/></rdf:Description></rdf:Description>

Knowledge Systems Laboratory, Stanford University

8

DAML Versions of SUMO, WMD, and TerrorismDAML Versions of SUMO, WMD, and Terrorism

Teknowledge’s DAML files provide a great starting point, but there are Teknowledge’s DAML files provide a great starting point, but there are a few problemsa few problems Syntactic errors and issues with resolving references across files -- these Syntactic errors and issues with resolving references across files -- these

problems are easy to fix.problems are easy to fix. A large amount of the original KIF content was dropped in the translation A large amount of the original KIF content was dropped in the translation

to DAML. Reincorporating some of this content is trivial, but it is generally to DAML. Reincorporating some of this content is trivial, but it is generally nontrivial.nontrivial.

(instance part TransitiveRelation)(instance part TransitiveRelation)

Example (trival): TransitiveRelationExample (trival): TransitiveRelation

<daml:TransitiveProperty rdf:ID=“part”><daml:TransitiveProperty rdf:ID=“part”> … …</daml:TransitiveProperty></daml:TransitiveProperty>

Knowledge Systems Laboratory, Stanford University

9

KIF -> DAML Example (nontrivial)KIF -> DAML Example (nontrivial)

(=> (and (instance ?SUBSTANCE BiochemicalAgent)(=> (and (instance ?SUBSTANCE BiochemicalAgent)(possesses ?AGENT ?SUBSTANCE))(possesses ?AGENT ?SUBSTANCE))

(capability BiochemicalAttack agent ?AGENT))(capability BiochemicalAttack agent ?AGENT))

Original KIFOriginal KIF

Translation of Translation of capabilitycapability [ternary to binary relation][ternary to binary relation]<rdfs:Class rdf:ID=“Capability”/><rdfs:Class rdf:ID=“Capability”/><rdf:Property rdf:ID=“capabilityRole”><rdf:Property rdf:ID=“capabilityRole”> <rdfs:domain rdf:resource=“#Capability”/><rdfs:domain rdf:resource=“#Capability”/> <rdfs:range rdf:resource=“#CaseRole”/><rdfs:range rdf:resource=“#CaseRole”/></rdf:Property></rdf:Property><rdf:Property rdf:ID=“capabilityProcess”><rdf:Property rdf:ID=“capabilityProcess”> <rdfs:domain rdf:resource=“#Capability”/><rdfs:domain rdf:resource=“#Capability”/> <rdfs:range><rdfs:range> <daml:Restriction><daml:Restriction> <daml:onProperty rdf:resource=“&rdfs;#subClassOf”/><daml:onProperty rdf:resource=“&rdfs;#subClassOf”/> <daml:hasValue rdf:resource=“#Process”/><daml:hasValue rdf:resource=“#Process”/> </daml:Restriction></daml:Restriction> </rdfs:range></rdfs:range></rdf:Property></rdf:Property><rdf:Property rdf:ID=“capability”><rdf:Property rdf:ID=“capability”> <rdfs:range rdf:resource=“#Capability”/><rdfs:range rdf:resource=“#Capability”/></rdf:Property></rdf:Property>

Knowledge Systems Laboratory, Stanford University

10

KIF -> DAML Example (nontrivial) cntd.KIF -> DAML Example (nontrivial) cntd.

(=> (and (instance ?SUBSTANCE BiochemicalAgent)(=> (and (instance ?SUBSTANCE BiochemicalAgent)(possesses ?AGENT ?SUBSTANCE))(possesses ?AGENT ?SUBSTANCE))

(capability BiochemicalAttack agent ?AGENT))(capability BiochemicalAttack agent ?AGENT))

Original KIFOriginal KIF

Translation of “?AGENT has BiochemicalAttack capability if it Translation of “?AGENT has BiochemicalAttack capability if it possesses a BiochemicalAgent”possesses a BiochemicalAgent”

<sumo:Capability rdf:ID=“BiochemicalAttackAgentCapability”><sumo:Capability rdf:ID=“BiochemicalAttackAgentCapability”> <sumo:capablityRole rdf:resource=“&sumo;#agent”/><sumo:capablityRole rdf:resource=“&sumo;#agent”/> <sumo:capabilityProcess rdf:resource=“#BiochemicalAttack”/><sumo:capabilityProcess rdf:resource=“#BiochemicalAttack”/></sumo:Capability></sumo:Capability><rdfs:Class rdf:ID=“AgentsWithBiochemicalAttackCapability”><rdfs:Class rdf:ID=“AgentsWithBiochemicalAttackCapability”> <rdfs:subClassOf><rdfs:subClassOf> <daml:Restriction><daml:Restriction> <daml:onProperty rdf:resource=“&sumo;#capability”/><daml:onProperty rdf:resource=“&sumo;#capability”/> <daml:hasValue rdf:resource=“#BiochemicalAttackAgentCapability”/><daml:hasValue rdf:resource=“#BiochemicalAttackAgentCapability”/> </daml:Restriction></daml:Restriction> </rdfs:subClassOf></rdfs:subClassOf> <daml:unionOf rdf:parseType=“daml:collection”><daml:unionOf rdf:parseType=“daml:collection”> <daml:Restriction><daml:Restriction> <daml:onProperty rdf:resource=“&sumo;#possesses”/><daml:onProperty rdf:resource=“&sumo;#possesses”/> <daml:hasClass rdf:resource=“#BiochemicalAgent”/><daml:hasClass rdf:resource=“#BiochemicalAgent”/> </daml:Restriction></daml:Restriction> … …

Knowledge Systems Laboratory, Stanford University

11

Query-Answering Example 1Query-Answering Example 1 ““What has the capability of being the agent of a biochemical attack?”What has the capability of being the agent of a biochemical attack?”

Query pattern: (capability ?agt Biochemical-Attack-Agent-Capability)Query pattern: (capability ?agt Biochemical-Attack-Agent-Capability)

Knowledge in the ontology:Knowledge in the ontology: A thing is an “Agent-With-Biochemical-Attack-Capability” if and only if it –A thing is an “Agent-With-Biochemical-Attack-Capability” if and only if it –

> Has a capability “Biochemical-Attack-Agent-Capability” orHas a capability “Biochemical-Attack-Agent-Capability” or> Possesses a “Biochemical-Agent”Possesses a “Biochemical-Agent”

An “Agent-With-Biochemical-Attack-Capability” has capability “Biochemical-Attack-Agent-An “Agent-With-Biochemical-Attack-Capability” has capability “Biochemical-Attack-Agent-Capability”Capability”

A “Nerve-Agent” is a “Biochemical-Agent”A “Nerve-Agent” is a “Biochemical-Agent”

If AGT has capability “Biochemical-Attack-Agent-Capability”, then AGT is capable of being If AGT has capability “Biochemical-Attack-Agent-Capability”, then AGT is capable of being an “agent” in a “Biochemical-Attack”an “agent” in a “Biochemical-Attack”

If C is the capability of playing role R in processes of type PT, and AGT is known to have If C is the capability of playing role R in processes of type PT, and AGT is known to have played role R in a process of type PT, then AGT has capability Cplayed role R in a process of type PT, then AGT has capability C

Knowledge from documents:Knowledge from documents: ““Al-Qaida” is a “Foreign-Terrorist-Organization” that possesses a “Nerve-Agent”Al-Qaida” is a “Foreign-Terrorist-Organization” that possesses a “Nerve-Agent”

““Aum-Supreme-Truth-Chemical-Attack-27-Jun-94 is a “Chemical-Attack” whose agent is Aum-Supreme-Truth-Chemical-Attack-27-Jun-94 is a “Chemical-Attack” whose agent is “Aum-Supreme-Truth”“Aum-Supreme-Truth”

Knowledge Systems Laboratory, Stanford University

12

Query-Answering Example 1Query-Answering Example 1

““What has the capability of being the agent of a biochemical attack?”What has the capability of being the agent of a biochemical attack?” Query pattern: (capability ?agt Biochemical-Attack-Agent-Capability)Query pattern: (capability ?agt Biochemical-Attack-Agent-Capability)

Answer: “Al-Qaida”Answer: “Al-Qaida” ““Al-Qaida” is a “Foreign-Terrorist-Organization” that possesses a “Nerve-Agent” Al-Qaida” is a “Foreign-Terrorist-Organization” that possesses a “Nerve-Agent”

[from documents][from documents]

A thing is an “Agent-With-Biochemical-Attack-Capability” if and only if it –A thing is an “Agent-With-Biochemical-Attack-Capability” if and only if it –> Has a capability “Biochemical-Attack-Agent-Capability” orHas a capability “Biochemical-Attack-Agent-Capability” or> Possesses a “Biochemical-Agent” [from ontology]Possesses a “Biochemical-Agent” [from ontology]

An “Agent-With-Biochemical-Attack-Capability” has capability “Biochemical-Attack-An “Agent-With-Biochemical-Attack-Capability” has capability “Biochemical-Attack-Agent-Capability” [from ontology]Agent-Capability” [from ontology]

If AGT has capability “Biochemical-Attack-Agent-Capability”, then AGT is capable of If AGT has capability “Biochemical-Attack-Agent-Capability”, then AGT is capable of being an “agent” in a “Biochemical-Attack”being an “agent” in a “Biochemical-Attack” [from ontology][from ontology]

A “Nerve-Agent” is a “Biochemical-Agent” [from ontology]A “Nerve-Agent” is a “Biochemical-Agent” [from ontology]

Knowledge Systems Laboratory, Stanford University

13

Query-Answering Example 1Query-Answering Example 1

““What has the capability of being the agent of a biochemical attack?”What has the capability of being the agent of a biochemical attack?” Query pattern: (capability ?agt Biochemical-Attack-Agent-Capability)Query pattern: (capability ?agt Biochemical-Attack-Agent-Capability)

Answer: “Aum-Supreme-Truth”Answer: “Aum-Supreme-Truth”

““Aum-Supreme-Truth-Chemical-Attack-27-Jun-94 is a “Chemical-Attack” whose Aum-Supreme-Truth-Chemical-Attack-27-Jun-94 is a “Chemical-Attack” whose agent is “Aum-Supreme-Truth” [from documents]agent is “Aum-Supreme-Truth” [from documents]

Playing the role “agent” in a “Biochemical-Attack” requires the capability Playing the role “agent” in a “Biochemical-Attack” requires the capability “Biochemical-Attack-Agent-Capability” [from ontology]“Biochemical-Attack-Agent-Capability” [from ontology]

If playing role R in a process of type PT requires capability C, and Agt plays role R in If playing role R in a process of type PT requires capability C, and Agt plays role R in a process of type PT, then Agt has capability C [from ontology]a process of type PT, then Agt has capability C [from ontology]

““Aum-Supreme-Truth” has capability “Biochemical-Attack-Agent-Capability”Aum-Supreme-Truth” has capability “Biochemical-Attack-Agent-Capability”

Knowledge Systems Laboratory, Stanford University

14

Query-Answering Example 2Query-Answering Example 2

““Who are the agents of attacks that used the same type of weapons as Who are the agents of attacks that used the same type of weapons as “Recent-Attack-001?”“Recent-Attack-001?” Query pattern: (type Recent-Attack-001 ?res) (onProperty ?res instrument) Query pattern: (type Recent-Attack-001 ?res) (onProperty ?res instrument)

(hasClass ?res ?inst-type) (type ?attack ?res) (agent ?attack ?agt)(hasClass ?res ?inst-type) (type ?attack ?res) (agent ?attack ?agt)

Must-bind variables: ?agt ?attackMust-bind variables: ?agt ?attack

Knowledge in the ontology:Knowledge in the ontology: A “Mortar-Attack” has an instrument of type “Mortar”A “Mortar-Attack” has an instrument of type “Mortar”

Knowledge from documents:Knowledge from documents: ““Recent-Attack-001” is a Thing that has an instrument of type “Mortar”Recent-Attack-001” is a Thing that has an instrument of type “Mortar”

““Revolutionary-Armed-Forces-Of-Colombia-Mortar-Attack-1-Jul-00” is a “Mortar-Revolutionary-Armed-Forces-Of-Colombia-Mortar-Attack-1-Jul-00” is a “Mortar-Attack” that has agent “Revolutionary-Armed-Forces-Of-Colombia”.Attack” that has agent “Revolutionary-Armed-Forces-Of-Colombia”.

Answer: “Revolutionary-Armed-Forces-Of-Colombia”Answer: “Revolutionary-Armed-Forces-Of-Colombia”

AQUA Program PlanAQUA Program Plan

Overview of the projectOverview of the project Goal is to create a system that can answer complex Goal is to create a system that can answer complex

questionsquestions With plus up funding, we now have an end-to-end With plus up funding, we now have an end-to-end

system. Makes use of KSL’s Ontolingua Knowledge system. Makes use of KSL’s Ontolingua Knowledge Server and Java Theorem Prover (JTP) to develop Server and Java Theorem Prover (JTP) to develop answers to queriesanswers to queries

Uses SAIC and other technology to automatically Uses SAIC and other technology to automatically populate KBs with information from new text sourcespopulate KBs with information from new text sources

Uses multiple extractors from multiple sources to Uses multiple extractors from multiple sources to answer queriesanswer queries

> KSL extractorKSL extractor> UMBC/NMSU extractorUMBC/NMSU extractor> IBM extractorIBM extractor

SAIC

KIF TMR

Mapper/Translator

NLQUESTION

CNSTEST DATA

MOQA

Text TMR

Translator

IBM

Text KIF

Translator

KIF-Formatted

Question

KIF Answer/

Proof tree

OntolinguaKnowledge Server

----------------JAVA Theorem

Prover

KSL generated

explanation

A

D

SAIC

TMR KIF

Mapper/Translator

CMOQA

NL TMR

Query Processor

B

MOQA

TMR NL

Answer Processor

NLANSWER

E

DATASAIC (A-F)MOQAKSLIBM

SAIC- Team

CNS Data

GenerationF

KSL

Extractor

AQUA Current PlansAQUA Current Plans

AQUA Initial ConceptAQUA Initial Concept

QUESTION

NL QueryInterlingua Query

KIF Q

uery

KIF Answer

Interlingua Answer

NL Answer

ANSWER

NMSU Query Processor

SAIC Interlingua KIF Translator

KSL Java Theorem Prover

SAIC KIF Interlingua Translator

NMSU NL Generator

Key Tasks - SAIC Key Tasks - SAIC Perform translation of Onyx/UMBC Perform translation of Onyx/UMBC

extracted TMRs to KIF (Item A)extracted TMRs to KIF (Item A) Align two disparate ontologiesAlign two disparate ontologies Translate terms once alignedTranslate terms once aligned

Both formalized queries and extracted text Both formalized queries and extracted text need to be translatedneed to be translated

Develop CNS WMD ontologyDevelop CNS WMD ontologyCo-ordinate subcontractors and develop Co-ordinate subcontractors and develop

system interfacessystem interfaces

Key Tasks - OnyxKey Tasks - OnyxProvide formalized translation of NL queries Provide formalized translation of NL queries

(MOCA – item B)(MOCA – item B)Perform extraction of CNS data into text Perform extraction of CNS data into text

(MOCA – item A)(MOCA – item A)

Key Tasks - IBMKey Tasks - IBMAssist in relations extraction from text into Assist in relations extraction from text into

WMB ontologyWMB ontology

KSL’s Current ActivitiesKSL’s Current Activities JTPJTP – Hybrid reasoning for query answering – Hybrid reasoning for query answering

Includes a temporal reasonerIncludes a temporal reasoner

Is a DQL (DAML Query Language) serverIs a DQL (DAML Query Language) server

Knowledge Base PartitioningKnowledge Base Partitioning – Enabling Q-A from large scale KBs using parallel – Enabling Q-A from large scale KBs using parallel heterogeneous reasoners heterogeneous reasoners

Inference WebInference Web – Providing understandable explanations for derived query answers – Providing understandable explanations for derived query answers

Knowledge extractionKnowledge extraction from semi-structured documents from semi-structured documents Tables, lists, outlines, property-value pairs, etc.Tables, lists, outlines, property-value pairs, etc.

SAIC Current Activities SAIC Current Activities SAICSAIC

In-house Ontolingua server with JTP now In-house Ontolingua server with JTP now installed and in use in development effortsinstalled and in use in development efforts

Ontology is available as part of demonstration Ontology is available as part of demonstration in the demo roomsin the demo rooms

Please visit the SAIC/KSL demo standPlease visit the SAIC/KSL demo stand

SAIC Current Activities (cont.)SAIC Current Activities (cont.)SAIC spearheading a federation of a WMD SAIC spearheading a federation of a WMD

ontology development effort, assisted by ontology development effort, assisted by Stanford KSLStanford KSL

Begun development of CNS ontology. Begun development of CNS ontology. Ontology is currently 700 terms and Ontology is currently 700 terms and viewable in our in-house version of viewable in our in-house version of Ontolingua. (Demo available)Ontolingua. (Demo available)

SAIC Current Activities (cont.)SAIC Current Activities (cont.) Discussions underway with Sergei to put Onyx Discussions underway with Sergei to put Onyx

under subcontract to SAIC. Subcontract to go out under subcontract to SAIC. Subcontract to go out as soon as possible.as soon as possible. Labor division is defined and agreed toLabor division is defined and agreed to Major issue – Due to subcontract issues Onyx is still not Major issue – Due to subcontract issues Onyx is still not

under subcontract. This affects Q?A ayatem under subcontract. This affects Q?A ayatem development rates as this task is on the critical path for development rates as this task is on the critical path for system development.system development.

Distributed ontology to KSL and IBM. Distributed ontology to KSL and IBM. Development of the ontology is critical in order to Development of the ontology is critical in order to

allow the extractors to function appropriatelyallow the extractors to function appropriately

WMD Ontology Creation Initial -Confederation WMD Ontology Creation Initial -Confederation AssignmentsAssignments

Stanford/KSL: NIS-Facilities (439 terms) and Stanford/KSL: NIS-Facilities (439 terms) and Russian-Naval-Facilities (365 terms)Russian-Naval-Facilities (365 terms)

IBM: MPT-Topic (771 terms)IBM: MPT-Topic (771 terms)Xerox-Parc: Missiles-Topic (765 terms)Xerox-Parc: Missiles-Topic (765 terms)Tecknowledge: NIS-Nuclear-Weapons-Tecknowledge: NIS-Nuclear-Weapons-

Aggregate (219 terms)Aggregate (219 terms)Battelle: Nuclear-Safety-Assistance (36 Battelle: Nuclear-Safety-Assistance (36

terms)terms)

Knowledge Systems Laboratory, Stanford University

26

Year Two Project GoalsYear Two Project GoalsComplete CNS ontology developmentComplete CNS ontology developmentParticipate in TREC Participate in TREC

System is still immature System is still immature Novel appoachNovel appoach Significant potential for further developmentSignificant potential for further development

Refine interfaces and determine system Refine interfaces and determine system metrics to ensure maximum performance in metrics to ensure maximum performance in future system iterationsfuture system iterations

Knowledge Systems Laboratory, Stanford University

27

TREC participationTREC participation SAIC is signed up for TREC participation this year.SAIC is signed up for TREC participation this year. A multi-pronged approach is possible with the A multi-pronged approach is possible with the

current architecturecurrent architecture With the SAIC/Onyx route and NL interface, gives With the SAIC/Onyx route and NL interface, gives

the initial capability for an end-to-end system with the initial capability for an end-to-end system with restricted domain and rangerestricted domain and range

Formatted queries possible for IBM extractionFormatted queries possible for IBM extraction System will be very immature in year 1 and likely System will be very immature in year 1 and likely

achieve poor TREC scores, but will mature in achieve poor TREC scores, but will mature in multiple and novel directions over timemultiple and novel directions over time

Knowledge Systems Laboratory, Stanford University

28

Future PlansFuture PlansContinue multi-pronged approach (running Continue multi-pronged approach (running

multiple extractors over a uniform KB)multiple extractors over a uniform KB)Plan further enhancements (Possibly add Plan further enhancements (Possibly add

more extractors or reasonersmore extractors or reasonersLeverage multiple KB approach to optimize Leverage multiple KB approach to optimize

research in multi-partition reasoningresearch in multi-partition reasoningDevelop effective metrics to determine Develop effective metrics to determine

efficacy of this approach and which efficacy of this approach and which pathways are optimalpathways are optimal

Knowledge Systems Laboratory, Stanford University

29

Future plans (Cont)Future plans (Cont)Work on implementing latter Proof tree to Work on implementing latter Proof tree to

NL mocha interface in the future (Reverse NL mocha interface in the future (Reverse TMR to KIF)TMR to KIF)

Transition from KIF to DAML format where Transition from KIF to DAML format where possiblepossible

Extend range and capabilities of question Extend range and capabilities of question answering. Initial participation will be answering. Initial participation will be limited in terms of domain and range of limited in terms of domain and range of questions.questions.