Date post: | 31-Dec-2015 |
Category: |
Documents |
Upload: | owen-rogers |
View: | 217 times |
Download: | 2 times |
PUBLIC RELEASE – DISTRIBUTION UNLIMITED
SPIE 2015
The International Technology Alliancein
Network and Information Sciences
Collaborative human-machine analysis using a
Controlled Natural Language
David Mott, Dave Braines, IBM UKDon Shemanski, PSU
Cheryl Giammanco, ARL
v4
2
Our Objective: Supporting the AnalystOur Objective: Supporting the Analyst
doc27doc27
doc27
Facts
Inference Rationale
Hypotheses
Query
Analysts’s Conceptual
Model
Assumptions
UncertaintyReasoning Tools
NLP
Requirements
Information
Otherdata
Referencedata
Sensor Data
Structured
Facts
Unstructured
We aim to provide techniques to support the analyst in problem solving
PUBLIC RELEASE – DISTRIBUTION UNLIMITED
Reasoning tools based on ITA Controlled EnglishReasoning tools based on ITA Controlled English
Controlled English (CE) is a Controlled Natural language, being a subset of English that is both human readable and machine interpretable
Represents the Conceptual Models defined by the users:– types of things, and the relationships between them–written language used to express facts about things and relations– logical rules allowing query and inference from facts
CE is just one component to support reasoning–Support for different problem solving strategies–Conversion to other digitised semantic languages, eg OWL–Use of rationale to express flow of reasoning–Visualisation techniques for showing CE facts Human
Reasoning
Visualisation
CNL Rationale
Digitised Semantics
PUBLIC RELEASE – DISTRIBUTION UNLIMITED 3
Controlled English: Models, Facts, AssumptionsControlled English: Models, Facts, Assumptions
there is a girl named Fiona that lives in the town Oxford and is married to the man John.
for what X is it true that the person Jane suffers from the illness X?
if ( the group A is the enemy of the group B ) and ( the group B is the enemy of the group C ) then ( the group A is the friend of the group C ).
it is assumed that the attack a1 involves the group Browngroup. PUBLIC RELEASE – DISTRIBUTION UNLIMITED 4
Analytic PitfallsAnalytic Pitfalls
Mirroring– assuming that the rest of the world shares the same motivations, goals and
concepts of rationality (including the meaning of words) as those that the analyst possesses
Layering – using assumptions made in one analysis as the factual basis for judgments in a
subsequent analysis without factoring in the uncertainties of the initial analysis
Logical Failures– failing to have the correct concepts to describe or reason about the situation
These are normal human characteristics but can sometimes lead to incorrect reasoning
PUBLIC RELEASE – DISTRIBUTION UNLIMITED 5
The “Analysis Game” and Analytic PitfallsThe “Analysis Game” and Analytic Pitfalls
Used in a analyst’s training course to highlight analytic pitfalls:
A problem that requires logical reasoning, and interpretation of sentence meaning
Mirroring:– ambiguous terms are given meaning according to the analyst’s frame of
reference and background: “cuddly pet”, “far end of the block”, “two doors down”
Layering– not accounting for uncertainties by losing track of what assumptions have
been made: “which way round is left and right”
Seven girls live on the same street. Each of the girls has a pet and lives in a different colored house. … Using the following clues, determine who lives in each house and the pet that lives with them.
… The monkey lives between the dog and the lizard. Cassie does not own a cuddly pet but she lives next door to someone who does and that person lives in a green house.…
PUBLIC RELEASE – DISTRIBUTION UNLIMITED 6
Analysis Game: Basic CE FormulationAnalysis Game: Basic CE Formulation
“types”, “properties”, “relations”, “rules” and “assumptions”
PUBLIC RELEASE – DISTRIBUTION UNLIMITED 7
Problem Solving StrategyProblem Solving Strategy
Problem is specified by constraints:– Brenda and Grace live next to each other – Two people cannot live in the same house
Reasoning by elimination– “If only one house left for the fish to live in, then that is where it must live”– if ( the pet P is a cuddly pet ) then ( the pet P cannot be the pet cassies_pet ).
Represented by negative concepts– the pet fish cannot live in the house red_house.– the pet ashleys_pet cannot be the pet fionas_pet.
Use of descriptive terms to handle unknown things:– the person Brenda owns the pet brendas_pet and lives in the house brendas_house.
Solution requires identification of unknowns and propagation of their properties– the pet brendas_pet is the same as the pet monkey.– the pet brendas_pet lives in the house orange_house.
=> the pet monkey lives in the house orange_house.
Constraint Satisfaction Variables
PUBLIC RELEASE – DISTRIBUTION UNLIMITED 8
Collaborative Problem Solving InterfaceCollaborative Problem Solving Interface
Analyst and Machine collaborates via Microsoft® Word documents and macros
Reasoning may be executed by communication between Word and CE
reasoning systemPUBLIC RELEASE – DISTRIBUTION UNLIMITED 9
Visualisation and RationaleVisualisation and Rationale
Note that Cassie’s pet must be the
lizard
So Cassie (and the lizard) must live in
the blue house!
Geographic informationRationale
PUBLIC RELEASE – DISTRIBUTION UNLIMITED 10
Interpretation of termsInterpretation of terms
“Cuddly” pets–Is a bunny cuddly? A dog? A lizard?–We decided to categorise only those that are “definitely” cuddly
“Two doors down”–One door in between or two doors in between?–Wrong definition led to an inconsistency and needed reformulation
No sentences mention “left” or “right”–Two possible solutions, which are mirror images–Reasoning can only limit the end houses to two possible locations
Making users define their terms carefully, and analyse their logic more deeply, may help to bring out possible preconceptions and thus reduce
mirroring?
PUBLIC RELEASE – DISTRIBUTION UNLIMITED 11
Rationale for “the girl Cassie is located on the middle plot p2”Rationale for “the girl Cassie is located on the middle plot p2”
• “Random choice” assumption made for location of the green end house
• But in reality might need to know which house Cassie actually lives in?
• By recording assumptions, we do not forget that the choice was random, if the real location is important Recording and tracking assumptions may help to retain knowledge of sources of uncertainty lying behind high value conclusions, and thus avoid
layering?PUBLIC RELEASE – DISTRIBUTION UNLIMITED 12
Mis-formulationMis-formulation
Initially “houses” were used to define the locations of girls and pets:– the house h1 is next to the house h2.– the girl Ashley lives in the house h2.
But this was VERY confusing as the location of the houses are initially unknown!
Thus we defined a “plot” to represent the fixed locations:– the plot p1 is a left end plot.– the plot p2 is immediately right of the plot p1.
Things could then be located on plots:– the girl Brenda is located in the plot p3.– the house orange_house is located on the plot p3.
This gave a more stable mental model of space!
Creating new concepts DURING
problem solving
Choosing the right conceptualisations during problem solving makes the reasoning easier to understand thus helping to reduce logical
errors
PUBLIC RELEASE – DISTRIBUTION UNLIMITED 13
Journey to the solution – Iterative FormulationJourney to the solution – Iterative Formulation
Important discovery was in the way that the man and machine could collaborate to gain an understanding of the problem
– Trying to make the machine “just provide the answer” is difficult to understand
– More natural, human, way is to engage in an iterative “journey of discovery”:• Partial formulation of concepts• Perform the reasoning• Review of results so far• Formulate new ways to conceptualise and represent the problem space
We must model the Problem Solving process itself
Reasoning
Initial Concepts
Revised Concepts
Problem Statement
Conclusions, Inconsistencies
& RationaleAssumptions
Solution
Domain ModelFormulation
PUBLIC RELEASE – DISTRIBUTION UNLIMITED 14
Maintaining consistency of rulesMaintaining consistency of rules
During the exploration process, new rules were added, BUT– Some were missed, some duplicated– Difficult to maintain consistency and ensure all inferences are drawn
Many relationships have a particular semantic nature which can be used to generate rules
Helps to ensure consistency and completeness of rules
the relation 'lives in' is a one-to-one-relationship.if ( the girl S lives in the house O1 ) and ( the girl S lives in the house O2 ) and ( the thing O1 # the thing O2 )then ( the house O1 is the same as the house O2 ).
if ( the girl S1 lives in the house O ) and ( the girl S2 lives in the house O ) and ( the thing S1 # the thing S2 )then ( the girl S1 is the same as the girl S2 ).
if ( the girl S1 lives in the house O1 ) and ( the girl S1 cannot be the girl S2 ) and ( the girl S2 lives in the house O2 )then ( the house O1 cannot be the house O2 ).
if ( the girl S1 lives in the house O1 ) and ( the house O1 cannot be the house O2 ) and ( the girl S2 lives in the house O2 )then ( the girl S1 cannot be the girl S2 ).
User defines the nature of a relationship
System uses meta-reasoning to construct rules automatically
PUBLIC RELEASE – DISTRIBUTION UNLIMITED 15
Conclusion and Future WorkConclusion and Future Work
Solved some fairly complex problems by collaborative human and machine reasoning using CE
– Iterative formulation seems a useful way to structure the collaboration– Microsoft® Word is useful for creating, organising, and executing the information– New ways to perform problem solving using a CNL have been developed
Analytic pitfalls– Formulating and solving the Analysis Game encountered analytic pitfalls– Exploring how CE-based problem solving could potentially reduce pitfalls
Experimenting on how humans could use CE for problem solving, modelling, and explanation
Augmenting CE reasoning capabilities and expressiveness
PUBLIC RELEASE – DISTRIBUTION UNLIMITED 16
Potential to reduce cognitive biases and improve critical thinking through iterative formulation and evaluation of conclusions and rationale
AcknowledgementAcknowledgement
This research was sponsored by the U.S. Army Research Laboratory and the U.K. Ministry of Defence and was accomplished under Agreement
Number W911NF-06-3-0001. The views and conclusions contained in this document are those of the author(s) and should not be interpreted as
representing the official policies, either expressed or implied, of the U.S. Army Research Laboratory, the U.S. Government, the U.K. Ministry of Defence or the U.K. Government. The U.S. and U.K. Governments are
authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation hereon.
PUBLIC RELEASE – DISTRIBUTION UNLIMITED 17