33
Page 1 of 33 Promoting Scientific Creativity by Utilising Web-based Research Objects Project acronym: Dr Inventor Deliverable No. 8.2 Initial Version of Benchmark Datasets
! ! !
Page!1!of!33!
!
Promoting Scientific Creativity by Utilising Web-based Research Objects Project acronym: Dr Inventor Deliverable No. 8.2 Initial Version of Benchmark Datasets
! ! !
Page!2!of!33!
!
Grant agreement no: 611383 !
Dissemination!Level! ! !PU! ! ! Public! !
PP! Restricted!to!other!programme!participants!(including!the!Commission!Services)! X!RE! Restricted!to!a!group!specified!by!the!consortium!(including!the!Commission!Services)! !CO! Confidential,!only!for!members!of!the!consortium!(including!the!Commission!Services)! !
!
COVER!AND!CONTROL!PAGE!OF!DOCUMENT! ! !Project!Acronym:! Dr!Inventor!Project!Full!Name:! Promoting! Scientific! Creativity! by! Utilising! WebQbased! Research!
Objects!Deliverable!No.:! D8.2!Document!name:! Initial!Version!of!Benchmark!Datasets!!Nature!(R,!P,!D,!O)1! P!Dissemination! Level! (PU,! PP,!RE,!CO)2!
RE!
Version:! 1!Actual!Submission!Date:! 19/12/2014!Editor:!Institution:!EQMail:!
Diarmuid!P.!O’Donoghue!(NUI)[email protected]!!
!
! !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
1!R=Report,!P=Prototype,!D=Demonstrator,!O=Other!2! PU=Public,! PP=Restricted! to! other! programme! participants! (including! the! Commission! Services),!RE=Restricted! to!a! group! specified!by! the! consortium! (including! the!Commission!Services),!CO=Confidential,!only!for!members!of!the!consortium!(including!the!Commission!Services)!
! ! !
Page!3!of!33!
!
!
ABSTRACT:!
This!deliverable!arises!from!inital!work!on!“T8.2&Build&up&metrics&or&indicators&and&benchmarks“.!In!this!report!we!present!a!number!of!different!collections!(datasets)!that!will!be!used!in!developing,!testing!and!benchmarking!!the!Dr!Inventor!creativity!Support!Tool!(CST).!One!collection!is!formed!of!(mostly)! handQwritten! (ROS! like)! descriptions! of! topics! derived! from! a! number! of! different!publications,!primarily!in!the!field!of!computational!models!of!analogy.!The!!second!collection!arises!from!the!texts!used!in!some!psychological!experiments!on!peoples!ability!to!reason!using!analogies.!These! texts!have!been!parsed!and!converted! into!ROS!structures.!The! final! collection!arises! from!the!processing!of!research!papers!on!computer!graphics!–!the!target!dscipline!for!Dr!Inventor.!!!
!
!
KEYWORD!LIST:!
Kilaza!corpus,!analogy!psychology!texts,!graphic!paper!collection,!ROS!datasets,!benchmark!!
!
The!research!leading!to!these!results!has!received!funding!from!the!European!Community's!Seventh!Framework!Programme!(FP7QICTQ2013.8.1)!under!grant!agreement!no!611383.!
The!author! is! solely! responsible! for! its!content,! it!does!not! represent! the!opinion!of! the!European!Community!and!the!Community!is!not!responsible!for!any!use!that!might!be!made!of!data!appearing!therein.!
! !
! ! !
Page!4!of!33!
!
!
!
MODIFICATION!CONTROL! !Version! Date! Status! Author!1.0! 24/11/2014! Draft! Donny!Hurley!1.1! 26/11/2014! Draft! Diarmuid!P.!O’Donoghue!2.0! 3/12/2014! Draft! Donny!Hurley!2.1! 4/12/2014! Draft! Diarmuid!P.!O’Donoghue!3.0! 5/12/2014! Draft! Donny!Hurley!3.1! 5/12/2014! Draft! Yalemisew!Abgaz!3.2! 17/12/2014! Draft! Feng!Dong!4.0! 19/12/2014! Final! Diarmuid!P.!O’Donoghue!
!
List!of!contributors!
− Donny!Hurley,!(NUI)!Maynooth!University,!Ireland.!− Diarmuid!P.!O’Donoghue,!(NUI)!Maynooth!University,!Ireland.!− Prof.!Feng!Dong,!University!of!Bedfordshire,!UK.!− Yalemisew!Abgaz,!(NUI)!Maynooth!University,!Ireland.!! !
! ! !
Page!5!of!33!
!
!
!
Contents-!
1! EXECUTIVE!SUMMARY!....................................................................................................................................!6!2! INTRODUCTION!...............................................................................................................................................!7!
2.1! PURPOSE!OF!THIS!DOCUMENT!.......................................................................................................................!7!
2.2! STRUCTURE!OF!THE!DELIVERABLE!..................................................................................................................!7!3! THE!BENCHMARK!DATASETS!..........................................................................................................................!8!
3.1! THE!KILAZA!COLLECTION!.............................................................................................................................!8!4! THE!PSYCHOLOGY!LITERATURE!TEXTS!..........................................................................................................!18!
4.1! PSYCHOLOGY!LITERATURE!Q!ROS!.................................................................................................................!18!
4.2! REFERENTS!RESOLVED!–!PSYCHOLOGY!ROS!..................................................................................................!20!
4.3! ANALOGY!AND!GENERALISATION!.................................................................................................................!23!
4.4! PSYCHOLOGY!TEXTS!Q!CONCLUSION!.............................................................................................................!23!5! DR!INVENTOR!CORPUS!.................................................................................................................................!24!
5.1! FULL!GRAPHICS!PAPERS!.............................................................................................................................!24!
5.2! GRAPHICS!PAPER!Q!ABSTRACT!ONLY!.............................................................................................................!26!6! CONCLUSION!................................................................................................................................................!29!7! REFERENCES!..................................................................................................................................................!30!8! APPENDIX!1!–!ABBREVIATIONS!AND!ACRONYMS!.........................................................................................!31!9! APPENDIX!2!–!EXAMPLES!OF!VAS!.................................................................................................................!32!
!! !
! ! !
Page!6!of!33!
!
1 Executive-Summary-
Dr! Inventor! encompasses! a! number! of! disparate! activities! aimed! at! producing! a! computational!model!of!creative!reasoning!that!is!also!a!creativity!support!tool.!!To!support!the!creative!aspect!of!the!project,!a!member!of!benchmark!data!sets!have!been!developed.!! Contained! within! these!datasets! are! a! number! of! pQcreative! analogical! comparisons,! derived! from! a! variety! of! different!sources!and!presenting!distinct!challenges!to!the!Dr!Inventor!model.!These!datasets!and!the!creative!analogies!they!contain!will!help!guide!the!development!of!Dr!Inventor.!Additionally,!the!progress!of!the!project!might!be!measured!in!terms!of!the!number!of!these!creative!analogies!that!can!be!(reQ)!discovered.! !Progress!may!also!be!assessed!in!how!well!each!of!these!creative!comparisons!can!be!elucidated.!
The!term!analogy!has!several!distinct!but!related!meanings.!These!meaning!can!be!useful!as!working!definitions!of!analogy.!!
1)! Firstly! it! is! the! human! ability! to! think! and! reason! using! two! seemingly! unrelated! systems! of!information.!This!is!a!process!centric!view.!!!
2)! Secondly,! an! analogy! is! an! inference! that! if! two! systems! of! information! are! similar! in! some!respects,!then!they!might!also!be!similar!in!yet!further!respects.!This!is!also!focused!on!the!analogy!process!and!its!ability!to!suggest!inferences.!!
3)! Thirdly,! an! analogy! is! a! comparison! that! has! been! identified! between! two! seemingly! different!systems!of! information!(as!generated!by!the!analogy!process).!This!third!view! is!more!data!centric!and!this!deliverable! is!primarily!focused!on!the!data!that!forms!an!analogical!comparison.!Gentner!(1999)! has! described! analogy! as! a! similarity! in! which! the! same! relations! hold! between! different!domains!or!systems.!!
This! deliverable! is! focused!more!on! the!data! centric! view!of! analogy,! rather! than! focusing!on! the!analogy!process.!The!various!benchmark!data!sets!contain!a!number!of!analogous!pairs!of!concepts!Q!including!the!Rutherford's!atom!is!like!the!solarQsystem!analogy!(illustrated!below).!
!
!
!
!
!
!
!
!
sun!
planet!
planet!
!
nucleus!
electron!
electron!
!
! ! !
Page!7!of!33!
!
2 Introduction-
2.1 Purpose+of+this+document++The! purpose! of! this! document! is! to! identify! and! specify! benchmarks! by! which! progress! on! Dr!Inventor! can! be! assessed! and! monitored.! To! both! assisting! in! and! assess! the! progress! of!development!and!Dr! Inventor,! three!different!benchmark! collections!of!data!are!presented! in! this!deliverable.!!!
Throughout!this!deliverable!we!use!the!term!“domain”!to!refer!to!a!thematically!related!collection!of!information!and! some!specific! topic.! ! Thus,!one!collection!of!data! sets! contains!over!150!of! these!domains! identified! from!a! variety!of! papers! and! computational!modelling!of! analogical! reasoning.!!Many!of!these!were!handQcoded!domain!descriptions!and!are!presented!in!the!data!set!in!the!form!of! ROS! (Research! Object! Skeletons).! ! Embedded! in! this! collection! are! a! number! of! leading!technologies! between! a! source! domain! and! another! distinct! target! domain.! ! An! example! of! a!creative! analogy! contained! within! this! collection! is! the! famous! Rutherford! solar9system& :& atom!analogy.!!
All!datasets!mentioned!in!this!deliverable!have!been!uploaded!to!the!Redmine!project!management!tool,!for!access!by!all!members!of!the!Dr!Inventor!project!team.!
2.2 Structure+of+the+Deliverable+This!deliverable!describes! three!different! collections!of! information! that!have!been!assembled! for!the!purposes!of!testing!and!benchmarking!the!Dr! Inventor!model.! !The!first!collection! is!called!the!Kilaza!collection,!containing!domains!retrieved!from!research!papers!on!computational!modelling!of!analogical!reasoning.!!
The! next! part! of! this! paper! describes! datasets! derived! from! the! text! of! psychology! experiments!published!to!clarify!the!human!ability!to!reason!using!analogies.! !This! includes!the!source!text!and!the!ROS!generated!for!those!texts.!
The! final! part! describes! a! benchmark! collection! of! graphics! papers,!which! is! being! used! in! the!Dr!Inventor! project.! ! This! includes! a! number! of! published! graphics! papers! and! also! results! and! the!structures!that!have!been!generated!for!those!papers.!
! ! !
Page!8!of!33!
!
3 The-Benchmark-Datasets--
To! assess! the! creative! potential! of! our! model,! we! assembled! various! domain! descriptions! from!several!sources.!We!expect!that!these!will!prove!useful!in!assessing!Dr!Inventor’s!performance!at!the!p9creative! task!of! reQcreating!some!wellQknown!h9creative&analogies.!The!domains! include!some!of!the! famous! examples! of! creative! analogical! comparisons,! including! the! Rutherford’s! solar9system:atom&analogy,! the!heat9flow:water9flow! analogy!and!Karl!Duncker’s! (1945)! tumour:fortress!analogy.!Our!descriptions!are!based!on! the! standard! representation!of! these!domains!as! found! in!the! analogy! literature.! In! a! few! instances,! we! include! more! than! one! description! of! the! same!domain,! where! it! was! thought! appropriate! to! cover! representational! differences! between! a! few!different!sources.!!
3.1 The+Kilaza+Collection+The! first! collection! of! data! is! representative! of! the! domains! currently! used! in! the! literature! on!computational!models!of!analogical! reasoning!–!and!on!computationally!creative!models!based!on!analogy!and!blending.!Most!(though!not!all)!of!these!domains!were!previously!used!in!O’Donoghue!and!Keane!(2012).!
3.1.1 The-Professions-Dataset-The!first!source!of!domain!descriptions!contained!in!this!collection!is!called!the!professions!domains!and!was!sourced!from!Tony!Veale!(Veale,!1995).!The!Professions!Dataset!consists!of!descriptions!of!14! distinct! professions! such! as! accountant,& scientist,& priest& and& butcher.! These! are! large! domain!descriptions!and!range!in!size!from!10!to!over!100!predicates.!Interestingly,!the!Professions!dataset!relies!on!many!different!instances!of!a!small!number!of!relational!predicates,!such!as!control,&affect,&depend,& and& part.! Another! important! feature! is! that! this! dataset! does! not! appear! to! use! clearly!identifiable!higherQorder! relations! (cause!or! inhibit)!between! firstQorder!predicates.!Both! the! large!size!of!these!domains!and!the!absence!of!distinct!highQorder!relations!appear!to!echo!the!initial!ROS!structures!that!have!been!created!from!the!“graphics!papers”!collection!discussed!below.!!
!Figure-1-Example-of-one-of-the-professions-domains-
! ! !
Page!9!of!33!
!
Each!pair!of!domains!in!the!professions!collection!is!capable!of!forming!an!analogy!Q!of!varying!sizes!due! to! their! use! of! a! small! vocabulary! of! relational! predicates.! Thus,! we! don't! highlight! specific!comparisons! from! within! this! collection.!However,!illustrative!analogies!include:!surgeon&:general,&chef&:accountant!and!the!banker&:vampire.!!
!
Target- Source-
surgeon! general!
chef! accountant!
banker! vampire!
!
3.1.2 The-Assorted-Dataset-The!Assorted!Dataset! consists!of!a! large!number!of! smaller!and!more!varied!domain!descriptions,!including!many!of! the! frequently! referenced!domains! in! the!analogy! literature;!solar9system:atom,&
heat9flow,& water9flow& and& tumor:fortress! comparisons.! It! also! includes! an! assortment! of! other!domains!describing!golf,! soccer!and!storyQtelling.!The!81!domains!of! the!Assorted!dataset!typically!use! unique! instances! of! relational! predicates.! Each! of! these! domains! contains! between! 1Q15!predicates!(M=4.16,!SD!=!2.9).!The!average!number!of!unique!relational!predicates!in!each!domain!is!M=3.48,!indicating!that!most!relational!predicates!are!used!just!once.!These!domains!therefore!may!present!a!different!challenge!to!Dr!Inventor!than!do!the!Professions!domains.!
The! figure! depicting! one! part! of! Rutherford’s! solarQsystem! to! atom! comparison! is! shown! in! two!different! formats.! On! the! left! is! the! typical! predicate! calculus! style! representation! found! in! the!literature!on!analogical!reasoning.!On!the!right!is!a!graph!based!depiction!of!that!same!information!–!but! this! image!was! hand! drawn! to! highlight! the! ROS! structure! implicit! in! the! standard! (predicate!style)! representations.! This! has! also! been! hand! coloured! for! consistency!with! the! other!machineQgenerated!ROS!images.!!
!
!
!!
!
!
!
!
revolves
nucleus electron
attracts heavier
and
cause cause and revolves and attracts heavier attracts nucleus electron heavier nucleus electron revolves electron nucleus
Figure-2-Representations-of-the-result-of-Rutherford’s-creative-comparison-between-the-atom-and-the-solar-system.+
! ! !
Page!10!of!33!
!
!Figure-3-Domain-representing-the-Renaissance-painter-Caravaggio.-This-graph-is-structured-very-like-the-“Leadbelly”-
domain-(Leadbelly-was-an-American-folk-and-blues-musician)-
A!great!many!potential!analogies!can!be!identified!from!within!this!collection,!including:!!
Target+problem+ Source+(analogous+solution)++
heat9flow&& water9flow&
tummor&& fortress&
solar9system&& atom&
Leadbelly&& Caravaggio&
Atom9Falkenhainer& &Solar9System9Falkenhainer&
General& &Surgeon&
Love9triangle& &Triangle9Directed&
Requited9love& &Love9Triangle&
Fish&& Bird&
Vampire&& Banker&
Cycling& &Driving&
Arturian9saga&& Kennedy9saga&
gunes9family&& gunes9orchestra&
gunes9fruit&& gunes9orchestra!
! ! !
Page!11!of!33!
!
Many!of!the!other!domains!in!this!collection!are!primarily!useful!it!terms!of!acting!as!potential!"false!positive"! analogies.! That! is,! they! pose! many! of! the! qualities! of! potential! analogs! Q! but! should!ultimately!be!rejected!by!an!analogy!based!model.!!
3.1.3 The-Alphanumeric-Dataset--
The! Alphanumeric! Dataset! contains! 62! domains! –! that! differ! primarily! in! their! topology! as! their!semantics!are!very!much!constrained.!These!were!partly!intended!to!test!the!recall!and!precision!of!the! analogy! process,! effectively! acting! as! “distractor”! domains! for! the! analogy! model.! These!domains!provided!a!counterpoint!to!the!semantic!richness!of!the!other!domains!and! it!was!hoped!they!might!uncover!some!novel!comparisons!and!inferences.!These!domains!represent!the!26!letters!in!upper!and!in!lower!case!forms.!The!10!digits!are!also!included.!!
The!structure!and!representation!of!one!example!is!shown!below.!We!highlight!that!these!domains!are! lacking! in!semantics!and!thus!are!used!to!represent!the!semantic!challenge!of!finding!a!source!(of! creative! inspiration)! when! the! semantics! of! the! problem! domain! do! not! appear! to! suggest!anything!novel.!
! !
! ! !
Page!12!of!33!
!
!
!
!
!
!
!
!
!
3.1.4 Additional-Datasets-The!final!collection!of!domains!is!much!smaller!with!a!couple!of!domains!each!being!sourced!from!a!variety!of!origins!including!(Bohan!and!Keane,!2005)!and!(Baydin!et&al,!2012).!The!domains!by!Bohan!and!Keane!(2005)!were!developed!for!the!purposes!of!examining!the!role!of!analogical!comparisons!within! argumentation.! The! Baydin! et& al! (2012)! collection! originated! from! experiments! on!computational!creativity!–!creating!new!source!analogs!for!some!presented!target!problem.!!
! !
1! 2! 3!
4! 5! 6!
7! 8! 9!
connected 1 2 connected 1 5 connected 2 3 connected 3 6 connected 5 6 connected 6 9 connected 8 9
Figure-4-A-semantically-impoverished-domain-that-possesses-rich-topological-structure.-
! ! !
Page!13!of!33!
!
!
!
!
Figures:-The!dotted!arcs!between!the!two!descriptions!indicate!some!of!the!items!that!should!be!paired!during!the!process!of!analogical!discovery.!
!
! ! !
Page!14!of!33!
!
The! analogies! sourced! from! Baydin! et& al! (2012)! each! involved! one! domain! that! was! handcrafted!while! the! corresponding! analogous!domain!was! the!machine! generated! –! achieved!by!mining! the!contents!of!ConceptNet!to!generate!novel!analogous!domains.! It!should!be!noted!that!these!were!generated! to! be! structurally! very! similar! to! the! input! domain! and! thus! the! generated! “source”!domains! of! these! comparisons! do! not! provide! additional! information! (inferences)! with! which! to!create! new! information! in! the! target! –! that! is,! these! comparisons! were! not! designed! to! support!analogical! inferences.! That! said! the! pair! of! domains! below! could! support! an! inference! to! the!“orchestra”!description!stating!that!both!drum!and!clarinet!are!both!partQof!the!same!concept.!
!
!!!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
!
Figure:-These!two!analogs!have!been!presented!so!that!the!graphQsimilarity!can!be!easily!seen.!A!few!of!the!interQdomain!mappings!are!heighted!with!dotted!lines.!
! ! !
Page!15!of!33!
!
!
The!previous!example!also!highlights!one!of! the! factors! that!we!expect! to!encounter! in!a! creative!analogy!–!finding!a!mapping!between!two!semantically!very!distant!items,!in!this!case!two!concept!nodes.! In!this!example!we!see!that;!“father”! is!mapped!with!“drum”!and!“human”! is!mapped!with!“instrument”.! !Many! creative! analogies! exhibit! this! quality! of!mapping! dissimilar! items! –!which! is!probably! why! people! find! such! analogies! are! difficult! to! discover.! Of! course! an! excess! of! these!dissimilarities!in!an!analogy!makes!the!comparison!too!difficult!to!understand.!Thus,!one!of!the!uses!of! ontological! information! is! to! monitor! the! number! (and! semantic! distance)! of! these! dissimilar!mappings.!
As!can!also!be!seen,!the!previous!example!only!places!identical!relations!in!correspondence!with!one!another.! This!may! give! a! clue! as! to! how! the! generating!model! operated! to! create! these! domain!descriptions.! In! many! natural! analogies! we! see! some! differences! between! some! of! the! paired!relations!of!an!analogy.!
! ! !
Page!16!of!33!
!
!
Figure:!A!second!depiction!of!the!gunes9family!to!gunes9orchestra!mapping!
! ! !
Page!17!of!33!
!
The! image! above! overlays! the! gunes9family! and! gunes9orchestra,! also! depicted& on! the! previous!page.&Information!related!to!the!family!is!presented!using!the!usual!green!and!yellow!colours,!while!an!alternate!colour!scheme! is!used!for! information!related!to!the!orchestra.& Items!placed& (almost)!on!top!of!one!another!represent!items!mapped!between!the!two!systems.!This!analogy!highlights!a!potentially! novel! comparison! between! a! family! and! an! orchestra.! (Apologies! that! this! includes! an!inappropriate! comparison! for! the! “windQinstrument”! above.! We! point! out! that! a! related! pairing!reports!the!more!pleasing!“mother”!&!“clarinet”!analogous!pair).!
!
Target+ Source+
gunes9family&& gunes9orchestra&
gunes9fruit&& gunes9solar9system!
!
These! collections! of! domain! descriptions! have! been! included! in! the! “Kilaza”! collection! as! many!(though!not!all)!of!these!domains!were!used!by!the!Kilaza!model!(O’Donoghue!and!Keane,!2012).!
! ! !
Page!18!of!33!
!
4 The-Psychology-Literature-Texts-
The! next! set! of! benchmarks! is! derived! from! the! texts! used! in! a! number! of! psychological! studies.!These! texts! formed! part! of! the! psychology! materials! presented! to! human! subjects! as! part! of! a!laboratory!style!test,!to!ascertain!the!precise!nature!of!different!factors!impacting!upon!the!process!of!analogical!reasoning.!!
4.1 Psychology+Literature+E+ROS+The!first!version!of!this!dataset!was!created!by!processing!the!original!textual!materials!and!by!using!the!resultant!information!to!generate!the!corresponding!ROS.!There!are!18!distinct!domains!in!this!collection.! This! collection! contains! several! related! domains,! covering! conditions! such! as! “True!Analogy”!and!“Mere!Appearance!Comparisons”.!!
!
!
!
!
!
!
!
!
!
!
!
!
!
Zerdia+True+Analogy:+
Once& there& was& a& small& country& called& Zardia& that& learned& to& make& the& world’s& smartest&
computer.&One&day&Zardia&was&attacked&by& its&warlike&neighbor,&Gagrach.&But&the&missiles&were&
badly& aimed& and& the& attack& failed.& The& Zardian& government& realized& that& Gagrach& wanted&
Zardian&computers&so&it&offered&to&sell&some&of&its&computers&to&the&country.&The&government&of&
Gagrach&was&very&pleased.&It&promised&never&to&attack&Zardia&again.&
Karla+the+Hawk:++
Karla,&an&old&hawk,& lived&at& the& top&of&a& tall&oak& tree.&One&afternoon,& she&saw&a&hunter&on& the&
ground&with&a&bow&and&some&crude&arrows&that&had&no&feathers.&The&hunter&took&aim&and&shot&at&
the& hawk& but& missed.& Karla& knew& the& hunter& wanted& her& feathers& so& she& glided& down& to& the&
hunter&and&offered&to&give&him&a&few.&The&hunter&was&so&grateful&that&he&pledged&never&to&shoot&
at&a&hawk&again.&He&went&off&and&shot&a&deer&instead.&&
! ! !
Page!19!of!33!
!
!
!
Target- Source(s)-
Zerdia&the&country& Karla&the&hawk&
Tumour! The& General,& The& Commander,& Red& Adair,& The&Fire&Chief&!
Tumour& Fortress&1&Dispersion&Attack&
!
It!should!be!noted!that!not!all!domains!in!this!collection!were!designed!to!support!the!identification!of! analogies.! Some! domains! were! explicitly! designed! as! “distractor! stories”! e.g.! ("The! Wine!Merchants"!and!"The!Identical!Twins").!These!stories!can!be!considered!as!potential!“false!positive”!sources! that! we! would! not! like! to! identify! as! source! domains! for! the! Tumour! problem.! These!additional!stories!were!matched!closely!in!length!with!the!Attack!Dispersion!story,!and!also!describe!problems! and! their! solutions.! However,! the! two! distractor! stories! were! intended! to! be! as! disRanalogous!to!the!radiation!problem!as!possible.!! !
! ! !
Page!20!of!33!
!
4.2 Referents+Resolved+–+Psychology+ROS+The!second!version!of!the!dataset!has!also!been!provided.!!This!differs!from!the!previous!version!in!only!one!respect!–!referents!such!as!“it”!and!“he”!have!been!(manually)!!replaced! by! the! correct!counterpart.! This! collection! also! includes! all! 18! domains! from! the! first! version! of! this! collection!(section!4.2!above).!This!version!was!produced!for!two!purposes.!Firstly,!to!identify!the!impact!that!resolving! these! referents!would!have!on! the! resultant!data.!Secondly,! to!assess! the! impact!on! the!performance!of!Dr! Inventor! that!might!occur! if! this! lexicoQsemantic! information!was! resolved!and!clarified.!!
Resolving!these!referents!has!a!few!obvious!impacts!that!can!be!highlighted!in!the!Zerdia!example.!In!the!Figure!in!Section!4.1!above!the!node!“it”!is!attached!to!both!the!“promise_to_attack”!and!the!“offer_to_sell”! nodes.!We! note! that! the!word! “it”! appears! as! a! single! node! in! the! resulting! ROS.!However,!by!correctly!replacing!“it”!by!its!correct!referent,!we!discover!that!one!instance!of!the!“it”!node!refers!to!“Zardia”!while!another!instance!of!this!node!refers!to!“Gagrach”.!The!resulting!ROS!is!a!better!description!of!the!semantic!content!of!the!KarlaQtheQHawk!story.!!
!
! ! !
Page!21!of!33!
!
!
After!the!mapping!process!has!been!applied!to!these!two!(resolved)!ROS,!we!expect!the!mapping!to!identify!the!following!subsets!of!information!from!each!of!the!two!input!domains.!The!pair!of!images!below!(including!the!dotted!lines!indicating!mapped!equivalents)!indicates!one!of!the!analogies!we!expect!to!identify!from!this!corpus.!!
These!following!pair!of! images!have!removed!all!unQmapped!parts!of!the! input!ROS!–!for!clarity!of!presentation.! Additionally,! they! have! been! laidQout! to! in! a! manner! that! highlights! the! structural!similarities! between! these! collections! of! information.! This! should!make! it! possible! to! structurally!identify!the!paired!items!between!the!two!domains.!! !
! ! !
Page!22!of!33!
!
!
!
!
!
!
!
!
!
!
! !
Karla
saw offer_to_give
hunter
want pledge_to_shoot arrow_have
feather
crude
Zardia
offer_to_sell promise_to_attack
gagrach
want be_attack learn_to_make
computer
world
! ! !
Page!23!of!33!
!
!
4.3 Analogy+and+Generalisation+Analogy!also!plays!a!role!in!generalisation.!The!table!below!lists!two!pairs!of!sources!together!with!a!related! target! problem.! There! a! two! pairs! of! sources! and! we! can! see! that! they! are! semantically!related!sources!Q!one!pair!relates!to!military!information!and!the!other!pair!of!domains!relate!to!fire!safety! information.! In!an! influential!paper! (Gick!and!Holyoak,!1983)!showed!that!a! target!problem!plus!two!(semantically!related!sources!can!form!the!basis!of!for!generalisation!–!or!the!induction!of!some!general!rule.!While!there!are!very! little!useable!benchmark!data!for! investigating!this! issues,!Dr! Inventor!might! be! able! to! provide! some! useful! insight! in! future! –! building! on! focused! corpusQsection!aimed!at!subQtopics!in!computer!graphics.!!
Target- Source-
Tumour! The&General,&The&Commander&&
(paired!sources!for!generalisation!!purposes)!
Tumour& Red&Adair,&The&Fire&Chief&&
(paired!sources!for!generalisation!!purposes)&
!
Both!pairs!of!sources!related!to!the!same!wellQstudied!inoperableQtumour!problem,!made!popular!by!studies! reported! in! Karl!Duncker! (1945).!However,! generalisation! requires! a!model! of! the! domain!information,!such!as!a!domain!specific!ontology.!!
4.4 Psychology+Texts+E+Conclusion+These!two!collections!contain!several!analogies,!as!well!as! the!Karla& the&hawk& :Zerdia& the&country.!For!the!tumour!problem!there!are!several!potential!analogs;!including!The&General,&The&Commander!(two!military!stories),!Red&Adair,&The&Fire&Chief!(two!firefighter!stories)!(Gick!and!Holyoak,!1983).!It!should! be! noted! that! there! are! (deliberately)! four! potential! source! domains! as! these! problems!introduce!the!potential!to!investigate!the!notion!of!analogy!based!generalisation!–!or!the!induction!of!generalised!schema.!These!texts!being!used!to!show!the!role!that!analogy!plays!in!the!induction!of!generalised! rules! upon! the! presentation! of! two! (semantically! related)! sources! for! a! single! target!problem.!Dr!Inventor!will!return!to!this!issue!when!addressing!the!process!of!conceptual!blending.!!
So!the!two!psychology!collections!present!a!different!challenge!to!Dr!Inventor.!They!incorporate!text!processing!as!well!as!ROS!based!analogizing.!Ultimately,!we!expect!Dr!Inventor!to!exhibit!the!same!performance!on! these! texts!as!has!been! recorded!by!human!subjects!–!using! the!very!same! input!materials.!!
!
!
! !
! ! !
Page!24!of!33!
!
5 Dr-Inventor-Corpus-
The!final!set!of!benchmarks!is!taken!from!a!corpus!of!graphics!papers!in!the!Dr!Inventor!project.!A!selection!of!papers!in!four!different!graphics!domains!was!made;!Cloth!Simulation,!Fluid!Simulation,!Skinning! and!Motion.! In! the! Dr! Inventor! corpus,! each! of! these! domains! contains! 40! papers.! The!initial! version! of! this! benchmark! data! in! this! deliverable! relates! to! an! initial! set! taken! from! this!corpus.!
5.1 Full+Graphics+Papers+This!contains!the!ROS!generated!from!10!of!these!papers!from!the!collection!related!to!Skinning.!The!ROS!are!generated!using!the!same!methods!as!the!ROS!for!the!psychology!texts!–!involving!activities!like!text!extraction!and!parsing.!Having!extracted!the!relationships!between!the!words! in! the!text,!the!ROS!is!then!created.!A!full!paper!contains!many!nodes!Q!typically!of!the!order!of!one!thousand!nodes.! The! following! graphs! are! illustrations! of! the! ROS! generated! from! the! paper! “Real9Time&
Weighted&Pose9Space&Deformation&on&the&GPU”.&The!first!image!focuses!on!one!small!section!of!the!entire!ROS!for!this!paper.!The!second!image!illustrate!where!this!portion!of!the!ROS!can!be!found!in!the!complete!ROS.!!
!
! ! !
Page!25!of!33!
!
!
! ! !
Page!26!of!33!
!
Due!to!the!very!large!size!of!these!ROS,!it!is!expected!that!future!ROS!will!need!to!be!generated!from!summaries!and!extracts!of!papers!–!and!not!the!entire!paper!as!above.!!
Work! is! currently! ongoing! to! identify! analogies! from! the!Dr! Inventor! corpus.! So! far!we! have! only!identified! “within! domains”! comparisons.! That! is,! analogies! where! both! the! source! and! target!domains! come! from! similar! discipline.! Not! surprisingly! perhaps,! many! graphics! papers! take!inspiration! from! the! “real!world”,! using!mathematically! based!models.! A! few! “between! domains”!analogies!have!been!identified,!but!not!among!the!papers!that!are!currently!within!the!Dr!Inventor!corpus.!
5.2 Graphics+Paper+E+Abstract+only++To!overcome!the!size!problems!associated!with!the!full!graphics!papers!we!also!generated!a!series!of!ROS,! based!only! the!Abstracts! of! these!papers.! These! abstracts! are! typically! around!200!words! in!length,!capturing!the!essential!topics!covered!in!that!paper.!!
!
!
We!have!generated!10!ROS!based!on!these! full!graphics!papers.!These!contain!an!average!of!26.6!(StDev=6.7)!yellow!concept!nodes!and!17!(StDev=5.0)!relations.!!
5.2.1 LowRlevel-Analogies---Efforts! have!been!made! to! identify! analogies! that!will! promote! scientific! and! research! creativities!among!the!sample!papers!from!the!field!of!computer!graphics.!Each!paper!is!represented!using!ROS.!Our! experiment!was! carried! out! based! on! the! verb! action! (VA)! structure! extended! from! the! ROS!
! ! !
Page!27!of!33!
!
triples! (subject,!verb,!object).! !More!details!of!VA!are!described! in!D3.1!(under!Section!3.4.1).!This!collection!contains!80!papers!and!546!individual!sentences.!
At! this! stage,! the!search! for!analogies!was!carried!out!based!on! the!semantic! similarities!between!the!ROSs.!The!abstracts!from!the!papers!contain!sentences!that!provide!summative!information!and!major!underlying! initiatives!of! the!papers.!Hence!comparing! their! semantic! similarity! for!analogies!may!bring!the!following!benefits!to!the!researchers:!
• They!will!receive!a!collection!of!papers!that!contain!semantically!similar!terms.!For!example,!in!computer!animation,!the!following!terms!are!strongly!linked!!
Animate!behavior!Control!behavior!!Synthesize!motion!!Compute!motion!!Create!animation!Create!motion!!
Hence,! retrieving!papers!whose!ROS! contain! the!above! terms!alongside!with! their! evaluation!criteria!(e.g.!citation!numbers,!impact!factors)!will!provide!the!researchers!with!a!broad!view!on!the!current!research!activities!and!work!that!are!of!relevance.!
In!addition,!ROS!visualization! techniques!will!be!made!available! (see!D6.1),!which!will! visually!present! the! main! initiative! of! the! papers! through! the! display! of! the! ROSs,! allowing! the!researchers! to!visually!capture!the!similarity!and!disQsimilarity!between!the!papers!and!hence!inspire!their!new!research!ideas.!!
• WithinQVAQanalogy:! Based! on! the! VA! structure,! we! will! be! able! to! provide! researchers! with!more! analytics! on! the! research! papers! at! the! semantic! level! w.r.t.! their! fundamentally!methodologies,!aims,!etc.!!For!example,!different!ways!to!carry!out!motion!synthesis!(by!using!the!triple!of!predicate,!SBJ,!OBJ);!different!purpose!of!applying!motion!synthesis!(by!using!PRP);!different! locations! where!motion! synthesis! is! applied! (by! using! LOC).! Further,! it! will! be! also!quite! valuable! to! explore! any! other! information! where! needed! using! temporal! (TMP! tag),!negation!(NEG),!cause!(CAU),!MNR!(manner),!etc.!
Notably,!here!the!analogy!will!only!take!place!within!the!same!VA!structure!by!generating!the!mandated!inferences!via!copying!with!substitution!and!generation!(CWSG).!!
Here!is!one!example,!
Sentence:!“Motion&synthesis&is&created&using&motion&captured&data&for&biomedical&research”.&
VA:! create! (predicate),! Motion! synthesis! (OBJ),! motion! captured! data(MNR),! biomedical!research!(PRP)!
Sentence:!“We&have&generated&an&animation&using&Maya”.&&
VA:!generate!(predicate),!we!(SBJ),!animation!(OBJ),!Maya!(MNR).!
By!carrying!out!the!withinQVAQanalogy,!we!can!gain!the!following!initiatives:!
1) Motion!synthesis!can!be!done!by!Maya.!!
! ! !
Page!28!of!33!
!
2) Animation!can!be!generated!by!motion!synthesis!
3) Animation!from!can!be!used!for!biomedical!research!
• CrossQVAQanalogy:! Also! based! on! the! VA! structure,!we!will! be! able! to! compare! and! shift! the!entire!VA!among!different!sentences! from!different!papers! to!gain!new!research! initiative.!By!doing!so,!the!researchers!will!be!able!to!see!problems!&!possible!solutions!raised!from!referring!to!the!settings!from!other!research!papers.!For!instance,!if!a!motion!synthesis!method!is!carried!out!under!certain!constraints,!this!leads!to!a!question!whether!these!constraints!can!be!applied!to! other! similar! methods! in! motion! synthesis.! If! a! motion! synthesis! is! performed! under! the!settings!of! texture!synthesis,! this! leads! to!a!suggestion! that! texture!synthesis!can!be!used! for!other!motion!synthesis!methods.!!
As!examples,!the!following!4!sentences!are!analysed!with!their!VA!features:!
Sentence!1:!We&have&used&our&model&in&conjuction&with&advecting&solid&textures&to&create&many&
fluid9like&animations&interactively&in&two9&and&three9dimensions.!
Sentence! 2:! A& spacetime& constraints& solver& computes& an& adapted& motion& that& re9establishes&
these&constraints&while&preserving&the&frequency&characteristics&of&the&original&signal.!
Sentence!3:!This&interpretation&induces&a&sequence&of&target&segments&from&the&database,&which&
is&concatenated&to&create&the&appropriate&target&motion.&
Sentence! 4:!We& combine& each& of& these& elements& to& produce& a& simulation& algorithm& that& is&
capable&of&creating&animations&at&high&maximum&resolutions&while&avoiding&common&pitfalls&like&
inaccurate&boundary&conditions&and&inefficient&computation.!
Their!extracted!VAs!is!listed!in!Appendix!2.!!
Notably,! here! the! analogy! will! take! place! across! different! sentences! through! copying! either!entirely! or! partially! the! VAs.! Hence,! by! carrying! out! the! crossQVAQanalogy,! we! can! gain! the!following!initiatives!
1) Textures!may!be!used!for!spacetime!constraints?!!
2) Captured!motion!data!may!be!used!for!spacetime!constraints?!
3) Does!Motion! synthesis! via!motion! database! need! to! reQestablish! constraint! and! preserve!frequency!characteristics?!!
4) Do! we! need! to! consider! how! to! avoid! pitfalls! like! inaccurate! boundary! conditions! and!inefficient!computation!for!spacetime!constraints!and!for!motion!data!synthesis?!!
While!these!analogies!are!only!suggested!at!the!semantic!level!that!belongs!to!the!category!of!“lowQlevel! analogies”,! they! provide! useful! information! to! the! researchers,! allowing! them! to! look! into!different!techniques!available,!their!workflows,!the!usage!of!these!techniques!and!their!application!fields.!!Through!the!use!of!the!analogy!techniques,!new!research!initiatives!can!be!generated.!!!
The!following!is!an!example!of!the!ROS!structure!generated!for!the!sentence:!“While&our&approach&
conserves&mass,&volume&loss&is&still&possible&but&only&temporarily”.!The!table!for!this!sentence!is!listed!at!the!end!of!Appendix!2.!!
! ! !
Page!29!of!33!
!
!
Firstly,!we!can!see!that!while!the!contents!of!these!ROS!differ!from!the!earlier!ones!in!this!inclusion!of!more!syntactic!relations!like!“AMQADV”.!However,!the!structure!of!these!ROS!still!conforms!to!the!graph!based!definition!of!ROS.!Furthermore,!mapping!between!these!syntactic!ROS!should!identify!low!level,!syntactically!rooted!similarities!and!should!support!low!level!creativities.!!
The! above! image! shows! a! full! paper! version! of! one! of! these! ROS.! This! one! illustrates! one! of! the!smaller!ROS!generated.!The!corpus!contains!80!ROS!created!from!the!abstracts!of!graphics!papers.!!
In! addition! to! the! Abstracts,! we! provided! an! individual! ROS! for! each! sentence! in! this! collection.!These!sentenceQbased!ROS!should!allow!more!“pieceQmeal”!comparisons!between!a!sentence!from!a!target!problem!and!several!distinct!source!ROS!–!each!representing!one!sentence!from!this!corpus.!!!
6 Conclusion-
This! document! describes! a! number! of! different! datasets! for! use! in! testing! and! the! Dr! Inventor!model.!The!datasets!come!from!a!number!of!diverse!backgrounds,!to!help!ensure!the!completeness!and!robustness!of!the!resulting!model.!Future!work!on!benchmarks!and!datasets!will!focus!primarily!(if!not!exclusively)!graphics!papers!and!their!ROS.!!
! !
! ! !
Page!30!of!33!
!
7 References-
[1] Baydin! AG,! López! de! Mántaras! R,! Ontañón! S.! (2012),! Automated& Generation& of& Cross9Domain&Analogies& via& Evolutionary& Computation.! In! Proceedings!of! the! Third! International!Conference!on!Computational!Creativity!ICCC;!Dublin,!Ireland.!P.!25Q32.!
[2] Bohan! A,! Keane! MT.! (2005)! Boosting& Analogical& Arguments:& The& Effects& of& Goodness& &&Complexity& on& Everyday& Arguments.! In! Proceedings! of! Cognitive! Science! Society;! 2005;!Stresa,!Italy.!P.!304Q309.!!
[3] Duncker,!K!(1945)!“On&Problem&Solving”,!Psychological!Monographs,!Vol.!58!(5),!1945,!iQ113.!
[4] Gentner,!D.!(1999).!Analogy.!In!R.!A.!Wilson!&!F.!C.!Keil!(Eds.),!The!MIT!Encyclopedia!of!the!Cognitive!Sciences!(pp.!17Q20).!Cambridge,!MA:!MIT!Press.!
[5] Gick!M,!Holyoak,!K,!(1980)!“Analogical&Problem&Solving”,!Cognitive!Psychology,!12,!306Q355.!
[6] Gick! M,! Holyoak,! K,! (1983)! “Schema& Induction& and& Analogical& Transfer”,! Cognitive!Psychology,!15,!1Q38.!
[7] O'Donoghue,!Diarmuid!P.! Keane,!Mark! T.! (2012)!A&Creative&Analogy&Machine:& Results& and&Challenges,!4th!International!Conference!on!Computational!Creativity!(ICCC);!May!30QJune!1,!UCD,!Dublin,!Ireland.!P.!17Q24.!!
[8] Veale,! T.! (1995),!Metaphor,& memory& and& meaning:& Symbolic& and& connectionist& issues& in&metaphor&interpretation;!Ph.!D.!dissertation,!Trinity!College!Dublin.!
! !
! ! !
Page!31!of!33!
!
8 Appendix-1-–-Abbreviations-and-acronyms-
!
ROS! Research!Object!Skeleton!
! !
! !
! !
! !
! !
!!
!
Page!32!of!33!
! !9 Appendix)2)–)Exam
ples)of)VAs))
We#have#used#our#m
odel#in#conjuction#with#advecting#solid#textures#to#create#m
any#fluid8like#animations#interactively#in#tw
o8#and#three8dim
ensions.!
!
A#spacetime#constraints#solver#com
putes#an#adapted#motion#that#re8establishes#these#constraints#w
hile#preserving#the#frequency#characteristics#of#the#original#signal.!
!
This#interpretation#induces#a#sequence#of#target#segments#from
#the#database,#which#is#concatenated#to#create#the#appropriate#target#m
otion.#
!
!!
!
Page!33!of!33!
!
We#com
bine#each#of#these#elements#to#produce#a#sim
ulation#algorithm#that#is#capable#of#creating#anim
ations#at#high#maxim
um#resolutions#w
hile#avoiding#com
mon#pitfalls#like#inaccurate#boundary#conditions#and#inefficient#com
putation.!
!
!
!!predict PO
S com
ponent core
args phrase
phraseId parentPhraseId
level W
hile our approach conserves mass, volum
e loss is still possible but only temporarily.
be
VB
A
M-A
DV
w
hile our approach conserve mass
2 0
1
be V
B
AM
-TMP
still
5 0
1
be V
B
AM
-TMP
only tem
porarily 7
0 1
be V
B
OB
J loss
A1
volume loss
4 0
1
be V
B
ATT
A
2 possible but
6 0
1
conserve V
B
SB
J approach
A0
our approach 1
2 2
conserve V
B
OB
J m
ass A
1 m
ass 3
2 2
!
