Date post: | 13-Jan-2016 |
Category: |
Documents |
Upload: | rosaline-stevenson |
View: | 213 times |
Download: | 0 times |
Enabling Natural Interaction
Randall Davis, Howard ShrobeAaron Adler, Christine Alvarado, Mark Foltz,
Tracy Hammond, Mike Oltmans, Metin Sezgin,Olga Veselova
MIT Artificial Intelligence Lab
Randall Davis
Consider This Device...
Randall Davis
Natural Interaction?
• What if we had to deal with one another as we deal with our software?
Randall Davis
Let’s Talk
Randall Davis
Our Vision
Randall Davis
Our Model
Randall Davis
Rationale Capture
• Rationale capture doesn’t happen.– Because it’s too much trouble
• Rationale capture tools don’t work– Because they’re too much trouble
• The key: it needs to be less trouble than it’s worth.
• The key to less trouble: break down the barriers
Randall Davis
Sketch Understanding
Randall Davis
Why Sketching?
• Informal
• Succinct
• Natural
• High in information content
• Preserve topological and spatial properties
Randall Davis
Sketch Understanding
• Requires intelligence– at the signal level: early processing– at the interpretation level: handling ambiguity– at the engineering level: intended behavior
Randall Davis
Other Research
• Additional domain: software diagrams
• Multi-modal interaction
• Describing and learning new icons
• Generating efficient recognizers
Randall Davis
Sketching Software
Randall Davis
Sketching Software
Randall Davis
Sketching Software
Randall Davis
Sketching Software
Randall Davis
Sketching Software//Source file: c:\\video\\Deck.javapublic class Deck extends Hand implements Dealable { public Card theCard; public Game theGame; /** * @roseuid 3C21348C0257 */ public Deck() { }}
//Source file: c:\\video\\Dealable.javapublic interface Dealable {}
//Source file: c:\\video\\Game.javapublic class Game { public Graphics theGraphics; public Deck theDeck; public Player thePlayer; /** * @roseuid 3C21348C02E3 */ public Game() { }}
Randall Davis
Multi-Modal Interaction“Three identical, equally spaced pendulums…”
• Graphical vs verbal communication: What’s best said, what’s best sketched?
Randall Davis
New Icons
• A shape description language
• A drawing sequence description language
Define AndGateline L1 L2 L3arc Asemi-circle A1orientation(A1, 180)vertical L3parallel L1 L2same-horiz-position L1 L2connected A.p1 L3.p1connected A.p2 L3.p2 meets L1.p2 L3meets L2.p2 L3
Randall Davis
Sketch Understanding Architecture
Mechanical
Engineering
devices
simple compositions
primitive objects
geometry rectangles, curves
Randall Davis
Sketch Understanding Architecture
Mechanical
Engineering
devices
simple compositions
primitive objects springs, joints
geometry rectangles, curves
Randall Davis
Sketch Understanding Architecture
Mechanical
Engineering
devices
simple compositions ratchet, latch
primitive objects springs, joints
geometry rectangles, curves
Randall Davis
Sketch Understanding Architecture
Mechanical
Engineering
devices mechanisms
simple compositions ratchet, latch
primitive objects springs, joints
geometry rectangles, curves
Randall Davis
Sketch Understanding Architecture
Mechanical Software
Engineering Engineering
devices mechanisms
simple compositions ratchet, latch
primitive objects springs, joints objects, methods
geometry rectangles, curves
Randall Davis
Sketch Understanding Architecture
Mechanical Software
Engineering Engineering
devices mechanisms
simple compositions ratchet, latch hierarchies
primitive objects springs, joints objects, methods
geometry rectangles, curves
Randall Davis
Sketch Understanding Architecture
Mechanical Software
Engineering Engineering
devices mechanisms modules
simple compositions ratchet, latch hierarchies
primitive objects springs, joints objects, methods
geometry rectangles, curves
Randall Davis
Design Rationale Capture
• Make it less trouble than it’s worth– Break down the barriers: eliminate
interfaces– Natural interaction is enabled by
intelligence• About sketching, gesturing• About the domain• About the task