Defense Of Research
Ian Smith
Overview
• Motivational scenario
• The problem
• Approach
• Tool explanation (example)
• Local state problem
• Contribution Summary
• Related work
Scenario
• Alice & Bob want to change the org chart together
• They worked together before but now Alice travels more often...
• Alice is remote, Bob is at his desk
• Alice has a pilot, Bob has a workstation.
Workstation Org Chart App
Org Chart App On A Pilot?(Bad Idea 1)
Pilot Display?
Org Chart App On A Pilot?(Bad Idea 2)
Pilot Display
Pilot Org Chart Application
Org Chart Reports To
President (John)
VP of Eng. (Jane)
Principal Sci. (Mary)
Staff Eng. (Bill)
VP of Finance (Fred)
The Problem?
• People want to work together, but have different devices
• Two structurally different views of same data– UIs have different underlying structures
• Views must be synchronized
Org Chart
President (John)
VP of Eng. (Jane)
Principal Sci. (Mary)
Staff Eng. (Bill)
VP of Finance (Fred)
PilotWorkstation
Window
Graphical FormulationReports To
Palette Scroller
Line RectRect...
Scroller
Tree Control
Tree Node
Tree Node Tree Node
Pilot App
Org Chart Reports To
President (John)
VP of Eng. (Jane)
Principal Sci. (Mary)
Staff Eng. (Bill)
VP of Finance (Fred)
PilotMy World
Scroller
Tree Control
Tree Node
Tree Node Tree Node
Pilot App
Workstation
Palette Scroller
Line RectRect...
Window
• Given a UI tree A, compute a new UI tree B from A
• If A is computed from a model, compute B from the same model
Problem Generalization
A BUltraman
Model
A B
Ultraman Ultraman
Approach
• End-programmer declares constraints on the structure of an interface– “Constraint” here is a function of a tree which
computes a tree (not values!)
• Constraints apply over an entire input tree– “When you see a node of type foo, create a node of
type bar and attach to this structure…”
• Constraints are applied dynamically, as the application runs
Using Ultraman
• “Ultraman” is a tool and a runtime
• Tool allows you to declare patterns and transformations
• Pattern is a structural relationship of nodes in the source tree
• Transformation is a computation to run when a pattern is found
Ultraman Tool Summary
• Several ways to specify computations and structural patterns
• Tool generates a lexer/parser specification which implements the pattern matching – Lexer/parser pair is built on ANTLR 2.20
• Generated parser calls into user code to implement transformations
• Transformations are run after each event
Example App
Transformations? But….
• Pull-Down menus have state: the currently displayed item.
• If we regenerate the menu how do we get the right item displayed in the Pull-Down?
Column
RadioButton*
Pull-Down Menu
MenuItem MenuItem...
Ultraman
Local State Problem
• Tree T1 produces tree T2 by some transformation F.
• User manipulates T2 in some way, modifying T2’s state.
• We make a change to T1 and rerun transformation F producing tree T3.
• How do we reconcile the freshly created T3 with the old, stateful tree T2?
Value Numbering
• Nodes created by the transformations are given a “label” called a value number
• The value number (VN) is a representation of “where and when” in the transformation node was created.
• Nodes with the same VN are from the same “spot” in the transformation process.
Shadow Objects
• Shadow objects are objects which “represent” interior nodes in the UI tree.
• Shadow objects mimic the API of an interactor, but actually buffer calls
• End programmers build dynamic structures underneath shadow objects.
• Effectively, shadow objects build a “shadow” version of the interactor hierarchy during the transformation.
Grafting Algorithm
• After the transformation is complete...• Walk the interface on the screen’s tree (T2)
collecting pairs (VN,node).• Walk the shadow interface’s tree (T3) and compare
VNs of its nodes to the table.• If a VN is in the table, exchange the corresponding
nodes.• Replace the interface on the screen with the
shadow.
Grafting Algorithm Example
N1 (A)
N3 (C)N2 (B)
Currently On Screen
N1
N6N2
Result
Nodes To VN Table
N2
N1
N3
B
A
C
N4 (A)
N6 (D)N5 (B)
Shadow Version
Issues With Grafting
• Value numbering is a sensitive and bottom-up change metric.
• Changes at the “top” of a tree can be missed due to shadow objects.
• End-programmer can be surprised when objects which are created never appear!
• Garbage collector can be a problem...
Contribution Summary
• Developed a taxonomy for multi-viewed transformation (4 categories) & identified salient characteristics of multi-viewed transformation system (7 properties)
• Implemented a specification tool for composing patterns and transformations
Contributions (cont’d)
• Developed and implemented algorithms for converting specifications into grammars
• Developed and implemented algorithms for handling the local state problem
• Implemented 2 demo applications– Cards: Transform to a very different device– Contact: Transform someone else’s data model
Related Work• UI Software (excluding toolkits)
– Patterson & Hill, Foley/Szekely, Olsen
• Constraints– Borning, Hudson, Myers, Vander Zanden
• Compilers/Parsing– Early, Parr, Graham & Glanville, Brotsky/Wills,
Alpern
• Transformation Systems– TXL, REFINE
Characteristics Of A Multi-Viewed Transformation System
• Semantic Predicates
• Aggregation
• Multiple & recursive patterns
• Input & output trees have different shapes
• Control the progress of the pattern match
• Multi-tree input
• Take input from a model
Taxonomy Of Transformations
• Filtering - projection (N to 1)– Aggregating - Computing a function of the
entire input
• Reordering - order change only (1 to 1)
• Decorating - adding nodes to the input (1 to N)