Defense Of Research

Ian Smith

Overview

• Motivational scenario

• The problem

• Approach

• Tool explanation (example)

• Local state problem

• Contribution Summary

• Related work

Scenario

• Alice & Bob want to change the org chart together

• They worked together before but now Alice travels more often...

• Alice is remote, Bob is at his desk

• Alice has a pilot, Bob has a workstation.

Workstation Org Chart App

Org Chart App On A Pilot?(Bad Idea 1)

Pilot Display?

Org Chart App On A Pilot?(Bad Idea 2)

Pilot Display

Pilot Org Chart Application

Org Chart Reports To

President (John)

VP of Eng. (Jane)

Principal Sci. (Mary)

Staff Eng. (Bill)

VP of Finance (Fred)

The Problem?

• People want to work together, but have different devices

• Two structurally different views of same data– UIs have different underlying structures

• Views must be synchronized

Org Chart

President (John)

VP of Eng. (Jane)

Principal Sci. (Mary)

Staff Eng. (Bill)

VP of Finance (Fred)

PilotWorkstation

Window

Graphical FormulationReports To

Palette Scroller

Line RectRect...

Scroller

Tree Control

Tree Node

Tree Node Tree Node

Pilot App

Org Chart Reports To

President (John)

VP of Eng. (Jane)

Principal Sci. (Mary)

Staff Eng. (Bill)

VP of Finance (Fred)

PilotMy World

Scroller

Tree Control

Tree Node

Tree Node Tree Node

Pilot App

Workstation

Palette Scroller

Line RectRect...

Window

• Given a UI tree A, compute a new UI tree B from A

• If A is computed from a model, compute B from the same model

Problem Generalization

A BUltraman

Model

A B

Ultraman Ultraman

Approach

• End-programmer declares constraints on the structure of an interface– “Constraint” here is a function of a tree which

computes a tree (not values!)

• Constraints apply over an entire input tree– “When you see a node of type foo, create a node of

type bar and attach to this structure…”

• Constraints are applied dynamically, as the application runs

Using Ultraman

• “Ultraman” is a tool and a runtime

• Tool allows you to declare patterns and transformations

• Pattern is a structural relationship of nodes in the source tree

• Transformation is a computation to run when a pattern is found

Ultraman Tool Summary

• Several ways to specify computations and structural patterns

• Tool generates a lexer/parser specification which implements the pattern matching – Lexer/parser pair is built on ANTLR 2.20

• Generated parser calls into user code to implement transformations

• Transformations are run after each event

Example App

Transformations? But….

• Pull-Down menus have state: the currently displayed item.

• If we regenerate the menu how do we get the right item displayed in the Pull-Down?

Column

RadioButton*

Pull-Down Menu

MenuItem MenuItem...

Ultraman

Local State Problem

• Tree T1 produces tree T2 by some transformation F.

• User manipulates T2 in some way, modifying T2’s state.

• We make a change to T1 and rerun transformation F producing tree T3.

• How do we reconcile the freshly created T3 with the old, stateful tree T2?

Value Numbering

• Nodes created by the transformations are given a “label” called a value number

• The value number (VN) is a representation of “where and when” in the transformation node was created.

• Nodes with the same VN are from the same “spot” in the transformation process.

Shadow Objects

• Shadow objects are objects which “represent” interior nodes in the UI tree.

• Shadow objects mimic the API of an interactor, but actually buffer calls

• End programmers build dynamic structures underneath shadow objects.

• Effectively, shadow objects build a “shadow” version of the interactor hierarchy during the transformation.

Grafting Algorithm

• After the transformation is complete...• Walk the interface on the screen’s tree (T2)

collecting pairs (VN,node).• Walk the shadow interface’s tree (T3) and compare

VNs of its nodes to the table.• If a VN is in the table, exchange the corresponding

nodes.• Replace the interface on the screen with the

shadow.

Grafting Algorithm Example

N1 (A)

N3 (C)N2 (B)

Currently On Screen

N1

N6N2

Result

Nodes To VN Table

N2

N1

N3

B

A

C

N4 (A)

N6 (D)N5 (B)

Shadow Version

Issues With Grafting

• Value numbering is a sensitive and bottom-up change metric.

• Changes at the “top” of a tree can be missed due to shadow objects.

• End-programmer can be surprised when objects which are created never appear!

• Garbage collector can be a problem...

Contribution Summary

• Developed a taxonomy for multi-viewed transformation (4 categories) & identified salient characteristics of multi-viewed transformation system (7 properties)

• Implemented a specification tool for composing patterns and transformations

Contributions (cont’d)

• Developed and implemented algorithms for converting specifications into grammars

• Developed and implemented algorithms for handling the local state problem

• Implemented 2 demo applications– Cards: Transform to a very different device– Contact: Transform someone else’s data model

Related Work• UI Software (excluding toolkits)

– Patterson & Hill, Foley/Szekely, Olsen

• Constraints– Borning, Hudson, Myers, Vander Zanden

• Compilers/Parsing– Early, Parr, Graham & Glanville, Brotsky/Wills,

Alpern

• Transformation Systems– TXL, REFINE

Characteristics Of A Multi-Viewed Transformation System

• Semantic Predicates

• Aggregation

• Multiple & recursive patterns

• Input & output trees have different shapes

• Control the progress of the pattern match

• Multi-tree input

• Take input from a model

Taxonomy Of Transformations

• Filtering - projection (N to 1)– Aggregating - Computing a function of the

entire input

• Reordering - order change only (1 to 1)

• Decorating - adding nodes to the input (1 to N)