8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 1/26

  rgarcia@alterwork.com

Aesthetics of Computation

Visualizing Software

Presented by Ray Garcia

March 7, 2007

University of Arizona

rgz@email.arizona.edu

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 2/26

 

Noesis Challenge

Noesis = Perception + Learning + Reasoning

Software is not yet a mature engineeringprofession and needs to strive to be an

artistic craft that satisfies utility (function) as

well as beauty (form) as co-evolving goals

while enabling anyone to learn the craft.

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 3/26

 

Software as Art

Produces Artifacts, sometimes unpredictably Malleable Material but limited by the tools Variety of interactive and temporal expressions of a concept User creates their own meaning

The process requires Understanding, abstraction, insight, and interpretation Mastery of a craft Representational patterns and forms Creativity and inventiveness

For general public appreciation it lacks Inaccessible as a communications medium Visibility of computation Aesthetic Sense and Critical Review Cultural constructs

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 4/26

 

Visible computation

“Programs should be written for people to read,

understand, and manipulate, and only incidentally

for computers to execute”

Visible computation allows for direct observationand interaction with the code execution supporting: Ease of consumption (reading)

Ease of comprehension (thinking)

Ease of construction (writing)

Software should be a communications medium of 

design and its intended utility

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 5/26

 

Computation is invisible

Programming is the craft of manipulatingcomputation

Programming systems enable abstract specificationof the solution in text and diagrams (code & UML)

The problem, intent, and context is lost in thespecification

The specification is not the execution The diagrams and the text are not inter-changeable

and require annotation IDE’s and Modeling tools are not enough Execution is what matters most yet it is obscured

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 6/26

 

Programming Systems

Programming tools prevent computation from being

visible and accessible by the novice or curious

spectator 

Programming concepts are generally easy tounderstand Data structures, state manipulation, conditionals, recursion,

algorithms, …..

Incomplete set of tools and representation for dealing with computation

The mysteries of computation should be exposed

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 7/26

 

Current Tools are Incomplete

Specification is not computation Anatomy is not a not alive

Blueprints are not a building

A score is not music Code is not computation

What is visible are the remnants or output of 

computation UI, reports Debuggers, tracing, footprints

Logs files, profilers

Static snapshots are not enough

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 8/26

 

Thinking of Computation

Human skim/scan faster than reading

Human process images and sound faster than text

People think in visual spatial terms for many

technical problems

The reader needs to reconstruct a mental map that

existed in the creators mind

Novice programmer construct their own internalrepresentation of computation through trial and error  Need to think of computation as a dynamic process

instead of as a static specification language

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 9/26

 

Unnatural Code

Computation is a multi-dimensional and non-linear 

dynamic structure

Code may have parallel or multiple execution paths

Navigating through one dimensional text is likeunraveling a contorted knot

Code relies on understanding written language

Even commented code is difficult to understandwithout the reader exerting a lot of effort

Code strays too far from natural language

Annotated diagrams don’t go far enough 

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 10/26

 

Representations of 

Computation

Why isn’t programming more like interactivehypertext with diagrams and animation that can bemanipulated directly?

The syntax of code has no relationship to theruntime process, it is a constraint of the parser  Generating names for identifiers is simpler than

creating a picture of the same information

Diagrams can more easily represent structure andflow The representations should be designed to conform

to the limits of Human perception

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 11/26

 

Programming as a Craft

Requires easy tools which makes the craft possibleand accessible to the novice

The craft needs to conform to the limitations of humancognition instead of forcing people to think likecomputers

Lack of visible computation negatively impact ease of use and communication

Visible computation would facilitate learning An aesthetic of computation is needed to advance the

craft of programming Craft = Literate programming + Visual Modeling +

Visible Computation + Aesthetics

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 12/26

 

Literate Programming

Program expressively to communicate to anaudience instead for a computer to execute

Tool Support Syntax directed IDE Text Mapping Code Navigation Code Hypertext Documentation Testing and tracing frameworks Code coverage Cross hatching with Aspect Oriented programming

Meta Coding Editorial Code Refactoring Code generators Domain Specific Language development Evolutionary genetic programming

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 13/26

 

Visual Programming

Visual programming Computation

material data

process

Visual Interface visual language

Construction

navigation Aesthetics

Patterns

critical theory

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 14/26

 

Computational Processes

Object oriented – interactions between objects thatcontain state and procedures (smalltalk) Imperative – execution of sequential commands with state Procedural – combination of imperative and functional can

affect state

Functional – calculation of mathematical functionswithout state (ocaml)

Constraint based – rules of truth and logicstatements (prolog)

Multi-paradigm languages (mozart) NLP = ontology + behaviors + natural language

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 15/26

 

Visual Design

Human Perception needs continuity Human mind is not configured to follow discrete

transitions and needs smooth continuous

representations i.e.. animations Gradual updates are easier to perceive

Computation is quantized therefore poses a

design challengeClear spatial relationships

Minimal context switching

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 16/26

 

Visual language

The visual language should be used for 

specification, interaction, and execution

Show cause and effect

Computation should appear as an animated

interactive film that may be edited

Challenge to invoke rhythm and drama in the

animation of computation

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 17/26

 

Visual Modeling (UML)

Remaining Problems UML is limited to a static representation of mixed

perspectives on the specification

Managing scale and levels of abstraction

Context and transparency of components

On the Horizon

Executable UML Pattern Languages

Model Driven Architecture

Model Driven Development

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 18/26

 

Possible Solutions

Invent a new visual language design and execution

environment

Extend UML to support code generation, execution

and interactive animation Parse and instrument or probe existing code to

generate animation code to view the computation

with annotation links to the source code

Create a language design environment that

supports human cognition using the full multimedia

capabilities

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 19/26

 

Complexity, Crowds, Chaos

Size and complexity

Software maintenance and evolution

History through versioning

Open source failure rate fosters innovation

Peer review and collaborations

Chaos due to rate and impact of change

Software development as a social phenomenon Learning theory applied to software development

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 20/26

 

Scope, Scale, and Perspective

Amplified perception Augmented human & social intelligence Audience support for users and creators Information hiding and abstraction Software comprehensibility through visualization

& sound synthesis of computation execution,interaction, and navigation

Algorithm animation Software Aesthetics Critical Theory for software

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 21/26

 

Algorithm Animation Examples

cyclic queue

train sorter 

dijkstra shortest path

Further study

http://www2.hig.no/~algmet/animate.html

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 22/26

 

Visual Programming Examples

Marten http://www.andescotia.com [Mac osx]

National Instruments LabView http://www.ni.com/labview/ 

[device control]Pictorial Janus http://users.encs.concordia.ca/~haarslev/pjmovies/

ToonTalk http://www.toontalk.com [Kids visual

programming]

PureData PD http://puredata.info/ [Media graphical

programming environment]

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 23/26

 

Pictorial Janus Syntax

Call Configuration

RuleAgentConstraint

Function

Port emptyList

Constant

Link Channel

abc List ([a,b,c | [])

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 24/26

 

Pictorial Janus Example

Append 2 List

Distributed Queue

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 25/26

 

Animated UML

OOP Eggs display – using biomimicry to

visualize software

Visio Example of animated UML diagram

Future possibility UML X3D

8/14/2019 Aesthetics of Computation

http://slidepdf.com/reader/full/aesthetics-of-computation 26/26