Julian KrengeVina Wibowo

OrganicUser Interfaces

Seminar on Post-Desktop User Interfaces

Seminar paper at theMedia Computing GroupProf. Dr. Jan BorchersComputer Science DepartmentRWTH Aachen University

Advisor:Max Möllers

Semester:Winter Semester 2008

Submission date:Jan 29th, 2009

iii

Contents

Abstract ix

Uberblick xi

1 Introduction 1

2 Related Work 3

2.1 Organic Computing . . . . . . . . . . . . . . . . . . . . . . 3

2.2 User Interfaces We Know So Far . . . . . . . . . . . . . . 4

3 Organic User Interfaces Defined 7

3.1 Properties of Organic . . . . . . . . . . . . . . . . . . . . . 7

3.2 Design Principles . . . . . . . . . . . . . . . . . . . . . . . 8

3.3 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.4 Tangible UI Vs. Organic UI . . . . . . . . . . . . . . . . . 9

4 On the Way Towards Organic User Interfaces 11

4.1 Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4.1.1 Interactive Surfaces . . . . . . . . . . . . . . . . . . 12

iv Contents

4.1.2 Deformation Tracking . . . . . . . . . . . . . . . . 13

4.2 Flexible Displays . . . . . . . . . . . . . . . . . . . . . . . 14

4.2.1 Digital Paper . . . . . . . . . . . . . . . . . . . . . 14

4.2.2 Bendable Screens . . . . . . . . . . . . . . . . . . . 15

4.3 Shape Actuation . . . . . . . . . . . . . . . . . . . . . . . . 16

4.3.1 Physical 3D Displays . . . . . . . . . . . . . . . . . 16

4.3.2 Ferrofluid Displays . . . . . . . . . . . . . . . . . . 17

4.3.3 Volumetric Displays . . . . . . . . . . . . . . . . . 18

4.4 Combination of Technologies . . . . . . . . . . . . . . . . 18

5 An Alternative Approach to Organic User Interfaces 21

5.1 Data Presentation . . . . . . . . . . . . . . . . . . . . . . . 21

5.2 Data Manipulation . . . . . . . . . . . . . . . . . . . . . . 22

6 Evaluation 23

6.1 OUI in Everyday Life . . . . . . . . . . . . . . . . . . . . . 23

6.2 Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

7 Summary and Future Work 27

7.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

7.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Bibliography 31

Index 35

v

List of Figures

4.1 SmartSkin . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4.2 TWEND . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4.3 E-Ink . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4.4 Flexible OLED . . . . . . . . . . . . . . . . . . . . . . . . . 16

4.5 Lumen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.6 SnOil, a ferrofluid display . . . . . . . . . . . . . . . . . . 18

4.7 Gummi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

6.1 Organic UIs interaction method . . . . . . . . . . . . . . . 24

7.1 The Nokia Morph Concept . . . . . . . . . . . . . . . . . . 29

vii

List of Tables

3.1 Differences between Tangible UIs and Organic UIs . . . . 9

ix

Abstract

Nowadays, user interfaces are rigid and do not utilise human’s manipulation skill.They force users to learn certain methods to interact with them. However, in orderto improve the richness of interfaces, they should conform to the way humans in-teract with their environment. This leads to the concept of Organic User Interfaces(UIs) in which interfaces are designed to imitate this interaction. This is achievedby developing interfaces so that they comply with the following principles: in-put equals output, form equals function and form follows flow. Currently, there is nointerface, which follows all three principles above. Nonetheless, there are severaltechnologies leading to the realisation of Organic UIs.

x Abstract

xi

Uberblick

Heutige User Interfaces sind starr und schopfen die Moglichkeiten menschlicherInteraktion nicht aus. Stattdessen verlangen sie dem Benutzer ab, sich Metho-den zur Kommunikation anzueignen. Um aber die Qualitat des Interfaces zuverbessern, sollten sie sich der menschlichen Art mit der Umwelt zu interagierenunterwerfen. Dies fuhrt zu dem Konzept der Organisches User Interfaces (Organis-che UIs), die diese Interaktion imitieren. Erreicht wird dies durch die Entwicklungvon Interfaces, die folgenden Prinzipien entsprechen: Eingabe gleicht der Ausgabe,Form gleicht der Funktion und Form folgt Funktion. Im Moment ist noch kein Interfaceverfugbar, was allen drei Prinzipien genugt. Nichtsdestotrotz sind bereits einigeTechnologien erhaltlich, die die Realisierung von Organischen UIs ermoglichen.

1

Chapter 1

Introduction

“Because you don’t have to be green to be green.”

—MTV Switch, The Green Song

Being organic is a trend that becomes more and more popular thesedays. From organic food to organic clothing, from organic agricultureto organic cleaner, everything is going green to save the earth environ-ment and make humans healthier.

Inevitably, the term organic has also inspired the computing field. Nev-ertheless, the purpose of organic differs from the purpose of organicin general. In organic computing, the term organic means to imitatethe properties owned by organic beings such as adaptation. Apartfrom organic computing, there has been a silent development of so-called Organic User Interfaces (UIs). In which, interfaces are built toresemble the shape of nature, which is flexible and deformable, and theinteraction between nature systems.

This seminar paper presents an overview of the development ofOrganic UIs. Several work, which are related to the concept of OrganicUIs, will be presented in chapter 2. The next chapter is dedicated to thedefinition of Organic UIs and their design principles. Chapter 4 givesseveral technologies that have a great contribution in the realisation ofOrganic UIs. In chapter 5 an alternative approach to Organic UIs isgiven. Chapter 6 discusses the interaction styles and issues that mayarise by implementing Organic UIs in everyday life. Last but not least,summary and possible future work will be given in the last chapter.

2 1 Introduction

3

Chapter 2

Related Work

“Art is either plagiarism or revolution.”

—Paul Gauguin

Research on Organic UIs is still in its infancy state. Currently, not manyor even hardly any real UI is introduced as Organic UI. Nevertheless,several work related to Organic UIs have been done. One work in-cludes organic computing, which has the same concept of imitatingorganic properties found in nature system. Others include several UIs,of which characteristics have similarity with of Organic UIs.

2.1 Organic Computing

Organic Computing and Organic UIs share the same organic term. Al-though both concepts are inspired by nature, they have different ap-proaches in realising the nature or organic properties in a system.

Organic Computing is a system that dynamically adapts to changes in DynamicAdaptationthe environment and probably interact with each other [Seebach et al.,

2007]. The objective is to use principles observed in nature systemfor building technical system with organic properties [Muller-Schloer,2004]. These include self-configuration, self-adaptation, self-healing Self-x

Propertiesand other self-x properties, as well as context-awareness. By combin-ing these properties, the system is given with more degree of freedomto react upon component failures or environmental changes.

4 2 Related Work

2.2 User Interfaces We Know So Far

Until today, a number of UIs have been introduced to the public. Al-though there are no revolutionary changes since good HCI design isevolutionary rather than revolutionary [Canny, 2006], each believesthat its interaction is more intuitive than any of its predecessors. Be-low are some of UIs that can be seen as the building blocks of organicUIs.

Graphical UIs, especially the WIMP Interfaces, are probably the mostWIMPInterfaces well-knowned interface ever. Since 1970s when WIMP Interfaces

were introduced, all applications running on PCs are built to supportthem [Canny, 2006]. A WIMP Interface consists of windows, icons,menus, and is equipped with a pointing device, in this case a mouse.Many argue that this interface does not reflect daily objects interaction.Rekimoto in his article [Rekimoto, 2008] mentioned that by using amouse users can only interact with ”one point” and either ”pressed”or ”hover”. Meanwhile, in reality people touch objects on ”multiplepoints” and even put a special ”pressure” to them, like handshakingfor example.

The popularity of Graphical UIs has made its way to smaller mobiledevices such as PDAs. However, PDAs’ small screen makes it difficultfor the users to browse through menus. Adaptive Interfaces have beenAdaptive

Interfaces introduced to address this issue. The idea is that the system adaptsthe interface according to the users’ needs, such as displaying only thefeatures used most at the front page rather than all features. Althoughthis proved to be viable in small displays, in large display these ”un-predictable menus” may lead to users’ confusion [Findlater and Mc-Grenere, 2008].

Other interfaces similar with Adaptive Interfaces called AmbientInterfaces adapt their interface not based on their users but on theirAmbient

Interfaces users’ physical environment such as light, sound, or movement [Gross,HCII, 2003]. They can be used for awareness information environmentthat aims to improve awareness among geographically separated teammembers. One example is by displaying pop-up window giving in-formation regarding what the other co-workers are currently workingon.

Although Adaptive and Ambient Interfaces are improvements of clas-sical Graphical UIs, the main interface is still in WIMP-style. Manyresearchers moved towards post-WIMP Interfaces to enrich user expe-Post-

WIMP

2.2 User Interfaces We Know So Far 5

rience in interacting with interfaces by introducing more natural wayof interaction.

Touching is the first natural thing people do when encountering newobjects. By utilising this human instinct, Tactile and Haptic Interfaces Tactile-

HapticInterfaces

were developed [Motamedi, 2007]. The idea of these interfaces is thatthe users are actually touching the interfaces when they interact withthem and getting haptic feedback, such as vibration. The later featureadds a value to Tactile-Haptic Interfaces especially to visually impairedusers [Kahol and Panchanathan, 2006].

The ability to grasp and manipulate physical objects inspired the devel- TangibleUserInterfaces

opment of Tangible UIs. Hiroshi Ishii, as one of the pioneers of TangibleUIs, defined Tangible UIs as interfaces, which give physical form to itsdigital counterpart [Ishii, 2008b]. Users manipulate the digital infor-mation by directly manipulating the physical object that represents thedigital information and learn what interactions are possible from thephysical affordances of the object itself. Both properties increase thedirectness and intuitiveness of interactions.

6 2 Related Work

7

Chapter 3

Organic User InterfacesDefined

“To make something look real and alive, nothing can besymmetrical because nothing in real life is symmetrical. You

have to make it look organic. ”

—John Kricfalusi

Take a look at today’s PC hardware, for example an LCD. It is built to beplanar and rigid in order to protect the electronics inside it. Comparethis with paper. It can be folded, wrapped around, torn, even recycled.Such thing cannot be imagined to be done on the current LCD [Holmanand Vertegaal, 2008].

3.1 Properties of Organic

Compared to Organic Computing, in which self-x properties of naturesystem becomes the fundamental key in developing a technical sys-tem, Organic UIs are inspired by the shapes of the nature system which Nature

Shapeare transformable, flexible, naturally adaptable, resilient and reliable[Vertegaal and Poupyrev, 2008]. Just like a leave, which bends insteadof breaks to accommodate the reception of sunlight. It also grows andadjusts its shape to flexibly adapt with the environment [Holman andVertegaal, 2008].

8 3 Organic User Interfaces Defined

Another emphasis is on the analogue, continuous and transitionalnature of physical and human interaction [Rekimoto, 2008] not on theNatural

Interaction physical objects or metaphors [Schwesig, 2008]. A successful OrganicUI makes the users forgetting that they are operating machines to ma-nipulate virtual data.

A mouse, which is the most popular input device, is considered as themost inorganic interface. It is a tool to point and manipulate a certain(x, y) location of an object which is located on the display, a differentdevice. It contradicts the true nature of human interaction in which atool is rarely needed and the fact that people manipulate the shape ofthe object directly at multiple points at the same time.

3.2 Design Principles

Holman and Vertegaal [Holman and Vertegaal, 2008]defined three de-sign principles, which can be used as guidelines, to develop OrganicUIs. These include input equals output, form equals function and formfollows flow.

A person draws directly on the paper and views his/her drawing di-rectly from the paper. This visualises a true physical interaction whereinput and output interaction happen at the same location, in this caseInput

EqualsOutput

the paper. To imitate this behaviour, Organic UIs should have a displaywhich can sense [Holman and Vertegaal, 2008]; or to put it differentlythe input device acts also as the output device.

The form of an object gives hints on what activities people can do withFormEqualsFunction

it [Holman and Vertegaal, 2008]. The flexible form of a piece of papersuggests that it can be folded, bent, crumpled or even torn up. Yet, itcan still serve its original purposes: to be read or written on. OrganicUIs should use their form as a physical representation of activity.

A spring changes its shape to follow the movement of the person’sFormFollowsFlow

hand, which extends it. As the person lets the spring go, it will goback to its original shape. This implies that Organic UI should be ableto either alter its shape to follow the flow of user interaction or to adapt itsshape automatically for better context of use [Holman and Vertegaal, 2008].

3.3 Definition 9

3.3 Definition

Concluding from the three design principles above, a definition ofOrganic UI can be formalised. An Organic UI is a UI that more closely re-sembles natural human-physical and human-human interaction by using non-planar displays, as input and output, that may actively or passively changeshape following analogue physical input to adapt to user’s needs. [Rekimoto,2008] [Holman and Vertegaal, 2008] [Vertegaal and Poupyrev, 2008]

Although they are said to be organic, Organic UIs do not need to bemade out of organic materials. The emphasis is to promote flexibility,enhance users satisfaction and allow users to be creative rather thanproductive. [Holman and Vertegaal, 2008].

3.4 Tangible UI Vs. Organic UI

By looking at the definition of Organic UIs defined above, similari-ties between Organic UIs and their predecessor, Tangible UI cannotbe avoided. Organic UIs are Tangible UIs but Tangible UIs are not always Physical

Manipulationorganic. Tangible UIs are said to be the gate to Organic UIs. They in-spire Organic UIs to use physical interaction to manipulate digital data.However, they lack of shape adaptation since the tool used to interact No

ShapeAdaptation

cannot change shape in real time. Adding shape actuating behaviourmakes Tangible UIs may be considered as Organic UIs [Ishii, 2008a]. Ta-ble 3.1 describes further differences between Tangible UI and OrganicUI [Rekimoto, 2008].

Characteristics Tangible UIs Organic UIsInteraction Metaphor using tool direct contactOrientation manipulation-oriented communication-orientedRepresentation tool = digital info shape = activityCoverage application specific generic

Table 3.1: Differences between Tangible UIs and Organic UIs

10 3 Organic User Interfaces Defined

11

Chapter 4

On the Way Towards OrganicUser Interfaces

“Technology: No Place for Wimps!”

—Scott Adams, Dilbert

As shown, Organic UIs are meant to be more intuitive than other UIs.To provide this, technologies enabling devices to feel organic are nec-essary. Input as well as output technologies are essential to make theinteraction between human and computer feel natural. In this section,some of these technologies are described.

4.1 Input

First of all, the users should be provided with an intuitive way to ma-nipulate data on the computer. Since nowadays most common inputdevices, such as mouse and keyboard, are very unnatural, a touch-screen can be said as an advancement of the nativeness of a userinterface, even though a touch-screen is bound to the rigid and planarshape of a computer. Since rigidity does not occur in nature, OrganicUIs should be without edges and corners as well.

12 4 On the Way Towards Organic User Interfaces

4.1.1 Interactive Surfaces

Three kinds of evolvement could be applied to a standard touch-pad.By these, a standard touch panel could be improved to an interactivesurface, which enables intuitive interaction.

Firstly, the bondage of rigidity has to be broken. Not only shouldFlexibleDevices touch pads be capable to fit any form or shape but also to be deformed

while being used. A first approach to this improvement is SmartSkinthat could be produced to be flexible. Additionally, it could also betransparent to be applied on top of a display [Rekimoto, 2002].

Secondly, multiple inputs should be allowed, so that more than oneMultipleInputs hand can be used and even multiple users are able to interact at the

same time. At the moment, multi-touch panels, which use differenttechnologies, are available. The best-known examples are Apple prod-ucts such as the iPhone or iPod touch.

Thirdly, the sensing capabilities should be improved. Hovering as wellGestureDetection as other gestures should be recognised. This would allow a more nat-

ural way of input as one can express complex instructions in uncom-plicated ways. SmartSkin , which uses capacitive sensing, [Rekimoto,2002] provide this capability. A more recent approach is ShapeTouch, in which the sensing is more precise [Cao et al., 2008]. By using op-tical recognition, ThinSight tracks hand gestures in short distance andthrough an overlying display [Hodges et al., 2007]. Toshiba recentlyreleased a notebook with a dedicated processor to recognise gesturesusing the built-in webcam [Toshiba DPD, 2008].

Figure 4.1: SmartSkin, an interactive surface - by Sony CSL

4.1 Input 13

Although these technologies are promising, there are still several is- Issuessues to be addressed. Up to now there is no way to provide directhaptic feedback. Also, the coupling of touch panel and display has tobe improved to enable thin devices. Using projectors decouples theinput and output. Additionally, the use of gestures is a newly discov-ered field and effective ways of interaction have yet to be found. Userstudies have to be performed to discover how users would like com-municating their commands.

4.1.2 Deformation Tracking

Pure bearing of deformation, as in the section before, might not be DeformationasInput

enough. For instance books allow input by deformation when search-ing for a specific page. It is likely to bend the pages in a way thatthey quickly flip over enabling the user to scroll through the bookwhile getting a glance at every page. Yet, this is not the only exam-ple. Deformation is a very common way of interaction with objects.This leads to the conclusion that organic devices should be capable oftracking its own actual shape.

TWEND, an input device, named by the terms ”twist” and ”bend”, Stateof theArt

provides this capability. Deformation regarding to the X- and Y-axiscan be recognised. In this approach, optical bending sensors are used.Based on this technology, gestures can be defined: flipping one cornerover for going to the next page or simulating a dog-ear for bookmarks[Herkenrath et al., 2008]. A more recent device providing similar fea-tures is Bookisheet [ichiro Watanabe et al., 2008].

Figure 4.2: TWEND, a device to track twisting and bending - by theMedia Computing Group at RWTH Aachen

14 4 On the Way Towards Organic User Interfaces

The problem when allowing users to interact via deformation is thatIssuesthe whole device has to be deformable. Flexible displays are availablebut the whole processing hardware has to be flexible as well. Usefulgestures have to be found similar to the multi-touch panels. User stud-ies on the interaction based on gestures have to be conducted to clarifyhow these features can be used efficiently.

4.2 Flexible Displays

Only creating devices, which are deformable, is not enough to createa flexible touch-pad or even to track the deformation. Since the wholedevice should be flexible, the display has to be bend- and twistableas well. In addition, this development is accompanied by a higherdurability. There are two highly promising technologies.

4.2.1 Digital Paper

Formerly, the most used medium to provide information was paper,HighContrastandDurability

which is very different to computer screens. On one hand, it lacks ofthe ability to change its information easily. On the other hand, it is flex-ible and has a very high contrast. While computer screens are not read-able any more when the sun shining on them, paper is immune to thateffect. Closing this gap is electrophoretic ink displays, which are oftenassociated with the brand E Ink. They combine advantages of both me-dia. Also, they can be equipped with background lighting easily. Inaddition, digital paper has less power consumption than common Liq-uid Crystal Displays (LCDs) because due to their technique, their stateis stable and only changes trigger the use of energy [Comiskey et al.,1998].

Recently, E Ink was improved such that it is unbreakable. It withstandsStateof theArt

extreme vibration as well as impacts of heavy objects. This broadensthe application fields of electrophoretic displays.

Electrophoretic displays are already far developed and applicable inIssuesproductive use. There are several products on the market using thisdisplay technology, usually developed as E-Book readers, such as theAmazon Kindle. However, digital paper still suffers from two main

4.2 Flexible Displays 15

Figure 4.3: A bendable E-Ink display - by E-Ink

problems. Firstly, its frame rate is low. Videos cannot be shown usingthis kind of display. This also hinders electrophoretic displays to beused in portable devices such as phones or PDAs because fluid menunavigation is not possible. Even so they convince their customers withtheir very low energy consumption. The Motorola F3 was the first cellphone using digital paper but had only basic features. Secondly, thechroma resolution is very low. Coloured displays can be producedby microcapsules containing red, green and blue droplets rather thanwhite ones. Three capsules could be combined to one pixel.

4.2.2 Bendable Screens

While digital paper enhances the way the information can be altered Full-colourandVeryThin

easily, Organic LED (OLED) displays improve nowadays screens. Byusing OLED technology, full-coloured computer displays can be verythin and robust and therefore flexible. Compared to an LCD, the en-ergy consumption is lower and the size of the borders is smaller. Thisenables OLED displays to be applied in the most convenient way.

Sony has already presented a flexible full-coloured display based on the Stateof theArt

OLED technology. Recently, Samsung SDI presented an OLED displaywith a thickness of only 0.05 mm, which actually flaps in the wind.These thin displays allow light shining through them and are flexible.

16 4 On the Way Towards Organic User Interfaces

Figure 4.4: A flexible Organic LED display - by Pioneer

Although OLED displays are far developed and already applicable inIssuesproductive use they still suffer from several issues. At present only rel-atively small displays are available. Because of this, OLED displays areonly available in small devices such as mp3-players. In further devel-opment, the screen size has to be enlarged. Another issue is the rela-tively short lifetime of Organic LEDs. Although they are still very highcompared to LCD- or Plasma-displays, they still cannot compete withLEDs.

4.3 Shape Actuation

Two-dimensional displays are not the only way to visualise informa-tion. In several applications, the shape of an actual object representsdigital data. Recent computers display a three-dimensional object byhaving several points of view at the object at the same time. This addsadditional information to the data. Additionally, direct manipulationof an actual object would be more efficient and natural.

4.3.1 Physical 3D Displays

The most efficient and convenient way of manipulating three-ActualObjects dimensional objects would be a physical representation of the object of

interest. This would grant direct haptic feedback and an instant viewon the result. This overlaps with the field of Tangible UIs.

4.3 Shape Actuation 17

One of technology for shape alternation is so called shape memory SimpleApproachalloys. Using this is Lumen. It is a 16x16 pixel display where a third

dimension is added by enabling the pixels to alter their physical height.Therefore, every pixel provides not only information by its red, greenand blue colour value, but also by its height [Poupyrev et al., 2004,2007].

Figure 4.5: Lumen, a simple three-dimensional display - by Sony CLS

At the moment the possibilities of shape alternation are limited. Lumen Issuesis a simple approach and cannot convey a lot of additional data viathe height of the pixels. More advanced approaches would need morecomplex shape alternation.

4.3.2 Ferrofluid Displays

While physical 3D displays are still bound to the shape of their ele- ShapeableLiquidments, ferrofluid displays are an approach to alternate shape in another

way. Ferrofluid acts similar to iron, but is liquid. When a magnet ap-proaches, it changes its shape.

An example for ferrofluid displays is SnOil. It is an implementation of Stateof theArt

the classic game Snake. Underlying electromagnets influence a basinfilled with ferrofluid [Poupyrev et al., 2007]. Another approach is Pro-trude, Flow. It is not displaying information but focusing on aesthetics[Kodama, 2008].

Although ferrofluid displays are more versatile than nowadays physi- Issuescal 3D displays, they are not capable of representing any shape. Theyare bound to the possibilities of electromagnets. In addition, they can-not be touched.

18 4 On the Way Towards Organic User Interfaces

Figure 4.6: SnOil, a ferrofluid display - by Martin Frey at UDK Berlin

4.3.3 Volumetric Displays

Volumetric displays are not part of the field of shape actuation but alsoHologramscapable of displaying information three-dimensionally. A precise handtracking could enable users to modify the projected objects directly us-ing their hands. Since volumetric displays are encapsulated in glass,this concept is yet unrealisable [Grossman and Balakrishnan, 2006].

4.4 Combination of Technologies

When building an Organic UI it is not necessary to include all organicBuildinganOrganicUI

technologies mentioned before. As already explained an Organic UIhas to be adjusted to its use very carefully. These technologies are toutilize that fit to the theme of the device that is about to be built.

Figure 4.7: Gummi, an organic digital map

4.4 Combination of Technologies 19

Gummi is the concept of a digital map as an Organic UI. The whole FirstOrganicUI

device is bendable and bending is used as input. In addition, a bend-able touch-pad is positioned on the backside. Gummi is the first devicecombining a lot of organic aspects. It is flexible and also tracks thedeformation to provide intuitive input. The user is not that aware ofit as a computer but recognises its capability to display a street map[Schwesig et al., 2003, 2004]. Another example for the concept of aOrganic UI is Morph, which will be explained in 7.2—“Future Work”.

Currently, Gummi is just a concept, the actual prototype does not con- Issuessist of all aimed features. The prototype already allows evaluating theeffectiveness of the interaction since all features are emulated. Gummiis not yet bendable but recognises applied pressure as if one was bend-ing it.

20 4 On the Way Towards Organic User Interfaces

21

Chapter 5

An Alternative Approach toOrganic User Interfaces

“A different language is a different vision of life.”

—Federico Fellini

Up to now, the focus lays on devices that enable users to interact withthem in a natural way. The most important aspect on this was the flexi-bility of the devices. However, there is a different approach in viewingOrganic UIs, that not only the device itself can be organic but also canbe the software. There is a possibility for software to provide informa-tion in a natural way and still running on an ordinary computer.

5.1 Data Presentation

Data could be provided and shown in a natural way. Nowadays, OrganicPresentationTool

presentation tools are based on the concept of a linear sequence ofslides. Obviously the human mind is not organised in a straight pro-ceeding. Dealing with this problem is Fly. It tries to map the thoughtsof the presenter to the presentation. The information to be described isorganised in any way the author wants it to be [Holman et al., 2006].

22 5 An Alternative Approach to Organic User Interfaces

Another section of the field of organic data presentation is the datastructuring. Apple recently developed iPhoto software to organise pho-tos in a more natural way than any other software before. Photos aregrouped into events that can be tagged by the users. By this, they do nothave to remember the exact date, but only the people they met there.This is a good representation of the human memory.

5.2 Data Manipulation

Digital data could be manipulated in a more natural way. WhileOrganicVideoNavigation

the common way to navigate within a video is the timeline, themanipulation of objects in the video to browse through time would bemore convenient. For example, one can jump to a previous part of themovie by dragging a person backwards in space and by this also intime. When a digital object is moved to the position where it was in aprevious part, all other objects are reset to their previous positions aswell. This is the concept of DRAGON [Karrer et al., 2008].

Although DRAGON is a very organic way of video navigation, it isIssuesnot practical. For instance, when the scene changes, the objects onthe frame change as well. Therefore, DRAGON could not be used fornavigation in movies.

23

Chapter 6

Evaluation

“I do not fear computers. I fear the lack of them.”

—Isaac Asimov

Several technologies in the field of Organic UIs have been described.However, it is questionable whether these approaches might be appli-cable to everyday life.

6.1 OUI in Everyday Life

A few years ago when PCs made its way to offices to store necessarybusiness documents, employers have dreamed to cut business cost byrealising a paperless office. Nevertheless until today, many employ- Paperless

Officeees still prefer to print out the document, hold the paper in hands andmark things up [Blevis, 2008]. With the emergence of thin and flexibledisplays, they are becoming more and more paper-like. In the near fu-ture, it is likely that displays can be treated as real paper and paperlessoffice can finally be realised.

By applying other continuous parameters such as pressure, one can im- ContinuousParametersprove the intuitiveness of the interface [Rekimoto, 2008]. Different pres-

sure applied to the interface should result in different possible action.An example would be the bending interaction, which means zoomingaction. A softer pressure to the interface means that the zooming actionis slower than when a harder pressure is applied.

24 6 Evaluation

Figure 6.1: Paper-like Organic UIs interaction method

The development of Organic UIs is still in its infancy. Therefore, notmany interaction methods could be thought up during this period.However, regardless on which interaction method introduced later,Emotionally-

involved interaction with Organic UIs should make people emotionally involvedin the interaction and forget that they are operating computers [Schwe-sig, 2008] and see them as ordinary everyday object [Holman and Verte-Everyday

Object gaal, 2008].

As for the alternative approach of organic UIs, the interaction methodAlternativeInteraction might be different with the Organic UIs mentioned above. However,

the emphasis is still to establish a natural interaction between the usersand the interface. DRAGON, for example, allows users to navigatethrough a video by selecting the object inside the video instead of thevideo timeline [Karrer et al., 2008]. While Fly presents a new way to or-ganise presentation slide as mind map rather than in linear order [Hol-man et al., 2006].

6.2 Issues

Nothing is perfect. The concept of Organic UIs gives a modernapproach on how computers can integrate seamlessly in everyday lifeso that people forget that they are actually computers. Nevertheless,issues arise inevitably.

Flexibility in Organic UIs arises issues concerning the hardware. Notonly the displays, which should be flexible but also the processor andFlexible

Hardware other electronics. Current technologies mentioned before revealed thatthere is no any approach, which tries to combine organic in- and out-put technologies so that they can be seen as one device. While TWENDdoes not possess a display, flexible displays such as E-Ink or Organic

6.2 Issues 25

LED supplies no input method. Gummi tried to combine these twotechnologies. However, they are still located in different positions(front and back).

Recently developed technologies in the field of shape actuation sufferfrom another problem. Either they are bound to certain types of formmanipulation and therefore cannot display any information or they are Untouchable

Interfacenot suitable for human interaction as they can not be touched.

The interaction techniques of Organic UIs are not complete yet. Onlysimple interaction techniques, such as bending for zooming action us- Incomplete

InteractionTechniques

ing both hands, have been introduced. It is questionable whether laterthe interaction can involve other modalities, such as eye gazing, blow-ing and entire body. Thus, more interaction techniques are still yet tobe discovered. [Rekimoto, 2008]

One property that makes Organic UIs differ from Tangible UIs is thatOrganic UIs are general rather than application-oriented. However, itdoes not mean that one Organic UI fits all since consistencies across ac- Interaction

Inconsistenciestivities and contexts might be difficult to realise [Holman and Vertegaal,2008]. In graphic application, bending might mean zooming; while invideo application bending might mean fast forwarding.

Last but not least, enhancing computers in a way that they are notrecognised as computers any more might lead to privacy problem. Peo- Privacyple might feel inconvenience by the thought of being observed by com-puters all the time and not being able to distinguish between digitallyimproved and ordinary objects in their everyday life.

26 6 Evaluation

27

Chapter 7

Summary and Future Work

“Our imagination is the only limit to what we can hope tohave in the future.”

—Charles F. Kettering

Coming to the end of this seminar paper, several questions are still re-mained. How is the future of Organic UIs? Will they be as successful asGraphical UI? What improvements can be done to guarantee the futureof Organic UIs?

7.1 Summary

Organic UI is a UI which more closely resembles natural human- Definitionphysical and human-human interaction by using non-planar displays,as input and output, that may actively or passively change shapefollowing analogue physical input to adapt to user’s need 3.3—“Definition”. The definition states that Organic UIs should follow threedesign principles, which are input equals output, form equals function and Design

Principlesform follows function 3.2—“Design Principles”.

Based on these definition and design principles, currently there is no NoOrganicUI

real UI which follows these principles. Nevertheless, several technolo-gies of input, flexible displays and shape actuators play an importantrole in the realisation of Organic UIs.

28 7 Summary and Future Work

Another approach in viewing Organic UIs from other point of viewAlternativeApproach has been introduced. Rather than focussing on the shape of the device,

the alternative approach focusses more on the interaction between theusers and the application interface.

Organic UIs aims to create interfaces, which are seamlessly integratedwith everyday life objects. To resemble everyday life objects, OrganicUIs need to be flexible. This property is fulfilled by the emergence offlexible displays and flexible input devices today. More research andIssuesexperiments still need to be done in order to couple these two tech-nologies. The seamlessly integration of the interface might raise pri-vacy problem. People are afraid of being watched since they cannotdifferentiate between which objects are computers and which are not.

7.2 Future Work

Organic UIs is a fresh concept in interaction design. Since its officialintroduction published by ACM in April 2008, there has been no pub-lication labelled “Organic UIs“ which leads to further development ofOrganic UIs. Nevertheless, by looking at the past and current technolo-gies presented before, a possible future work can be concluded.

The in- and output technologies which lead to the realisation of organicInput +Output UIs have been developed. However, there has not been any attempt to

put these two technologies together such that they can be seen as onedevice. One possibility is by combining TWEND which is an inputdevice and Flexible OLED which an output device. Both are flexibleand possible to be put into one device.

Introduced earlier this year was a concept by Nokia called Morph,NokiaMorph [Nokia, 2008] which uses Nanotechnology as the basis for future mobile

phones. Nanotechnology is a development and research on materialsNanotechof which size ranges from 1 -100 nanometer (1 nm = 10−6 mm = 10−9 m)[Paull and Lyons, 2008]. By using Nanotechnology, the Nokia Morphdemonstrates the possibility to have a mobile phone, which is flexible,stretchable and transparent. It is charged using solar power and hasintegrated sensors, which can sense the environment around the users.

7.2 Future Work 29

Figure 7.1: The Nokia Morph Concept

Further user studies should also be done to discover new interaction NewInteractionTechniques

techniques for using organic UIs and to address the privacy issues,which might come out when a computer does not feel like a computeranymore.

30 7 Summary and Future Work

31

Bibliography

Eli Blevis. Sustainability implications of organic user interface tech-nologies: an inky problem. Commun. ACM, 51(6):56–57, 2008.

John Canny. The future of human-computer interaction. Queue, 4(6):24–32, 2006.

Xiang Cao, Andrew D. Wilson, Ravin Balakrishnan, Ken Hinckley, andScott E. Hudson. Shapetouch: Leveraging contact shape on interac-tive surfaces. Horizontal Interactive Human Computer Systems, 2008.TABLETOP 2008. 3rd IEEE International Workshop on, pages 129–136,Oct. 2008.

Barrett Comiskey, J. D. Albert, Hidekazu Yoshizawa, and Joseph Jacob-son. An electrophoretic ink for all-printed reflective electronic dis-plays. Nature, 394:253–255, May 1998.

Leah Findlater and Joanna McGrenere. Impact of screen size on perfor-mance, awareness, and user satisfaction with adaptive graphical userinterfaces. In CHI ’08: Proceeding of the twenty-sixth annual SIGCHIconference on Human factors in computing systems, pages 1247–1256.ACM, 2008.

Tovi Grossman and Ravin Balakrishnan. The design and evaluation ofselection techniques for 3d volumetric displays. In UIST ’06: Proceed-ings of the 19th annual ACM symposium on User interface software andtechnology, pages 3–12. ACM, 2006.

Gero Herkenrath, Thorsten Karrer, and Jan Borchers. Twend: Twistingand bending as new interaction gesture in mobile devices. In Ex-tended Abstracts of CHI 2008 Conference on Human Factors in Comput-ing Systems, Florence, Italy, April 2008. ACM Press. Student ResearchCompetition 2nd Place.

Steve Hodges, Shahram Izadi, Alex Butler, Alban Rrustemi, and BillBuxton. Thinsight: versatile multi-touch sensing for thin form-factor

32 Bibliography

displays. In UIST ’07: Proceedings of the 20th annual ACM symposiumon User interface software and technology, pages 259–268. ACM, 2007.

David Holman and Roel Vertegaal. Organic user interfaces: designingcomputers in any way, shape, or form. Commun. ACM, 51(6):48–55,2008.

David Holman, Predrag Stojadinovic, Thorsten Karrer, and JanBorchers. Fly: an organic presentation tool. In CHI ’06: CHI ’06 ex-tended abstracts on Human factors in computing systems, pages 863–868.ACM, 2006.

Jun ichiro Watanabe, Arito Mochizuki, and Youichi Horry. Bookisheet:bendable device for browsing content using the metaphor of leafingthrough the pages. In UbiComp ’08: Proceedings of the 10th internationalconference on Ubiquitous computing, pages 360–369. ACM, 2008.

Hiroshi Ishii. The tangible user interface and its evolution. Commun.ACM, 51(6):32–36, 2008a.

Hiroshi Ishii. Tangible bits: beyond pixels. In TEI ’08: Proceedings of the2nd international conference on Tangible and embedded interaction, pagesxv–xxv. ACM, 2008b.

Kanav Kahol and Sethuraman Panchanathan. Distal object perceptionthrough haptic user interfaces for individuals who are blind. SIGAC-CESS Access. Comput., (84):30–33, 2006.

Thorsten Karrer, Malte Weiss, Eric Lee, and Jan Borchers. Dragon:a direct manipulation interface for frame-accurate in-scene videonavigation. In CHI ’08: Proceeding of the twenty-sixth annual SIGCHIconference on Human factors in computing systems, pages 247–250.ACM, 2008.

Sachiko Kodama. Dynamic ferrofluid sculpture: organic shape-changing art forms. Commun. ACM, 51(6):79–81, 2008.

Nima Motamedi. The aesthetics of touch in interaction design. In DPPI’07: Proceedings of the 2007 conference on Designing pleasurable productsand interfaces, pages 455–460. ACM, 2007.

Christian Muller-Schloer. Organic computing: on the feasibility ofcontrolled emergence. In CODES+ISSS ’04: Proceedings of the 2ndIEEE/ACM/IFIP international conference on Hardware/software codesignand system synthesis, pages 2–5. ACM, 2004.

Nokia. The morph concept, December 2008. URL http://www.nokia.com/A4852062.

Bibliography 33

John Paull and Kristen Lyons. Nanotechnology: The next challenge fororganics. Journal of Organic Systems, 3(1):3–22, 2008.

Ivan Poupyrev, Tatsushi Nashida, Shigeaki Maruyama, Jun Rekimoto,and Yasufumi Yamaji. Lumen: interactive visual and shape dis-play for calm computing. In SIGGRAPH ’04: ACM SIGGRAPH 2004Emerging technologies, page 17. ACM, 2004.

Ivan Poupyrev, Tatsushi Nashida, and Makoto Okabe. Actuation andtangible user interfaces: the vaucanson duck, robots, and shape dis-plays. In TEI ’07: Proceedings of the 1st international conference on Tan-gible and embedded interaction, pages 205–212. ACM, 2007.

Jun Rekimoto. Organic interaction technologies: from stone to skin.Commun. ACM, 51(6):38–44, 2008.

Jun Rekimoto. Smartskin: an infrastructure for freehand manipulationon interactive surfaces. In CHI ’02: Proceedings of the SIGCHI con-ference on Human factors in computing systems, pages 113–120. ACM,2002.

Carsten Schwesig. What makes an interface feel organic? Commun.ACM, 51(6):67–69, 2008.

Carsten Schwesig, Ivan Poupyrev, and Eijiro Mori. Gummi: userinterface for deformable computers. In CHI ’03: CHI ’03 extendedabstracts on Human factors in computing systems, pages 954–955. ACM,2003.

Carsten Schwesig, Ivan Poupyrev, and Eijiro Mori. Gummi: A bendablecomputer. In CHI ’04, pages 24–29. ACM, 2004.

Hella Seebach, Frank Ortmeier, and Wolfgang Reif. Design and con-struction of organic computing systems. In IEEE Congress on Evolu-tionary Computation 2007, pages 4215–4221. IEEE, 2007.

Toshiba DPD. Toshiba qosmio world’s first laptop with cell proces-sor technology, July 2008. URL http://explore.toshiba.com/pressrelease/423413?fromPage=editorials.

Roel Vertegaal and Ivan Poupyrev. Organic user interfaces. Commun.ACM, 51(6):26–30, 2008.

Typeset January 28, 2009