Advisor: Prof. Francesco Tisato Tutor: Prof. Carla Simone September 23th 2009

Advisor: Prof. Francesco Tisato Tutor: Prof. Carla Simone

September 23th 2009

Diego BerniniPh.D. research status and thesis hypothesis

Software Architectures forResponsive Environments:

the case of Interactive Art & Design

University of Milano-Bicocca, Doctoral School of Science Ph.D. Program in Informatics, Cycle XXIV

Department of Informatics, Systems and Communication

September 23th 2009 D. Bernini, Ph.D. research status 2

Outline

• Reference scenario• Research topic, status and thesis hypothesis• State of the art• Plan of the activities• Miscellanea

Software Architecture

• Having a sound software architecture is a key aspect of ICT systems

• In different classes of ICT systems relevant high-level concepts are not turned into proper elements in their software architecture– need of proper architectural abstractions– examples: time and space in surveillance systems and Geographic

Information Systems (Micucci et al., 2006) (Micucci et al., 2009)

• This leads to several issues– intermix of conceptual, design aspects and implementation

details– strong dependency on specific technologies



Responsive environments

• Real physical spaces equipped with ICT technologies capable of sensing and responding accordingly to entities that inhabit them (Silva and Vasconselos 2007) (Pinhanez and Bobick 2003)

• Different aims– From more concrete …

• Automation of offices, shops, houses (Elrod et al., 1993)– …to more aesthetics

• Interactive Art & Design (Bullivant 2007) (Bullivant 2006)

• Various related research areas, different characterizations– From Software Engineering, Ubiquitous (Pervasive) Computing– to Human-Computer Interaction, Ambient Intelligence


..in Interactive Art

• ICT-supported interactive art installations– Art Installations: artworks where space is central

• “art that appropriates space to its own artistic ends” (Bestor 1996)– Interactive Art installations: the viewer become an active player – ICT-supported: some interactions are achieved by ICT

• For itself: they are integrant part of the installation• For the exhibition: valorization of traditional artworks/installations

– Interactive museums and galleries

(Studio Azzurro,www.studioazzurro.com) (Rafael Lozano Hemmer,

http://www.lozano-hemmer.com)


.. in Interactive Design

• Various classes of interactive small-scale architectural & design interventions (Castle 2007) (Bullivant 2007) (Bullivant 2006)

• Interactivity: the environment enhance the user experience• Interactive Architecture & Design, urban-open & closed spaces

(Digital Pavillion,ONL [Oosterhuis_Lénárd] design office

http://www.oosterhuis.nl)

(Interactive tunnel, Vito Acconci Studio and CSAI,

(Bandini et al 2008))


High-level concepts: examples• Time

– real-time responsiveness: the interactions may require strong time constraints

– temporal input analysis and time-driven output

• Space– locating objects in multiple spaces (e.g. sensor network space vs.

environment topology)

• Multi-modality– Multiple heterogeneous sensing and responding

• Sensing: – Strict physical environmental properties

» Pressure, temperature, humidity, lighting condition, noise levels– Entity properties

» Presence, identification, motion, contact, physiological

• Responding: audio and visual effects, commanding of specific devices


Current practice

• Various development tools, but– no explicit treatment of the previous high level concepts!– low level of abstraction

• Hardware computing devices: “low level” programming– E.g. Arduino, input, output & computing device, C programming

– oriented to specific technologies• Different visual programming language & tools, but too specific

– E.g. Cycling’74 Max, oriented to audio and image processing

• … consequently, they hardly address issues like:– Management of complexity: do not add complexity to the

inherent complexity of the desiderata interactions– Integration: how to integrate new and heterogeneous

technologies – Scalability: from few homogenous sensors to a lot of

heterogeneous sensors


Research topic

Filling the gap between low-level, technology oriented aspects and high-level concepts of the previous

response environments

• What are suitable architectural abstractions for the development of responsive environments in Interactive Art & Design? (Trifonova 2008) (Edmonds et al., 2005) (Machin 2002)


What I Have Done

• Characterization of a thesis hypothesis• Investigation of the state of the art • Identification of case studies and collaborations (in

progress)

• GAS (“Grandi Attrezzature Scientifiche”) project (UniMiB-DISCo): location aware, multimodal applications – my overall collaboration: architectural abstractions for the

integration of heterogeneous components• they allow the establishment of communication flows based on

space and time– my current specific task: design and implementation of a

supporting platform (in progress)


Thesis hypothesis (what)

• Devising effective architectural abstractions for responsive environments in Interactive Art and Design that:– Goal 1: abstract from the specific technologies

(hardware-input/output and computing devices- and software- specific programming languages & components-)

– Goal 2: go beyond ad-hoc approaches, fulfillment of real-time and multimodal requirements in an effective, robust and scalable way

– Goal 3: are usable by artists & designers, not only technologists– Goal 4: are applicable to other purposes, e.g. Video-surveillance,

Domotics

• Design and development of experimental applications based on the proposed abstractions– One for Interactive Art & Design, another for a different context


Thesis hypothesis (how and who)

• Core approach: Software Architectures (Prof. Tisato) enhanced with other disciplinary perspectives– Ambient Intelligence– Interaction Design/Human-Computer Interaction– Collaborative work and Knowledge Management – Prof. Tisato, Prof. Archetti, Prof. Bandini, Prof. De Michelis, Prof.

Schettini, Prof. Simone...

• Identified case studies, collaborations and abroad period– A specific workgroup among CSAI center, DISCo-ITIS and DISCo-

SAL labs– Studio Azzurro (ref. Prof. De Michelis)– GAS project – Abroad period: acquiring knowledge & skills in interactivity vs. Art

& Design vs. ICT• e.g. The Mixed Reality Lab, University of Nottingham (ref. Prof. De

Michelis)


Physical environment

OU

T DE

VICE

S

IN D

EVIC

ES

State of Art: a reference schema

– It does NOT imply a centralized design organization and/or deployment

ICT System

Computing & Communication

Input flow Output flowResponsive environment

ICT System organization: from (purely) reactive,

stateful/reflexive, to emergent behaviors

centralized-federated

highlydistributed

Deployment

Different conceptualmodels are adopted:from Data flow modelsto Agents


State of the Art: enabling technologies

• Input: sensors, vision & tracking systems, tangible & wearable devices– Values of strict physical environmental properties and entity

properties, a survey in (Cook and Das 2007)

– Common technologies:• Wireless sensor networks, a survey in (Yick et al., 2008)• Cameras, image detection and tracking systems

– Object tracking, human motion capture ad analysis, surveys in (Yilmaz et al., 2006) (Moeslund et al., 2006)

– Face recognition (Colombo et al., 2007) and other systems: DISCo-Imaging and Vision Lab

• Tangible interfaces and wearable devices (Harrison and Hudson 2008)

• Output: lights, projectors, displays, speakers, sound synthesizers and other specific devices– Some common protocols: DMX (lights), MIDI (sound), Power Line

Communication (PLC) and X10 (general electronic devices)


State of the Art: tools & programming languages

• Arduino– “an open-source electronics prototyping platform

based on flexible, easy-to-use hardware and software”– http://www.arduino.cc/

• Cycling ’74 Max– Visual programming language, http://www.cycling74.com– System as computational graph of objects (patches) representing

input/output devices, processing and data containers• Objects: self-contained program, they communicate by messages

– Data flow oriented model, high extensible– MSP package: audio synthesis– Jitter package: video processing– Support for execution in local networks

• Others: Pure Data (PD), vvvv, Quartz Composer– Core-model similar to MAX, but different focus


State of the Art: architectural abstractions• Software Components

– (Heineman and Council 2001), (Shaw and Garlan 1996), component styles (Garlan 1996) (Maier and Rechtin 2000)

– Specific component typologies: components with ports and connectors (Caflisch et al., 2005), (Senthil et al., 2009), targeted to responsive environments: context components (Jacquet et al., 2005), e-Gadgets (Markopoulos et al., 2004)

• Time and space awareness– time (De Paoli and Tisato 1995), (Micucci et al., 2006), space

(Micucci et al., 2009), GAS project

• Emergent conceptual and architectural models– Multi-Agent models, Cellular Automata (CA) models, suitable for

emergent and lifelike behaviors– (Bandini et al., 2008), (Silva and Vasconcelos 2007), (Sparacino et

al., 2001)


State of the art: conceptual frameworks

• Time and space– time, space and audience vs. interactivity (Benford et al., 2009)

• Multi-modality– multi-modality categories (Bongers and Veer 2007)– typologies of sensing (Benford et al., 2005)– human movement representation and responsive environments

(Loke and Robertson 2009)


Plan of Activities

• Phase 1 (investigation) - 12 months– Investigation of the state of the art (completed)– Identification of case studies & collaborations (currently in

progress)

• Phase 2 (research) – 12 months– Devising and development of the architectural abstractions, by

generalizing the analysis of the identified case studies

• Phase 3 (experiment) – 8 months– Design and development of experimental applications based on

the proposed abstractions

• Phase 4 (research) - 4 months– Refinement of the architectural abstractions on the basis of the

previous experimental applications


Wishful thinking

• How artists/designers would like to conceive the interactivity, the audience/user participation, independently from available technologies?– From collaborations with artists/designers!– This is NOT A GOAL of the thesis


Target Conferences and Journals

• Conferences (and related Symposia)– IEEE/IFIP Conference on Software Architecture (WICSA) and European

Conference on Software Architecture (ECSA)– ACM Special Interest Group on Graphics and Interactive Techniques

(SIGGRAPH) conferences• ACM’s Creativity & Cognition conferences• ACM’s International conference on Digital Interactive Media in

Entertainment and Arts – ACM Special Interest Group for Computer–Human Interaction (SIGCHI)

conferences– ACM International Conference on Multimedia– International Conference on Tangible and Embedded Interaction (TEI)

• Journals– ACM Transactions on Computer Systems (TOCS)– Pervasive and Mobile Computing, Elsevier (PMC)– ACM Transactions on Computer-Human Interaction (TOCHI)– ACM Transactions on Multimedia, Applications, and Computing

(TOMCCAP)


Courses

• Attended courses:– “Knowledge Management: conoscenza, condivisione,

tecnologia”, Prof. Simone– “Interaction Design”, Prof. De Michelis– Others

• “Laboratorio di Comunicazione” (from QUASI_SI Ph.D. Program)• “Comunicazione della Scienza” (Doctoral School of Science)• “Gestione dei progetti di ricerca” (Doctoral School of Science)• English course (DISCo), ongoing

• Next courses:– 2/3 internal courses

• “Aspetti epistemologici dell’informatica”, DISCo, Prof. Bandini• Enabling technologies: courses of Prof. Bisiani & Schettini?

– 2/1 external courses/Ph.D. schools


Other collaborations

• F.I.R.B. - Integrated Systems for Emergency project– two papers related to my Master’s degree thesis

• Bernini, D., Toscani, D., and Frigerio, M. A software architecture for the deployment of executable transformation models. Proceedings of the 2009 International Conference on Wireless Communications and Mobile Computing: Connecting the World Wirelessly, ACM (2009), 47-51.

• Toscani, D., Frigerio, M., and Bernini, D. Dynamic update of data analysis models in emergency systems. Proceedings of the 2009 International Conference on Wireless Communications and Mobile Computing: Connecting the World Wirelessly, ACM (2009), 37-41.

– workshop participations and talks• Leipzig, 21-24 June 2009, International Workshop on Advanced

Topics in Mobile Computing for Emergency Management: Communication and Computing Platforms (MCEM 2009), presentation of (Bernini et al., 2009)

• Rome, 3-4 April 2009, InSyEme internal workshop, talk “From Application Requirements to Application Architecture”


References (1)

• Overview of Responsive Environments, Interactive Art & Design– Bullivant, L. Responsive Environments: architecture, art and design. Victoria and

Albert Museum, 2006.– Bullivant, L. 4dSocial: Interactive Design Environments. John Wiley & Sons, 2007.– Castle, H. Editorial. Architectural Design 77, 4 (2007), 4-5.– Elrod, S., Hall, G., Costanza, R., Dixon, M., and Rivières, J.D. Responsive office

environments. Commun. ACM 36, 7 (1993), 84-85.– Edmonds, E.A., Weakley, A., Candy, L., Fell, M., Knott, R., and Pauletto, S. The

studio as laboratory: combining creative practice and digital technology research. Int. J. Hum.-Comput. Stud. 63, 4-5 (2005), 452-481.

– Machin, C.H.C. Digital Artworks: Bridging the Technology Gap. Proceedings of the 20th UK conference on Eurographics, IEEE Computer Society (2002), 16.

– Trifonova, A., Jaccheri, L., and Bergaust, K. Software engineering issues in interactive installation art. International Journal of Arts and Technology 1, 1 (2008), 43-65.


References (2)

• Enabling technologies, tools & programming languages– Bestor, C. MAX as an overall control mechanism for multidiscipline installation art.

Computers & Mathematics with Applications 32, 1 (1996), 11-16.– Cook, D.J. and Das, S.K. How smart are our environments? An updated look at the

state of the art. Pervasive Mob. Comput. 3, 2 (2007), 53-73. – Colombo, A., Cusano, C., and Schettini, R. Face^3 a 2D+3D Robust Face

Recognition System. Proceedings of the 14th International Conference on Image Analysis and Processing, IEEE Computer Society (2007), 393-398.

– Harrison, C. and Hudson, S.E. Scratch input: creating large, inexpensive, unpowered and mobile finger input surfaces. Proceedings of the 21st annual ACM symposium on User interface software and technology, ACM (2008), 205-208.

– Pinhanez, C.S. and Bobick, A.F. Interval scripts: a programming paradigm for interactive environments and agents. Personal Ubiquitous Comput. 7, 1 (2003), 1-21.

– Moeslund, T.B., Hilton, A., and Krüger, V. A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 104, 2 (2006), 90-126.

– Yick, J., Mukherjee, B., and Ghosal, D. Wireless sensor network survey. Comput. Netw. 52, 12 (2008), 2292-2330.

– Yilmaz, A., Javed, O., and Shah, M. Object tracking: A survey. ACM Comput. Surv. 38, 4 (2006), 13.


References (3)

• Software architectures and architectural abstractions– Caflisch, L., Savigni, A., Schettini, R., and Tisato, F. A software architecture for real-time,

embedded monitoring systems. Advanced Video and Signal Based Surveillance, 2005. AVSS 2005. IEEE Conference on, (2005), 540-545.

– De Paoli, F. and Tisato, F. Architectural Abstractions for Real-Time Software. Proceedings of the Second Asia Pacific Software Engineering Conference, IEEE Computer Society (1995), 199.

– Heineman, G.T. e Councill, W.T., cur. Component-based software engineering: putting the pieces together. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2001.

– Garlan, D. and Shaw, M. An Introduction to Software Architecture. Carnegie Mellon University, 1994.

– Jacquet, C., Bourda, Y., and Bellik, Y. An architecture for ambient computing. Intelligent Environments, 2005. The IEE International Workshop on (Ref. No. 2005/11059), (2005), 47-54.

– Markopoulos, P., Mavrommati, I., and Kameas, A. End-User Configuration of Ambient Intelligence Environments: Feasibility from a User Perspective. In Ambient Intelligence. 2004, 243-254.

– Maier, M.W. and Rechtin, E. The art of systems architecting (2nd ed.). CRC Press, Inc., 2000. – Micucci, D., Oldani, M. and Tisato, F. Time-Aware Multi Agent Systems. In Proocedings of

Multiagent Systems and Software Architecture (MASSA). D. Weyns and T. Holvoet (Eds.), 2006, 71-78.

– Micucci, D., Tisato, F., and Adorni, M. Engineering spatial concepts. Knowl. Eng. Rev. 24, 1 (2009), 77-93.

– Senthil, R., Kushwaha, D.S., and Misra, A.K. An improved component model for component based software engineering. SIGSOFT Softw. Eng. Notes 32, 4 (2007), 9.

– Shaw, M. e Garlan, D. Software architecture: perspectives on an emerging discipline. Prentice-Hall, Inc., Upper Saddle River, NJ, USA, 1996.


References (4)• Emergent conceptual and architectural models

– Bandini, S., Bonomi, A., Vizzari, G., et al. A CA-Based Approach to Self-Organized Adaptive Environments: The Case of an Illumination Facility. Proceedings of the 2008 Second IEEE International Conference on Self-Adaptive and Self-Organizing Systems Workshops - Volume 00, IEEE Computer Society (2008), 1-6.

– Silva, F.S.C.D. e Vasconcelos, W.W. MANAGING RESPONSIVE ENVIRONMENTS WITH SOFTWARE AGENTS. Appl. Artif. Intell. 21, 4-5 (2007), 469-488.

– Sparacino, F., Davenport, G., and Pentland, A. Media in performance: interactive spaces for dance, theater, circus, and museum exhibits. IBM Syst. J. 39, 3-4 (2000), 510, 479.

• Conceptual frameworks– Benford, S., Giannachi, G., Koleva, B., and Rodden, T. From interaction to

trajectories: designing coherent journeys through user experiences. Proceedings of the 27th international conference on Human factors in computing systems, ACM (2009), 709-718.

– Benford, S., Schnädelbach, H., Koleva, B., et al. Expected, sensed, and desired: A framework for designing sensing-based interaction. ACM Trans. Comput.-Hum. Interact. 12, 1 (2005), 3-30.

– Bongers, B. and Veer, G.C. Towards a Multimodal Interaction Space: categorisation and applications. Personal Ubiquitous Comput. 11, 8 (2007), 609-619.

– Loke, L. and Robertson, T. Design representations of moving bodies for interactive, motion-sensing spaces. Int. J. Hum.-Comput. Stud. 67, 4 (2009), 394-410.

Date post:	25-Feb-2016
Category:	Documents
Upload:	lilia
View:	28 times
Download:	0 times

Advisor: Prof. Francesco Tisato Tutor: Prof. Carla Simone September 23th 2009

Documents