Date post: | 05-Apr-2016 |
Category: |
Documents |
Upload: | sebastian-alvarado |
View: | 235 times |
Download: | 0 times |
PROTOTYPE GINGATHE DANCER BECOMES MUSIC
ADVANCED INTERACTION STUDIO.
Sebastián Alvarado Grugiel
Costa Rica
June 2014
Master in Advanced Architecture2013-2014
PROTOTYPE GINGATHE DANCER BECOMES MUSICADVANCED INTERACTION STUDIO.
STUDENTSebastián Alvarado Grugiel
Costa Rica.
FACULTY
Carlos Gómez
Xavi González
Master in Advanced Architecture2013-2014
‘‘I would only believe in a God who could dance.’’
Friedrich Nietzsche
INDEX
Pre-studyAdvanced ArchitectureSound, Color and Interaction
1 9
Introduction 7
References
Special thanks
42
44
2 17Interaction
Input/Output - Looking for LoopsAbout HumansSynesthesiaDesigning Behaviors for New Humans
3 28Prototype GINGA
Context - CapoeiraReading the BodySensor SetupTurning Data into Sound
4 36ExhibitionsSONAR 2014IAAC
The dancer becomes music. Ginga is a wearable technology that turns body movement into sound.
Our body is the result of thousands of gen-erations of intense evolution. When we use modern technologies our bodies are reduced to a chair and a finger click. De-sign creates behaviours. A new approach is needed to develop the technologies of the future, starting from our physical re-ality, giving emergent technologies a hu-man shape.
Prototype Ginga is an experiment con-ceived in Capoeira, the Brazilian dance/martial arts.
Introduction
IaaC MAA_01 2013-2014
11 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014
PRE-STUDY1
Advanced Architecture
Now a days different disciplines of knowledge are find-ing points of coincidence as they develop and get more specific. There is no more an expert in architecture, but an architect who is an expert in some area of the field, within asymmetric groups of experts. Disciplines mix as they get complex and creativity and intelligence result in hybrid new fields of knowledge, linking and under-standing in new ways, as sciences look for answers in-side each other.
The first personal interest in interaction came from an interest in music, trying to find a link between personal passions. Architecture in evolving incredibly fast now, and is demanding of new links between knowledge. If we compare the evolution of architecture and the evo-lution of life, we see that architecture have evolved in form, while life evolves in form and intelligence. Will ar-chitecture become intelligent?... Advanced Architecture is intelligent, through interaction and communication.
12 IaaC MAA_01 2013-2014
Prototype GINGA
13 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014
14 IaaC MAA_01 2013-2014
Prototype GINGA
Sound, Color and Interaction
Sound is essential for the interaction of the most com-plex kind of life. Language is one of the most important elements of human interaction and intelligence. It could have evolved from music, as exposed by Rousseau, Herder, Humboldt, and Charles Darwin. Complex organ-isms like humans use also color/light as information to interact. Can intelligent Architecture use color/light and sound to interact and communicate?
Color and sound can be measured, and turned into bits of information. Bits can be transmitted and under a cer-tain code, they can become messages for interaction. This way, color becomes sound and sound becomes color. They are both just channels for information. This can be used to develop Intelligence in Architecture de-signing new lightscapes and soundscapes.
15 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014
16 IaaC MAA_01 2013-2014
Prototype GINGA
Experiment 1
Using seven basic colours and assigning them to each musical note, we can trans-late sound information into colour information. How would that change how we ex-perience music, space, atmospheres?
17 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014
IaaC MAA_01 2013-2014 18 IaaC MAA_01 2013-2014
Prototype GINGA
MORE information at
https://www.youtube.com/watch?v=C-jtWJiO6SI
19 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014 IaaC MAA_01 2013-2014
INTERACTION2
Input/Output - Looking for Loops
Interaction is the action that occurs as two or more ob-jects have an effect upon one another. Object 1 sends an output to Object 2, which receives it as an input. Ob-ject 2 reacts sending an output that Object 1 receives as input. This modifies the behavior of Object 1 and its next output. Interaction sets a loop of inputs and out-puts that modify the behavior between objects.
Interaction Design focuses on the way people interact with each other using external objects. Interaction is as good as the experiences that people can share. Tech-nology is a tool to make this interaction more complex and intriguing.
20 IaaC MAA_01 2013-2014
Prototype GINGA
Kids playing in a square. The metallic pyramid is noth-ing particularly special, but the people’s interaction gives it an important meaning. The interaction between people, its complexity, and the experiences we learn and share, is what makes interaction meaningful.
21 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014
A loop of inputs and outputs.
Input
IaaC MAA_01 2013-2014 22 IaaC MAA_01 2013-2014
About Humans
Interaction is as good as what people share and learn from each other. People should be the target of any in-teraction design. Technology improved how we commu-nicate today and proved to be a great tool for exploring new ways of interaction, making it more complex by linking new interests.
Modern humans have been existing for 200 000 years. This means that there are around 6000 generations of humans between us and our first ancestros. We are the result of such evolution, resulting in the biological tech-nologic marvel that is our body. We are capable of in-credible things: we play music, dance, sing, paint, build, swim, glide, surf, you name it; and our brains backup all our creativity and understanding of the world.
23 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014 IaaC MAA_01 2013-2014
The human body is a powerful result of evolution.
IaaC MAA_01 2013-2014 24 IaaC MAA_01 2013-2014
Synesthesia
This project of interaction was based on how to add complexity to the human natural behavior. The main in-terest was in the human factor of interaction. We define reality using several channels, we call senses, to regis-ter information from what we call the world around us.
“This music crept by me upon the waters”
From “The Tempest” by William Shakespeare.
25 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014 IaaC MAA_01 2013-2014
The data from the environment is picked up by recep-tors that are specialized to respond to a specific physi-cal stimulus. The data is transferred to the brain as sig-nals with the same amplitude but different frequencies. The brain doesn’t determine the type of stimuli since all arrive in the same form, but from the place the stimuli came from. This means that any stimuli received by the photoreceptor will be decoded in the visual center of the brain as light information; if by any chance it came to the sound center, it would be interpreted as sound. This ability/weakness of the brain to mix sensory sig-nals some times lead to SYNESTHESIA.
Understanding interaction through the human experience.
26 IaaC MAA_01 2013-2014
Prototype GINGA
Synesthesia is a neurological phenomenon that occurs when the stimulation of one sensory or cognitive path-way is experienced in a second sensory or cognitive pathway. People with synesthesia may see each letter in a particular color when they are actually black and white and even if they are blind, experiment taste in their mouth when they listen to a particular word, or count by arranging number in space and color. Studies on synesthets reveal that our brain establish sensory associations during our childhood, all the mixed sense experience of synesthesia is defined when we are very young. Synesthets appear to have connections between different areas of the brain that are typically completely separate.
Although synesthesia is present in a minority of people and with very diverse manifestations, it might help to explain the complexity of human intelligence. Everyday we use synesthetic principles to understand complex concepts, like number sequences. Synesthesia helps to explain another critical skill of the human condition: creativity.
27 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014
Genetic studies on synesthesia have found that synes-thetic experiences are 8 times more common among artists, poets and novelists. This increased creativity could be caused by the perception of a sense disguised as another and the ability to perceive hidden informa-tion. The basis of creativity is finding unexpected links.
The use of metaphors in language evidence our nat-ural ability to link our senses to create and describe concepts. We might say things as loud green, sharp cheese, bright sound, where we mix sound, color, shape and physical properties together. Synesthetic processes could even explain the origins of human language, ac-cording to several researchers. In a way, we are all syn-
Some kinds of synesthesia allow people to see letters in color, even thought they are printed in black and white.
IaaC MAA_01 2013-2014 28 IaaC MAA_01 2013-2014
Behaviors for new humansDesign makes behavior. Designers have to be conscious that every time we design a product, we are also de-sign how people are going to behave around it. This is a huge responsibility, challenge and opportunity.
Modern technologies, even though powerful, tend to constrain the body into not natural behaviors. Mod-ern humans suffer from diseases related to the lack of movement and forced sedentary live styles. Future technologies will take a human form so that instead of limiting its potential, it will amplify it.
How should technology attach to the body?The picture shows the work of Heather Hansen and how she creates several drawings based on the body move-ment and proportions. New technologies can improve the human interaction by finding new ways of under-standing the relationship between the human reality and the technological interfaces.
29 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014 IaaC MAA_01 2013-2014
Physical Movement Translated into Symmetrical Charcoal. Emptied
Gestures by Heather Hansen.
Neil Harbisson uses this color sensor that detects color frequency
and produces a sound in his head. This way he can listen to color.
The paintings shown are his interpretation for Für Elise by Beethoven
and Triumphal March from Aida by Giuseppe Verdi, from left to right
IaaC MAA_01 2013-2014 30 IaaC MAA_01 2013-2014
PROTOTYPE GINGA3
Choosing a context - Capoeira
Based on the previous research, this experiment was run to clarify the viability for technology to take a hu-man form, reading its natural behavior to develop new interfaces between humans and machines/computers.A context was chosen in order to define limits and direct the project.
There was a special interest since the beginning of the studio in music for interaction. This was added to the interest in the human experience and its physical real-ity. The perfect context would be that in which the body could express freely and creatively in relation to mu-sic, using all the body potential. Several contexts were considered specially contemporary dance, break dance and martial arts. The perfect combination was found in Capoeira, a Brazilian dance/martial art.
GINGA is the name of the main movement in the Ca-poeira dance/martial art.
31 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014 IaaC MAA_01 2013-2014
Capoeira is a Brazilian martial art that combines ele-ments of dance, acrobatics and music. It originated in Brazil in the 16th century. When Portugal reclaimed Bra-zil, they brought slaves from Africa to work the lands in inhumane conditions. Slaves developed several fighting techniques hoping to fight their way to freedom. They trained with music trying to disguise their fighting tech-niques as a dance or a game, in order to avoid punish-ments.
Modern Capoeira is practiced as a performance dance, using the traditional instruments and songs. Even though it still uses several fighting movements, it is not a combat martial art. It is a dance that improves your body abilities and reinforce social relations.
IaaC MAA_01 2013-2014 32 IaaC MAA_01 2013-2014
Reading the bodyThe project developed a set of sensors capable of read-ing the body movement, in this case, the Capoeira dance. The traditional music of Capoeira will play and the ‘‘players’’ will start dancing. Each player will have a set of sensors. As they move following the music, the sensor will read their movements and translate them into data that will alter the base music, adding sounds or modifying it.
The resulting music and sound will alter again the be-havior of the dancer, which will react to it with new movements that will again alter the base music. In this way, there is a loop of action-sound-reaction in which the dancers not only listens to the music but make it. The set of sensors amplify the human output adding complexity to the experience. This results in an interac-tion that adds complexity to the experience.
The device amplifies the human power. The dancer listens to the music and dances, the sensors turn the dance into data, the data is transmitted to a computer that uses it to modify the music. The new music is rein-terpreted by the dancer, who adapts the movement to the new sounds. The loop is closed and flowing.
33 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014 IaaC MAA_01 2013-2014
Human
Flex sensorAccelerometer
Wireless micro controller.Arduino Fio + Xbee
Computer + MAX/MSPMusic programming
Sound/Music
Flowing loop of interaction.
IaaC MAA_01 2013-2014 34 IaaC MAA_01 2013-2014
Sensor SetupThe device works by attaching a flex sensor in the el-bow to detect the extension of the arm and the act of opening the body. An accelerometer in the wrist reads the start and end of any wrist displacement in space. It reads in x,y,z axis.
The sensors send those inputs to an Arduino Fio board and turns it into numbers. The data is sent wirelessly via Xbee radios to a computer. The numbers are put into a programming language for music called MAX/MSP, which creates several frequencies with each input.
The real achievement of the prototype was to turn the body movement into data that can be used, remapped and modified as parameters in different computer ap-plications. This means that the device can be used in several fields such as music composition, music perfor-mance, dance performance, medical uses, virtual real-ity interfaces, robotics and sports.
All the technologies used to develop this prototype are open source.
35 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014 IaaC MAA_01 2013-2014
First sketch.
Accelerometer Flex sensor Arduino FIO + LIPO battery
Prototype GINGA
IaaC MAA_01 2013-2014 36 IaaC MAA_01 2013-2014
Turning movement/data into sound
Numbers are information that can be used to do any-thing. Movement is turned into data using a sensor set-up. The numbers are used to produce sound and mod-ify music, through a programming language for music (max/msp).
Arduino Fio+XbeeRadio signalto computer
Flex Sensor
AccelerometerX, Y, Z axis
37 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014 IaaC MAA_01 2013-2014
Data received in four channels. Shown as a wave.
GINGA - Basic movement in Capoeira
X
Y
Z
Flexion
IaaC MAA_01 2013-2014 38 IaaC MAA_01 2013-2014
Exhibitions
As part of the study programme for the Interaction Studio we had the chance to present our project in SONAR+D, a international conference for creativity and technology, designed to promote talent and business opportunities in the digital culture environment. It was also shown in the final presentation of the master in IAAC.
People had the chance to try out GINGA and the feed-back got from them was of great value. The sound of GINGA is very experimental, so as people move they change basic frequencies. People immediately recog-nized the sound changing as they moved, which proved that when technology turns into human shape we can instinctively adapt it to our natural behavior.
Several people described the sound as robotic, and sug-gested different kinds of sound like nature sounds of water, rain, or more recognizable sounds. Even though, the reaction was generally very positive and many of the visitors really engaged into the experience of listen-ing to their own body as it moved.
39 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014 IaaC MAA_01 2013-2014
From the feedback and the experiences learned, this project could grow in several directions such as medi-cal applications for rehabilitation, dance performances, music composition and robotics. It could also comple-ment alteration to traditional musical instrument or even become one by itself.
MORE information at:
instagram.com/gingasonar2014
facebook.com/gingasonar2014
40 IaaC MAA_01 2013-2014
Prototype GINGA
SONAR 2014
41 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014
42 IaaC MAA_01 2013-2014
Prototype GINGA
IAAC Final Presentation
43 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014
44 IaaC MAA_01 2013-2014
REFERENCES
Connecting – Trends in
UI, Interaction, & Experience
Design
by Bassett & Partners
youtube.com
2014
What color is Tuesday? -Exploring Synesthesia
ted.com
2013
I Listen to color Ted Talk by Neil Harbisson
ted.com
2012
How architecture helped music evolveTed Talk by David Byrne
ted.com
2010
The 4 ways sound affects us
Ted Talk by Julian Treasure
ted.com
2009
When Senses Collide
-Synesthesia Origins
BBC
2008
Software (as) artTed Talk by Golan Levin
ted.com
2004
45 2013-2014 MAA_01 IaaCIaaC MAA_01 2013-2014
IMAGES REFERENCE
Ear by Jerry Wang
Page 25
The Noun Project
Eye by Jean-Philippe Caba-
roc
Page 25
The Noun Project
Nose by Rachel Healey
Page 25
The Noun Project
Hand by Maico Amorim
Page 25
The Noun Project
Mouth by Sergi Delgado
Page 25
The Noun Project
SynesthesiaPage 28
wikipedia.org
Emptied Gestures By Heather Hansen
Page 30
heatherhansen.net
I Listen to color By Neil Harbisson
Page 30
ted.com
CapoeiraPage 32
By Jacek Dyląg
46 IaaC MAA_01 2013-2014
Special thanksCarlos Gómez
Xavi González
Guillem Camprodon
Apostolos Mouzakopoulos
John Giraldo
Giombattista Areddia
Rodolfo Parolin
Pablo Marcet
Raphael Libonati
Michele Braidy
Mary Katherine Heinrich
Esther Marquez
Carmen Aguilar
Daniel Giraldo
Marina Castan
Rhys Duindam
Tomico Plasencia
IaaC MAA_01 2013-2014
www.iaac.net
Accredited by:
www.upc.edu
fablabbcn.org www.valldaura.net
valldauraSelfS ufficientLab
The dancer becomes music. Ginga is a wearable technology that turns body movement into sound.
Our body is the result of thou-sands of generations of intense evolution. When we use mod-ern technologies our bodies are reduced to a chair and a finger click. Design creates behav-iours. A new approach is need-ed to develop the technologies of the future, starting from our physical reality, giving emergent technologies a human shape.
Prototype Ginga is an experi-ment conceived in Capoeira, the Brazilian dance/martial arts.