CCRMA - 2011

Álvaro Barbosa Research Center for Science

and Technology of the Arts CITAR (UCP–Porto, Portugal)

Álvaro Barbosa http://www.abarbosa.org/

2000/2006 – MTG, UPF-Barcelona, Spain

http://mtg.upf.edu/

Since 2006 – CITAR, UCP-Porto, Portugal

http://artes.ucp.pt/citar/

Since June 2010 – CCRMA, Stanford-CA, USA

http://ccrma.stanford.edu/

Networked Music

Networked Music “(…) All Music Is networked. You can think about an Orchestra as client-server network, where a conductor is “serving” visual information to the “client” musicians, or a peer-to-peer networking model in an improvising Jazz Combo, where there is no one directing, and the musicians are all interacting, so, any performance context we can think of in some way there is a network connecting the performers (…). Networked Music with capital N and capital M (the kind we are talking about) is about performance situations where traditional acoustic and visual connections between participants are augmented, mediated or replaced by electronically-controlled connections.” (From Jason Freeman’s lecture opening at the 1st Networked Music Workshop during ICMC 2005)

Displaced Soundscapes

Displaced Soundscapes is a metaphorical description of the way lively generated sounds can be perceived over the Internet.!!(Book) Barbosa, A. 2008. “Displaced Soundscapes”!VDM Verlag (ISBN: 978-3-8364-7154-1)!!-  Extensive Survey of NM Applications !-  Research on Latency and Internet Acoustics!-  Models for Networked Music Systems!-  Implementation “Public Sound Objects” !

Latency and Networked Music

In a vocal conversation it is possible to maintain it even with one-way delays of up to 500 ms.

Holub, J., Kastner, M., Tomiska, O. 2007

In order to maintain a synchronized and smooth musical interaction reduces drastically to the order of tens of milliseconds.

Schuett, N. 2002 Chafe C., Gurevich M, 2004 Lago, N and Kon, F. 2004 Barbosa, A., Carôt, A. 2005 Chew, E., Sawchuk, A., Tanoue, C., and Zimmermann, R. 2005 Bartlette, C., Headlam, D., Bocko, M., Velikic, G. 2006 Farner, S., Solvang, A., Sæbo, A., Svensson, U. P. 2009 Chafe, C., Cáceres, J-P., Gurevich, M., 2010

Background on Network Collaborative Practices


For the Human ear to perceive the order of two simultaneous sounds, they should not be displaced in time over 20ms (Hirsh, 1959)

A difficulty in discerning the order of sounds events

ê hard to maintain a synchronous musical interaction.

The ability to perform music synchronously is strongly dependent on:

•  music style (rhythmic, melodic,…)

•  musicians background and experience

•  visual or physical feedback

•  Sound pitch, loudness and particularly timbre (Bregman 1990)



What Happens?



Basic Principles (1) Ensemble Performance Threshold (EPT)

(2) Echo Feed-Back (Self Delay)

(3) Inverse Proportion to Tempo

(4) Slow Attack Times

(5) Reverb and Complementary Modalities


(1) Ensemble Performance Threshold (EPT)

(CCRMA 2004, USC 2005)


DELAY (ms) 0 5 10 15 20 25 30 35 40 45 50 75 100 150

Difficult toCompensate

Comfort Zone for Untrained Users

EPT Interval forUntrained Users

Comfort Zone for Expert Musicians EPT Interval for Expert Musicians

ExtremelyDifficult to

Compensate

AlmostImpossible toCompensateUnnaturally

Low Latency

(2) Echo Feed-Back (USC 2005)


(2) Echo Feed-Back (UCP-Porto 2005)


(3) Inverse Proportion to Tempo (UCP-Porto 2005)


(3) Inverse Proportion to Tempo (CCRMA/Banff 2006)


St. Lawrence String Quartet

(over 50ms Delay)

(4) Slow Attack Times


Bregman (1990) Auditory Scene Analysis

(5) Reverb and Complementary Modalities


Studies on the effect of Reverberation (Farner 2009) The Influence of Delay and Various Acoustic Environments Studies on the effect of Visual Feed-Back Pilot experiments at CCRMA since 2006 Studies on the effect of Haptic Feedback ???

Latency at the global level

Jitter and delay asymmetry introduce further disruption in Network Communication. Furthermore, Latency is not a technological condition that can be overcome in the near future. Consider mobile or satellite technology or a communication setup that spans worldwide.

Distance = 14.141 Km

Data Transfer = Light Speed Latency= 94,3 ms

Computer-Supported Cooperative Work for music Applications | Álvaro Barbosa, 2006

Applications for Collaborative Music

Based on Tom Rodden’s CSCW Classification Space (1992)

•  Going beyond the paradigm in which we connect two or more remote spaces for a performative session by creating a shared space in a computer network, inside which users can achieve a certain degree of immersion and flexibility in their behaviour

•  Users can freely join or leave this space, choosing to participate or just to listen to an on-going acoustic piece

•  It is not oriented towards a time limited event scenario

Shared Sonic Environments

The Shared Nature of the Internet

A musical instrument that can be played using only a web browser, enabling multiple performers to play the instrument in a collaborative way.

Jorge Herrera

CCRMA, 2009

The HORGIE: Collaborative Online Synthesizer

Shared Sonic Environments

Shared Sonic Interface on the Web

From the Web to the Real World

Public Sound Objects

Public Sound Objects: 2004

Original PSOs – 2004/2006, MTG (BCN)


Original PSOs – 2004/2006, MTG (BCN)


PSOsSERVER

(...)

PSOs CLIENTS

STREAMINGAUDIO SERVER

HTTP-SERVER

INTERACTIONSERVER

LOCAL VISUALREPRESENTATION

ENGINE

Public Installation Site

INTERNET

Global Audio Performance(Continuous Streaming Connection)

Performance Commands(Discrete Connection triggered by client events)

SOUNDOBJECTS

DATA-BASE

WEB BROWSER

Streaming Audio Client

Controller Interface

WEB BROWSER



WEB BROWSER



Apache + Custom Developed Servlet

ICECAST Streaming Server

Pure-Data

Pure-Data

Pure-Data + GEM



Small Fish (Fujihata e Furukawa 1999)







PSOs Interface

It is based on Pitch, Dynamics, Tempo and Timbre.

It tolerates Latency because: • The resulting soundscape is musical. Yet, it does not require a strict synchronization between sound events •  Behavior driven interactive Interface (loose coupling interaction) •  Adaptive speed and dynamics (to network conditions) •  Individual Delayed Feed-Back (same piece to everyone) •  Coherent with visual cues (spatialization of sound helps)

Public Sound Objects: Panorama Cues

t 2 Timet 1 t 3

R

L

t 1 t 2 t 3

t 2

Timet 1

t 3

R

L

t 1 t 2 t 3

t 2

Timet 1

t 3

R

L

t 1 t 2 t 3


Local Network of PSOs – CITAR (Porto) commissioned by “Casa da Musica” http://en.wikipedia.org/wiki/Casa_da_musica






Client Server









Permanent Installation Casa da Música (Renaissance Hall)

Thank You!!!!

[email protected]

or [email protected]

http://www.abarbosa.org/pso/

Date post:	24-May-2015
Category:	Technology
Upload:	alvaro-barbosa
View:	233 times
Download:	1 times