Date post: | 20-Dec-2015 |
Category: |
Documents |
View: | 219 times |
Download: | 1 times |
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 11
Sound Composition Sound Composition for Sensing/Speaking for Sensing/Speaking
SpaceSpace
UCSB Center for UCSB Center for Research in Electronic Research in Electronic
Art TechnologyArt Technology
Stephen Travis Stephen Travis [email protected]@create.ucsb.eduedu
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 22
OutlineOutline Sensing/Speaking Space
Introduction The Design Principles (Policies) Sound Layers The Rumble The Singer The Speakers Responsiveness and Interactivity
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 33
The PoliciesThe Policies Structure and Noise
Speech and Text
Use of Old Materials
Responsiveness and Interactivity
“Contemplative Place”
(Ritual Place) (Zen Garden)
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 44
The ToolsThe Tools Programs written in Smalltalk, Python, C,
C++, and SuperCollider over the span of 18
years (Mostly PD or OpenSrc tools)
Some GUIs, some text-based programming
languages; delivery system is camera-only
Separate manipulation of sound material,
gesture-level score, and interactive control
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 55
The Tools (2)The Tools (2)
All running on Apple Macintosh G4
(Desktops and PowerBooks)
Camera-to-Director
(TrackThemColors), Director Plug-
in-to-OSC (by GarryK), SuperCollider
programs reading OSC
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 66
Siren Development Siren Development EnvironmentEnvironment
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 77
Siren Sound/Spectrum Siren Sound/Spectrum EditorsEditors
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 88
SuperCollider Debugging SuperCollider Debugging GUIGUI
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 99
Delivery EnvironmentDelivery Environment
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 1010
The Sound LayersThe Sound Layers Rumble* -- low constant bell drone
Ripples -- mode 1, resonated water
sounds
Tingles -- mode 2, resonated hand bells
Singer* -- constant chanting bells
Speakers* -- phoneme streams triggered
by audience motion
Transition Pipes -- short transition effect
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 1111
The RumbleThe Rumble Tibetan Dorji Bell
Time-stretched and transposed from 18 Hz - 18 kHz using the “Loris” analysis/synthesis package and several Python programs
Mixed for Rumble and Singer sources
Examples• Original bell
• Transposed to extremes
• Various layer mixes
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 1212
The SingerThe Singer “Cross-synthesis” of bell clusters and
processed T’ang-dynasty poetry/chant
Use PhaseVocoder, Loris, and
SuperCollider
Examples
• Vocoded voice
• Bell texture
• Cross-synthesis
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 1313
The SpeakersThe Speakers The Siren speech segmenter and
phoneme database• Segmentation
• Analysis
• Similarity metrics
The Database// Format: name folder start stop dur maxAmpl rmsAmpl spectralCentroid
// spectralWidth [spectralBands]
#[ '1.snd', 'Content:Sound:YYYJD:ec:r3:src', 6144, 17280, 0.252517, 17901, 385.762652, 714.563192, 0.404395, [0.059394, 0.063983, 0.078587, 0.125629, 0.415792, 0.823278, 0.901691, 0.389382, 0.225568, 0.185267, 0.166998, 0.169787, 0.192624, 0.161891, 0.168954, 0.136718] ]
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 1414
Siren Speech SegmenterSiren Speech Segmenter
I CE M E L T S
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 1515
Phoneme Stream Phoneme Stream
ExamplesExamples Selecting Phoneme Streams
• Selection criteria (distance metric)• Dynamic control: mode, threshold, density,
stretch factor, position, volume, etc. Triggering and Tracking
• Not completely deterministic Examples
• Source Texts
• Stream 1
• Stream2
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 1616
Spatial Sound: The Spatial Sound: The
CreatophoneCreatophone
Sensing/Speaking SpaceSensing/Speaking Space 04/18/2304/18/23 1717
Sensing and InteractivitySensing and Interactivity Through-composed Version Completed
in Dec. 2001
Lab set-up with Monitoring GUI in Jan.
for testing of interactivity and
intermedia integration
Different Sociology of the Layers
Major Transitions and Scenes