Post on 11-Apr-2015
description
transcript
Changes Over Time:
Practice A portfolio of collaborative electroacoustic works demonstrating the heterogenous reapplication of jazz
improvisational and time-feel techniques and models.
2
Contents
Introduction
Portfolio Contents
Compositional Notes
1. Hidden Music
2. Extended Solo Instruments
3. Reworked Ensemble
4. Research Output (Performances, Recordings and Events)
5. Ongoing and Upcoming Projects
References
3
Introduction
This portfolio comprises seven CDs and five DVDs from which several
representational extracts are compiled (CD 2 Core Works). Although there are many
compositional mechanisms and themes in common among all the works, the selected
pieces have been separated into three main categories. These are:
1) Hidden Music. Acousmatic electronic works that employ the mapping of
external physical phenomena to generative quasi-improvisational - M-Space
and expressive contour – systems: Primal Sound (2004, 2007), Head Music
(2004, 2007), Bloodlines (2005), Microcosmos (2006), Dendro (2009) and
Membrane (2009).
2) Reworked Ensemble. Works for a variety of mixed ensemble, employing
electronics alongside traditional instruments, M-Space modelling and
progressive interactive improvisational strategies. Selfish Theme (2006) and
extracts from the Rat Park Live (2007), 11th Light (2008), Rumori (2008) and
Jazz Reworked (2010) projects.
3) Extended Solo Instruments. Pieces for solo instruments and live electronics
involving a range of improvisational elements and technological extensions
to the instrument: String Theory (2006), Event Horizon (2007), Omnia 5:58
(2008), Assini (2009), Dark:Light (2009) and Blue Tension (2010).
This categorization allows for a more logical narrative through the works while
maintaining a chronology within each category. References are also made to other pieces
in the portfolio and ongoing projects. Score notation is not often used in these works,
and on the occasions when it is, its chief function is usually as an aide memoire, concealing
a range of concepts born from collaborative dialogue in rehearsal and implicit stylistic
mechanisms - not unlike those found in a jazz chart. Many pieces in this portfolio were
4
conceived - and are performed - with little, or no, use of notation. Graphic
representations, electronic flow diagrams and software screenshots of some works are
used here as illustrations of compositional approaches. However these are often a form
of post-hoc illustration of compositions that originated in well-defined but unwritten
compositional ideas, technological experimentation or collaborative improvisation; and
are rarely made before the work is performed or recorded. Even the idea of a completed
piece is somewhat nebular; pieces are often reworked and include major improvisational
elements, electronic systems are transplanted and spliced into other works, and no
identical, or even ‘ideal’ performance of any work exists. The work itself, borrowing
terminology from M-Space, may be seen as a higher-level field grouping a myriad of
varying, but related, possible performances. In this way, the composer in these works
migrates between roles, from the traditional conceiver of the entire conception of the
piece, to a facilitator deferring various portions of creative responsibility to performers,
random input and external physical phenomena. Within this portfolio, ‘jazz composer’,
Varèse’s notion of an ‘organiser of sound’ and ‘generative sound artist’ are all roles that
contribute in varying measures to that of the traditional composer.
The commentaries of each work vary in the level of detail and particular technical
focus so as to introduce compositional techniques in a logical sequence, maintain a
narrative and avoid unnecessary duplication of similar material. The works have been
selected in order to present a range of compositional approaches and musical contexts,
which may be explored more deeply across the whole portfolio at the examiner’s
discretion.
Following the compositional notes on is a list of performances, research events and
exhibitions that feature works from the submitted portfolio, and a survey of ongoing and
upcoming creative research projects.
5
Portfolio Contents
CD 1 Audio Examples (Supporting audio material)
CD 2 Core Works (Selected works from the portfolio 2004-10)
CD 3 Hidden Music (Lucid 2008)
CD 4 21st Century Bow: New Works for HyperBow (RAM 2007)
CD 5 SPEM (CoffeeLoop 2007)
CD 6 Tensions (Electronic works for harp, cello and guitar 2004-10)
CD 7 Jazz Reworked (DeWolfe 2010)
CD 8 Terminal (Mute 2010)
CD 9 Nexus (Selected ensemble works 2004-9)
DVD 1 Martino:Unstrung (Feature-length documentary, Sixteen Films 2007)
DVD 2 Microcosmos (Video installation, Wellcome Trust 2006)
DVD 3 Wake Up And Smell The Coffee (Planetarium Movie, SEEDA 2009)
DVD 4 Rat Park: Live (Live ensemble performance, Lucid 2006)
DVD 5 Organisations of Sound (Selected performances from UK, USA, Sardinia,
China, Japan)
6
Compositional Notes
1 Hidden Music
1.1 Primal Sound
1.2 Head Music
1.3 Bloodlines
1.4 Microcosmos
1.5 Membrane
2 Reworked Ensemble
2.1 The Selfish Theme
2.2 Rat Park Live
2.3 11th Light
2.4 Rumori
2.5 Double Back
3 Extended Solo Instruments
3.1 String Theory and Event Horizon
3.2 Omnia 5:58
3.3 Assini
3.4 Dark:Light
3.5 Blue Tension
7
1. Hidden Music Electronic quasi-improvisational works guided by external physical phenomena
1.1 Primal Sound (2004-7)
Artist & sculptor Angela Palmer has received much acclaim and publicity for her
science/art crossover works, which include MRI images of an Egyptian mummy, a
public exhibition of felled Amazonian trees in Trafalgar Square and a photography
exhibit of the most, and least, polluted places on earth (Palmer 2010). She approached
the author in January 2004 with a particular project in mind; having been introduced to a
piece of writing from Rainer Rilke’s Ur-Geräusch (1919, p 1085-1093) which set out an
irresistible challenge.
The coronal suture of the skull (which should now be chiefly
investigated) has let us assume a certain similarity to the
closely wound line that the needle of a phonograph cuts into
the receptive, revolving cylinder of the machine. Suppose, for
instance, one played a trick on this needle and caused it to
retrace a path not made by the graphic translation of a sound,
but self-sufficing and existing in nature – good, let us say it
boldly, if it were (e.g.) even the coronal suture – what would
happen? A sound must come into being, a sequence of
sounds, music…Feelings of what sort? Incredulity, awe, fear,
reverence yes, which of all these feelings prevents me from
proposing a name for the primal sound that would then come
to birth?
Ur-Geräusch (Rilke 1919, p 1087)
8
Palmer hoped to commission a piece of music inspired by this text, to be used in a
gallery exhibition alongside the skull of an unknown Victorian woman (Figure 1.1.1).
Figure 1.1.1 Image of the coronal suture of an unknown Victorian woman, source material for Primal Sound’s expressive contours (©2004 Angela Palmer).
The translation of physical shapes to musical parameters, or millimetrization, has been
explored in composition, perhaps most famously by Villa Lobos in New York Skyline
Melody (Frey 2010), and the idea of melodic contour in general is well-researched (Adams
1976). Rilke proposes something unusual in the field by suggesting that a physical shape
could represent amplitude - rather than Villa Lobos’s approach of melody - against time.
With the resources available, a simple ‘phonographic’ rendering of this contour alone did
not provide enough complexity for a piece of any significant length, so the decision to
map its shape to a variety of parameters was made. This may be thought as a form of
reverse engineering of the expressive contours concept (see Changes Over Time: Theory 1.6
p 56-68). As opposed to many contours emerging as spontaneous gestures during a jazz
improvisation, in this scenario one particular contour is employed to manipulate a host
9
of parameters. This concept may be termed isokinetos (‘equal gesture’) (see Changes Over
Time: Theory 1.4 p 20) and allows in this case the creation of a quasi-improvisational work,
whereby compositional decisions are deferred to a physical pattern. Although the types
of relationships between gesture and particular parameters are determined by the
composer, the bulk of the resulting musical outcome is not known. There is a sense of
discovery in this compositional process quite unlike the experience of tinkering at a
skeletal melodic concept, for example.
Rilke’s mapping was achieved electronically using Max/MSP to inscribe a virtual
record groove with a scan of the suture (Figure 1.1.2 shows a mapping together with an
electron microscopic image of a record groove for comparison). The resulting sound,
though short, is rather effective (CD1.47).
Figure 1.1.2 ‘Phonographic’ translation of the coronal suture in Figure 1.1.1 shown above an electron
microscopic image of a record groove (lower image ©2005 Chris Supranowitz, University of Rochester). The resulting audio is played 3 times on CD1.47.
10
To create more material, this same contour is also rendered as a microtonal melody.
CD1.48 plays the musical translation, of the sample indicated in Figure 1.1.3, as a sine
wave following the contour within the frequency range indicated.
Figure 1.1.3 The coronal suture in Figure 1.1.1 rendered as a microtonal melodic expressive contour
(CD1.48).
In order to form a harmonic backdrop, a just intonated scale (Pythagorean A
Lydian) is mapped against a tracing of the suture (Figure 1.1.4). This slow translation
introduces single sine-wave tones that split as the loops of the contour curve away
creating a gradually converging and diverging harmonic texture. The musical translation
that emerges from the sample indicated in Figure 1.1.4, with its characteristic beating
frequencies, becomes a recurrent motif in the piece (CD1.49).
Figure 1.1.4 Sine waves released by the coronal suture as it hits predefined trigger points, forming a harmonic texture (CD1.49).
11
Musical events in Primal Sound are created - and also organized structurally -
according to systems guided by the coronal suture itself. For example, new events are
triggered by each looping point of the contour, as illustrated in Figure 1.1.5 (many of the
events have been omitted on the figure for visual clarity). The pan position of the
triggered event is determined by the vertical position of the loop point, moving through
the stereo field until it is faded at the crossing of the central line. Since, on occasion,
adjacent loops occur on the same side of the central line, multiple musical events may
coexist.
Figure 1.1.5 Compositional structure of Primal Sound. Musical events are triggered at loop points at the pan positions determined by their vertical position (many trigger points are omitted for visual clarity). The suture is employed forwards, and then backwards, following a complete circumnavigation of the skull
to form a continuous piece.
The initial commission of Primal Sound was for an art installation piece (running in
one exhibition for five months continuously at the Royal College of Surgeons) and so its
‘loopability’ needed to be considered. The compositional structure was determined by
following the coronal suture over the top of the skull, and back around the inside to end
at the starting point for the next repetition of the piece. This is represented in Figure
1.1.5 by the mirrored contour, reflected at the midpoint of the piece.
12
All visual-audio translations were created using MAX/MSP and Jitter, and recorded
into Logic 7 for compositional structuring, panning & mixing. Some modulation effects,
using fragments of the suture as automation data, was also incorporated. An audio
extract of the piece appears on Core Works (CD2.1) and the complete work on Hidden
Music (CD3.1).
Primal Sound is essentially an isokinetic expressive contour study, and a
demonstration of the extensive use of tightly limited musical material (See Changes Over
Time: Theory 1.4 p 17-24). However, despite its specificity, it has been employed
successfully in contexts divorced from its aesthetic origin - most notably in
Martino:Unstrung (DVD 1) where the piece was requested by both director Ian Knox and
Pat Martino as a representation of Martino’s geometrical musical vision and prolific
creativity. A complete list of its use in art installations and soundtracks, along with the
other works from the portfolio, appears in Section 5 (Research Output).
13
1.2 Head Music (2004, 2007)
Primal Sound represented the first in a series of compositions employing biological
phenomena as source material. Head Music (2004), another commission by Palmer, was
written to accompany an exhibition of works created from MRI scans. Palmer’s
ingenuous art works were produced by inscribing MRI images of the human head on to a
series of glass tiles. MRI imagery is performed in multiple cross-sectional ‘slices’ which
are consolidated to form a three-dimensional model. Similarly, the stacking of multiple
inscribed glass tiles would recreate the three-dimensional model of magnetic resonances.
When underlit in a darkened gallery, a ghostly sculpture would emerge, making
permanent the transitory resonances engendered by a momentary brain state (Figure
1.2.1).
Figure 1.2.1 Self-Portrait. MRI images inscribed on a series of glass tiles and underlit (©2004 Angela Palmer).
MRI imagery uses a strong magnetic field to align, and rotate, the hydrogen atoms in
an object. The resulting magnetic fields of hydrogen nuclei will differ for the various
constituent soft tissues. In effect, tissue consistency is translated into patterns of light
and shade, and are captured by Palmer as art works. A parallel translation process was
performed to create the accompanying music. Slivers of the original MRI images were
employed as expressive contours and a translation system (using Max/MSP/Jitter
programming) mapped these to various musical parameters. Whereas Primal Sound
14
employed just one flat contour, this work has three dimensions of material with which to
work providing countless contours for musical mapping (Figure 1.2.2). Expressive
contours were derived electronically by translating a range of lightness values (0-127) for
each sliver into settings for 1) LFO (low frequency oscillator) frequency, 2) Cut off
frequency for low-pass filter parameters, 3) a pitch and rhythmically quantized ‘lattice’
melody, 4) pan position and 5) gliding microtonal monophonic melody. The derivation
of the first three of these expressive contours is illustrated in Figure 1.2.2. The melody in
contour 3 is formed using a ‘Villa-Lobos lattice’ with only a few allowable rhythms and
equal tempered scale notes. Melodic fragments like this appear at 3:09 in the piece
(CD2.2 and CD3.2), and this process also forms the long skeletal notes at the start of the
piece to which LFO, filtering and pan position contours are applied. The microtonal
glides may be heard from 0:34 on CD2.2.
Figure 1.2.2 Illustration of expressive contour derivations from three-dimensional slices of MRI
images in Head Music (CD2.2 and CD3.2). The top image is a scan of Pat Martino’s head, from which a reworked 2007 version of the piece was derived (Image ©2007 Sixteen Films).
15
The LFO frequency contour provides a special case in its treatment of time.
Whereas most contours simple follow the absolute level of a musical variable, LFO
frequency determines the rate at which a parameter is altered – in this case the amplitude
of a synthesized note. This process may be heard from the start of CD2.2 (and CD3.2)
together with image-derived filtering and panning contours. At 1:40-2:00 the effect of a
dark area of the image is heard with a very slow rate of amplitude modulation. Multiple
contours and mappings for various three-dimensional slivers were combined and
overlaid, and an intuitive mixing and focusing was undertaken in real-time, guided by the
listening experience.
Head Music has been performed within the context of Palmer’s exhibitions and, as a
purely audio work, in scientific and musical conferences and events. In 2007, it was
reworked using the startling MRI images of Martino’s devastating head trauma, and
appears on the soundtrack of Martino:Unstrung (DVD 1).
16
1.3 Bloodlines (2004-5)
While undergoing treatment for leukaemia in an isolated ward at Charing Cross
Hospital in 2004-2005, the author underwent daily tests for a range of blood cells types.
These were monitored closely for signs of infection, relapse and immunity level. This
close inspection revealed patterns of population growth, and the idea of mapping these
to a musical composition formed quickly. Primal Sound and Head Music used relatively
few contours in order to create multiple musical layers, but the 14 blood cell types in
Bloodlines allowed for a parallel approach, with each contour controlling just one 14
consynchronous musical elements (Figure 1.3.1).
Figure 1.3.1 A screenshot of blood cell tests, part of the source material for Bloodlines, with expressive contours derived from their changing values. CD2.3 and CD1.48 play audio extracts from two versions of
the work with, and without, guitar respectively.
These contours are derived in a variety of ways from fairly simple microtonal
interpretations (white blood cells) and MIDI pitch translations of values (platelets) to
17
complex mappings of digits to audio samples. For example, haematocrit (HCT) testing
of red blood cells was provided in the form of four digit numbers, each digit calling up
one of ten samples (0-9) for each semiquaver subdivision of a beat.
Each day of treatment is translated into one second of music, and the undulations in
health can be heard musically as the piece progresses. In particular, the prominent
microtonal swell can be heard to descend as the white blood cell count starts extremely
high due to the leukaemic cells, and is massively reduced by chemotherapy until the body
reaches a vulnerable neutropaenia (0:30 – 0:40 on CD1.50). The ‘autobiological’ nature
of the work engenders an emotive response and memory of the journey through
treatment. This human response was later added into the piece itself with a completely
improvised guitar part, responding to the memories of each point in time as it progresses
through the work. In this way, the contours of each blood cell type and the improvised
responsive ‘human’ expressive contours coexist. Extracts of the 2005 version with guitar
appears on CD2.3 and CD3.3, and from the purely electronic version on CD1.50.
Bloodlines has been disseminated at conferences, used in exhibitions, and the guitar
part improvised live in concert (see section 5 for details).
18
1.4 Microcosmos (2007)
A 2007 Wellcome Trust grant provided the opportunity to explore yet more
complex ‘hidden’ musical concepts in a large scale work, and with scientific
collaboration. Microcosmos (DVD 2) is an audio/video installation that presents high-
quality images of bacterial colonies (shot by photographer Steve Downer (Blue Planet)) set
to an original 4.1 surround sound track. Scientific supervision of the microbacterial
colonies and collaborative discourse was provided by Dr. Simon Park (University of
Surrey). The project’s aims included the construction of an electronic compositional
system that could create a satisfying sound design from overt visual aspects of the video
material (colour and form) as well as hidden source material (DNA sequences, protein
production and sound grains).
Rather than impose an anthropocentric ‘emotional’ film score to the video footage,
there was an incentive to design a system that would automatically respond to video and
biological data in a way that was both aesthetically satisfying, complex and not
distractingly predictable. The sound design aimed to mirror aspects of the growth of
microbacterial colonies: emergent large scale structures from the interactive behaviour of
simple components.
The electronic system designed for the work became absolutely integral and
indispensable to the sound design (the interactions being far too complex to be
undertake ‘manually’) and once constructed, allowed for a virtually ‘hands-off’
compositional process. The translation of colour, shape and DNA sequences into
musical material in Microcosmos is centred around the concept of M-Space mapping: the
superimposition of a set variables (e.g. the red, green & blue content of a pixel on screen)
on to a set of musical parameters (e.g. cut-off frequency, resonance & effects
modulation). However, unlike Bloodlines where physical parameters controlled coexisting
19
yet independently generated musical layers, Microcosmos constructs a complex
interrelationship between variables. This interdependence of parameters does not set up
a simple one-to-one response between one particular input value and one isolated
musical event. Small changes in one colour can pass a tipping point and trigger a series
of non-linear events in a way that feels improvisational, reactive and unpredictable.
The core of the compositional system is dependent on patches written in Cycling
‘74’s MAX/MSP 4.5 and Jitter 1.6 (What’s The Point? ©2007 Mermikides and Gene Genie
©2007 Mermikides). Ableton Live 6 was also used in synchronization with Max/MSP
for audio synthesis and manipulation. Recording, editing and mastering was conducted
with Logic Pro 7, an Audient Mixing Console and an M&K 5.1 monitoring system.
IMovie and DVD Studio Pro were employed in the finalising process.
The installation was first exhibited in the Lewis Elton Gallery, University of Surrey
6-23 March 2007 as part of the Guildford International Music Festival using a Samsung
52” plasma screen, four Eclipse TD Monitors and an Eclipse sub-bass speaker. It was
later shown (as large scale projections) in the 2008 Digiville event (Brighton), a 2008
York Gate research event (Royal Academy of Music, London), Dana Centre (London
Science Museum) - as part of the 2008 Infective Art series - and the 2010 Art Researches
Science conference (Antwerp, Belgium).
Here follows a comprehensive technical description, followed by an illustrative
representation of the compositional system of Microcosmos. The complete video
installation (in surround format) used in performance is found on DVD 2 An audio
extract of the work appears on Core Works (CD2.4) with the full 19 minutes of audio on
Hidden Music (CD3.4). Audio examples in this commentary are found on the Audio
Examples CD (CD1.51-58).
The sound design is divided into four layers namely:
1) The DNA Code Layer
20
2) The Protein Layer
3) The Background Layer
4) The Grain Layer
These four layers exist simultaneously and, though independent, are manipulated by
a common set of parameters namely:
a) The red colour content (R) of various ‘hotspots’ of the video (0-255).
b) The green colour content (G) of various ‘hotspots’ of the video (0-255).
c) The blue colour content (B) of various ‘hotspots’ of the video (0-255).
d) The DNA coding from the 16s RNA sequence of the particular bacteria on
screen. This is coded in strings of A,C,T and G.
e) The protein manufactured by the correlation of three adjacent DNA codes (or
one ‘codon’). This is one of the 20 common amino acids as determined by the codon’s
particular sequence of codes. For example the sequence AAA produces lysine, while
GAA produces glutamic acid. Some code sequences (e.g. TGA) sends a STOP message,
which produce silence in Microcosmos.
The process by which input parameters a-e are translated into each of the four
musical layers is described below.
DNA Layer
DNA is a string of information comprising only four distinct units termed A, C, T
and G (in RNA, the code U is used in the place of T). The video display a series of
microbacterial colonies (their names are available on the ‘subtitles’ menu) and for each,
the relevant 16s RNA sequence is employed. In this layer, these codes are translated into
MIDI pitches via their ASCII code value (Figure 1.4.1).
DNA Code A C T G ASCII 65 67 84 71 MIDI Pitch F3 G3 C5 B3
Figure 1.4.1 Translations of DNA codes into MIDI pitches via ASCII (CD1.51).
21
These four derived notes are played on CD 1.50. DNA code is read in groups of
three (code1, code2 and code3), which allows 64 permutations of ACTG. These are
heard sequentially on CD1.52 at a fixed tempo. However, Microcosmos employs a variable
tempo determined by the brightness of the central point in the screen, with brighter
colours producing faster tempi (Figure 1.4.2).
Tempo (ms) = 1010 - (330*(R+G+B))
or
Tempo (bpm) = 60000/(1010 - (330*(R+G+B)))
Figure 1.4.2 Tempo determination of DNA layer via brightness of centre point in screen.
Note length and velocity (0-127) of each code is also determined by colour data
(Figure 1.4.3). CD1.53 demonstrates the effect of colour change on the permutations of
CD1.53.
Code1: Note length(ms) = G Velocity = R/2
Code2: Note length(ms) = B Velocity = G/2
Code3: Note length(ms) = R Velocity = B/2
Figure 1.4.3 Note length and velocity determination of Code Layer (CD1.53).
The timbre of each code is a triangle wave but with a band-pass filter (+10dB)
applied according to the rules of Figure 1.4.4, with an audio demonstration on CD1.54.
Code1 filter frequency(Hz) = 20+ (75 * R) Code2 filter frequency(Hz) = 20+ (75 * G) Code3 filter frequency(Hz) = 20+ (75 * B)
Figure 1.4.4 Timbral filtering of Code Layer in Microcosmos (CD1.54).
22
The three codes are positioned around the four speakers in a triangular
configuration, which is slowly rotated until it reaches its starting point at the next repeat
of the video sequence (Figure 1.4.5).
Code 1 Surround angle Video position (seconds) * 2.91
Code 2 Surround angle 120 + (Video position (seconds) * 2.91 (mod 360))
Code 3 Surround angle 240 + (Video position (seconds) * 2.91 (mod 360))
Figure 1.4.5 Spatialisation of Code Layer in Microcosmos.
Protein Layer
In nature, every group of three codes (or one codon) produces a specific amino acid.
The 16s RNA coding of each bacteria is decoded into a series of amino acids (of which
20 are used) – the specific amino acid that is produced may be checked against a codon
usage table. An analogous process is constructed in Microcosmos where each amino acid is
represented by a specific chord, and a codon usage table is employed in the electronic
system to select automatically one of a library of 20 chords. Codons that in biology
Speaker 3 Speaker 4
Code 1
Code 2 Code 3
Speaker 1 Speaker 2
23
produce a STOP message, here, create no chord. These ‘protein’ chords are manipulated
timbrally by RGB parameters from a continuously definable ‘hotspot’ on screen, which
allows the viewer to track the centre point, or representative colour of a bacterial colony.
The Protein Layer uses an additive synthesis program with parameters controlled by
RGB colour content (Figure 1.4.6).
Low Pass Filter Frequency (Hz) 60+ (60 * R)
Low Pass Resonance (Range 0 (flat)-127 (+12dB)) G/2
Modulation Effects Level (0 (Silent) - 127
(100% relative to dry signal))
B/2
Delay Feedback Percentage on Low Frequency Content R/2.5
Delay Feedback Percentage on Mid Frequency Content G/2.5
Delay Feedback Percentage on High Frequency Content B/2.5
Figure 1.4.6 Synthesis manipulation of Protein Layer by colour content in Microcosmos.
Background Layer
A background is also created to provide a low frequency foundation to each
bacterial colony, and comprises a microtonally intonated low-frequency arpeggiated
three-note chord for each key image. A low pass filter, whose parameters are controlled
by the overall lightness of the image, modulates this chord. The specific notes in the
chord is derived from the RGB data of the background colour of each image. The MIDI
notes for each voice in the triad are calculated via the What’s The Point? software patch
(Figure 1.4.7).
Voice 1 MIDI Note 10 + (0.6 * R)
Voice 2 MIDI Note 11 + (0.13 * G)
Voice 3 MIDI Note 15 + (0.35 * B)
24
Figure 1.4.7 Three-note chord construction algorithm of Protein Layer.
The constructed chords are rounded to discretely identified pitch values. However,
RGB values are taken at both the beginning and end of the allotted time-frame for each
chord, and any colour discrepancies that occur within these limits are interpreted as
microtonal glides. Four examples of the colour-chord translation process of specific
images are shown in Figure 1.4.8 and may be heard on CD1.55-58. This process may be
thought of as the dynamic superimposition of a three-dimensional visual subset (RGB)
on to a three-dimensional subset of M-Space.
Figure 1.4.8 Example three-note chord translations of images in Background Layer (CD1.55-58).
A low-pass filter is applied to the chord with the specific settings laid out in Figure
1.4.9. For this, RGB values are averaged across five hotspots on the screen in order to
provide an overall measure of brightness. With this process, a simple synaesthetic
relationship is created: a very dark image, for example, would only allow very low
frequencies to be heard, while a bright image would open the filter to its fullest extent,
with a continuum of timbral signatures between these extremes.
Low Pass Filter Cut (Hz) 60+ (20 * R * G * B) Low Pass Resonance [0 (flat) – 127 (+12dB)] 0.15 * R * G * B
Figure 1.4.9 Cut-off and resonance algorithm in Background Layer.
25
Grain Layer
DNA is digital information, but is stored in an organic medium and its instructions
are followed in the biophysical world. This digital/biological duality is reflected in
Microcosmos with the blending of synthetic and naturally occurring sounds; the Grain
Layer is constructed to manipulate the acoustic sounds of various materials using the
RGB data within the images. Each bacterial species is assigned a particular signature
timbre: a fragment (or ‘grain’) of acoustically occurring sound.
1. Chromobacterium violaceum A human voice
2. Serratia marcescens A single bell tone
3. Vogesella indigofera A resonating glass
4. Bacillus atrophaeus A struck stone
5. Pseudomonas aeruginosa A struck cymbal
6. Staphylococcus aureus (MRSA) A resonating piece of wood
These sound grains are manipulated by the colour content of the bacterial colonies.
The duration, pitch and starting point of each of these grains are controlled by the red,
green and blue colour content of the bacterial colonies respectively. More specifically, a
relationship between frequency in the colour and aural spectra is identified. This simple
mapping procedure creates highly complex and evocative musical responses to the
evolving imagery1. Between one and four grain layers are applied simultaneously
depending on points of colour interest, selected by the author in the rendering process.
1 For an overview of the use of grains in electronic music see Roads (2004).
26
Furthermore, the locations of selected colour points on screen are interpreted spatially
among the four speakers (Figure 1.4.10).
Figure 1.4.10 An illustration of Grain Layer manipulation and spatialisation in Microcosmos.
The sound grains are manipulated in terms of the speed they are played (defining
their pitch), the duration of each grain and the position in the sample from which the
grain is derived (affecting timbral elements). These three elements are controlled by the
RGB colour content of the selected on screen points as defined in Figure 1.4.11.
Grain Quantization (% of sample duration) R/5.1 Grain Duration (ms) G * 2 Grain Frequency (pitch) multiplier (0-2) (B/127) - 1
Figure 1.4.11 Grain manipulation via RGB content in Microcosmos.
27
Summary
The translation of DNA and colour information in Microcosmos may be conceived as
the transplantation of multi-dimensional parameters from one medium (DNA, protein
synthesis, colour, location etc.) to another (musical parameters). In other words, M-
Space exploration is guided by physical phenomena rather than a human improvisational
performance. An impression of the mappings is given in Figure 1.4.12.
Figure 1.4.12 The mapping of visual and biological data to M-Space subsets in Microcosmos.
Although there is fairly limited data input, the interactive complexity is sufficient to
have an interesting emergent response. Some points of interest:
1) The bell theme caused by the close red tones (DVD2 0:10-0:31).
28
2) The slow coding & subdued timbre with the dark screen and blue microbacteria
(DVD2 6:15-6:35).
3) The dramatic change in activity from a white screen (fast tempo and high
frequency timbres) to the instant when the vogesella indigofera colony crosses the centre
point of the screen (6:50-7:03).
Microcosmos represents a complex M-Space/expressive contours mapping system
evolved from experience with Bloodlines, Primal Sound and Head Music. This mapping
technique is employed in a more stylistically accessible context in Membrane (1.5, p 29)
and is developed further in Palmer’s ongoing Ghostforest (see Section 4). The relationship
between live visual and musical material is also explored in Rat Park Live (3.2), this time
in reverse, whereby musical improvisations create corresponding visual imagery.
29
1.5 Membrane (2009)
An opportunity to explore M-Space mapping in a different stylistic context to
Microcosmos was presented in the Wake Up and Smell the Coffee project with collaborators
Dr. Anna Tanczos and Dr. Martin Bellwood (University of Surrey). The immersive
planetarium movie medium, together with a now well-established theoretical framework
allowed an appropriate context in which to employ a direct mapping of the M-Space
model throughout the film. This section focuses on a short segment of the film as a
demonstration of the type of techniques employed.
Wake Up and Smell the Coffee (DVD3) is an educational film explaining the effect of
caffeine on the human body, and includes an extended animated sequence following the
journey of a caffeine molecule from ingestion to its docking with an adenosine receptor
in the brain. The Membrane clip (DVD3 11:12-13:15) follows the molecule’s passage
through the cell membrane of the villi into the nucleus of the cell, and its return
trajectory in a different trajectory. The molecule follows a clear flight path above and
through the cell membrane and this three-dimensional trajectory is mapped musically
using M-Space concepts. As well as a pad and bass-line, the sound design in this scene
includes two layers of a virtually modeled instrument (reminiscent of a marimba). These
layers each consist of simple two bar phrases (consisting of Coltranesque three note
pentatonic cells (see Changes Over Time:Theory 1.5 p 25-26) with three subdivided zones of
proximity - derived from the spatial position of the molecule (Figure 1.5.1 p30). Four
zones are identified (A-D) with corresponding phrases for Layers 1 and 2 (1a-d and 2a-
d). Each phrase represents a proximal musical relationship based on the physical
position of the molecule. Fields B and D are within the same physical zone and so have
similar corresponding phrases, however for variation: Phrase 1d is a retrograde
transformation of 1b, while 2d and 2b are identical, at least in terms of notational pitch
30
and rhythm. These two layers are each transformed independently in terms of
continuous (timbral and time-feel) parameter subsets based on the molecule’s three-
dimensional position. The avoidance of overly obvious parameters, such as chromatic
transposition and volume, helps keep the correlations subliminal and unearths otherwise
unvisited parameter modulations and interactions. The Membrane sequence may be seen
on DVD3 11:12-13:15 and heard, without narration on CD2.5 and CD3.5.
Figure 1.5.1 M-Space mappings on Layers 1 and 2 in Membrane (from Wake Up and Smell the Coffee) derived from the motion of the caffeine molecule through the membrane of the villi. Four zones are
identified to create proximal relationships in each layers, and the molecules motion is mapped to continuous musical parameters as indicated (CD2.5, CD3.5 and 11:12-13:15).
Hidden Music presented a brief survey of the heterogenous mapping of unorthodox
source material on to a concept of musical space derived from jazz improvisational
theory. Some of these techniques may also be redeployed in the context of acoustic (and
electric) instrumental performance again with the extensive use of electronics. The next
section presents a selection of works employing solo instrumentation with electronics
31
and the use of M-Space and expressive contour concepts in performance, improvisation
and composition.
32
2. Reworked Ensemble Improvisational works for ensemble and live electronics
2.1 The Selfish Theme
33
3. Extended Solo Instruments Works for solo instruments and live electronics
3.1 String Theory (2006) and Event Horizon (2007)
Tod Machover’s long established and distinguished career with electronic music,
starting with his collaborations at IRCAM with Pierre Boulez, is well documented2
(Machover 1992, 2004, 2006 and 2010). In 2004, Machover set up a project with the
Royal Academy of Music and MIT Media Lab to develop repertoire and research output
for the Hyperbow for cello - an electronic cello bow - designed and built by Dr. Diana
Young3. As director of the Royal Academy Music wing of the Hyperbow project, the
author had the opportunity to work closely with Machover, Young and the Royal
Academy composers and cellists. This led to the staging of a concert at the Duke’s Hall
in 2007 and the production of a CD of new works (CD3 21st Century Bow: New Works for
Hyperbow RAM 2007) from which Event Horizon is taken.
The Hyperbow is a modified cello bow fitted with electronic sensors that detect
seven parameters of motion and force. These are: distance from a bridge sensor of the
bow tip (1) and frog (2), the downward (3) and lateral (4) force on the bow, and three
dimensions (X, Y, Z) of acceleration (5-7). The location of these sensors is indicated in
Figure 2.1.1 (p 33). Values of the seven parameters are sent wirelessly to a computer for
recording, analysis or mapping. Young’s Hyperbow for violin was used extensively for
performance research, in particular the cataloguing bowing techniques and the analysis of
their relationship to sound production (Young 2007). However, the outputted parameter
2 See Machover (1992, 2004, 2006, 2010). 3 See Young (2002, 2003), Young & Serafin (2003) and Young, Nunn & Vassiliev (2006).
34
values during performance may also be used to control and manipulate musical
parameters, and it is this creative mapping approach that was taken in the RAM/MIT
Hyperbow project, and in Event Horizon.
Figure 2.1.1 Sensors on Young’s Hyperbow for cello (Image ©2006 Yael Maguire)
The use of continuous electronic controllers in musical manipulation is of course
well established; from early electronic instruments like the Theremin, the guitarist’s wah-
wah pedal, to the huge range of midi controllers now available. However these may all
be thought of as auxiliary controls that do not interfere significantly with normal
instrumental execution (in the case of the guitarist’s foot pedal for example) or as the
sole focus in performance as demonstrated by the new breed of laptop musician. Tools
such as the Hyperbow have a unique function as the control signals generated have an
interrelationship with normal performance. Furthermore, the nuanced control of these
parameters is aided by the performer’s technical discipline rather than a novel skill to be
learned. The electronic systems in Event Horizon rely on sophisticated bow control, both
away from the instrument and during sound production, and so it may be seen as a piece
that requires an extended instrumental technique rather than the learning of a auxiliary
parallel skill.
strain gauge
force sensors
accelerometerswireless
module
battery
tip position
signal
frog position
signal
35
What is also apparent from Figure 2.1.2 (A screenshot from Hyperactivity (2006)4 a
MAX/MSP by the author routing hyperbow data to midi controller values and triggers)
is the familiarity of the sensor data; clearly there are expressive contours here readily
available for mapping to musical parameters.
Figure 2.1.2 A screenshot from Hyperactivity, a MAX/MSP patch that routes hyperbow data to midi controller values. Thresholds for parameter values may also be specified that when exceeded send out
midi notes to trigger discrete events.
The cellist may consciously control these contours while the cello is not played, or
they may be teased out during performance with for example, a slight twisting of the
bow or a little more downward force during a passage. Other contour relationships may
occur with little potential control or conscious awareness, but may still have musical
effect. Hyperactivity allows the specification of threshold values for Hyperbow data that,
when exceeded, may trigger discrete events via MIDI notes. For example a sharp flick of
the bow may trigger a (predetermined or randomised) harmonic change in the electronic
backing, or when downward force surpasses a certain level, a filter may be engaged with
4 Hyperactivity includes a MAX/MSP object for data calibration programmed by Patrick Nunn (2006).
36
its characteristics controlled continuously by Hyperbow data. A demonstration of the
types of mappings and trigger events possible with the Hyperbow and Hyperactivity, most
of which are used in Event Horizon, is presented as a video on DVD 4.1 with the cellist
Peter Gregson playing an Eric Jensen five-string electric cello. The video represents an
early technical sketch of Event Horizon, dubbed String Theory, and the first recorded audio
take appears in the soundtrack of Martino:Unstrung (DVD1 43:23-44:03) with additional
versions on CD2.2 and CD4.2. String Theory is an example of a work that, once the
electronic system is constructed, allows an open-ended improvisational approach by the
soloist who is free to control structure, melodic detail and extreme timbral
characteristics, even if some of the backing material and mapping relationships are
prepared. Another example of this approach is found in Dublicity – a solo guitar
improvisation by the author (CD4.3). With this system the electric guitar’s effects may
be selected with a foot pedal, but the output of the guitar is also mapped to MIDI data,
which is separated into overlapping zones depending on string and pitch range. This
allows for the simultaneous performance and control of electric guitar, MIDI controlled
keyboard, bass, effects and rhythmic fragments. As in String Theory, the electronic system
may be exploited with instrumental proficiency - in this case techniques such as two-
handed fretboard and string-crossing mechanics – and jazz sensibilities, rather than the
typical pedalboard, assistant technician or mouse and laptop approach.
The Hyperbow project is the author’s first of many collaborations with Gregson - an
enthusiastic champion of the use of electronics in contemporary music - which include
four CD releases, research events and international concert performances of cello and
electronics programmes.
Event Horizon
References
Adams, C. (1976) Melodic Contour Typology in Ethnomusicology 20, 179-215. Frey, D. (2010) New York Skyline. [online] Red Deer Public Library. Available: http://www.villalobos.ca/ny-skyline [Accessed 6 February 2010]. Palmer, A. S. (2010) Projects [Online] Available: http://www.angelaspalmer.com/category/press/ [Accessed 10 January 2010]. Rilke, R. M. (1919) Ur-Geräusch in Sämtliche Werke, vol. 6. 1987 Revised Edition. Frankfurt: Insel. Roads, C. (2004) Microsound. Paperback edition. Cambridge: MIT Press. Young, D. (2002) The Hyperbow Controller: Real-Time Dynamics Measurement of Violin Performance in 2002 Conference on New Instruments for Musical Expression, Dublin, Ireland. 24-26 May 2002. Young, D. (2003) Wireless Sensor System for Measurement of Violin Bowing Parameters in Stockholm Music Acoustics Conference (SMAC 03) Stockholm, 6-9 August 2003.
Young, D. & Serafin, S. (2003) Playability Evaluation of a Virtual Bowed String Instrument in Conference on New Interfaces for Musical Expression (NIME). Montreal, 26 May 2003.
Young, D., Nunn, P. & Vassiliev, A. (2006) Composing for Hyperbow: a collaboration between MIT and the Royal Academy of Music in 2006 Conference on New Instruments for Musical Expression (NIME). Paris, 4-8 June 2006.
Young, D. (2007) A Methodology for Investigation of Bowed String Performance through Measurement of Bowing Technique. PhD Thesis. Boston: MIT. Machover, T. (1992) Hyperinstruments - A Progress Report 1987 – 1991. [Online] MIT Media Laboratory, January 1992. Available: http://opera.media.mit.edu/hyper_rprt.pdf [Accessed 7 January 2010]. Machover, T. (2004) Shaping Minds Musically [Online] BT Technology Journal, 22.4, 171-9. Available: http://www.media.mit.edu/hyperins/articles/shapingminds.pdf [Accessed 10 October 2009].
Machover, T. (2006) Dreaming a New Music [Online] Available: http://opera.media.mit.edu/articles/dreaming09_2006.pdf [Accessed 2 January 2010].
Machover, T. (2010) On Future Performance [Online] The New York Times, 13 January 2010. Available: http://opinionator.blogs.nytimes.com/2010/01/13/on-future-performance/ [Accessed 6 February 2010].