Framework for Live Algorithms
Tim Blackwell
Michael Young
Goldsmiths College, London
www.timblackwell.com
Organised Sound 9(2): 123–136
The contextConsider (for now, because it’s easier) improvised performance
i.e. free of compositional directives such as harmony, rhythm, form…
Conventionally such improvisations are explorations of timbre and texture
The contextConsider (for now, because it’s easier) improvised performance
i.e. free of compositional directives such as harmony, rhythm, form…
Conventionally such improvisations are explorations of timbre and texture
What is a Live Algorithm?
A Live Algorithm is
Interactive
Autonomous
Ideas generator
Idiosyncratic
Comprehensible
InteractiveSympathetic, supportive, makes appropriate contributions, tacets
AutonomousNot merely automatic, mechanical and predictable. Must be comprehensibly interactive and capable of novelty
Ideas machineContradictory, individualisticsource of novelty and leadership
Idiosyncratic
Contributions limited by instrument and experience but can be unusual, individualistic
Comprehensible
Suitable for collaboration, at least by humans. Not opaque, but not too transparent either
And, importantly…
Closure
…knows when to stop!
Principles
Expectation of generating form might not be necessary
Local events can be structuringFor example, spatio-temporal self-organisation depends only on local interactions of a certain complexity and on positive and negative feedback.
Here, the space is (interpreted as) a musical/sonic parameter space at some level.
Parameterisations possible at micro (sample, grain), mini (note) and meso (phrase) levels. The problem of emergence might necessitate a need a multi-level system
Knowledge of music rules might not be necessary
LA need not be aware of what it is doing
In the model, LAs and humans interact with meaningless sounds, populating an inert environment.This builds on a former XY model, which expresses the paradox of interaction
(In our nature-inspired systems, The sounds are organised by stigmergy)
Knowledge of music rules might not be necessary
LA need not be aware of what it is doing
Desirability of an Interactive Model and a Conceptual ArchitectureIn the model, LAs and humans interact with meaningless sounds, populating an inert environment.This builds on a former XY model, which expresses the paradox of interaction
(In our nature-inspired systems, The sounds are organised by stigmergy)
The XY model
The model
A B
Conceptual (and Actual?) Architecture
P: Listening, analysingTypically thins degrees of freedom; from real timeIN -> p(t) -> pi
F(p): Ideas enginePatterning in a hidden space H; generative/algorithmic/iterative xi -> xi+1
Q: Interpretation, Playing, synthesizingTypically expands d.o.f.; into real timexi+1 -> q(t) -> OUT
PfQ Architecture
Analysisis typically projection from one level to another, higher level. E.g. samples to event parameters pP: (t) -> p(t) -> pi
We observe that this projection is determined by cultural, personal, genre-specific and even political forces
P might be a map into H and hence hidden state x is a possible parameterisation of the sonic environment.
Interpretationis the process of relating internal states to event parameterisations, and eventually to soundQ: xi+1 -> q(t) -> OUT
A technical complication surrounds the relationship between real and algorithmic time
It is unlikely that both will flow at the same rate: the computational update time xi -> xi+1 might not correspond with the desired time interval between events (t), (t+t...
Internal states might be sampled at a given rate irrespective of iterative time or timing information could even derive from x itself
Information might be derived from averages over a population {x1, x2, x3, ...}
All these complications and possibilities are represented by a single interpretative function Q
Ideas generatorF is parameterised by p and determines state flow of hidden variables xF(p): xi -> xi+1
xi+1 determined F, p(t) and xi
F need not derive from any musical concern, and this may be advantageous
F only concerns patterning and structure
Autonomous 1. x ≠ 0 even if p = 0
2. x = 0 even if p ≠ 0
1. This is usual, expresses state flow in a dynamical system
2. Harder to achieve, but could arrange for Q( x[x-dx, x+dx] ) = q.Alternatively, include dissipation and re-energising so that x 0 between injections of energy (determined by p?)
Idiosyncratic Q and P concern the mappings to and from sound – herein lies the machine's idiosyncrasies and characteristics
Can be quite limited, since interactivity and autonomy are the major prerequisites
Potentially P, Q may cross levels
Interactive autonomy + modification of system state
Analysis parameters p ensure interaction i.e. affects, but does not fully determine, x
For example, p might be an attractor in H. In a dynamic system, x orbits p, but precise trajectory depends on initial conditions
Alternatively, hidden variables x can be regarded as parameters which select mappings Fx: p -> q
The collaborative interface is the asynchronous mapQ Fx P = Mx
OUT(t+t) = Mx IN(t)
Comprehensible:Transparency > Comprehensibilty > Opacity
Very transparent if p H, Q = P-1 and dim(H) is small enough. Easy to establish correlations between human (incoming sound, deposited in the environment by a human) and LA (outgoing sound deposited by the LA). Might become too predictable unless enhanced by some stochastic adjustment (a degenerate solution)
Very opaque if QFP is very complex and dim(H) is big. Very hard to establish correlations or any measure of causal relations between human and LA. Too unpredictable and therefore not collaborative
Closure:Raises issues of knowledge of form
Could argue that LA should not be deprived of knowing the modus operandi of the performance
In humans, closure is influenced by time constraints, visual cues and an increasing capacity to interpret gestures as possible sonic cadences
However, in an LA closure might possibly emerge from lower level dynamics e.g. self-organised criticality
Other desirable properties
MemoryDynamical states do not conventionally posses this
Short term (repetitions, anticipations based on recognition..) >>> attractor persistence, pheromone trailsE.g. Swarm Techtiles, Ant Colony simulations
Long term (draws on previous musical engagements) >>> current states could have memories of previous states… E.g. Particle Swarms (particles have a memory and participate in social networks)…AND the sequence of conditions that caused these states (a contextual memory)E.g. ??
Links with existing research fields:
P: -> pReal-time music analysis and informatics
Q: x -> Real-time audio synthesis
FGenerative Music (Neural, Cellular Autonoma, Genetic Algorithms, Swarms
Links with existing paradigms
Live Electronics:ideas engine F replaced by human volition. State flow is adjustment of “controls” x
Live CodingF = I and Q is adjusted (in software) manually
Generative/algorithmic music:Set p = 0 (turn P off)(t) = F(xi) = FN( x0 )
The system is self-contained - the composer chooses F and the initial condition, x(0)
Reactive SystemsIn a reactive system, IN will necessarily trigger certain transitions xi -> xi+1
Mx = QFP express a causal and necessary chain – a rule-based system
It is conjectured that a reactive system might not be sufficient for a Live Algorithm because it is not sufficiently interactive
This touches on various issues in cognitive science and machine intelligence
Possibly, a large enough rule set might do the job, and might also be indistinguishable from a discrete dynamical system (i.e. an algorithmic model of a continuous DS)
It is hard to see how we might achieve this complexity without drawing on inspiration from dynamical, and other natural, systems
(Our) Experience
Swarm Music:
Very transparent, P and Q operate at the note level
Swarm Granulator:
Less transparent, functions largely at the grain level, although some level-crossing
Swarm Tech-tilesP actually expands into a 2D landscape
F integrates ALife and optimisation Q is a rendering of parts of the landscape visited by x
ArgrophyllaxP: FFT analysisf: stochastic functionH: Fourier spaceQ: coefficients of inverse FFT transform Q
Evaluation Irrelevance of Turing Test? TT is more relevant for the performers than the listeners. A metric: do the performers find the LA to be stucturing?
Applicability: The attributes of a LA should be useful in other musical contextsE.g. intelligent effects pedal, synth plug-ins, accompaniment programs, genre improvisation