University of Calgary PRISM: University of Calgary's Digital Repository Graduate Studies The Vault: Electronic Theses and Dissertations 2016 AAIM: Algorithmically Assisted Improvised Music Fay, Simon Fay, S. (2016). AAIM: Algorithmically Assisted Improvised Music (Unpublished doctoral thesis). University of Calgary, Calgary, AB. doi:10.11575/PRISM/24629 http://hdl.handle.net/11023/3073 doctoral thesis University of Calgary graduate students retain copyright ownership and moral rights for their thesis. You may use this material in any way that is permitted by the Copyright Act or through licensing that has been assigned to the document. For uses that are not allowable under copyright legislation or licensing, you are required to seek permission. Downloaded from PRISM: https://prism.ucalgary.ca
Transcript
University of Calgary
PRISM: University of Calgary's Digital Repository
Graduate Studies The Vault: Electronic Theses and Dissertations
2016
AAIM: Algorithmically Assisted Improvised Music
Fay, Simon
Fay, S. (2016). AAIM: Algorithmically Assisted Improvised Music (Unpublished doctoral thesis).
University of Calgary, Calgary, AB. doi:10.11575/PRISM/24629
http://hdl.handle.net/11023/3073
doctoral thesis
University of Calgary graduate students retain copyright ownership and moral rights for their
thesis. You may use this material in any way that is permitted by the Copyright Act or through
licensing that has been assigned to the document. For uses that are not allowable under
copyright legislation or licensing, you are required to seek permission.
Downloaded from PRISM: https://prism.ucalgary.ca
UNIVERSITY OF CALGARY
AAIM:
Algorithmically Assisted Improvised Music
by
Simon Fay
A THESIS
SUBMITTED TO THE FACULTY OF GRADUATE STUDIES
IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE
4.1 Classification of the AAIM system using the categories presented in Chapter 3. . . . 76
5.1 Examples of the relationships between various i-o-i’s and the number of repetitionsof both the ‘base i-o-i’ and the i-o-i itself necessary for synchronisation, using one18 th note as the ‘base’ i-o-i. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.2 The relationships between interval names used by the AAIM.melodyVary moduleand the chromatic intervals they represent. . . . . . . . . . . . . . . . . . . . . . . 112
ix
List of Figures and Illustrations
1.1 The scope of the research presented in this thesis. . . . . . . . . . . . . . . . . . . 71.2 Graphical representation of the methodology used for the research presented in this
2.2 Example of a simple rhythm, ‘3 against 4,’ using the Schillinger method. . . . . . . 50
4.1 Illustration of the placement of the AAIM system between the interface and soundsource. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
4.2 Expansion of Figure 4.1, illustrating the fundamental role played by the AAIM.rhythmGen mod-ule in triggering each of the other primary modules. . . . . . . . . . . . . . . . . . 79
5.1 Example of a two bar phrase output by the rhythm generator. Blue arrows signifythe choosing of a new i-o-i, and red arrows signify triggers output by the rhythmgenerator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
5.2 Output of Fig 5.1 in musical notation, with the ‘base’ i-o-i equalling 18note. . . . . 90
5.3 Simple example of the AAIM.rhythmGen ‘ioiMaker’ method. . . . . . . . . . . . . 935.4 Examples of the AAIM.rhythmGen ‘complexity determination’ method. . . . . . . 935.5 The decision-making process used by each AAIM.rhythmGen voice choosing new
i-o-i values and outputting triggers. . . . . . . . . . . . . . . . . . . . . . . . . . . 945.6 A schematic diagram demonstrating two alternative groupings created by the AAIM.rhythmGen ‘gen-
Grouping’ algorithm using the same input. . . . . . . . . . . . . . . . . . . . . . . 965.7 A schematic diagram illustrating the AAIM.rhythmGen ‘indispensability’ algorithm. 975.8 The decision-making process used by the AAIM.patternVary when using the output
of the AAIM.rhythmGen to vary patterns. . . . . . . . . . . . . . . . . . . . . . . . 995.9 The decision-making process used by the AAIM.loopVary module when playing
and varying samples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1035.10 Example of how the AAIM.loopVary algorithm segments a sample into n equal
parts, and how the module maps triggers onto these segmentations. . . . . . . . . . 1045.11 Example of mapping the output of the AAIM.rhythmGen module onto a sample
using the AAIM.loopVary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1055.12 Variations of a sample using the AAIM.loopVary ‘reverse’ method. . . . . . . . . . 1065.13 Variations of a sample using the AAIM.loopVary ‘retrigger’ method. . . . . . . . . 1065.14 Variations of a sample using the AAIM.loopVary ‘jump’ method. . . . . . . . . . . 1075.15 Demonstration of how the AAIM.melodyVary module converts input of pitch and
trigger type pairs into a melodic line (using the nursery rhyme ‘Mary had a littlelamb’). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
5.16 Demonstration of how the AAIM.melodyVary module maps the list of ‘AvailablePitches’ onto the list of ‘Actual Pitches’ to create new versions of melodic lines. . . 110
5.17 Demonstration of the modal mapping method used by the AAIM.melodyVary module.1115.18 The AAIM.melodyVary analysis of the intervallic content of the melody used in
5.19 Expansion of the possible notes used by the AAIM.melodyVary scalar note method. 1135.20 Example of the AAIM.melodyVary ‘invert’ method . . . . . . . . . . . . . . . . . . 1155.21 Example of the AAIM.melodyVary ‘retrograde’ method . . . . . . . . . . . . . . . 1165.22 Example of the AAIM.melodyVary ‘expansion’ method . . . . . . . . . . . . . . . 1175.23 Example of the AAIM.melodyVary ‘sequence’ method . . . . . . . . . . . . . . . . 1185.24 Demonstration of the AAIM.melodyVary ‘scalar note’ method. . . . . . . . . . . . 1195.25 Demonstration of the AAIM.melodyVary ‘scalar note’ method, but using the entire
C pentatonic scale rather than just the notes present in the original melody. . . . . . 1215.26 Demonstration of the AAIM.melodyVary ‘scalar melody’ method. . . . . . . . . . . 1225.27 Screenshot of the AAIM.app application main screen. . . . . . . . . . . . . . . . . 1235.28 Screenshot of the AAIM.app application AAIM.rhythmGen interface. . . . . . . . . 123
6.1 The “There is pleasure...” Feedback Frequency Modulation (FFM) synthesis engine.1286.2 The interface used to control “There is pleasure...” during live performances. . . . 1296.3 The ‘Vamp’ section of under scored . . . . . . . . . . . . . . . . . . . . . . . . 1336.4 The ‘A’ section of under scored . . . . . . . . . . . . . . . . . . . . . . . . . . 1356.5 The ‘B’ section of under scored . . . . . . . . . . . . . . . . . . . . . . . . . . 1366.6 The ‘transition’ section of under scored . . . . . . . . . . . . . . . . . . . . . . 1366.7 The form used by the Aspect ensemble when performing under scored at the
2014 joint ICMC and SMC conference. . . . . . . . . . . . . . . . . . . . . . . . 1386.8 The, suggested, skeletal drum pattern for the primary theme of 5Four . . . . . . . . 1406.9 The, suggested, skeletal drum pattern for the secondary theme of 5Four . . . . . . 1416.10 The rhythmic pattern to be used during the ‘shout chorus’ section of 5Four . . . . . 142
A.1 The AAIM system window on load . . . . . . . . . . . . . . . . . . . . . . . . . 180A.2 The AAIM Setup window on system load . . . . . . . . . . . . . . . . . . . . . . 181A.3 The AAIM system performance interface . . . . . . . . . . . . . . . . . . . . . . 184A.4 The central section of the AAIM system performance interface . . . . . . . . . . . 185A.5 The audio section of the AAIM system performance interface . . . . . . . . . . . 185A.6 The polygon interface section of the AAIM system performance interface . . . . . 186A.7 The AAIM.rhythmGen interface on system load . . . . . . . . . . . . . . . . . . . 188A.8 The AAIM.rhythmGen controls on the main window . . . . . . . . . . . . . . . . 190A.9 The AAIM.patternVary interface on system load . . . . . . . . . . . . . . . . . . . 192A.10 The AAIM Setup following initial toggling of the patternVary algorithm . . . . . . 193A.11 patternVary performance interface on AAIM System main window . . . . . . . . . 193A.12 Interface used to create patterns which are subsequently played/manipulated using
the patternVary algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194A.13 Interface used to determine the chance of additional notes being inserted into the
given pattern for each individual voice . . . . . . . . . . . . . . . . . . . . . . . . 194A.14 Interface used for output of MIDI from AAIM . . . . . . . . . . . . . . . . . . . . 195A.15 Interface used for output of OSC messages from AAIM . . . . . . . . . . . . . . . 195A.16 Interface used for AAIM.samplePlayer, a patch which allows the triggering of an
arbitrary number of samples-with each sample being triggered by a single voice . . 195A.17 Global interface used for AAIM.samplePlayer2 . . . . . . . . . . . . . . . . . . . 196A.18 Sample load section of the AAIMsamplePlayerInterface2 patch . . . . . . . . . . . 196
xi
A.19 Sequence section of the AAIMsamplePlayerInterface2 patch . . . . . . . . . . . . 197A.20 Multisliders section of the AAIMsamplePlayerInterface2 patch . . . . . . . . . . . 197A.21 The AAIM Setup following initial toggling of the looper algorithm . . . . . . . . . 198A.22 AAIM.looperMultiVoiceInterface used for control of all AAIM.looper voices si-
Although algorithmic performance systems that are intended to create music within
pre-existing styles have proved successful (e.g. [16], [127]), their flexibility is often
limited by the narrow scope of the music they attempt to emulate. In the design of
any musical system or instrument limitations are inevitable; for example, a six-
7
string guitar can play a maximum of six simultaneous notes. However, a central
tenet of this research is that musical performance systems should strive to afford as
broad a variety of styles, aesthetics, and approaches as acoustic instruments, or to
quote Morton Subotnick: “the medium should always be at the service of the artist”
[168, 113]. To again use the guitar as an example, although the performer is limited
to six simultaneous notes, this has not limited the range of styles that musicians
have explored on the instrument. As such, a significant challenge for this research
was developing algorithms that do not attempt to emulate individual styles but are,
in fact, applicable to a broad variety.
2. Develop multi-voice algorithms for rhythm, pattern, melody, and sample play-
back
To maximise the flexibility and scope of the AAIM system, I needed to develop
algorithms that address four features common to many styles of music:
• Rhythm - addressing the temporal nature of music.
• Pattern - the triggering of a sequence of discreet events over time
(e.g. a drum beat).
• Melody - a series of individual pitches, again triggered over time.
• Sample playback - addressing the widespread use of audio record-
ings, i.e. samples, in electronic music.
It was also necessary for these algorithms to facilitate the usage of multiple voices
that could act independently but within the same overarching structure, thus simu-
lating the interactions between different performers in an ensemble.
3. Address Simoni and Dannenberg’s four evaluation criteria for musical algo-
rithms
In parallel to the above concerns, it was also necessary for the algorithms to be
8
performable, i.e. usable in real-time musical scenarios. In striving to meet this
challenge I used Simoni and Dannenberg’s four evaluation criteria for musical al-
gorithms [160, 4] as a model:
(a) Simplicity - each algorithm should be simple enough to be both run
and controlled during live performances.
(b) Parsimony - each algorithm should require only a minimal amount
of input from the user.
(c) Elegance - each algorithm should also afford a broad a range of
styles and aesthetics using only the users-input.
(d) Tractability - the relationship between the user’s-input and the
algorithms’-output should be as intuitive as possible.
However, in many scenarios these criteria were in conflict, either with each other or
with the above challenges, and as such a balance often had to be struck.
1.6 Methodology
The methodology employed in this thesis is a form of practice-based research, defined by Candy
as:
Practice-based Research is an original investigation undertaken
in order to gain new knowledge partly by means of practice and the
outcomes of that practice. Claims of originality and contribution
to knowledge may be demonstrated through creative outcomes that
may include artefacts such as images, music, designs, models, dig-
ital media or other outcomes such as performances and exhibitions
[25, 3].
9
Proceeding from this definition the methodology employed had four primary components: study,
development, testing, evaluation. However, as can be seen in Figure 1.2, the pathway through these
stages is circular, where evaluation augmented the knowledge gained through study and inspired
the next phase of development.
Figure 1.2: Graphical representation of the methodology used for the research presented in thisthesis.
1. Study
To understand the state of algorithmic performance systems I reviewed the litera-
ture to find relevant related work. This research into the current state of the field
enabled me to understand previous approaches to algorithmic performance systems.
In turn, this allowed me to imagine new approaches that could be taken and possible
augmentations or alterations to previous methods. My years of personal experience
as a performing and improvising musician of both guitar and electronic music aug-
mented this research even further.
2. Develop
Having understood the related work, and applied this to prior knowledge and expe-
rience of performance and improvisation, I designed and programmed algorithms
that facilitated the variation and manipulation of user-defined musical materials.
3. Testing
To test the ability of the algorithms to facilitate live performance and improvisa-
tion, I wrote, and subsequently performed, a series of musical works. Each of these
10
works serves as a ‘framework’ within which the performer is free to improvise by
varying and manipulating composed musical materials, allowing me to evaluate the
improvisational affordances of the algorithms. Each piece also explores different
aesthetics, thus examining the ability of the algorithms to facilitate a broad range
of styles and approaches. Although some of the performances of these works in-
volved the real-time creation of the work in the studio, many also received concert
performances.
4. Evaluate
Following testing, I evaluated the affordances of the algorithms. This evaluation
examined not only successes and shortcomings of the algorithm in affording the
performance in question, but also the extent to which the algorithm addressed the
challenges presented in Section 1.5. The knowledge gained from this evaluation
added to the knowledge already acquired during the study stage, and this then led
to a new phase of development.
1.7 Contributions
This thesis contains some significant research contributions, which I achieved by applying the
methodology described in Section 1.6 to meet the challenges discussed in Section 1.5. The primary
contributions are the AAIM system, discussed in Chapters 4 and 5, and the portfolio of creative
works created using AAIM described in Chapter 6.
• Portfolio of algorithmic modules for the variation of musical materials [64]
The primary contribution of this thesis is the collection of algorithms comprising the
AAIM system, described in Chapter 5. These algorithms cover some of the common
features of electronic music: rhythm, patterns, melodies, and sample playback.
Implementations of the AAIM system are available here:
11
https://simonjohnfay.com/aaim/. As will be discussed later, these downloads
do not represent definitive versions of the AAIM system, but rather serve as exam-
ples of some of the potential uses of AAIM.
The main contributions of the AAIM system are:
– Variation of user-defined materials11
Rather than generate entirely new musical elements, each of the
AAIM modules is concerned with affording the user the ability to
vary and manipulate their musical ideas. This maximises the con-
nection between the music the user inputs and the output of the sys-
tem.
– Facilitate Live Performance12
I designed AAIM for use during live performances. I achieved this by
only developing algorithms that could run in real-time, refining the
algorithms to improve their speed and efficiency, and incorporating
divergent mapping techniques, where one variable influences many
features, allowing control of the systems with a minimal number of
inputs.
– Facilitating Improvisation13
As will be discussed in Section 2.2, there are many different ap-
proaches to improvisation in music, and the extent to which an elec-
tronic music system affords improvisation “probably depends on the
11The use of AAIM to vary user-defined materials can be heard in any of the works contained in the portfolio ofcreative works. For example, in the variation of the melodic and harmonic material in under scored [58], or thevariation of prerecorded samples in 5Four[61].
12AAIM has been used in many concert performances. Both by the author: [70],[66],[69],[68], and [71]; and byother performers: [65] and [67].
13All of the works contained in the creative portfolio demonstrate AAIM’s potential to facilitate improvisation.Perhaps the most obvious examples are radioAAIM[63], BeakerHead[72], and new beginnings[57], all of which wereperformed with no preplanned score or progression.
2.2.1 Improvisation and composition as two points on a continuum
Grove Music defines improvisation as “the creation of a musical work, or the final form of a
musical work, as it is being performed,” noting that “it may involve the work’s immediate com-
position by its performers, or the elaboration or adjustment of an existing framework, or anything
in between,” and also that “to some extent every performance involves elements of improvisation,
although its degree varies according to period and place, and to some extent every improvisation
rests on a series of conventions or implicit rules” [121]. In discussing improvisation, Bruno Nettl
hypothesised that “the juxtaposing of composition and improvisation as fundamentally different
processes is false, and that the two are instead part of the same idea” [120, 6]. Margaret Kartomi
has echoed this belief that composition and improvisation are just two points on a continuum, stat-
ing that “since improvisation and composition both involve workings and reworkings of creative
ideas, they are essentially part of the same process” [92, 55]. Similarly, Berliner notes that in an
attempt to distinguish composition and improvisation by drawing up a “precise list of their exclu-
sive properties” he soon realised that his “grasp on them was less strong than [he] had thought, and
that “in fact, their characteristics seemed to overlap hopelessly at the margins” [13, 4].
Dean has also echoed these thoughts, writing that “improvisation and composition in sound
form a continuous spectrum, with mutual interfaces and slippages no more obvious than those
between red and orange” [43, xii]. He continues to differentiate the two on the basis that “im-
provisation is usually distinguished as involving substantial fresh input to the work at the time of
each and every performance” [43, xii]. Numerous scholars have noted the idea that “improvisation
involves some degree of spontaneity in musical decision making during the act of performance”
[170, 71]. For example, in his study of jazz, Berliner wrote that musicians often “reserve the
term for real-time composing - instantaneous decision making in applying and altering musical
materials and conceiving new ideas” [13, 221].
To address the continuum between improvisation and composition, Dean proposes two broad
categories into which improvisation can be said to fall:
40
1. “Applied” improvisation - the improvisation that occurs when a musician/composer
performs on their instrument or computer system in the process of composing and
then arranging or modifying these results into a codified piece of music later.
2. “Pure” improvisation, “an analytical fiction,” occurs during public performances,
particularly that part which is least predetermined. [43, xix]
Dean also notes that “it can be argued that all composition is, or involves, applied improvisation”
[43, xix]. Or as Nettl states:
We must abandon the idea of improvisation as a process separate
from composition and adopt the view that all performers improvise
to some extent. What the pianist playing Bach and Beethoven does
with his models - the scores and the accumulated tradition of perfor-
mance practice - is only in degree, not in nature, different from what
the Indian playing an alap in Rag Yaman and the Persian singing the
Dastgah of Shur do with theirs [120, 19].
2.2.2 The use of ‘referents’ or ‘models’ in improvisation
Of particular interest in the context of this paper is the use of a “model” [120, 11] or “referent” [43,
xii] upon which the improviser(s) base their performance. In jazz, blues, rock, and similar Western
musical traditions, this often takes the form of a harmonic and rhythmic structure, for example a
12-bar blues or popular song, and the improviser then spontaneously creates new melodic lines that
fit within this framework. So common is the use of referents in jazz that the harmonic and rhythmic
structures of numerous songs/pieces have gained new life, often in a modified form, as the basis
for new compositions, i.e. new melodic lines. In some cases these new compositions using an
old structure have become so numerous and common that they become a ‘standard’ progression
(‘referent’), often divorced from the original melody. Perhaps the most prominent example of this
41
is the chord changes for the George Gershwin song I Got Rhythm,98 which has been used as the
basis for so many compositions99 that they are often simply known as Rhythm Changes [90, 415].
However, numerous variations on this idea have been explored in different musical cultures, and
the ‘referent’ could even be a “series of arhythmic pitch or sound structures” [43, xii]. So, even
though some Western improvisers have attempted “to avoid the use of any model” [120, 17], in
truth “total nonreferentiality is almost impossible” [43, xiv].
Although much Western Art music written after 1750 has required a strict “adherence to the
notes of a composition as written down by the composer” [83, 3], earlier music, such as that
of the Renaissance and Baroque periods, often relied heavily on improvisation of the performer.
While scores from these eras often looked simple, “what is seen on the printed page was often
merely an outline to be amplified in performance according to regularized patterns of improvised
embellishment” [83, 3]. This is, for example, evidenced by the widespread use of figured bass, a
technique whereby the composer would indicate the bass note, and the intervals to be played above
it, but leave the actual voicing of the chord to the performers discretion.
The use of melodic lines as a ‘referent’ for improvisation forms the basis of much traditional
Irish folk music. Instrumental musicians often learn a repertoire of relatively simple melodic
lines,100 and then in performance will embellish these melodies using ornamentations, both rhyth-
mic and melodic, varying accents, and microtonal changes [39] [177]. While instrumentalists often
learn music intended to accompany dancing, and therefore very strict rhythmically, singers often
sing unaccompanied in a much more rhythmically free manner, a style called sean nos,101 or ‘old
style,’ singing. Although rhythmically the instrumental and vocal styles are quite different, sean
nos singing also relies on the performer improvising ornamentation on extant melodic lines [177,
628].98https://www.youtube.com/watch?v=fSTkz1BvrXY99For example: Cotton Tail by Duke Ellington, Rhythm-a-Ning by Thelonious Monk, Anthropology by Charlie
Parker, and Meet the Flintstones by Hoyt Curtin100https://www.youtube.com/watch?v=2Z_TheGgFWI101https://www.youtube.com/watch?v=Qvus7IFyFMA
mathematics, some on rhythmic structure and the power of repeti-
tion...” [32, 173]
Giant Steps,129 perhaps Coltrane’s most famous work demonstrates this mathematical fascina-
tion. The piece only modulates by intervals of a major 3rd, moving between the keys of ‘B Major,’
‘G Major,’ and ‘Eb Major,’ thus dividing the octave into three equally spaced divisions [8, 93].
Although Giant Steps is known for being based entirely on this technique of modulating in a cycle
of major or minor thirds, commonly referred to in jazz as Coltrane Changes, Coltrane had already
used the technique on multiple occasions to ‘extend’ extant harmonic progressions. For exam-
ple, Countdown130 uses Coltrane Changes to extend the harmonic progression of the Miles Davis
composition Tune Up,131 26-2132 uses Coltrane Changes to extend the harmonic progression of the
Charlie Parker piece Confirmation133 [8, 92], and Satellite134 uses Coltrane Changes to extend the
harmonic progression of the jazz ‘standard’ How High the Moon135 [182, 137]. In addition to its
use as a compositional tool, both Coltrane and numerous jazz musicians since have ‘superimposed’
Coltrane Changes onto chord progressions during their improvisations [8, 92] [105]. This super-
imposition of Coltrane Changes can be found in much of Coltrane’s own playing, for example,
Limehouse Blues,136 Summertime,137 and Fifth House138 [182, 137].
The use of a cycle of modulations by a major 3rd had already been incorporated into jazz pieces
prior to the composition of Giant Steps, for example during the bridge of the jazz standard Have
You Met Miss Jones139 [182, 136]. However, Coltrane was also inspired by Nicolas Slonimsky’s
Thesaurus of Scales and Melodic Patterns [162] [182, 136], in which Slonimsky derives 1,330
scale patterns by mathematical “methods similar to [Joseph] Schillinger” [139, 602]. Numerous129https://www.youtube.com/watch?v=30FTr6G53VU130https://www.youtube.com/watch?v=lJ7QTRzV9RM131https://www.youtube.com/watch?v=iiSDqJIQk8k132https://www.youtube.com/watch?v=JG9ygPjWdj0133https://www.youtube.com/watch?v=yXK0pZx92MU134https://www.youtube.com/watch?v=gITxsklDH58135https://www.youtube.com/watch?v=djZCe7ou3kY136https://www.youtube.com/watch?v=eTAjh1pjsFI137https://www.youtube.com/watch?v=NEftw9o1joo138https://www.youtube.com/watch?v=ngvgWUGTKYU139https://www.youtube.com/watch?v=2G8UTMtY0JM
ities” [43, 73] of their instrument, without the need for the equivalent level of technical virtuosity.
As such, AAIM shifts the performers focus from technical virtuosity, the physical ability to
perform music, to musicality, the creation of materials suitable in the context of the performance.
For example, if a very fast or ‘complex’ rhythmic subdivision is as simple to produce as an 18 th
note, then each are also equally impressive/unimpressive when taken in isolation. As such, the only
concern becomes whether or not these more ‘complex’ and fast subdivisions are suitable within the
musical context of the piece being performed. Similarly, if it is just as easy to play a melody in the
original key as it is to play the melody in a different key, using a different scale, and in retrograde,
then the physical/technical act of performing the modified melody is no more impressive than that
of performing the original melody, and the only concern becomes whether or not the manipulations
are suitable to the musical context.
Although AAIM is intended to shift a user’s focus away from technical virtuosity, and towards
musicality, its use does not necessitate a performance practice devoid of technical virtuosity. The
history of electronic music, and in fact all music, is full of examples of pioneers ‘squeezing’ more
expressivity out of an interface than had previously been thought possible. For instance, the afore-
mentioned examples of King Tubby, and his innovative use of a mixing board, or DJ Kool Herc
and Grandmaster Flash, and their transformation of a turntable from a machine for reproducing
sound into a musical instrument in its own right (see Sec. 2.1). As mentioned above, AAIM has the
potential to be controlled using any number of interfaces, the only criteria being that the interface
is capable of transmitting digital information which is within, or can be scaled to values within,
the required input range (usually 0-1, see Ch. 5). With this in mind, it is conceivable, not only
that a virtuosic performance practice with AAIM could develop using one particular interface, but
that multiple practices could develop, with each practice using a different interface and allowing
users to focus on different elements of the performance, or to use different interactions, but with
AAIM as an intermediary between the interface and sound source in each case.
This shifting of focus is also linked to another fundamental facet of the AAIM approach, the
77
previously mentioned broad musical problem space with which the system engages. As noted
above, AAIM does not attempt to emulate any one particular style, but instead affords a wide variety
of manipulations applicable across a broad number of styles. In other words, AAIM specifically
focusses on the ‘fuzzy divide’ between various electronic music practices (see Ch. 2.1). Evidence
of this is apparent in the variety of styles presented in Chapter 6, for example the contrast in style,
usage, and approach between “There is pleasure...”2 (6.1) and under scored 3 (6.2), or between
5Four4 (6.3) and radioAAIM5 (6.4.3). However, these pieces only scratch the surface of ways
AAIM can be used.
4.3 Design Philosophy
In addition to the ideas discussed above, other ideals and concepts also inspired the design phi-
losophy of the AAIM system. Firstly, AAIM makes no major differentiation between disparate
musical materials. This approach is evident on multiple levels within the system. At a surface
layer, AAIM treats all materials of one kind in the same manner. For example, exactly the same
rules are used by the AAIM.patternVary module to govern the variation of a pattern of triggers
in 54 as are used to vary a pattern in 4
4 . Likewise, when the AAIM.melodyVary module receives a
melodic line there is no additional logic governing the variation of a melodic line in the whole-tone
scale6 than there is governing the variation of a melody in ‘C Major.’7
On a deeper level, while each module affords variations unique to the type of musical ma-
terials that it engages with, the fundamental approach to how each module varies materials is
the same. This underlying similarity is based on another feature of the AAIM design philoso-
phy, that rhythm is fundamental and, therefore, AAIM.rhythmGen is the primary module of the
system (see Fig. 4.2). Although each module can work if disconnected from the rest of the sys-
2https://soundcloud.com/ps_music/there-is-pleasure3https://soundcloud.com/ps_music/under_scored-aspect_demo4https://soundcloud.com/ps_music/5four_demo_0904155https://soundcloud.com/ps_music/radioaaim-2-cheesencrackers6A scale of pitches that are each 2 semitones, 1 whole-tone, apart, e.g.: C, D, E, F#, G#, Bb7A scale containing the notes: C, D, E, F, G, A, B
tem, each has been designed with the intention that they connect to the AAIM.rhythmGen mod-
ule, and the most basic variations each module facilitates involves mapping the triggers output
by the AAIM.rhythmGen module to the materials input by the user. This can be seen in how the
AAIM.loopVary module maps triggers to the playback of a sample, shifting playback to the begin-
ning of the ‘grain’ closest to the current position, as output by the AAIM.rhythmGen module, and
how the AAIM.patternVary module varies a pattern, outputting triggers from the nearest beat in the
pattern to the current position.
Figure 4.2: Expansion of Figure 4.1, illustrating the fundamental role played by theAAIM.rhythmGen module in triggering each of the other primary modules.
While AAIM treats all musical materials in similar ways, it also engages as broad a musical
problem space as possible, and therefore must acknowledge that some variations will be better
suited to some styles than others. As such, the nature of the variations is always at the user’s
discretion. This is demonstrated by the fact that none of the weights governing an individual
AAIM.rhythmGen voice’s choice of i-o-i, inter-onset-interval, are hardcoded. Consequently, not
only can different pieces/performances use different rhythmic subdivisions, but individual voices
can generate patterns using a unique collection of i-o-i’s. This also applies to the overall prob-
79
ability of variation in the i-o-i used in, or the insertion of rests into, the rhythmic lines of each
voice. As such, the AAIM.rhythmGen module not only allows the fine tuning of rhythmic varia-
tions on a macro level, but also allows the fine tuning of variables for each rhythmic line. Thus
AAIM.rhythmGen facilitates every scenario from: an ensemble of voices creating an unvarying
rhythmic line, to one varying rhythmic line performed by an entire ensemble, to one ‘soloist’
varying its rhythmic line while the rest of the ensemble performs their original material without
variation, to an entire ensemble of voices each producing their own rhythmic lines. This applies
equally to each voice of the AAIM.patternVary, AAIM.loopVary and AAIM.melodyVary modules,
i.e. each voice can reproduce the materials input by the user, all can vary their material in unison,
or each voice can use discreet settings for each variable, and at no time are any variation methods
enforced by the system.
In their discussion of mapping strategies for computer music interfaces, Rovan et al. propose
three possible categories:
• One-to-One Mapping: The simplest, but generally least expressive, mapping strat-
egy, whereby each input controls one musical parameter.
• Divergent Mapping: One input value controls multiple parameters. Although ini-
tially more expressive than one-to-one mapping, they note that it may nevertheless
prove limited as it does not allow control over any micro features.
• Convergent Mapping: Multiple controls combine to control one parameter. Al-
though initially harder to master than other mapping strategies, ultimately it proves
to be the most expressive approach [145, 69].
Although Rovan et al. originally proposed these categories as strategies for mapping the output
of an interface to some sound source, they are equally applicable when determining strategies for
mapping input to algorithmic systems. As AAIM triggers all micro events, it makes no attempt
to provide any opportunities for one-to-one mapping, although it is possible to include one-to-one
80
mappings that bypass the system. Instead, AAIM focusses entirely on divergent and convergent
mapping possibilities. Examples of divergent mapping include the global variables for Complex-
ity and Rests, which determine the maximum variation of i-o-i’s and probability of rests for all
AAIM.rhythmGen voices simultaneously. As mentioned above, the AAIM.rhythmGen module also
affords discreet Complexity and Rests values for each voice, resulting in convergent mapping,
where both a discreet and global value couple to determine the result. The use of convergent map-
ping is even more pronounced during the process of a voice choosing different i-o-i’s, as the chance
of each i-o-i occurring is determined by an individual variable in addition to the Complexity value;
thus, numerous variables combine to produce the parameters that determine the variation in i-o-i.
An additional bonus of this focus on divergent and convergent mapping is that it facilitates both
a ‘low threshold’ and ‘high ceiling.’ In other words, divergent mappings, such as the global values
for Rests and Complexity, allow users to immediately get results from the system, while convergent
mappings, such as the discreet voice specific values for Rests and Complexity, allow experienced
users to fine tune these results to their own specifications. In turn, this facilitates many of the
previously discussed features and goals of the system, such as the shift in focus from technical
virtuosity to musicality.
4.3.1 Algorithmic Approach
As mentioned above, the algorithms within the AAIM system are based primarily on Mathematical
Models. As discussed in Section 3.1.1, algorithms based on Mathematical Models tend to be
less complex than other algorithmic approaches, and are therefore more suitable to use in live
performance. In addition to this, the fact that they do not require hardcoding of musical rules means
that they also facilitate the use of far broader, less genre specific, approaches. For these reasons,
the algorithms used within the AAIM system are primarily based on Mathematical Models, and
more specifically weighted random choices. The AAIM.rhythmGen module, for example, uses two
main mathematical models to generate rhythmic lines:
81
• The indispensability (5.1.4) algorithm, which determines the velocity of each trig-
ger, and the likelihood of rests occurring in the rhythmic line, and
• The ioiChooser (5.1.3), which determines the i-o-i’s used by each
AAIM.rhythmGen voice.
In both cases, outputs of these algorithms are determined by weighted random choices. The in-
dispensability algorithm first determines the grouping of beats in the bar, either by analysis of
the input materials, direct user input, or random choice. All of these methods of determining the
grouping rely on dividing the total number of beats into subgroups of 2 or 3 beats, and then further
grouping each stratification of subgroups into groups of 2 or 3. At each level, a simple collection
of just two rules determines the importance, indispensability, of the beats, and thus the likelihood
of rests occurring on said beats. The velocity of triggers occurring on each beat is then determined
by simply scaling the indispensability values. This approach allows the creation of rhythmic lines
with any number of beats, and with a number of possible groupings for each. This subgrouping of
total beats into groups of 2 and 3 is typical in much Western music. For example, 54 is often played
as 10 18 th notes, grouped as 3 + 3 + 2 + 2.8
Similarly, the ioiChooser algorithm takes an equally broad approach. A voice repeats its chosen
i-o-i until the i-o-i synchronises with the underlying pulse, and the choices are limited only by
the remaining beats in the phrase, the grouping of beats used by the indispensability algorithm,
and user-defined variables. The algorithm thus ensures that all of the independent rhythmic lines
generated synchronise, both with the underlying pulse and with each other, and that the variations
in i-o-i emphasise the underlying grouping. This is also typical of much Western Music. For
example, jazz ‘standards’ often consist of phrases of 4, 8, or 16 bars, and the performers will almost
always synchronise at those points, often marked by the drummer performing a cymbal crash, and
emphasise the underlying grouping (usually 2 + 2 + 2 + 2 with a swing feel). However, they
are also free to perform different rhythmic subdivisions that do not interfere with these moments8For example Take Five by the Dave Brubeck Band, Seven Days by Sting, or the Mission Impossible Theme Song
both a semitone and a tone are considered a “2nd,” and both three and four semitones are regarded
as a “3rd” etc., giving a list of diatonic intervals. The analysis also records the trajectory of each
intervallic jump, that is whether the pitch rose or fell. The system is then capable of inserting addi-
tional or alternative notes into the melodic line, limiting its choices to those that fit the underlying
scale of the melodic line, and the results of the analysis. Although this restricts the system with
regards to facilitating ideas like “start using more Major 3rds (4 semitones) than Minor 3rds (3
semitones),” it does have advantages over a fully chromatic approach. Firstly, a fully chromatic
approach would, at the very least, require either separate variables for each chromatic note and in-
terval, or enough material to determine the relationships between each chromatic note. In contrast,
the AAIM.melodyVary module only needs the melodic line itself and a variable for the chance of
inserting additional notes. Combined with the direct link between the varied output and the user’s
input, this means that AAIM affords a far broader range of styles with far more ease. For example,
if a user gives the module a melodic line containing only small intervals, e.g. “2nds” and “3rds,”
the additional and alternative note choices will also retain this feature. Similarly, when varying a
melodic line containing only large intervals, e.g. “6ths” and “7ths,” the AAIM.melodyVary module
will only insert additional or alternative notes that retain this feature.
Each of the other variation methods afforded by the AAIM.melodyVary module, playing the
melody backwards (retrograde), inverting or expanding the intervals of the melody, and creating
sequences from fragments of the melody, are also limited to the scale of the underlying pattern
and their probability is determined by a user-defined variable. Again, this limits the system with
regards to some compositional approaches, although this limitation can be diminished by changing
the underlying scale to, for example, a chromatic scale. However, by treating all melodic lines as
fundamentally ‘tonal,’10 and then affording the use of chromatic, and other, scales, it is argued that
the AAIM.melodyVary module facilitates a far broader range of styles than would be afforded if,
for example, the chromatic scale was used as the basis for all variations regardless of user input.
10In this sense ‘tonal’ is intended to imply the use of any fixed collection of 12 or fewer pitch classes, rather thanindicating the use of the Western ‘Major’ or ‘Minor’ scales in particular.
84
4.4 Improvisation with AAIM
This section situates the AAIM system within the context of the improvisational approaches pre-
sented in Section 2.2.
4.4.1 Composition - Improvisation Continuum
As discussed in Section (see Sec. 2.2.1), improvisation and composition should not be thought
of as separate processes, but should instead be viewed as a “continuous spectrum” [43, xii]. This
belief is evident on many levels within the design and approach of the AAIM system, e.g. the fact
that the user needs to input some musical material before the system outputs any music, i.e. users
must ‘compose’ some music before the system allows them to improvise.
Dean’s division of ‘pure’ and ‘applied’ improvisation (see Sec. 2.2.1) is also of particular im-
portance to the AAIM approach to improvisation, and further demonstrates the system’s dedication
to treating composition and improvisation as “part of the same idea” [120, 6]. The avoidance of
stylistically distinct algorithms allows users to focus on their own compositional ideas, rather than
forcing them to work within the limits of an existing style. The ability to fine tune the variables
for each individual voice facilitates a significant degree of control over the variations. In addition,
the intuitive mapping allows users to work in the knowledge that although the variations may dif-
fer, similar settings will always produce similar results. As such, users can explore compositional
ideas, without the need to record every moment in fear of missing a singular event that will never
happen again. Consequently, although the system is designed to facilitate ‘pure’ improvisation,
it is equally adept in ‘applied’ situations. While one could claim this of most algorithmic music
systems, the ‘stylistically neutral’ nature of the algorithms, in addition to the mapping strategies
used, makes this especially true of the AAIM system.
85
4.4.2 Referent-based Improvisation
Improvisation with AAIM is explicitly referent-based, while simultaneously being non-referential
with regards to any singular style or approach. At the most basic level, this is evident in the
use of an underlying phase ramp, upon which the AAIM.rhythmGen module creates all of the
rhythmic variations. Each of the pieces presented in Chapter 6 display this feature, however, a
number also demonstrate methods for subverting this same fact. For example, in 5Four11(6.3) the
underlying phase ramp is subverted through the use of a metric modulation from 10 equally spaced
pulses during a bar, to the impression of 4 equally spaced pulses over an equal amount of time.
5Four also features the use of ‘cross-rhythms,’ resulting in 7, 8, and 12 beats being played over
the underlying 10 pulse pattern. The subversion of the basic pulse is also evident in BeakerHead,
which is based solely on live recordings of the audience, each of which has an arbitrary length and
rhythmic character. This results in numerous sections where the underlying pulse is concealed by
the texture of overlapping samples.
The reliance on referents or models also reveals itself through the use of user-defined musical
materials. As has already been discussed, all of the variations created are based on the input
musical materials. Improvisation with AAIM is, therefore, similar in nature to improvisational
practices based on embellishing pre-composed musical materials, such as a jazz improviser does
when improvising melodic lines over a harmonic progression, rather than generating entirely new
musical materials (see Sec. 2.2).
4.4.3 Improvisation through the re-use, variation, and manipulation of ‘building blocks’
In many cases, the user-defined materials that serve as the foundation for the variation-based im-
provisation with AAIM, also provide the system with information about the nature of the embellish-
ments/variations that occur. For example, the analysis of user-defined patterns by the AAIM.patternVary mod-
ule provides information about rhythmic emphases in the materials, which is then used to determine
Figure 5.1: Example of a two bar phrase output by the rhythm generator. Blue arrows signify thechoosing of a new i-o-i, and red arrows signify triggers output by the rhythm generator.
Figure 5.2: Output of Fig 5.1 in musical notation, with the ‘base’ i-o-i equalling 18note.
90
i-o-i Repetitions of ‘base’ Repetitions of i-o-i i-o-i ratioto synchronise to synchronise
Table 5.1: Examples of the relationships between various i-o-i’s and the number of repetitions ofboth the ‘base i-o-i’ and the i-o-i itself necessary for synchronisation, using one 1
8 th note as the‘base’ i-o-i.
5.1.2 Trigger Output
The AAIM.rhythmGen module requires a repeating phase ramp equivalent in time to the length of
one ‘base’ inter-onset-interval (i-o-i).2 The module treats the ‘base’ i-o-i as 1.0, and every other i-
o-i as some multiplier, allowing the AAIM.rhythmGen to create any i-o-i, irrespective of the length
of the ‘base’ i-o-i (i.e. the tempo).
At the global level, the AAIM.rhythmGen module converts the repeated phase ramp into a
‘position’ within the metric framework:
• Position = (current beat + phase) % (number of beats * number of bars), where:
– Current beat is incremented at the beginning of each phase
The module then uses this position to facilitate the choosing of i-o-i’s as described below (5.1.3).
When a voice chooses a new i-o-i, it records the position within the rhythmic framework. It then
subtracts this beginning position from the current position, allowing each voice to treat its position
as being a distance from 0 (the beginning of it’s current i-o-i) for the purpose of outputting triggers,
whilst still retaining the rhythmic framework.
Having determined the distance from ‘0,’ each AAIM.rhythmGen voice then uses a modulus of
the ratio of:2An ‘inter-onset-interval’ is the length of time between successive musical triggers or events.
91
• The repetitions of the ‘base’ i-o-i that are necessary for synchronisation, to
• The repetitions of the chosen i-o-i necessary for synchronisation (See Tab. 5.1 and
Fig. 5.1)
At each wrap around point, the voice outputs triggers in the form:
• 〈Voice number, velocity, i-o-i, current position〉,
and a new i-o-i is chosen when the wrap around point synchronises with the beginning of a phase
ramp. This approach allows the AAIM.rhythmGen module to create any i-o-i, provided it is re-
peated until it synchronises with the ‘base’ i-o-i.
5.1.3 ioiChooser
In order for each voice of the AAIM.rhythmGen to create different rhythmic lines, the algorithm
requires a list of i-o-i’s to choose from, in the form:
〈repetitions of ‘base’ i-o-i, repetitions of the i-o-i needed to synchronise, relative complexity of the
i-o-i〉 (see Tab. 5.1).
Figure 5.3 demonstrates the mathematical method by which the AAIM.rhythmGen creates a list of i-
o-i’s consisting of any n equally spaced triggers across any m beats, while Figure 5.4 demonstrates
the AAIM.rhythmGen method for determining the relative complexity of each i-o-i. In practice,
however, there is no advantage to using either of these algorithms in real-time, and as such both
were used to generate a list of i-o-i’s and their respective complexities that is stored in the code
and loaded at launch.
To choose a new i-o-i (see Fig. 5.5), the AAIM.rhythmGen voice first discounts all of the i-o-i’s
that require more ‘base’ i-o-i’s to synchronise than the remaining beats in the phrase. Then the
algorithm scales the user set chance, or weight, for each i-o-i by: that i-o-i’s complexity level, the
complexity of the output of each of the other voices, and the maximum complexity allowed for
that voice as set by the user. The voice then further scales these weights to increase the likelihood
92
Figure 5.3: Simple example of the AAIM.rhythmGen ‘ioiMaker’ method.
Figure 5.4: Examples of the AAIM.rhythmGen ‘complexity determination’ method.
93
Figure 5.5: The decision-making process used by each AAIM.rhythmGen voice choosing new i-o-ivalues and outputting triggers.
of i-o-i’s that synchronise with the grouping of the current time signature (see Sec. 5.1.4). A user-
defined value ‘gScale’ determines the extent to which this grouping scale is enforced. Once all of
the i-o-i weights have been scaled accordingly, the algorithm removes any i-o-i with a weight of
‘0.0’ and uses the remaining list of weighted i-o-i’s as a weighted probability table from which it
chooses a new i-o-i.
5.1.4 Indispensability
The AAIM.rhythmGen module determines both the velocity of individual triggers and the possibil-
ity of rests being included in the final rhythm using a modification of Barlow’s beat indispensability
algorithm [10], “indispensability” being the importance of each individual beat within a specific
time signature. Barlow’s original algorithm stratifies the number of beats in each bar according
to prime factors of that number, using an additional algorithm to determine the ‘indispensability’
94
for each beat in a prime number. Rather than approach time signatures as a prime factor problem,
the AAIM.rhythmGen algorithm instead stratifies every time signature, and thus every number, as
groups of 2 and 3, which are in turn grouped in 2s and 3s, continuing until only a single group of
2 or 3 remains.
The AAIM indispensability algorithm first groups the number of beats in the bar by randomly
choosing 2 or 3, storing each selection, until the initial number minus the sum of the random
choices is equal to 4 or less. At this point, the only choices remaining are: 〈2,2〉 (if the remainder
equals 4), 〈3〉, or 〈2〉 (see Fig. 5.6). The advantage of this approach is that it allows the creation
of multiple groupings for any number larger than 4. For example, in Barlow’s algorithm 8 beats
can only be grouped as 〈2,2,2,2〉, while grouping in 2’s and 3’s also allows groupings of 〈2,3,3〉,
〈3,2,3〉, and 〈3,3,2〉.
A secondary advantage of this approach is that it requires only two indispensability rules,3 one
rule for a group of 2, and one rule for a group of 3, rather than a separate set of indispensabilities
for every prime number. The rules used by the AAIM.rhythmGen module are:
• Indispensability of 〈1,0〉 for a group of 2, i.e. an indispensability of 〈1〉 for the first
beat in a group of 2, and 〈0〉 for the second.
• Indispensability of 〈1,0,0.5〉 a group of 3.
Once the indispensability for each individual beat is determined, the algorithm repeats the process,
grouping the resulting number of values from the previous iteration, and adding the indispensability
for each group to the first beat of that group, repeating the entire process until only a single group
remains (see Fig. 5.7). Finally, the indispensability of each beat is normalised between 0 and 1.
This gives the indispensability for each beat in a bar of n beats, and the AAIM.rhythmGen module
then scales these values (e.g. between 1 and 127 for MIDI) to determine the velocity of individual
triggers.3To expand the possibilities of the algorithm 3 rules are actually used, allowing groups of 1 to be enforced manually.
This is not strictly necessary for any time signature with more than 1 beat, and, as such, the algorithm does not creategroups of ‘1’ unless specified by the user.
95
Figure 5.6: A schematic diagram demonstrating two alternative groupings created by theAAIM.rhythmGen ‘genGrouping’ algorithm using the same input.
96
Figure 5.7: A schematic diagram illustrating the AAIM.rhythmGen ‘indispensability’ algorithm.
97
The AAIM.rhythmGen module uses this algorithm across three rhythmic levels: to determine
the indispensability of each beat in a tuplet, the indispensability of each beat in a bar, and the in-
dispensability of each bar in a phrase of bars, with the indispensability of individual beats within
tuplets and for whole bars within phrases being scaled to have a lesser effect on the output than the
indispensability of beats within a bar. At each trigger point the AAIM.rhythmGen voice multiplies
these three values, giving the overall ‘indispensability’ with respect to phrase, bar, and tuplet. The
voice then compares this value to a random number between 0.0 and 1.0, and if the ‘indispens-
ability’ is greater than the random number a trigger is output, otherwise a rest is inserted (see Fig.
5.5). The aforementioned ‘rests’ value enables users to scale the impact of the indispensability,
increasing the lower bound of each indispensability to above 0.0 while keeping the upper bound at
1.0, thus making the insertion of rests less likely.
5.1.5 Additional Features
To expand the possibilities of the AAIM.rhythmGen module two additional features were included.
5.1.5.1 Tempo Changes
Each voice of the AAIM.rhythmGen module can use a different tempo. A voice achieves this by
multiplying the current position by the desired tempo change, and using a modulo value of the
number of beats per bar multiplied by the number of bars to ensure that the length of the rhythmic
phrase remains the same.
5.1.5.2 Timing Variations
The module also allows the timing of each rhythmic line to be offset from the repeating phase ramp
by adding or subtracting the desired deviation, set by the user, to the voices current position, again
using a modulo of the total number of beats in the phrase.
98
5.2 AAIM.patternVary
AAIM.patternVary4 is a multi-voice algorithm inspired by a standard drum machine, and was de-
signed to facilitate the variation and manipulation of patterns of triggers.
Input materials are limited to patterns based on the ‘base’ i-o-i, with each voice essentially
being a monophonic step sequencer with a range of only one note (i.e. each voice’s pattern con-
sists of a series of ‘1’s and ‘0’s, either a note-on or not). Each voice of the AAIM.patternVary is
assigned a corresponding AAIM.rhythmGen voice (see Sec. 5.1), and maps the output of said
AAIM.rhythmGen voice to its own, user-defined, sequence (see Fig. 5.8).
Figure 5.8: The decision-making process used by the AAIM.patternVary when using the output ofthe AAIM.rhythmGen to vary patterns.
5.2.1 Input and Interaction
The AAIM.patternVary module expects input triggers in the form:
〈Voice number, velocity, i-o-i, current position〉 (i.e. the same as the triggers output by the4https://www.youtube.com/watch?v=4urUEwgXfjg&feature=youtu.be
Figure 5.9: The decision-making process used by the AAIM.loopVary module when playing andvarying samples.
103
Figure 5.10: Example of how the AAIM.loopVary algorithm segments a sample into n equal parts,and how the module maps triggers onto these segmentations.
104
Figure 5.11: Example of mapping the output of the AAIM.rhythmGen module onto a sample usingthe AAIM.loopVary.
tion of loop playback:
5.3.2 Grain Size
The ‘Grain Size’ variable allows samples to be varied by causing the sample to fade-out before the
end of a grain, creating a gating effect. The user set variable is a value between 0 and 1, where 1
plays the full length of each grain. Any other value results in the AAIM.loopVary voice choosing
a value between x2 and√
x , where x is the user set variable, and multiplies this value by the ‘beat
length’ to give the grain size.
5.3.3 Reverse
The AAIM.loopVary ‘reverse’ method creates variations of sample playback by reversing the play-
back of individual grains/segments (see Fig. 5.12). The user set variable sets the chance of the
AAIM.loopVarymodule playing a grain in reverse.
105
Figure 5.12: Variations of a sample using the AAIM.loopVary ‘reverse’ method.
5.3.4 Retrigger
The AAIM.loopVary ‘retrigger’ method varies the sample playback by forcing the playback posi-
tion to jump back to the beginning of the last segment played (see Fig. 5.13), resulting in an effect
much like a DJ “scratching” a record. Again the user set variable determines the chance of the
AAIM.loopVarymodule retriggering a segment.
Figure 5.13: Variations of a sample using the AAIM.loopVary ‘retrigger’ method.
106
5.3.5 Jump
The AAIM.loopVary ‘jump’ method produces a similar, although more extreme, result to that of
the ‘retrigger’ method. However, rather than only jumping the playback to the beginning of the last
segment played, the ‘jump’ method affords both forward and backward jumps in the playback. A
second, ‘jump size,’ variable sets the maximum size of these jumps as a percentage of the sequence
length (see Fig. 5.14).
Figure 5.14: Variations of a sample using the AAIM.loopVary ‘jump’ method.
5.4 AAIM.melodyVary
The AAIM.melodyVary6 module is designed to allow users to create, manipulate, and vary melodic
lines, and makes some assumptions about the nature of the sounds that it will create, i.e. that the
material contains 12 or fewer ‘pitch classes.’7 As with both the AAIM.patternVary and AAIM.loopVary mod-
ules, each voice of the AAIM.melodyVary module is assigned a corresponding AAIM.rhythmGen voice.
The AAIM.melodyVary module is also differentiated from both the AAIM.patternVary and AAIM.loopVary mod-
6https://youtu.be/fFGgOfY545U7This does not, however, need to be 12 even divisions of the octave as the 12 pitch classes may be derived from
ules by the fact that it includes variation methods that unfold over the length of the original melody,
rather than acting upon single triggers.
5.4.1 Input and Interaction
In contrast to the AAIM.patternVary module, the nature of melodic materials means that each voice
of the AAIM.melodyVary requires a more detailed input. Each note in a melodic line requires:
• Position
• Pitch
• Length
To achieve this the AAIM.melodyVary algorithm requires input that provides information regarding
both pitch and rhythm for each step in the sequence. As such the input needs to be a list of pairs of
numbers:
• One for pitch, and
• One for trigger type, where:
0. is a rest
1. ties a note to the last note (if the pitch does not change)
2. triggers a new note
The algorithm then converts this list to a melodic line recording the beat at which a note begins,
the pitch of this note, and how many base i-o-i’s the note is to be played for (see Fig. 5.15).
Each voice of the AAIM.melodyVary module uses two separate scales to facilitate the manip-
ulation of melodic materials. The module automatically sets both to the pitch classes within the
melody. However, the user can then manually change either, or both, to expand the pitch classes
used during variations.
108
Figure 5.15: Demonstration of how the AAIM.melodyVary module converts input of pitch andtrigger type pairs into a melodic line (using the nursery rhyme ‘Mary had a little lamb’).
5.4.1.1 Available Pitches
The ‘available pitches’ are the scale upon which the AAIM.melodyVarymodule bases its variations
of the melody. For example, the melodic line in Fig. 5.15 only uses the pitches:
• 〈C, D, E, G〉
but by including the pitch:
• 〈A〉
the variations created will use the entire C pentatonic scale. Also including the pitches:
• 〈F, B〉
will result in variations using the full C Major scale etc.
5.4.1.2 Actual Pitches
The ‘actual pitches’ are the pitches onto which the AAIM.melodyVary voice maps it’s output. For
example, if, continuing from the above, the ‘actual pitches’ are set to:
• 〈C, D, Eb, F, G, Ab, Bb〉
109
the output will be in C Minor rather than C Major. To achieve this mapping the AAIM.melodyVary al-
gorithm maps each note of the ‘actual pitches’ onto the nearest pitch in the‘actual pitches’ (see Fig.
5.16).
Figure 5.16: Demonstration of how the AAIM.melodyVary module maps the list of ‘AvailablePitches’ onto the list of ‘Actual Pitches’ to create new versions of melodic lines.
5.4.1.3 Modal Mapping
To expand on the affordance of the ‘Available’ and ‘Actual’ pitches system described above, the
AAIM.melodyVary module also includes a modal mapping feature. This method rotates the inter-
vals of the ‘available pitches’ list to create a ‘mode’ of the original scale, and the module then
maps the melody and all variations onto this scale. The algorithm first records the intervals, in
semitones, in the original scale and its lowest note. It then places the first interval of the original
scale as the last interval in the new mode, and adds each interval in the new mode to the lowest
pitch in the original, giving the new mode (see Fig. 5.17).
5.4.2 Melodic Analysis
In order to facilitate the manipulations described below, each voice of the AAIM.melodyVary
performs a simple analysis of its melody (see Fig. 5.18). The module determines the distance,
and direction, between each pitch and the pitch that follows it, with the first pitch in the melody
considered to be following the last, and then converts the intervals into ‘diatonic’ scale degrees,
110
Figure 5.17: Demonstration of the modal mapping method used by the AAIM.melodyVary module.
Figure 5.18: The AAIM.melodyVary analysis of the intervallic content of the melody used in Fig.5.15.
111
retaining the list of both chromatic and diatonic intervals for future use. This approach allows the
system to capture, and therefore make use of (see Sec. 5.4.8 and 5.4.9), some of the most notable
features of a melodic line:
5.4.2.1 Interval Content
A melodic line using only small interval jumps (e.g. 2nds and 3rds) can, therefore, be varied
using only 2nds and 3rds. But by using diatonic in addition to chromatic intervals, the sys-
tem can use 2nds and 3rds that are in the scale but not in the original melody. Table 5.2 shows
the relationships between semitones/chromatic intervals and diatonic intervals as used within the
AAIM.melodyVary module. The algorithm divides any compound intervals (those greater than 11
semitones) by 12, the remainder giving the equivalent interval within an octave. The algorithm
then multiplies this remainder by 7 and adds this value to the interval to determine the actual name
of the compound interval. The algorithm discards any note repetitions and octave jumps from the
final analysis to prevent feedback loops where the AAIM.melodyVary module continuously repeats
Figure 5.25: Demonstration of the AAIM.melodyVary ‘scalar note’ method, but using the entire Cpentatonic scale rather than just the notes present in the original melody.
121
Figure 5.26: Demonstration of the AAIM.melodyVary ‘scalar melody’ method.
122
Figure 5.27: Screenshot of the AAIM.app application main screen.
Figure 5.28: Screenshot of the AAIM.app application AAIM.rhythmGen interface.
123
Figure 5.27 shows the main interface of the AAIM.app application with each of the four pri-
mary modules added. The primary performance controls for each of the modules are also clearly
shown in this figure: in the top left of the screen are the ‘complexity’ and ‘rests’ variables for
the AAIM.rhythmGen, directly below these is the ‘extra’ variable for the AAIM.patternVary. To
the right of this is a global scale for each of the AAIM.loopVary variations, and to the right of
the AAIM.loopVary variations control is the global scale for each of the AAIM.melodyVary varia-
tions. The local values, which are scaled by these global controls, for each of the primary modules
can also be fine-tuned within secondary interface screens. For example, Figure 5.28 shows the
AAIM.app AAIM.rhythmGen interface. On the right-hand side of this screenshot, one can see the
multi-sliders controlling discreet ‘complexity’ and ‘rest’ values for each voice. The simplicity of
the performance interface is augmented through a Python script that turns a simple XY controller
into an n-sided polygon, with each corner of the polygon representing a different preset. The XY
controller can then be used as normal, but with the output fading smoothly between n presets,
rather than just outputting two values. One such instance of this polygon interface is visible in
the top righthand corner of Figure 5.27. This particular instance controls the global variables for
each of the four primary modules, facilitating both swift and smooth changes between drastically
different settings using just one interface. Appendix A contains a more detailed description of
AAIM.app and its use. Sections 6.4.3 and 6.4.6 discuss examples of musical works created using
AAIM.app.
124
Chapter 6
Portfolio: Creative Practice and Selected Works
This chapter will present a portfolio of musical works that have been created and performed us-
ing the AAIM system. The chapter begins with a description of “There is pleasure...” (6.1), a
piece composed in 2014 for solo laptop performer, with an intentionally digital aesthetic combin-
ing elements of electroacoustic music and electronic dance music. under scored (2013), a piece
influenced by jazz-funk fusion, composed for either solo laptop performer, or oboe, guitar, and lap-
top performer, is discussed in Section 6.2. Section 6.3 then presents 5Four (2015), a composition
for drum set and laptop performer combining elements of electroacoustic music, electronic dance
music, and jazz. Within each of these sections, an initial explanation about the concepts and goals
of the individual piece is followed by a description of the musical materials used and the manner
in which the AAIM system facilitates the performance of the piece.
Section 6.4 presents 6 further pieces created using the AAIM system. In contrast to Sections
6.1, 6.2, and 6.3, the pieces in Section 6.4 are discussed in less detail, reflecting the fact that
the processes involved in their creation were also somewhat less in depth. However, they have
been included herein to provide additional evidence of the broad “problem space” [181, 118] with
which AAIM engages, and the variety of performance scenarios within which AAIM could be used.
Section 6.4.1 presents a second example of how AAIM can be used in a jazz fusion setting. Section
6.4.2 presents an example of the use of AAIM sound art improvisation without any predetermined
plan. Section 6.4.3 presents an ongoing series of pieces in which AAIM is used as a DJ-ing device.
In Section 6.4.4 an example of AAIM being used as a live remixing device is discussed. Section
6.4.5 presents an ambient soundscape intended as the backing for an art exhibition. Finally, Section
6.4.6 discusses a piece created solely with the AAIM.app application.
125
6.1 “There is pleasure...”
2015 International Computer Music Conference, Denton, TX [64]
2015 Forms of Sound Festival, Calgary, AB
There is pleasure in recognising old things
from a new viewpoint.
Dr. Richard Feynman
“There is pleasure...”1 is a structured improvisation, with an intentionally digital aesthetic, com-
bining elements of modern electronic dance music and computer music, with the use of melodic
and rhythmic forms as a framework for improvisation - as is common in jazz and many other mu-
sics. The performer explores the materials and sounds used from multiple viewpoints, while the
musical materials themselves explore ‘modern’ approaches to ‘old’ techniques. The goal being to
create a variety of moods, ambiances, and textures, using a minimal amount of basic materials -
or to view a limited amount of musical materials from multiple viewpoints. During the improvi-
sation, the AAIM system enables the performer to manipulate and vary the pre-composed musical
materials (see Fig. 6.2).
All of the sounds used during “There is pleasure...” are created with Frequency Modulation
(FM) synthesis, and a constant underlying pulse is used to arrange these sounds in time, two
approaches that are often viewed as dated within the genre of computer music. However, the
underlying pulse is obscured through the use of overlapping tuplets, resulting in a wide variety of
multi-layered rhythmic textures and a sense of rhythmic freedom. These variations in the basic
rhythmic patterns, together with variations in the setting of the FM synthesizers, and the use of
signal processing and sound diffusion techniques, allow the performer to also explore the sounds
from multiple perspectives. Finally, the musical materials used are also viewed from multiple
also assigned a user-defined amplitude envelope, i.e. attack, decay, sustain, and release values, and
an XY interface allows for smooth crossfading between of each of the wavetables and interpolating
between each associated amplitude envelope.
Within the synthesiser, each of the four waveforms is looped continuously. Each ‘note on’
message the synthesiser receives opens an individual voice of a variable delay pitch shifter tuned
to the pitch of the desired note, through which the combined wavetables passed. The transposed
sound is then fed through the interpolated amplitude envelope to create the note. The final feature
of the synthesiser allows the ratios that the pitch shifter uses to be changed, thus enabling a wide
variety of alternative tuning systems.
6.2.2 Musical Materials
As mentioned previously, under scored is based on four sections in the style of jazz funk fusion,
each of which is in an 114 time signature, written as 6+6+6+4
8 in the score (see Fig. 6.3). The title of
the piece was inspired by the intentionally minimal materials used, consisting simply of harmonic
progressions and melodic fragments, mimicking the nature of ‘lead sheets’ for jazz standards,
which themselves usually only contain harmonic progressions and melodic lines. The reason I
took such a minimal approach was to maximise the possibilities for improvisation, while still
ensuring a clear referent throughout.
Figure 6.3: The ‘Vamp’ section of under scored
The ‘vamp’ section (see Fig. 6.3) acts as both the beginning and ending of the piece, and an
133
improvisatory refrain returned to at various points during the piece. This section also serves as
an introduction to many of the ideas explored throughout. For example, the previously mentioned
use of an 114 time signature. Although the use of time signatures other than 4
4 are used in jazz
fusion pieces, they are often more common as a single bar within a phrase of 44 rather than an
entire phrase. In more traditional jazz styles, the majority of pieces tend to only use a 44 time
signature. The nature of the rhythmic grouping used within this section also informs much of
the later material. As mentioned above the 114 time signature is notated as 6+6+6+4
8 in the score,
indicating the cellular ‘call and response’ nature of the rhythmic grouping. This cellular approach
continues in the materials of the latter sections. However, at all times this was only used as a
conceptual guide to the materials rather than acting as a mathematical structure for the piece. Also
apparent in the ‘vamp’ section is the absence of ascending perfect fourths in the root movements
of the chords. Again, this idea is explored throughout the piece, and places it at odds with many
traditional jazz styles - wherein these changes are amongst the most commonly used, and form the
basis of two of the most common ‘jazz progressions’: ‘ii - V - I’ and ‘iv - ii - V - I.’ This approach
lends a more ‘modal’ feel to the music and allows shifts between distant modalities, in this case,
the most prominent being ‘E minor,’ but the harmony shifts to modalities linked with ‘Bb’ and ‘F’
major. Finally, the diminution of the chord durations as the bar progresses, 6 quarter notes for the
‘E minor,’ followed by 112 quarter notes for the ‘G’ and ‘Eb’ chords, and 1 quarter note each for
‘Bb’ and ‘D minor,’ is echoed in the diminution and augmentation of subsequent sections.
The ‘A’ section of under scored (see Fig. 6.4) expands upon the harmonic movements used
during the ‘vamp’ section, again avoiding ascending perfect fourths in the roots and shifting from
‘E minor’/‘G major’ to more distant modalities linked to ‘Bb major.’ As mentioned above, the
material is intended to be cellular in nature, forming a hierarchy of ‘call and response’ phrases.
The first five notes of the melody forming a ‘call’ to the ‘response’ of the following five, these
ten notes then forming the ‘call’ to the ‘response’ of the final five notes in the first bar. The first
bar then forms a ‘call’ to which the second bar is the ‘response.’ This call and response phrase
134
Figure 6.4: The ‘A’ section of under scored
is then repeated and followed by the final bar, forming another call and response phrase. The
diminished nature of this final response phrase again links to the cellular character of the rhythm
itself, mimicking the final four beat grouping in the time signature, which is diminished with
respect to the groups of 6 that precede it.
The ‘B’ section of under scored is primarily based on a descending chromatic base line
variant of a passus duriusculus, beginning on ‘B’ and ultimately ending a tritone below on ‘F.’
Beginning on a ‘B, minor,’ the harmony moves away from the ‘E minor’-centric nature of both the
‘vamp’ and ‘A’ sections. The ‘call’ of the initial ‘B minor’ chord is answered with a ‘Bb major,’
and then this whole ‘call and response’ phrase is transposed down one whole step, ‘A minor’ and
‘Ab major,’ thus forming another ‘call and response.’ No melody is written for these first 2 bars of
the ‘B’ section, allowing either a break in the melodic line or a brief improvised phrase to form a
‘call’ before the subsequent ‘response’ of the melody. The melodic line again forms a hierarchical
order of calls and responses, and as with the ‘A’ section, the final bar of the ‘B’ section acts as
the ‘response’ to the ‘call’ of the preceding materials. However, in contrast to the ‘A’ section,
the harmonic movement in the ‘B’ section is far slower, changing only once every 5 or 6 quarter
135
Figure 6.5: The ‘B’ section of under scored
notes during the first 4 bars, and once every 3 or 2 quarter notes during the final bar, echoing
the previously mentioned diminution and augmentation of the chord durations during the ‘vamp’
section.
%% %%%%% %%%% %%%% %%%%% %%%%% %%
%%
%% %%%% %%%
%
8
''8
''
((((
( $($( $( $* $( * $* $*$((( $($( $( $
($( $ ($($($( $($( $(( $ ((( $($( $
($($($($( $( $($( $ ,
----------
2
13
Transition10
Figure 6.6: The ‘transition’ section of under scored
The ‘transition’ section of under scored (see Fig. 6.6) is again based on a hierarchical ‘call
and response’ approach, and echoes the descending whole tone transposition used during the ‘B’
section. Although in this case, the transposed ‘response’ is not an exact replication of the preced-
ing material - a ‘Bb major’ chord substituting for the expected ‘G minor’ chord. The harmonic
movement again avoids steps of an ascending perfect fourth, instead favouring descending perfect
136
fourths and movements of a third, in addition to a descending semitone movement during the final
bar. The final bar of this section also provides another example of the diminution and augmentation
of chord lengths, while the total length of this section, three bars, acts as a diminution of the five
bar phrases in sections ‘A’ and ‘B.’ This section contains no melodic line, as it serves as a transition
during improvisations, either between the ‘vamp’ and ‘A’ sections or between two ‘A’ sections.
Although, as previously stated, the laptop performer is given the ability to arrange the sections
in any order he/she chooses during performance, the sections themselves are intended to imply a
similar cellular ‘call and response’ form to that of the materials contained within. Both the ‘vamp’
and ‘A’ sections are suitable for repetition, while both the ‘B’ and ‘transition’ sections are more
suited to acting as a ‘response’ to repetitions of either the ‘A’ or ‘vamp’ sections. Such an approach
can be seen in Figure 6.7, which is a rough reproduction of the form used by the Aspect ensemble,4
when performing under scored at the 2014 joint International Computer Music Conference and
Sound and Music Computing conference in Athens.
6.3 5Four
2015 Music for Sounds Concert, Calgary, AB
5Four is written for drum kit and laptop performer,5 and draws influence from jazz, electronic
dance music, and computer music. 5Four explores the interaction between a live drummer and
the digital equivalents: drum machines and samplers, aiming to explore the interplay between the
fluidity of a live drummer and the “exaggerated virtuosity of the machine” [42, 2]. Sonically, the
piece combines traditional drum kit performance practice with sounds generated through extended
techniques. This accentuates both the quasi-pitched and inharmonic sounds the drum kit is capa-
ble of producing, while still retaining the rhythmic approach that has seen the drum kit become
one of the primary instruments in many 20th and 21st century styles of music. Throughout the
4Aura Pon - oboe and electronics; Simon Fay - guitar and electronics; Lawrence Fyfe - tablet controller and theAAIM system. http://innovis.cpsc.ucalgary.ca/Research/Aspect
radioAAIM is a series of pieces12 created using the AAIM.loopVary module, within the AAIM.app ap-
plication, as a ‘DJ’ device. Each piece in the series uses a selection of samples taken from various
pieces by other composers/bands. These samples were then mixed live to create a quasi DJ set
or mash-up, similar to Grandmaster Flash’s piece The Adventures Of Grandmaster Flash On The
Wheels Of Steel.13 In contrast to many, although by no means all, DJ sets and live mash-ups, each
of the radioAAIM series focuses not only on playing and mixing the original materials but also on
the variation and manipulation of the source material live during the performance.
Rather than use a score or plan of any kind, I create the form of each of these pieces organi-
cally in performance, a result of practice with both the loops and the system combined with im-
provisatory intuition. Instead, the ‘referent’ for each piece consists of a ‘bank’ of loops, which all
‘work’ together in the context of that particular radioAAIM piece. For example radioAAIM 2am
uses a collection of samples taken from both ‘popular’ and ‘art’ electronic music compositions,
an attempt to combine the ‘art’ and ‘popular’ styles of electronic music. In contrast, both ra-
dioAAIM cheeseNcrackers and radioAAIM cheeseNcrackers2 consist of a selection of funk and
soul pieces. These stylistic differences in the materials used also influenced a different approach
to the use of the AAIM system during performance. The nature of the material used in ra-
dioAAIM 2am suggested a rather extreme use of the AAIM.rhythmGen and AAIM.loopVary mod-
ules when varying the loops, while a more subtle approach was used in both radioAAIM cheeseNcrackers
and radioAAIM cheeseNcrackers2. Further differences to a typical DJ set afforded by the AAIM sys-
tem include the facility to create polyrhythmic textures using the original materials, examples of
which can be heard in both radioAAIM cheeseNcrackers and radioAAIM 2am.12https://soundcloud.com/ps_music/sets/radioaaim13https://www.youtube.com/watch?v=8Rp1oINrEuU
The main contribution of the AAIM system is the portfolio of algorithms covering the generation
of musical rhythms, and the mapping of these rhythms onto musical materials: drum patterns,
melodies, and sample playback. By using these rhythms to manipulate and vary user-defined
materials in real-time, AAIM facilitates the live performance and improvisation of electronic music
in a vast range of genres with widely differing degrees of complexity/variation.
This chapter will evaluate the AAIM system’s success in realising these contributions. Section
7.1 reiterates the approaches used in the AAIM system, which have been discussed in detail in
previous chapters. Section 7.1.1 then addresses how AAIM achieves the four criteria for musical
algorithms given by Simoni and Dannenberg: simplicity, parsimony, elegance, and tractability
[160, 4]. Section 7.2 presents evidence of the AAIM system’s ability to facilitate live performance.
Section 7.3 then discusses evidence of the system’s ability to afford improvisation. Finally, Section
7.4 discusses how the pieces discussed in Chapter 6 demonstrate the broad problem space with
which AAIM engages.
7.1 The AAIM system
The approach used by the AAIM system is firstly evident in the algorithms used by the AAIM.rhythmGen mod-
ule. The expansion of Barlow’s Indispensabbility algorithm [10] to work with groups of 2 and 3
beats, as opposed to prime number subdivisions, enables the AAIM.rhythmGen module to create
a far more comprehensive range of rhythmic groupings (see Sec. 5.1.4). The AAIM.ioiChooser
algorithm (see Sec. 5.1.3), is also, to the authors knowledge, unique, in that it makes possible any
rhythmic subdivision, in any time signature, and determines which i-o-i to use purely on the basis
of user set variables.
148
The AAIM.rhythmGen module’s approach to generating rhythms necessitates a complementary
approach in mapping these rhythms to musical materials, and this can be seen in how the other three
primary modules generate their most basic variations (see Sec. 5.2, 5.3, 5.4). Even as the simplest
of the primary modules, the AAIM.patternVary module’s Extra feature (see Sec. 5.2.3) provides a
method for inserting additional triggers into a pattern, while still retaining the fundamental nature
of the original.
The mapping approach used by AAIM is most apparent in the AAIM.loopVary module (see Sec.
5.3). The module’s treatment of every sample as a sequence of equally spaced and sized grains not
only allows the sequence, and thus the sample, to be easily rearranged, but also allows the module
to easily translate the rhythmic lines output by the AAIM.rhythmGen module into changes in the
sample playback position without changing the total length of the loop. In addition to this, it means
that the playback of individual grains can be easily reversed, without needing to reverse the entire
loop.
The AAIM.melodyVary module’s melodic analysis (see Sec. 5.4.2), Scalar Note (see Sec.
5.4.8), and Scalar Melody (see Sec. 5.4.9) methods demonstrate the module’s unique approach
to melodic variation. Rather than analyse melodic lines according to chromatic intervals, the use
of tonal intervals and trajectory allows the module to infer more musical information from less
musical input. In turn, this analysis enables the Scalar Note method to insert additional, or alter-
native, notes into a melodic line, which retains the fundamental features of the original line. The
Scalar Melody method is even more unique, in that it expands upon the Scalar Note method to
create entire melodic lines that retain the fundamental features of the original.
7.1.1 Criteria for Musical Algorithms
In his 1973 monograph, Donald Knuth presents a list of aesthetic criteria for the evaluation of an
algorithm [97]. From Knuth’s list, Simoni and Dannenberg selected four in particular that musical
algorithms should “strive to meet” [160, 4]:
149
• Tractability
• Simplicity
• Parsimony
• Elegance
In this section, the AAIM system will be evaluated according to its success in meeting these criteria.
It is important to note, however, that in many scenarios these criteria were in conflict, either with
each other or with the goals of the software, and as such a balance often had to be found.
As discussed in Chapter 4, one of the primary ideals of the AAIM system is that the relationship
between the user’s input and the system’s output be as intuitive as possible. As such it can be seen
that ‘tractability’ has been a guiding ideal in the design of AAIM since its earliest conception.
This has been achieved by ensuring that all variations are directly linked to user inputs, be they the
original musical materials or variables. Although the system primarily uses weighted probabilities,
these weights are always set by the user.
The ‘simplicity,’ ‘elegancy,’ and ‘parsimony’ of the AAIM system has also been discussed at
various points herein. For example, in addition to affording a far greater number of rhythmic
groupings, the use of groups of 2 and 3 in the AAIM.rhythmGen Indispensability method (see Sec.
5.1.4) also simplifies Barlow’s original algorithm, as it requires only two rules, rather than an in-
dividual rule for every prime number. This reliance on just two rules is also clearly parsimonious,
and the method’s ability to use these two rules to generate groupings in any time signature demon-
strates the elegant nature of the method. The use of groups of 2 and 3 exemplifies the occasional
conflicts resulting from using these four criteria. The method could be further simplified by only
using groups of 1. However, this would limit every number to only one grouping (1+1+...), thus
severely limiting the capabilities of the system. It could also be argued that such a limitation would
also make the algorithm both less elegant and less parsimonious.
150
The AAIM.rhythmGen method for creating i-o-is also meets these criteria. The modules’s equal
treatment of every i-o-i enables it to create any i-o-i, but only requires knowledge of:
• The number of repetitions of the underlying pulse needed for synchronisation, and
• The number of repetitions of the i-o-i itself required for synchronisation.
The AAIM.rhythmGen ioiChooser (see Sec. 5.1.3) method further demonstrates how the system
has met these four criteria. When using this method to choose an i-o-i, each AAIM.rhythmGen voice
only needs knowledge of the above constants, the number of beats left in the rhythmic phrase, and
user-defined variables.
Each of the other primary modules also fulfil these criteria. For example, the
AAIM.patternVary module only requires the user’s pattern and variables for the probability of
additional notes being inserted to create variations of any pattern in any time signature. The
AAIM.loopVary module only requires information about the number of grains to divide a sample
into to afford variations of any audio recording, and the AAIM.melodyVary module allows varia-
tions of any melodic line to be created, but only requires the melodic line itself and a handful of
variables determining the nature of the variations.
7.2 Live Performance
As all of the pieces presented in Chapter 6 were created in real-time using AAIM, they all demon-
strate the AAIM system’s capability to facilitate live performance. However, AAIM has also been
employed in many concerts. For example, I was invited to perform“There is pleasure...” (6.1) at
both the 2015 International Computer Music Conference in Denton, TX, and the 2015 Forms of
Sound Festival in Calgary, AB. I was also invited to perform 5Four (6.3) at the Music for Sounds
concert in Calgary, AB.1 The Aspect ensemble2 were invited to perform under scored (6.2) at
1Ethan Cayko - drumset; Simon Fay - the AAIM system2Aura Pon - oboe and electronics; Simon Fay - guitar and electronics; Lawrence Fyfe - tablet controller and the
AAIM system
151
the 2014 joint ICMC and SMC conference in Athens, Greece, and I was also invited to perform
under scored as a solo laptop performer, at the 2014 UofC Graduate Students Composer Concert
in Calgary, AB. As part of an artist exchange with Sonic Arts Waterford,3 I was invited to perform
at the opening of Polish artist Mateusz Bykowski’s exhibition You see what you want to see4 at
Kolonia Artystow, in Gdansk, Poland, in 2013 (see Sec. 6.4.5). Finally, myself and Ethan Cayko
were invited to perform at the 2015 Beakerhead festival in Calgary, AB (see Sec. 6.4.2).5
7.3 Improvisation
Each of the piece presented in Chapter 6 also demonstrate the AAIM system’s capacities to fa-
cilitate improvisation. However, the improvisational capabilities of the system are perhaps most
apparent in those pieces that do not rely on any predetermined form or material. For example, the
performance of BeakerHead involved no preconceived notions of form or materials, as the entire
performance used recordings of the audience at the 2015 Beakerhead festival. As such improvisa-
tion was essential in turning otherwise non-musical materials, i.e. people speaking, into the basis
for a musical performance. The radioAAIM (6.4.3) series also demonstrates the improvisational
affordances of the AAIM system. While the materials used in the series are inherently more ‘mu-
sical’ than those employed in BeakerHead, the lack of any predetermined score necessitated an
improvisational approach to both the materials used and the overall form. Similarly, as no prede-
termined form was used in new beginnings, the improvisational capacities of AAIM were vital in
facilitating the performance.
While each of the other pieces discussed in Chapter 6 rely on more defined forms and materials,
the improvisational capabilities of AAIM are evident in the variations of the underlying materials
that can be heard throughout. This form of improvisation, the elaboration and ornamentation of
musical ‘building blocks,’ has been discussed in detail in 2.2, and its relationship to improvisation
3http://www.sonicartswaterford.com/4http://www.kolonia-artystow.pl/article/mateusz-bykowski/2245Ethan Cayko - MIDI controller and the AAIM system; Simon Fay - tablet controller and the AAIM system
This thesis has described the development of the AAIM performance system, a portfolio of in-
terconnectable algorithmic software modules designed to facilitate improvisation and live perfor-
mance of electronic music. The primary inspiration for this research was a desire to exploit the
“potential of computer music to exceed human limitations” [34, 142] within live performances,
and a belief that algorithmic performance systems have the capacity to augment human creativity
in live performance and improvisation. To examine the effectiveness of the AAIM system in meet-
ing these goals, I created a portfolio of musical works using the system. These works explored
the affordances of the system in a variety of styles and genres, and many were performed in live
concerts.
The development of AAIM led to a number of contributions. This chapter describes those
research contributions (8.1) along with some ideas for future research (8.2). The chapter ends with
some personal reflections (8.3) and closing remarks (8.4).
8.1 Contributions
During the course of this research I answered the challenges described in Sec. 1.5. Those answers
resulted in the contributions of this research.
• Portfolio of algorithmic modules for the variation of musical materials
The main contribution of the research presented in this thesis is the AAIM system, a
portfolio of algorithms facilitating the variation and manipulation of rhythms, pat-
terns, melodies, and sample playback in electronic music. The main contributions
of the AAIM system are:
154
– Variation of user-defined materials1
A fundamental tenet of the AAIM system is that the musical output
should be directly linked to the aesthetics of the user, or to quote
Morton Subotnick “the medium should always be at the service of
the artist. [168, 113]. To this end, each of the AAIM modules is
concerned with allowing the user to vary and manipulate their own
musical ideas.
– Facilitate Live Performance2
A further principle of AAIM is that the system should be capable of
working in live performances. This was achieved by only develop-
ing algorithms that could run in real-time, refining the algorithms to
improve their speed and efficiency, and incorporating divergent map-
ping techniques, where one variable effects many features, allowing
users to control the systems with a minimal number of controls. To
evaluate this capability of the AAIM system I wrote and performed
a portfolio of creative works using the system (see Chap. 6).
– Facilitating Improvisation3
As discussed in Section 2.2, there are a wide range of different ap-
proaches to improvisation in music, and the extent to which a elec-
tronic music system affords improvisation is debatable and “prob-
ably depends on the individual as well as the software” [43, xvii].
The form of improvisation facilitated by the AAIM system involves
1The use of AAIM to vary user-defined materials can be heard in any of the works contained in the portfolio ofcreative works. For example, in the variation of the melodic and harmonic material in under scored [58], or thevariation of prerecorded samples in 5Four[61].
2AAIM has been used in many concert performances. Both by the author: [70],[66],[69],[68], and [71]; and byother performers: [65] and [67].
3All of the works contained in the creative portfolio demonstrate AAIM’s potential to facilitate improvisation.Perhaps the most obvious examples are radioAAIM[63], BeakerHead[72], and new beginnings[57], all of which wereperformed with no preplanned score or progression.
155
the embellishment and variation of musical materials, an approach
that is common in many styles (see Sec. 2.2.2 and 2.2.3). As Dean
noted, interaction and improvisation with algorithmic systems form
a continuum, and one relevant factor is “the degree to which users
can learn to exploit the capacities of the algorithm (versus the de-
gree to which the algorithmic changes are indifferent to the nature
of user input)” [43, 73]. Despite these inherent uncertainties the
AAIM strives to maximise its improvisational potential through in-
tuitive mapping of variables and the resulting variations. The port-
folio of creative works presented in Chapter 6 also demonstrate this
improvisational potential, as discussed in Section 7.3.
– Stylistically Neutral/Broad Problem Space4
The final contribution of the AAIM system is the broad problem
space with which it engages. Rather than attempt to create one in-
dividual style, each of the modules comprising the AAIM system
instead use generic methods suitable to many styles. By allowing
the user control over what variation methods are possible, and fo-
cussing on varying user-defined materials, the AAIM system max-
imises the extent to which the user is in control of the aesthetic out-
come. Evidence of this broad problem space is apparent in the range
of styles and approaches presented in the portfolio of creative works
(see Chap. 6).5
• Portfolio of creative works6
In addition to providing evidence for the extent to which the system meets the con-
4The range of styles AAIM facilitates is apparent in the differences between under scored [58], “There is plea-sure...”[59], and 5Four[61].
• Use the AAIM Setup window (see Fig. A.2) to load the three main modules of the
AAIM system:
– AAIM.patternVary (see Sec. A.1.2)
– AAIM.looper (see Sec. A.1.3)
– AAIM.melodyVary (see Sec. A.1.4)
———————————— NOTE ————————————
Any changes made using “AAIM Setup” patch, any changes to the number of voices
for any of the main modules, and all those made using the
AAIM.FXModule1 (see Sec. A.1.6), are scripted automatically. It is not advis-
able to do this, or to create/delete voices within any of the modules, during live
performance as it will likely cause disontinuities in audio and timing
181
• Press the “esc” key on your keyboard or click the button with the picture of the
speaker in the main window/performance Interface (see Sec. A.0.2 and Fig. A.3)
to turn on both audio and the system.
• In all patches/interfaces voices are counted from left to right, or from bottom to top.
• AAIM.rhythmGen (see Sec. A.1.1) always organises the voices of the other mod-
ules in the same way:
– AAIM.patternVary voices first
– AAIM.looper voices second
– AAIM.melodyVary voices third
• Time Signatures
– The AAIM system makes no differentiation between time signatures
- i.e. there is no special treatment/consideration given to any particu-
lar time signature. Therefore, variables which work well for certain
material in 44 will work equally well with similar material in 5
4 ,98 ,
etc.
– Generally it works best to treat bars of 44 as bars of 8
8 , bars of 54 as 10
8
etc.
– Although AAIM does not currently afford phrases within which the
time signature changes(i.e. 3 bars of 44 and a bar of 3
4 ) equivilant re-
sults can be achieved by summing the total beats desired and treating
it as one bar (i.e. one bar of 154 ).
– Both the “patternVary” and “melodyVary” algorithms attempt a low
level analysis of the rhythm of the materials they are given in order
to determine the grouping of beats in the material. This information
182
is then used by the “rhythmGen” algorithm to influence the choice of
inter-onset-intervals and insertion of rests (see Fig. A.7), in addition
to the velocity of each individual note/trigger.
183
A.0.2 Performance Interface
Figure A.3: The AAIM system performance interface
Although each of the modules have individual interfaces, the AAIM system performance in-
terface (see Fig. A.3) is intended as the main performance interface. The left hand side of the
interface provides controls over variable relating to each the 4 main modules and are discussed in
Sections A.1.1, A.1.2, A.1.3, and A.1.4.
• The central section of the interface provides some non performance related controls
(see Fig. A.4) .
– The panel at the top of this section of the interface displays the cur-
rent beat and bar (i.e. current position in the rhythmic phrase) in
addition to giving control over the basic tempo in beats per minute
(bpm) and the relative tempo.
– Below this, within the red panel, are buttons which open the “Audio
Settings” patch, the “keyboard Shortcuts” patch, general “Notes”
184
patch, and the AAIM Setup patch (see Fig. A.2). There is also but-
ton labelled “Recorder” which opens a patch that allows recording
of all audio output from the system.
Figure A.4: The central section of the AAIM system performance interface
• At the bottom right of the interface (see Fig. A.5) are the gain sliders and FX buttons
for each of the audio modules, in addition to buttons for opening the AAIM.FXModule1
(see Sec. A.1.6) for each.
Figure A.5: The audio section of the AAIM system performance interface
• The top right hand corner of the AAIM system performance interface provides the
main performative controls (see Fig. A.6).
– The polygon interface (see Sec. A.1.7) enables interpolation be-
tween multiple ‘preset’ states for the sliders labelled “complexity,”
“rests,” “patternVary extra,” “looper Variations,” and “melodyVary
185
Figure A.6: The polygon interface section of the AAIM system performance interface
Variations.” This allows swift changes between a wide variety of
system states.
· Pressing the button labelled “Presets” opens the AAIM.polygonInterface
presets patch (see Sec. A.1.7).
· Each preset contains 5 values:
· complexity
· rests
· patternVary Variations
· looper Variations
· melodyVary Variations
– To the right of the polygon interface the grey numbered buttons
recall ‘Section Presets’ (see Sec. A.0.3), while the red numbered
boxes store the systems current settings as a Section Preset. The
button labelled “write” saves the current Section Presets to file, and
the button labelled “read” opens a Section Preset file.
186
A.0.3 Saving/Loading Section Presets
• Section Presets can be saved and loaded using the top right corner of the perfor-
mance interface (see Fig. A.6).
• Once a section preset file has been read it is necessary to press the red button to
the right of of the “read” button in order to initialise all of the scripting (i.e. to
load each module). This may take 4/5 presses of the button in order to ensure all of
the information has been loaded (unfortunately automatic loading does not work...),
however once all of the modules used are loaded another file of section presets can
be loaded without this second step being necessary provided it uses an identical
collection of modules...
• Data included in these presets include all scripting data, therefore care should be
taken when saving presets. If a preset, for example, uses a different FX module to
previous presets this FX module will be created each time that preset is loaded and
destroyed each time another preset, which does not use said FX module, is loaded,
likely causing both discontinuities in the audio and delays in timing. As such each
section preset for a piece should use all of the same modules!
• If preset sections involve changes of time signature/tempos etc. best results will be
garnered if said sections are changed at the very end of a phrase (determined by the
“nBars”).
• Sample folders are not loaded with Section presets. Therefore once all of the mod-
ules have loaded it is necessary to manually load samples (see Sec. A.1.2.4, Sec.
A.1.2.6, and Sec. A.1.3), and then press the red button a number of times again.
• AAIM system comes with some example presets to demonstrate some of the fea-
tures of the system (see Sec. A.2)
187
A.1 AAIM Modules
A.1.1 AAIM.rhythmGen
Figure A.7: The AAIM.rhythmGen interface on system load
AAIM.rhythmGen is an algorithmic system for the generation of rhythms, the output of which
is used to trigger sounds using the other modules of the AAIM system.
To open the main interface click the button labelled “rhythmGen Interface” on the main screen/performance
interface (see Sec. A.0.2).
A.1.1.1 Global Controls
The top left of the interface provides control over global aspects of the module (see Fig. A.7). The
number of rhythmic lines generated are determined using the number box labelled “nVoices” (this
will automatically update when modules/voices within modules are created or destroyed). The
number box labelled ‘nBars’ determines the number of bars in the rhythmic patterns created(all
voices will always synchronise on the first beat of the first bar), and the number box labelled
‘nBeats’ determines the number of beats in each of these bars. The drop down menu labelled ‘base
ioi’ determines the fundamental rhythmic values of all of the rhythmic lines - all rhythmic lines
will be related to a constant pulse of this value in the selected tempo (see Sec. A.0.2).
188
A.1.1.2 inter-onset-interval probabilities
The final global control is a multislider labelled ‘inter-onset-interval probabilities.’ An inter-onset-
interval (ioi) is the length of time between successive musical onsets or triggers. In the AAIM
system these are represented as a fraction or ration of the ’base ioi,’ with 1.0 being equal to the
base ioi. This multislider sets the probability of each ioi occurring during a rhythmic pattern.
Whenever a voice chooses an ioi it is repeated until it synchronises with the underlying base ioi
- i.e. a ioi of 0.8 will be repeated produce 5 equally spaced triggers of the course of 4 beats
(quintuplets), a value of 0.75 will produce 2 triggers over the course of 3 beats (dotted rhythms),
0.667 will produce 3 triggers over the course of 2 beats (triplets), 0.571 will produce 7 triggers
over 4 beats etc (septuplets). Higher values will produce fewer triggers over a longer time, and
lower values will produce more rapid fire triggers.
The AAIM.polygonInterface (see Sec. A.1.7) provides a selection of preset i-o-i probabilities:
1. Basic i-o-i’s
2. Basic i-o-i’s 2
3. Triplets
4. Basic i-o-i’s 2
5. Quintuplets
6. Triplets 2
7. Septuplets
A.1.1.3 Individual Voice Controls
The controls in both the bottom left and top right of the interface correspond to values for each
individual rhythmic line generated.
189
A.1.1.4 tempo
The multislider labelled ‘tempo’ in the top right corner of the interface shifts the overall tempo of
individual voice in relation to the base ioi. The values available are:
• 0 - regular tempo
• 1 - half time
• 2 - double time
• 3 - triple time
Figure A.8: The AAIM.rhythmGen controls on the main window
A.1.1.5 rests
The multislider labelled ‘rests’ sets the probability of rests being inserted(individual trigger points
not resulting in triggers) into each rhythmic line. These values are then scaled using the ‘rests’
slider in the performance interface (see Fig. A.8 and Sec. A.0.2).
A.1.1.6 complexity
The multislider labelled ‘complexity’ scales the probability of individual voices inserting ioi’s not
equal to 1.0 into their own rhythmic lines. These values are then scaled using the ‘complexity’
slider in the performance interface (see Fig. A.8 and Sec. A.0.2).
190
A.1.1.7 timing deviations
The multislider labelled ‘timing deviations’ (bottom left of the interface) sets micro deviations in
timing for each voice - shifting the timing of each voice to land slightly before or after the under-
lying beat. These values are then scaled using the ‘timing deviations’ slider in the performance
interface (see Fig. A.8 and Sec. A.0.2).
A.1.1.8 voice on/off
The multislider labelled ‘voice on/off’ (bottom left of the interface) turns individual rhythmic
voices on or off (NOTE: This may result in minor delays when turning voices on in order to ensure
synchronisation)
191
A.1.2 AAIM.patternVary
Figure A.9: The AAIM.patternVary interface on system load
The patternVary algorithm is used for playing and manipulating drum machine style patterns
(see Fig. A.9)
• In the “AAIM Setup” window (see Fig. A.2) check the box labeled “patternVary”
• The “AAIM Setup” window should change to reveal a menu (Initially showing
“MIDI”) and a button(initially labelled “MIDI Out Interface”)(see Fig. A.10).
Use this menu to choose the destination of the messages(see Section A.1.2.1)
• In the main window(which should have resized to reveal a new section of the inter-
face) click the button labelled “patternVary Interface”(see Fig. A.11)
• Create a pattern using the interface provided(voices are counted from bottom to
top, while time (i.e. beats) are counted from left to right) (see Fig. A.12)
• In addition to mapping rhythms/triggers generated by the AAIM.rhythmGen algo-
rithm AAIM.patternVary is also capable of inserting additional triggers into a pat-
tern. The likelyhood of this happening is influenced by 3 factors:
– The number of notes in the given pattern for an individual voice (see
Fig A.12). i.e. if no notes are given for a particular voice no notes
192
will be added, and the more notes there are for an individual voice
in a given pattern the more likely extra notes will be triggered in that
voices.
– The chance of extra notes being added for each individual voice as
determined by the “voice Extra” multislider within the patternVary
interface (see Fig. A.13)
– Finally the overall chance of additional notes being inserted into the
pattern is scaled using the “patternVary Extra” slider in the main
window (see Fig. A.11 and Sec. A.0.2). This value is the final word
on the extent of the variations (i.e. if the value is 0 no additional
notes will be inserted regardless of the other factors!!!)
Figure A.10: The AAIM Setup following initial toggling of the patternVary algorithm
Figure A.11: patternVary performance interface on AAIM System main window
193
Figure A.12: Interface used to create patterns which are subsequently played/manipulated usingthe patternVary algorithm
Figure A.13: Interface used to determine the chance of additional notes being inserted into thegiven pattern for each individual voice
A.1.2.1 AAIM.patternVary trigger destinations
Output from the AAIM.patternVary algorithm can be used to trigger sounds using a number of
methods. The particular method used is chosen using the drop down menu in the AAIM Setup
window (see Fig. A.10).
A.1.2.2 MIDI
Triggers from the AAIM.patternVary algorithm can be sent to any MIDI device/program using the
AAIM.midiOut subpatcher (see Fig. A.14)
A.1.2.3 OSC
Triggers from the AAIM.patternVary algorithm can be sent to any OSC capable device/program
using the AAIM.OSCOut subpatcher (see Fig. A.15)
194
Figure A.14: Interface used for output of MIDI from AAIM
Figure A.15: Interface used for output of OSC messages from AAIM
A.1.2.4 AAIM.samplePlayer
The AAIM.samplePlayer patch (see Fig. A.16) uses the output of the AAIM.patternVary to trigger
samples from a given folder. The user must drag and drop the folder of samples onto the designated
area of the interface (NOTE: Only audio samples should be present within the folder as otherwise
the system will attempt, and fail, to treat other files as audio files!!!) and each sample will be
automatically assigned to an individual voice.
Figure A.16: Interface used for AAIM.samplePlayer, a patch which allows the triggering of anarbitrary number of samples-with each sample being triggered by a single voice
195
A.1.2.5 sequenceMaker
The sequenceMaker output of the patternVary uses the AAIM.sequenceMaker patch (see Sec.
A.1.5) to further extend the capabilities of the patternVary algorithm.
A.1.2.6 AAIM.samplePlayer2
Figure A.17: Global interface used for AAIM.samplePlayer2
Figure A.18: Sample load section of the AAIMsamplePlayerInterface2 patch
AAIM.samplePlayer2 uses a separate AAIM.samplePlayer (see Sec. A.1.2.4) and
AAIM.sequenceMaker (see Sec. A.1.5) for each individual voice of the patternVary algorithm.
This allows each voice to trigger samples from different collections of samples (i.e. one voice
196
Figure A.19: Sequence section of the AAIMsamplePlayerInterface2 patch
Figure A.20: Multisliders section of the AAIMsamplePlayerInterface2 patch
triggering from a collection of bass drum samples and one triggering from a collection of snare
samples).
The collection of numbered buttons along the top of the interface opens the individual AAIM.samplePlayer
and AAIM.sequenceMaker for each voice. Samples can either be loaded by using the individual
AAIM.samplePlayer patches or by dragging and dropping sample folders onto numbered boxes
at the bottom of the interface (see Fig. A.18). Once a sample folder has been loaded using this
method the interface will display the name of the folder loaded. The number of ‘notes’ for each
individual AAIM.sequenceMaker (see Sec. A.1.5) is automatically set to the number of samples
in the folder for that corresponding voice.
The 2 “live.tab” interface objects in addition to the 2 number boxes (see Fig. A.19) correspond
to the equivalent controls on each of the AAIM.sequenceMaker interfaces (see Fig. A.47), and
triggers these operations on each of the individual voices simultaneously.
The multisliders labelled “sample voice panning,” “sample voice pitch,” and “sample voice
volume” (see Fig. A.20) correspond to the equivalent controls on each of the AAIM.samplePlayer
interfaces (see Fig. A.16), changing the corresponding value for each of the samples in that voice
to the same value.
197
A.1.3 AAIM.looper
AAIM.looper is an algorithm for mapping the output of AAIM.rhythmGen (see Sec. A.1.1) onto
playback of an audio sample or loop. The AAIM system provides an interface for control of both
individual voices (individual samples) and control of all AAIM.looper voices simultaneously.
The algorithm divides an audio sample into n equally spaced segments, numbers these segments
sequentially, and then plays these segments according to a user set sequence.
Figure A.21: The AAIM Setup following initial toggling of the looper algorithm
• In the “AAIM Setup” window (see Fig. A.2) check the box labeled “looper.”
• The “AAIM Setup” window should change to reveal a number box and a button
labelled “looperMultiVoice Interface” (see Fig. A.21).
• Press this button to open the AAIM.looperMultiVoiceInterface patch (see Fig. A.22).
• Drag and drop a folder of audio samples onto the designated area at the bottom of
the AAIM.looperMultiVoiceInterface patch (see Fig. A.22).
• Use the drop down menu to load a sample from the folder for one voice, and open
that AAIM.looper interface by clicking the correspondingly numbered button at the
top of the interface (see Fig. A.22).
• The top section of the interface displays the waveform of the sample loaded (see
Fig. A.23), and also also the selection of a specific section of the sample
198
• Using the section of the interface directly below the waveform and to the right of
the interface set the number of equally divided samples by either setting the original
bpm of the sample (using the number box labelled “original bpm of sample”) or the
number of segments (using the number box labelled “beats in sample”).
• Use the AAIM.sequenceMaker (see Sec. A.1.5) labelled “sample segment sequencer”
on the left of the interface to set the playback sequence of the sample.
• Use the multislider at the top right of the AAIM.looper voice interface (see Fig.
A.23) to create variations of the playback, or the multislider at the top right of
the AAIM.looperMultiVoiceInterface (see Fig. A.22) to set a default setting for all
looper voices and then use the slider at the top right of the AAIM.looperMultiVoiceInterface
or looper variations slider on the
main/performance interface (see Fig. A.24) to set the chance of these variations
occurring.
A.1.3.1 AAIM.looperMultiVoiceInterface
AAIM.looperMultiVoiceInterface gives the user simultaneous control over all AAIM.looper voices.
However, when a sample is loaded for the first time changes will need to be made using the indi-
vidual AAIM.looper voice interface (see Sec. A.1.3.2).
The interface is divided into 4 sections (see Fig. A.22).
• The numbered buttons along the top of the interface open the individual AAIM.looper
voice interfaces (see Sec. A.1.3.2).
• Immediately below these are 4 multisliders (see Fig. A.25) which correspond to
the respective controls on the individual AAIM.looper voice interfaces (see Sec.
A.1.3.2), and a final slider on the right which controls the likelihood of these looper
variations occurring.
199
Figure A.22: AAIM.looperMultiVoiceInterface used for control of all AAIM.looper voices simul-taneously
200
Figure A.23: AAIM.looper used for control of an individual AAIM.looper voice
Figure A.24: looper performance interface on AAIM System main window
201
Figure A.25: Multislider section of the AAIM.looperMultiVoiceInterface
• Below these are two AAIM.sequenceMaker (see Sec. A.1.5) interfaces which con-
trol the play back of the sample (see Sec. A.1.3.2). The one on the right creates
identical playback sequences for each AAIM.looper voice (see Fig. A.26). The one
on the left creates individual playback sequences for each AAIM.looper voice. At
the bottom of this interface is a drop down menu which allows multivoice control
of playback:
– Choosing one of the voice numbers will result in the output of that
voice being mimicked in the playback of all of the other voices.
Figure A.26: AAIM.sequenceMaker section of the AAIM.looperMultiVoiceInterface
• At the bottom of the interface are controls which allows the loading of samples
and turning on/off of each individual AAIM.looper voice (see Fig. A.27). Drag-
ging and dropping a folder of audio samples/loops onto the designated box will
result in the drop down menus below automatically filling with the contents of
the folder. Loops/samples for each voice can then be loaded using these menus,
202
or by pressing one of the buttons labelled “load” and choosing the file for an in-
dividual voice using the resulting dialogue. Once a sample is loaded using the
AAIM.looperMultiVoiceInterface the interface will change to reveal the name of
the sample loaded beneath the controls for that individual voice. Each of the tog-
gles correspond to turning on/off an individual voice.
Figure A.27: Sample load section of the AAIM.looperMultiVoiceInterface
A.1.3.2 AAIM.looper
The AAIM.looper interface (see Fig. A.23) is divided into 5 main sections:
• At the top left of the interface is a toggle to turn on/off the algorithm. beside this is
an interface object which displays the waveform of a sample once it is loaded and
allows selection of a specific section of the sample (see Fig. A.28).
Figure A.28: Waveform section the AAIM.looper interface
• Below this and to the right of the interface is a section labelled “variations” (see
Fig. A.29).
– Pressing the button labelled “replace” opens a dialogue which loads
a sample.
203
– Pressing the button labelled “normalize” normalises the sample.
– The number box labelled “beats in sample” sets the number of seg-
ments the sample is divided into.
– The multislider object in this section of the interface sets the chance
of algorithmic variations of the sample playback:
· grainsize - the average length of each sample seg-
ment during playback.
· reverse - the chance of individual segments being
played in reverse
· retrigger - the chance of the sequence of segments
being ignored and individual sample segments being
repeated
· jump - the chance of playback jumping
backwards/forwards in the sequence
· jumpsize - the maximum size of these jumps (relative
to the number of jumps in the sequence)
Figure A.29: Variations section the AAIM.looper interface
• To the left is an AAIM.sequenceMaker (see Sec. A.1.5) interface labelled “sam-
ple segment sequencer” which determines the sequence of segments used during
playback of the sample (see Fig. A.30).
204
– A step set to ‘0’ results in an automatic pause being inserted into the
pattern.
– Every other value corresponds the beginning of one of the segments
of the sample and will cause playback to jump to that point of the
sample.
Figure A.30: Sample segment sequence section the AAIM.looper interface
• To the bottom left of the interface is a second AAIM.sequenceMaker (see Sec.
A.1.5) interface labelled “pitch shift sequencer” which sets a second sequence which
controls pitch changes during playback (see Fig. A.31).
– At the very bottom of the interface is a section labelled “pitch shift
range.”
· The slider allows selection of a range in semitones
within which pitch shifts occur.
· The number box to the upper left of this slider sets
the range to any equal distance above and below the
the original pitch, while the number box at the very
bottom left of the interface shows the exact pitch shift
of each step in semitones.
205
– The number box labelled “notes” sets the number of even divisions
of the pitch shift range
– Any step in the pitch shift sequence set to ‘0’ results in no pitch shift
occurring, every other value results in a pitch shift occurring.
Figure A.31: Pitch shift sequence section the AAIM.looper interface
• The final section of the interface provides volume and panning controls, and a but-
ton which opens an AAIM.FXModule1 patch unique to the AAIM.looper voice (see
Fig. A.32).
Figure A.32: Audio section the AAIM.looper interface
206
A.1.4 AAIM.melodyVary
AAIM.melodyVary is an algorithm designed to play, manipulate, and vary melodies. The AAIM
system provides an interface for control of both individual voices and control of all AAIM.melodyVary
voices simultaneously.
Figure A.33: The AAIM Setup following initial toggling of the melodyVary algorithm
• In the “AAIM Setup” window (see Fig. A.2) check the box labeled “looper.”
• The “AAIM Setup” window should change to reveal a number box and a button
labelled “melodyVaryMultiVoice Interface” (see Fig. A.33).
• Press this button to open the AAIM.melodyVaryMultiVoiceInterface patch (see Fig.
A.34).
• Press one of the numbered buttons along the top of the interface to open the corre-
sponding AAIM.melodyVary interface (see Fig. A.35).
• First set the range of melody using the number box in the top right corner of the
interface labelled “minimum pitch.”
• Second set the number of beats in the melody using the number box labelled “nBeats.”
• Use the colour coded multislider to set the pitches of the melody.
207
• Finally, use the lower multislider object to set the rhythmic content of the melody,
where:
– 0 - rest - inserts a rest into the melody
– 1 - tie - either results in the length of a note being extended (if pitch
doesn’t change), or triggers a new note (if pitch changes or previous
step was a rest)
– 2 - new note - always triggers a new note
• Use the variations section of the AAIM.melodyVary interface (see Fig. A.36 and
Sec. A.1.4.2) or the variations section of the
AAIM.melodyVaryMultiVoiceInterface (see Fig. A.37 and Sec. A.1.4.1) interface
to create variations of the melody.
Figure A.34: The AAIM.melodyVaryMultiVoiceInterface patch.
208
Figure A.35: The AAIM.melodyVary interface.
Figure A.36: The variations section of the AAIM.melodyVary interface.
Figure A.37: The variations section of the AAIM.melodyVaryMultiVoiceInterface interface.
209
A.1.4.1 AAIM.melodyVaryMultiVoiceInterface
The AAIM.melodyVaryMultiVoiceInterface patch provides control over all of the
AAIM.melodyVary voices simultaneously (see Fig. A.34).
• The numbered buttons at the top of the interface open each of the corresponding
AAIM.melodyVary interfaces.
• Immediately below this is a multislider object which sets the velocity range within
which each trigger is mapped for each voice.
• Below this a 7 further multisliders, labelled ‘original,’ ‘inversion,’ ‘retrograde,’
‘retrograde-inversion,’ ‘sequence,’ ‘scalar melody,’ and ‘scalar notes,’ which cor-
respond to the respective variables within each AAIM.melodyVary voice (see Sec.
A.1.4.2).
These values are then scaled using the slider along the right side of the interface,
or using the ‘melodyVary variations’ slider on the performance interface (see Fig.
A.38)
Figure A.38: melodyVary performance interface on AAIM System main window
• At the bottom of the interface are 3 objects which relate to the pitch collection used
for the melodies (see Fig. A.39).
210
– At the very bottom is a button which labelled ‘individual scales’
when off, or ‘common scale’ when on - in which case all voices use
the same collection of pitches.
– This collection of pitches is shown/set using the multislider which is
labelled “C, C# Db, D, etc.”
– Finally the slider between these 2 interface objects rotates the inter-
vallic relationships of the pitch collection being used.
e.g. if the pitch collection is a C major scale, the rotations will be
each of the modes of the major scale (ionian, dorian, etc.) starting
on C, with both the original melody and all variations being mapped
onto this new pitch collection.
Figure A.39: Pitch Collection section of the AAIM.melodVaryMultiVoiceInterface
A.1.4.2 AAIM.melodyVaryPoly
The AAIM.melodyVaryPoly interface provides direct control over the melodic content and varia-
tion of each of the AAIM.melodyVary algorithms. It is divided into 4 main sections:
• The left side of the interface sets the melodic content used by the AAIM.melodyVary
algorithm (see Fig. A.40). The upper multislider sets the melodic/pitch content - it
is colour coded so each pitch class (note of the chromatic scale) is given a unique
colour. The lower multislider sets the rhythmic content of the melody, where:
– 0 - rest - inserts a rest into the melody
211
– 1 - tie - either results in the length of a note being extended (if pitch
doesn’t change), or triggers a new note (if pitch changes or previous
step was a rest)
– 2 - new note - always triggers a new note
Figure A.40: Melodic content section of the AAIM.melodVaryPoly interface
• The top right of the interface (see Fig. A.41) consists 1 slider, and 2 number boxes:
Figure A.41: The top right section of the AAIM.melodVaryPoly interface
– The slider labelled “velocity range” sets the range of velocities within
which each trigger received is mapped.
– The number box labelled “nBeats” sets the number of beats in the
melody.
– The number box labelled “minimum pitch” sets the minimum pitch
displayed on the left of the interface.
212
• Below this is the section which sets the chance of each of the variations occuring
(see Fig. A.36):
– original - play the original melody
– inversion - invert the intervals of a melody (i.e. all upward steps
become downward steps and vice versa)
– retrograde - play melody in reverse
– retrograde-inversion - both retrograde and inversion
– expansion - expands each interval to keep overall shape of melody
but spread of a greater range
– sequence - creates a melodic sequence out of a selection of notes
from the original melody
– scalar melody - creates a new melody based on the intervallic con-
tent of the original melody
– scalar - sets the possibility of notes from the scale being inserted
into the melody - choices are based on the intervallic content of the
original melody
• Finally, the bottom right section of the interface sets the pitch content used in output
of the melody (see Fig. A.42).
Figure A.42: The pitch class section of the AAIM.melodVaryPoly interface
– The upper multislider sets the pitch classes (i.e. scale) used in the
melody and by the ‘scalar’ variations
213
– The lower multislider sets the pitch classes (i.e. scale) onto which
the the output is mapped (i.e. the actual pitches which are played)
– Finally the slider at the bottom rotates the intervallic content of the
pitch classes in the upper multislider.
e.g. if the pitch collection is a C major scale, the rotations will be
each of the modes of the major scale (ionian, dorian, etc.) starting
on C, with both the original melody and all variations being mapped
onto this new pitch collection.
A.1.4.3 AAIM.melodyVary trigger destinations
Output from the AAIM.patternVary algorithm can be used to trigger sounds using a num- ber of
methods. The particular method used is chosen using the drop down menu in the AAIM Setup
window (see Fig. A.33).
A.1.4.4 MIDI
Note on/off messages from the AAIM.melodyVary can be sent to any MIDI device/program using
the AAIM.midiOut patch (see Sec. A.1.2.2). One instance of the patch is used for each voice -
thus allowing different voices to be sent to different MIDI devices.
A.1.4.5 OSC
Note on/off messages from the AAIM.melodyVary can be sent to any OSC device/program using
the AAIM.OSCOut patch (see Sec. A.1.2.3). One instance of the patch is used for each voice -
thus allowing different voices to be sent to different OSC destinations.
A.1.4.6 AAIM.synth
AAIM.synth (see Fig. A.43) is a simple synthesiser which uses a feedback frequency modulation
synthesis algorithm. As with both the MIDI and OSC output options a separate synthesiser is used
for each AAIM.melodyVary voice.
The interface for each synthesis voice consists of 3 main sections:
214
Figure A.43: Interface for AAIM.synth
• The left hand side of the interface contains controls for the amplitude envelope (see
Fig. A.44).
– A - attack - length of time it takes a note to reach maximum ampli-
tude.
– D - decay - length of time it takes for a note to lower in amplitude to
the sustain level after maximum has been reached.
– S - sustain - amplitude level which each note rests at.
– R - release - length of time it takes until amplitude goes to 0, after a
note off has been received.
• The central section of the interface provides controls over the FFM synthesis algo-
rithm (see Fig. A.45).
215
Figure A.44: Envelope section of AAIM.synth interface.
Figure A.45: FFM section of AAIM.synth interface.
– The multislider labelled “index,” “ratio,” and “feedback” (in addi-
tion to the three number boxes directly below) set initial states for
the three basic parameters of the FFM synthesiser.
– The three number boxes below these provide a means of extending
the sounds created by setting a range within which the correspond-
ing parameter will change. This change is directly linked to the am-
plitude envelope, with the current amplitude value (0 — 1) being
multiplied by this value and added to the initial state.
– The slider at the bottom of the interface sets the extent to which
either low or high pitches are low-pass filtered (technically this is not
actual filter, but rather the effects of both the “ratio” and “feedback”
variables are lessened thus giving the effect of low pass filtering).
– The slider to the left of this section of the interface sets the extent of
pitch sliding (glissandi) between each note.
– The toggle at the bottom right sets whether or not a synthesis voice
216
is in legato mode.
· In legato mode a voice is automatically monophonic
(no release section of notes if another note is trig-
gered), and a new note trigger will result in the ampli-
tude envelope always beginning for the current am-
plitude.
• Finally, the far right of the interface (see Fig. A.46) provides control over the output
volume and an individual instance of the AAIM.FXModule1 (see Sec. A.1.6).
Figure A.46: Audiosection of AAIM.synth interface.
217
A.1.5 AAIM.sequenceMaker
Figure A.47: Interface for the AAIM.sequenceMaker patch
AAIM.sequenceMaker (see Fig. A.47) is a patch which allows the creation and modification
of simple step sequences. Sequences can either be created manually using the 2 multislider objects
or using the various standard manipulations described below.
Figure A.48: Upper section of AAIM.sequenceMaker interface
The upper section of the interface creates the initial sequence (see Fig. A.48). The number box
labelled “nBeats” determines the number of steps/beats in the sequence, this can then be altered
using the number box labelled “sequenceLength,” the value of which is multiplied by the number
of beats to give the total number of steps in the sequence. The number box labelled “notes” sets
the number of possible values for each step. The “live.tab” interface at the very top of the interface
gives 4 basic preset sequences which can be created using the combination of values from the three
number boxes described above:
218
• off
– This sets all of the steps(nBeats * sequence Length) to 0
• modulo
– This set the first slider to 1, the value of each subsequent step in-
creases by 1 until the value of a step is equal to the number of ‘notes’
at which point the count returns to 1 for the subsequent slider.
• stop
– The first slider is set to 1, the value of each subsequent step increases
by 1 until the value of a step is equal to the number of ‘notes,’ after
which point all the steps are set to 0
• random
– Each step is set to a random number between 1 and the number of
‘notes.’
Figure A.49: Lower section of AAIM.sequenceMaker interface
The lower section of the interface, labelled “output sequence,” provides controls which allow
further manipulation of the sequence (see Fig. A.49). The “live.tab” on the right of this section of
the interface provide more advanced manipulations of the sequence to those afforded by the upper
section of the interface:
219
• original
– This sets the ‘output sequence’ to be identical to that of the ‘original
sequence.’
• shuffle
– This divides the sequence into groups of 2/3 steps and rearranges
these groups randomly
• undoShuffle
– This returns the ‘output sequence’ to the sequence which was in use
immediately before the shuffle operation was used.
• rotate
– This shifts all of the values one step to the right, and places the last
step at the beginning of the sequence.
• retrograde
– This reverses the sequence of values - placing last value first, first
value last, etc.
• invert
– This inverts each of the values - i.e. sets a value of 1 to equal the
maximum value, sets a maximum value to equal 1, etc.
• retrogradeBar
– This manipulation results in a retrograde of each ‘bar’ in the sequnce
- where a ‘bar’ is equal to ‘nBeats’ (see Fig. A.48).
220
• palindrome
– This manipulation doubles the length of the sequence by adding a
retrograde of the sequence to the end of the sequence.
• generatePattern
– This final manipulation works in conjunction with the two number
boxes labelled “generatePattern Length” and “generatePattern Pause
Ratio.” It produces a ‘shuffled version of the sequence where the
possibility of pauses being included in the pattern equals the “gen-
eratePattern Pause Ratio,” and the length is equal to:
