GrainTrain: A Hand-drawn Multi-touch Interface forGranular Synthesis

Anıl ÇamcıDepartment of Performing Arts Technology

University of Michiganacamci@umich.edu

ABSTRACTWe describe an innovative multi-touch performance tool forreal-time granular synthesis based on hand-drawn waveformpaths. GrainTrain is a cross-platform web application thatcan run on both desktop and mobile computers, includingtablets and phones. In this paper, we first offer an analy-sis of existing granular synthesis tools from an interactionstand-point, and outline a taxonomy of common interac-tion paradigms used in their designs. We then delineate theimplementation of GrainTrain, and its unique approach tocontrolling real-time granular synthesis. We describe practi-cal scenarios in which GrainTrain enables new performancepossibilities. Finally, we discuss the results of a user study,and provide reports from expert users who evaluated Grain-Train.

Author KeywordsGranular synthesis; hand-drawn; multi-touch; cross-platform;interaction design

CCS Concepts•Human-centered computing →Web-based interac-tion; Sound-based input / output; •Applied com-puting → Sound and music computing;

1. INTRODUCTIONGranular synthesis as an audio production technique hasa robust history dating back to the 1950s [6]. With theintroduction of real-time granular synthesis systems in the1980s [10], this technique made its way into performancepractices. Today, there are numerous software and hard-ware tools for granular synthesis that adopt different com-putational and interactive approaches.

In this paper, we first offer an interaction taxonomy ofreal-time granular synthesizers to contextualize our work.By analyzing 20 granular synthesizers, we elaborate 3 com-mon paradigms of interaction design across these instru-ments. We discuss the properties of these paradigms, andoutline their implications in terms of musical style and ex-pression.

We then introduce, GrainTrain, an innovative implemen-tation of real-time granular synthesis based on hand-drawn

Licensed under a Creative Commons Attribution4.0 International License (CC BY 4.0). Copyrightremains with the author(s).

NIME’18, June 3-6, 2018, Blacksburg, Virginia, USA.

Figure 1: Image of GrainTrain in use on a tabletcomputer. The performer is using 5 fingers to in-teract with 4 waveform paths that are hand-drawnin various shapes and spatial configurations.

waveform paths. Designed as a cross-platform web appli-cation, GrainTrain opens up interesting performance possi-bilities. In addition to mouse interaction on desktop com-puters, it supports multi-touch interaction, which has beenapplied to granular synthesis relatively recently with theintroduction of mobile devices that utilize capacitive touchscreens.

As its primary contribution, GrainTrain enables users tocreate custom hand-drawn interfaces for granular synthesis.We discuss the implications of this approach in terms ofmulti-touch interaction ergonomics and performance tech-niques. We then offer the results of a user study, and ananalysis of the feedback gathered from expert users.

2. RELATED WORKIn one of the earliest applications of granular synthesis, thecomposer Iannis Xenakis spliced together short pieces oftape, and played them back at high speeds to create a densecloud of sound grains [7]. With advances in digital comput-ing, it became possible to program a computer to output agranulated version of an audio file based on pre-determinedparameters. Since then, numerous real-time applications ofgranular synthesis that adopt different interfaces and pro-cessing techniques have been developed. Although theseapplications rely on a similar principle of synthesis basedon the micro-organization of sound elements, they can im-plement vastly different approaches to how the user controlsthis process. The mouse being the predominant input de-vice on modern computers, most granular synthesis applica-tions are designed around point, click and drag interactions.

On the other hand, the widespread adoption of multi-touch screens in consumer-grade mobile devices has ushered

Figure 2: Interface archetypes for the three interaction paradigms for granular synthesis: parameter control(left), keyboard performance (middle), waveform scrubbing (right).

in a flurry of new musical applications, including those ofgranular synthesis. Some of these applications emulate themouse-based desktop interfaces, where touch actions essen-tially serve as individual mouse cursors. However, thereare also applications that rely on the unique possibilities ofmulti-touch interaction, which we will discuss in the follow-ing sections.

In addition to software UIs, other interaction modalitiesare used for granular synthesis as well. The PebbleBox, forinstance, relies on tactile interactions, where the sounds ofa user playing with pebbles in a box are analyzed in real-time to extract parameters for the granulation of arbitrarysounds [5]. ShakeStick is a physical interface in the form ofa wooden stick equipped with an accelerometer; by wavingthe stick, the user can control various granulation parame-ters that are mapped to the pitch, yaw and roll values of thestick [13]. In Node Kara, a depth camera is used to trackhuman movements in 3D space, which are then used to drivevarious parameters of a granular synthesis engine [3].

3. AN INTERACTION TAXONOMY OFGRANULAR SYNTHESIZERS

Despite the great variety of UIs implemented in granularsynthesis applications, common threads across these can beidentified. We reviewed 20 real-time granular synthesizersthat have been released over the past 18 years. These in-clude SoundGrain 6 (2017), Generative 2 (2015), Border-lands Granular 2 (2015), iDensity 2 (2015), The Mangle(2015), Granulator II (2013), GrainProc (2012), MegaCur-tis (2012), SAMPLR (2012), HourGlass (2012), Grain Sci-ence (2011), Granite (2011), SampleWiz (2011), SampleToy(2010), Narrativas Sonoras II (2010), MetaSynth 5 (2009),CataRT (2007), Partikel (2007), Emission Control (2004),and Granulab (2000). Among this list are desktop ap-plications that are designed for mouse or keyboard input,and mobile applications that are designed for touch input.Moreover, some of these applications can be controlled ex-ternally via MIDI or OSC.

All of these applications offer creative interfaces that sup-port unique forms of musical expression. Examining theseinterfaces, we elaborate three main interaction paradigmsfor real-time granular synthesizers: parameter control, key-board performance, and waveform scrubbing. In Fig.2, weoffer illustrations of archetypal interfaces for these 3 cat-egories. Although we define these interaction paradigmsseparately, many granular synthesizers combine these ap-proaches with different emphases. In Fig. 3, we providefew examples of how existing applications can be situatedwithin this taxonomy.

3.1 Parameter ControlControl of granulation parameters is intrinsic to most gran-ular synthesizers; common granulation parameters includewindow size and type, density, pitch, and amplitude. Someof the applications we reviewed prioritizes the performanceof changes in these parameters. In such applications, knobs,sliders, and X-Y controllers take up most of the interface.Examples of these applications include Emission Control [7]and Partikkel [1]. GrainProc similarly emphasizes slider-based control of parameters, but with the intent of enablingthe use of toes to manipulate the output of an instrument,which the performer is playing with their hands [8].

These interfaces commonly facilitate the detailed manip-ulation of continuous grain streams. Although a slider foreither explicit or probabilistic control of the playhead posi-tion can also be found in these interfaces, this slider oftenoccupies the same level of interaction hierarchy as those ofother parameters.

The focus on the manipulation of continuous granulationmake these interfaces especially powerful for exploring grad-ually evolving qualities of microsound. This form of inter-action renders this UI paradigm particularly suitable forgenerating textural sounds. However, more sparse or ges-tural sounds can also be achieved with specific parametercombinations and by using sound sources that display suchqualities.

3.2 Keyboard performanceApplications under this category utilize an interface thatis based on the keyboard instrument model. The interfaceoften includes a virtual piano keyboard, or can be controlledexternally with a midi keyboard. Pressing a key triggers agrain stream that is transposed accordingly. Knobs andsliders in these UIs serve a more ancillary role.

In these applications, granular synthesis serves a simi-lar function as the oscillator of a subtractive synthesizer,rather than as a means to explore the emergent qualitiesof microsound processing. From a stylistic point of view,these interfaces are often used for playing pad-like soundsthat consist of chordal combinations of grain clouds. Whilethese can be better suited for performing textural sounds,the user can also perform more gestural sounds by playingphrases with shorter notes.

For instance, in MegaCurtis, the user can move a playheadon a waveform to determine the point of granulation. Theuser then plays a virtual keyboard, which takes up most ofthe UI, to synthesize granular streams that are transposedaccording to the keys pressed. Similarly, in SamopleWiz,the user can either play a virtual keyboard or scrub over apiano-roll to granulate a file at different transpositions.

3.3 Waveform ScrubbingIn these applications, mouse or touch interactions are usedto move a playhead across the waveform of an audio file. Byscrubbing through the file, the user can change the pointat which the grain windowing is applied. In this mode ofinteraction, the emphasis is put on the temporal explorationof an audio file by scanning it for parts that respond togranulation in different ways. Scrubbing movements canoften lead to gestural sounds due to changes in spectral andtransient characteristics of a sound over time. Sustainedor textural sounds can also be achieved by maintaining theposition of the playhead.

For instance, in Borderlands Granular for iOS, the usercreates visual granulation nodes, under which they can placemultiple waveforms. By controlling the parameters of anode, the user can change how it interacts with the wave-forms that it coincides with. Waveforms can also be resizedand rotated the to alter the range of granulation. The de-veloper of this application describes his approach as puttingthe emphasis on ”gestural interaction over knobs and slid-ers” [2].

Figure 3: 4 examples of existing applications sit-uated within the interaction taxonomy discussedhere. 1: Borderlands Granular emphasizes wave-form scrubbing while also offering a separate inter-face state for the control of grain parameters. 2:MegaCurtis interface heavily relies on a virtual pi-ano keyboard but the user can also move the play-head on the waveform to change the point of gran-ulations triggered with the keyboard. 3: EmissionControl gives the user a complex combination ofsliders and X-Y controllers for the fine manipulationof a grain cloud, 4: Granulator 2 plug-in for Able-ton Live offers an interface consisting of multipleknobs and sliders. The grains can be set to loop orbe triggered with a MIDI keyboard. Although theplug-in does not offer a direct waveform-scrubbinginteraction, the playhead position can be controlledwith a knob. The pink dot represents GrainTrain.

4. GRAINTRAINGrainTrain is a novel application of granular synthesis basedon multi-touch interaction. Within the context of the in-teraction taxonomy described above, GrainTrain offers aninnovative approach to the waveform-scrubbing paradigmwith hand-drawn waveform paths as seen in Fig. 4. Thisallows the user to create custom interfaces that open upunique expressive possibilities, some of which are describedlater in this paper, and demonstrated in our video abstract. 1

1https://vimeo.com/graintrain/video

Figure 4: An interactive waveform in GrainTrain.The point of interaction is highlighted with gradualchanges in bar color and thickness. The extent ofthese changes is based on the spread parameter.

4.1 System DesignGrainTrain is designed for the web browser using the We-bAudio API, the WebGL library Three.js, HTML5 and CSS.GrainTrain therefore runs on multiple hardware platforms(i.e. desktop and mobile computers) and operating systems(e.g., iOS and Android). It adopts a fully client-side op-eration, which does not require a network connection afterthe system has been loaded. The user can load any audiofile from their local file system without a need for uploadingfiles to a remote server. On mobile devices that do not offera user-accessible file system, cloud storage services such asiCloud or Google Drive can be used for loading files intoGrainTrain. The native file input UI of the operating sys-tem is used for file selection on all platforms.

4.2 InterfaceOn start-up, GrainTrain offers a simple interface with 3buttons and 5 sliders. In a sense, GrainTrain launches witha lack of a UI, which the user can gradually construct andcustomize during the course of a performance as seen inFig. 1. The 3 buttons at the bottom of the screen are usedfor adding, moving, and deleting sound files. Upon pressingthe add button, the user is prompted to select a sound filefrom local or cloud storage. After selecting a file, the usercan draw an arbitrary path anywhere on the screen. Oncethe drawing is complete, the waveform of the selected fileis drawn on this path. After pressing the move button, theuser can relocate any of the waveforms already drawn onthe screen. Finally, the delete button is used for removingexisting waveforms. A similar mapping of an audio file toa hand-drawn path is used in Different Strokes, where thedrawing speed determines the speed of the file playback [14].

Waveforms in GrainTrain are made up of bars that rep-resent momentary amplitudes in the audio file at regularintervals as seen in Fig. 4. Each waveform, regardless of itspath length, represents the entirety of the file. The num-ber of bars in a waveform depends on the length of the pathdrawn. This implies that a longer path for the same file willoffer a higher temporal resolution for interaction. When amouse or touch action collides with any of the bars in awaveform, the file gets granulated at the corresponding po-sition. The range around this point from which grains areselected is represented with gradually changing colors.

The 5 sliders at the top of the screen gives the user con-trol over size, amplitude, pitch, density, and spread of thegrains. The spread parameter controls the range from whichgrains are selected around the point of interaction on awaveform. Additionally, touch force is mapped to the den-sity parameter on touchscreen devices. Although we posi-tion GrainTrain within the waveform-scrubbing paradigmoutlined earlier, we believe that these five fundamental pa-rameters of granulation unlock a range of sonic possibilities,and are therefore integrated into our UI albeit with mini-

mal footprint. Unlike most parameter-control applications,where the user manipulates a continuous stream of grains,a stream in GrainTrain is generated only when the userscrubs a waveform.

4.3 Ergonomics of Touch InteractionOver the past decade, touch interfaces have become a stan-dard in mobile computing. While most of our interactionswith touchscreen devices still rely on single-point controlssimilar to those performed with a mouse, modern phonesand tablets also implement multi-touch interactions [12].However, multi-touch interactions with 2D surfaces bringabout ergonomic restrictions [4]. This is primarily why mostmulti-touch interactions rely on either the duplication ofsingle-point interactions (e.g., two-finger swipes and taps),or the imitation of interactions with physical objects (e.g.,pinch and spread gestures).

Most granular synthesizers within the waveform-scrubbingparadigm utilize a 1-dimensional timeline, where the play-head is used to traverse an audio file by moving a cursorhorizontally on this timeline. Some synthesizers also mapthe vertical position of the cursor to various parameters,such as pitch or amplitude. A mouse interaction is perfectlysufficient to perform these one or two-dimensional actions.With touch surfaces, concurrent control of multiple time-lines or parameters becomes possible. However, multiplehorizontal timelines can pose ergonomic problems due tovariations in how different fingers can be extended [11]. Aprimary goal in designing GrainTrain was to enable thecreation of non-linear waveform trajectories that can bettersuit the anatomy of a user’s hand. This way, the user canmore easily perform simultaneous granulations of multipleaudio files with multi-touch gestures.

4.4 Synthesis EngineGrainTrain’s synthesis engine is based on a windowed play-back of asynchronous sample buffers. The hop size betweenwindows is determined by the grain size and density pa-rameters. Each mouse or touch interaction with a wave-form instantiates a new voice, which is maintained until theinteraction is completed. The user can create concurrentstreams of granulation through multi-touch or by interact-ing with overlapping portions of two or more waveforms asseen in Fig 5a.

The user’s interaction with a waveform is quantized tothe bars that make up a waveform. With each interaction,the bar index is used to determine the position in the au-dio buffer to be sampled. This position is updated as theuser moves their mouse or finger from one bar to another.Grains are picked from a user-controllable range around thisposition in a randomized fashion, which alleviates repetitionartifacts that might arise due to the quantization.

5. PERFORMANCE TECHNIQUESGrainTrain enables a combination of new and existing inter-action techniques that expand the expressive possibilities ofgranular synthesis on mobile and desktop computers. Here,we outline a few of the performance techniques that arebased on hand-drawn waveform paths and multi-touch in-teraction.

5.1 Superimposition of multiple waveformsAn arbitrary number of waveforms can be drawn on top ofeach other. When the user interacts with a point where twoor more waveforms intersect, grains from all audio files thatcorrespond to these waveforms will be picked. BorderlandsGranular facilitates a similar interaction via overlapping au-dio files with different orientations [2].

Even on a desktop browser that offers a single point ofinteraction, unique expressions can be achieved by superim-posing multiple waveforms as seen in Fig. 5a. At the top,three different waves are intersected. At the bottom, a sin-gle file is drawn twice from left to right and right to left.This allows the user to mix forward and reverse playbacks ofa file. When applied on a multi-touch device, this techniquegreatly expands the number of concurrent granulations, andopens up distinct mixing possibilities.

5.2 Forking interaction pathsDrawing partially overlapping curved paths for differentsound files enables an interesting gesture, where using a sin-gle swipe on a forking path will cross-fade between multipleaudio files. In, Fig. 5b, the user starts granulating one filebut switches over to another waveform with a single gestureas indicated by the dashed line.

5.3 Discontinuous scrubbingWaveform scrubbing on a linear timeline often results inphrases that are continuous regardless of the direction ofscrubbing. On a curved waveform, such as the one seenin Fig. 5c, the user can perform a continuous scrubbingaction on the dashed line that would result in discontinuousgranulations that would have required discrete interactionson a linear timeline.

5.4 Short-touch interactionsIn addition to continuous mouse or touch gestures, the usercan also play the waveforms with brief, trigger-like interac-tions. Using an ergonomically shaped arc as seen in Fig. 5d,the user can perform gestures that resemble playing trills ona piano.

5.5 Indeterminacy through complex pathsBy drawing complex curves as seen in Fig. 5e, the user canobfuscate the relationship between the temporal progres-sion of a sound and the waveform that represents it. Thiscreates a degree of interaction indeterminacy that leads toimprovisational possibilities. Especially when the drawingtakes up a large portion of the screen, it approximates theeffect of the audio being randomly scattered over the UI.This allows the user to perform arbitrary gestures that willresult in unpredictable combinations of grains but withinthe confines of a single audio file.

5.6 Intra-mixingIntra-mixing (i.e. the mixing of an audio file with itself) isenabled by drawing paths that loop in winding or circularshapes. In Fig. 5f, the user has drawn several circles in aloop. When the user interacts with any point on the draw-ing, grains from various parts of the waveform that corre-spond to that point are selected, effectively mixing discretetimes in the same audio file. When used with multi-touchinteraction, this extracts complex temporal structures froma single audio file.

5.7 Temporal resolution through path lengthThe user can control the temporal resolution at which theyinteract with a file by altering the length of the path thewaveform is drawn onto. In Fig. 5g, the same audio filehas been mapped to two drawings. While the top spiralallows the user to explore the file in finer temporal detail,the shorter line facilitates the picking of grains from a widertemporal range. In Fig. 5d, the four paths underneath thearc similarly allows the user to trigger grains from the entirespan of an audio file. Resizable waveforms in BorderlandsGranular enables a similar technique.

a b

c d

e f

g h

Figure 5: Illustrations of various performance tech-niques enabled by GrainTrain’s hand-drawn wave-form paths, and multi-touch interactions.

5.8 Multi-user interactionWhile the multi-touch capabilities of a device inherentlyimply that multiple users can interact with it simultane-ously, a UI that is geared towards a single user often ob-structs multi-user operation. The hand-drawn UI elementsin GrainTrain offers the flexibility to create custom layoutsthat can facilitate multi-user interactions. In Fig. 5h, fourusers have demarcated their corners on a tablet using curvedwaveforms. Additionally, two diagonal waveforms serve asshared UI elements between pairs of users.

5.9 SustainDue to the way GrainTrain’s synthesis engine implementsthe relationship between user interaction and grain genera-tion, vibrato-like gestures result in the triggering of multiplegrain streams while finger position is maintained. Especiallywhen the spread parameter is set to minimum, this gestureprolongs playback in a way that is similar to how the samegesture sustains the sound of a stringed instrument.

6. EVALUATIONTo evaluate the usability and the expressive capabilities ofGrainTrain, we conducted a study with novice users, andreached out to expert users for feedback.

6.1 User StudyTo evaluate certain use cases of GrainTrain, and its effec-tiveness in facilitating the use of waveform scrubbing forgranular synthesis, we conducted a study with 10 users. 3of the participants described themselves as having no priorknowledge of granular synthesis. Each study was performedon a 10.9” iPad Pro, and took approximately 20 minutes.

6.1.1 MethodEach user was given a 1-minute tutorial on the operation ofGrainTrain. They were then asked to use the applicationin the following scenarios: 1) a single linear waveform, 2) 2linear waveforms on two separate rows, 3) a single archedwaveform, 4) 2 arched waveforms side by side. In structuredinterviews following each scenario, the users were asked toreport their general impressions, the maximum number offingers from a single hand that felt comfortable to use, andhow two-handed performance fared on a tablet screen. Fi-nally, they were asked to create a free-form interface, playwith it, and verbally describe their overall experience.

6.1.2 Results and DiscussionWith a single linear waveform, users unanimously reported3 as the maximum number of fingers that felt comfortableto perform with. 7 users reported that 4 fingers were pos-sible to place on a linear timeline but uncomfortable toscrub with. While a few of the users attempted placing5 fingers on the linear timeline, none reported this as a vi-able interaction method. We observed that all users wereable to perform naturally with one or two fingers, and mostusers comfortably maintained 3 fingers on a single linearwaveform as they scrubbed through it. With a fourth fin-ger placed, although keeping them in position was possible,users struggled to keep the little finger in contact with thewaveform when scrubbing.

With two linear waveforms spaced apart in rows, userstried scrubbing with two hands. While the reports on thenumber of fingers that can be placed comfortably on a wave-form remained similar to that from the first scenario, someusers also attempted cross-placement of fingers (e.g., thethumb of the top hand touching the bottom waveform).The most problematic aspect in this scenario was reportedas hands getting in the way of each other when moved in op-posite directions. Users reported that being able to furtherseparate the waveforms vertically made it more comfortableto perform this action, implying that even with more tradi-tional waveforms, some flexibility in the UI is preferred.

With a single arched waveform, most users immediatelyplaced all 5 fingers on the waveform. Furthermore, they re-ported that scrubbing with 4 or 5 fingers were comfortable.Being able to draw arcs that fit their hand size was found tosignificantly improve their ability scrub with multiple fin-gers. Another common gesture was playing the arc like apiano with brief touch events.

With two arched waveforms, most users were able toplace all 10 fingers comfortably, with the exception of 2users whose hands were too large to fit on the tablet sur-face. These users instead used two arcs crossing each otherroughly were the thumbs would be placed. One of theseusers performed a zipper-like finger interlocking as two handsmoved in opposite directions on the arcs.

During the free-form design task, all users came up withdistinct interfaces, implying that GrainTrain can supporta variety of approaches to musical expression. All of theusers drew abstract non-linear paths using 2 or more soundswithout any supervision. Only 3 users interacted with thegrain-parameter sliders. Almost all of the users experi-mented with paths that intersect in various ways. Once

a user settled on a design, their interaction with it rangedfrom brief playful explorations to extended performance-likesession. Furthermore, given that these were all first-timeusers, some of whom did not have previous experience withgranular synthesis, we find the complexity of their designsparticularly encouraging in terms of GrainTrain’s potentialto afford a low-barrier and a high ceiling for usability, whichis deemed a desirable trait for interfaces that support cre-ativity [9]. All users reported having enjoyed playing withGrainTrain.

6.2 Expert FeedbackAdditionally, we reached out to 5 expert users, who de-scribed themselves as having used real-time granular syn-thesis in performance contexts, or offered instruction on thetopic. These users were briefed on the functionality and thedesign of GrainTrain. They were then asked to evaluateit in their own time, and document their experiences withscreenshots, recordings, and written notes. Further feed-back was gathered via follow-up interviews.

One of the experts described that they used GrainTrainto alter the way they physically interacted with their tabletdevice. By drawing waveforms to the left and right edgesof the screen, they were able to hold the device like an ac-cordion with the screen facing outwards, and perform withshort-touch interactions. This user also mentioned exper-imenting with representational drawings, and that it wasinteresting to think about such visual metaphors as UI ele-ments.

Another expert mentioned that they preferred drawingwavy lines as these functioned as space-filling curves thatincreased the extent of a waveform within the confines of atablet screen. They mentioned that while complex shapescame with a mental cost, wavy lines allowed them to exploresounds easily and intuitively.

Another expert argued that being able to scan complexshapes for interesting sound combinations which they couldthen riff on was the most engaging affordance of the UI.They described that exploring how various shapes and soundswound up together often lent itself to powerful contrasts be-tween gestural and textural sounds. Similarly, another ex-pert described that creating random scribbles and playingthem with multiple fingers were at times more expressivethan playing with a predictable visual pattern such as aline.

Another expert mentioned that the ability to fill up thescreen with many interface elements of different shapes andsizes (e.g., regions, spirals, buttons) was inspiring from aperformance stand point. They’ve also expressed that mov-ing waveforms in space to have them gradually overlap eachother proved to be an interesting way to think spatiallyabout the temporal progression of a performance.

7. FUTURE WORK AND CONCLUSIONIn the short term, we aim to address user requests suchas automatic dynamics management, waveform duplication,scene saving and loading, and separate color schemes forindividual waveforms. We also plan to map the saturationof each bar’s color to the spectral centroid of the audio itrepresents to create another channel of visual feedback. Afeature which we believe will enhance the live performancecapabilities of our system is the ability to swap the file thatis attached to a path. Additionally, we are investigating UIschemes for implementing per-waveform control over grainparameters. As we expand the features of our UI, we fullyintend to maintain its low barrier of entry for novice users.

Since GrainTrain is a cross-platform web application, on-line collaboration is a natural next step in our development.

With this feature, users will be able to access a common in-stance of GrainTrain remotely to mutually interact with thesame set of waveforms.

In this paper, we presented an interaction taxonomy forgranular synthesizers that can support the analysis of ex-isting applications as well as the design of new ones. WithGrainTrain, we introduced a new approach to the waveform-scrubbing paradigm with interactive waveforms that aredrawn onto custom paths created by the user. We identi-fied some of the performance techniques that this approachenables. Furthermore, the feedback we gathered from usersindicated the potential of GrainTrain in facilitating othernew and interesting expressive possibilities for granular syn-thesis.

8. REFERENCES[1] Ø. Brandtsegg, S. Saue, and T. Johansen. Particle

synthesis–a unified model for granular synthesis. InProceedings of Linux Audio Conference, 2011.

[2] C. Carlson and G. Wang. Borderlands: Anaudiovisual interface for granular synthesis. InProceedings of International Conference on NewInterfaces for Musical Expression (NIME), 2012.

[3] A. Camcı and A. Forbes. Node kara: An audiovisualmixed reality installation. In IEEE VR Workshop onMixed Reality Art, pages 1–4, 2016.

[4] E. Hoggan, M. Nacenta, P. O. Kristensson,J. Williamson, A. Oulasvirta, and A. Lehtio.Multi-touch pinch gestures: Performance andergonomics. In Proceedings of ACM InternationalConference on Interactive Tabletops and Surfaces,pages 219–222, 2013.

[5] S. O’Modhrain and G. Essl. Pebblebox andcrumblebag: Tactile interfaces for granular synthesis.In Proceedings of International Conference on NewInterfaces for Musical Expression (NIME), pages74–79, 2004.

[6] C. Roads. Microsound. MIT press, 2004.

[7] C. Roads. Grains, forms and formalization. InS. Kanach, editor, Xenakis Matters. Pendragon Press,2012.

[8] M. Sanganeria and K. Werner. Grainproc: Areal-time granular synthesis interface for liveperformance. In Proceedings of InternationalConference on New Interfaces for Musical Expression(NIME), pages 223–226, 2013.

[9] B. Shneiderman. Creativity support tools:Accelerating discovery and innovation.Communications of the ACM, 50(12):20–32, 2007.

[10] B. Truax. Real-time granular synthesis with a digitalsignal processor. Computer Music Journal,12(2):14–26, 1988.

[11] W. Westerman. Hand tracking, Finger Identification,and Chordic Manipulation on a Multi-touch Surface.PhD thesis, University of Delaware, 1999.

[12] W. Westerman, J. G. Elias, and A. Hedge.Multi-touch: A new tactile 2-d gesture interface forhuman-computer interaction. Proceedings of theHuman Factors and Ergonomics Society AnnualMeeting, 45(6):632–636, 2001.

[13] A. Wilson and A. Hindle. The shake stick. Technicalreport, PeerJ PrePrints, 2015.

[14] M. Zadel and G. P. Scavone. Recent developments inthe different strokes environment. In Proceedings ofInternational Computer Music Conference (ICMC),2008.