A Wireless, Network-based Biosensor Interface for Music

Gilles Dubostgilles.dubost@noos.fr

Atau TanakaSony Computer Science Laboratories

atau@csl.sony.fr

AbstractWe present a new hardware systemusing biosignals for musicalapplications. Architecture and designdecisions made in its realization aredescribed as well as initial experimentsthat have been conducted. The device ismodular, wireless, and network based,permitting a wide range of applications.It is a reasonable cost, open platform forresearch in performance, musical gesturerecognition, and wearable musicsystems.

KeywordsWireless, wearable, eletrcomyogram,sensor instrument, gesture recognition

IntroductionIn this paper we present a biosignalbased musical interface. The interface isbased on active EMG electrodes, whosesignal is transmitted over a small areadigital wireless network to itsbasestation. The basestation thentranslates the signal into a variety offormats, including MIDI, RS232, andEthernet to transmission to a synthesizer,host computer, or network of recipients.The interface is intended for musicalapplications, including use as aperformance instrument, as a gestureanalysis device, and wearable musicalcomputing. We describe the design ofthe interface, and discuss issuesconfronted in its realization.

BackgroundUse of biosignals for music hasprecedent dating to the 60’s[Rosenboom]. Biosensors as humaninterface devices came into their own inthe early 90’s [Knapp-Lusted]. While thefirst efforts to harness biosignals were inthe analog domain, this second wave ofbiosignal use was digital. This had thebenefit of allowing a shift of focus frombio-feedback to bio-control. Musicalpractice based on this techniques hasbeen reported [Tanaka1993,Tanaka2000], and has been expanded togo beyond a unidirectional controlparadigm [Knapp-Tanaka].

While the first evolution in the fieldwere enabled by the arrival ofspecialized digital signal processinghardware, this paper presents a furtherevolution of treating biosignals nowmade possible by recent developmentsin general purpose microprocessors andwidely accessible networkinfrastructures. These changes allow thedevelopment of a system that is at oncemore portable (for concert use) and moreopen (for gesture analysis research). Theportable and network orientation of thedevice allow investigation of new areasin wearable musical instruments[Nishimoto]

Prior systems can be divided into twocategories: special purpose, and general

data acquisition. General purpose dataacquisition systems have been used formusical applications with biosignals[Marrin]. However, as these systems aredesigned to accept any kind of inputsignal, they may have features notneeded at the same time lack specialneeds particular to biosensors.Furthermore, these data acquisitionsystems tend to be ungainly for liveconcert use.

Specialized hardware is able to addressthe specificities of the biosignal: arelatlively weak voltage, with with highsignal/noise ratio and nontrivialgrounding issues between body andcircuit [Cram]. However, until now, thesignal processing tasks necessary toextract a useful information from thebiosignal has required specialized analogcircuitry and dedicated digital signalprocessors (DSP). This has resulted insystems that were highly capable ofcertain defined tasks, but not easilymodified.

MotivationsRecent developments have permitted usto merge the advantages of an opengeneralized system with those of bio-signal specific hardware. Analogamplification and filtering circuitry hasbeen optimized and miniaturized toallow the development of active dryelectrode systems. Increased power inrecent microprocessors allows the use ofgeneral purpose processors for basicsignal treatment. Such a system can bequickly reconfigured in software writtenin C to allow different application types.

In the present project, we exploit theserecent devepments to create a highperformance, open system at reasonablecost. We define a device architecture that

leverages these advantages to create aportable re-configurable system. Thesystem needs to be compact yet robustand be wireless, for performanceapplications, yet provide high qualitysignal acquisition for data analysis andgesture recognition studies.

BiosignalsThe electromyogram (EMG) signal is anelectrical voltage generated by the neuralactivity commanding muscle activity.Surface electrodes pick up this neuralactivity by making electrical contactthrough the skin. Typically increasedmuscle tension results in higher energyin the biosignal. The EMG signal is inthe millivolt range and has as frequencyrange from DC to 2KHz, where thesignal is down 50dB [Putnam].

There is a high variance of biosignalfrom person to person and in one personfrom muscle group to muscle group. Inaddition, the human body is a floatingconductive electrical system. As soon asit is connected to a fixed system, it seeksground through the fixed system, andserves as an antenna for electricalartifacts. Any EMG measurementsystem must account for these problems.In addition a musical system mustdisplay consistent behavior across awide range of different environmentalcontexts and conditions.

ArchitectureThe architecture of the system can bedivided into several distinct modules:- active electrodes- analog stage- microprocessor 1 (beltpack)- UART/Bluetooth circuit- Microprocessor 2 (basestation)- Host system or networkThis section describes each of thesemodules in detail.

fig. 2. System architecture

Active ElectrodesThe electrodes are dry active surfaceelectromyogram electrodes. They aredifferential electrical circuits: there arethree points of contact with the body, 1.A ground reference, then two poles at aclose spacing. The biosignal is definedto be the difference in electricalpotential across the two poles withrespect to the reference. As thedifferential signal is symmetric (like amicrophone cable), it exhibits commonmode rejection and is less susceptible tointerference.

Previous systems have been passive wet-gel electrodes. The gel was required toassure electrical conductivity with thebody. The passive nature of previous

electrodes meant that the unamplifiedbiosignal traveled up to 5 meters beforebeing treated. With the active electrode,amplification and filtering circuitry is onthe electrode itself, minimizing thedistance traveled by the signal, andtherefore minimizing the accumulationof noise artifacts. This makes possible toforego the conductive gel, andimplement a dry contact with the body.There are various manufacturers of dryelectrode systems on the market,generally requiring 5V power for 1000xamplification and filtering at 4,000Hz.This is an off the shelf component thatwe have integrated into our system.

ModulesThe system is divided into two units, abattery operated wireless beltpack and abasestation. The electrodes enter thebeltpack, are digitized and pretreated,and sent out a Bluetooth connection tothe basestation. The basestation furthertreats the signal and sends the data to ahost computer, synthesizer, or network.

Unit 1Analog stageThe analog stage can be divided intothree subsections – multiplexer,programmable amplifier, andanalog/digital convertor (ADC). Themultiplexer is a digitally controlled 8channel analog signal multiplexer. Thismakes possible the use of up to 8electrodes simultaneously. Themultiplexer is controlled by themicroprocessor, sequentially switchinginputs in a synchronized fashion.

The output of the multiplexer,representing the signal of between 1 and8 electrodes, enters the programmableamplifier. The amplifier is a 12 bitdigital/analog convertor (DAC) and

analog multiplier. A control signal fromthe microprocessor becomes the analoggain. This complements the fixedamplification onboard the electrode witha programmable gain stage.

Finally, the multiplexed, amplifiedsignal enters a 16 bit ADC. The totalsampling rate of the ADC is 40KHz,giving an effective sampling rate perelectrode of 5KHz when using eightelectrodes. (Aliasing????) Thesynchronous signal then enters themicroprocessor for signal treatment androuting.

CPU ATwo microprocessors are used in thesystem at different stages, both of them 8bit microprocessors running at 22.1MHz.These processors are programmed in C,with the software stored in onboardFlash-ROM. The first processor is usedat the output of the analog stage as asignal pre-treatment processor. Thisprocessor controls the analogcomponents – the multiplexer, amplifier,and ADC. It works in a syncrhonousfashion with the muliplexer to set thenumber of inputs and sequentiallysample the currently active input. A 12bit word from the processor serves asgain setting for the programmableamplifier. Finally the signal ispreconditioned and prepared for serialtransmission to the basestation.

BluetoothThe output of the first processor enters aUniversal AynchronousReceive/Transmitter (UART)compoment. The 16 bit electrode data isnibblized into 8 bit bytes and istransmitted at 115.2kbps. Aserial/Bluetooth component follows,creating the wireless link to the

basestation. The Bluetooth standardallows for maximum datarate of 723kbpsand range of 10meters.

Unit 2CPU BA second Rabbit processor receives thebiosignal over the Bluetooth link. It canperform data processing on the signal(such as RMS power calculations,filtering). It’s primary role is to formatthe signal for various outputs and thenroute the signals to these outputs.

OutputThe final output stage consists ofmultiple parallel outputs in standardformats: RS232, MIDI, and Ethernet.The RS232 port (UART) provides directserial connections to a host computer.The MIDI port allows direct connectionto commercial synthesizers. TheEthernet port allows the biosignal to bepublished to a local area network, to beexploited by any other device on thenetwork. The basestation has an IPaddress on the network and sends dataover various Ethernet protocols, either toa specific destination IP address, or inbroadcast mode to all nodes on anetwork.

fig. 3. The completed systemDiscussionThe system was conceived as an openarchitecture with modules that can beused in a variety of different

configurations. Unit 1 can be used alonein a direct serial link to a host computer,or over Bluetooth to a host. When MIDIor Ethernet are needed, Unit 1 and Unit2 are used together. They can beconnected directly via RS232. In aperformance setting, the Bluetooth linkbetween the two units allows theperformer to be wireless onstage withthe back end offstage connected to thebasestation. In a distributed system,Ethernet transmission can be used todeliver the same biosignal to mutliplecomputational or analysis nodes.

As both processors in the system areidentical and programmable in C, acoherent development environment iseasily established. The C code iscompiled on an outboard computer, anddownloaded to the nonvolatile memoryof the microprocessors. A decision wasmade at the conception of the device tostay with simple microprocessors.Although this is has less calculationpower than a dedicated DSP, we felt thatthe cost benefits and programmingsimplicity justified the decision.

The use of simple micoprocessors led usto conceive of a distributed processingarchitecture. There are two identicalmicroprocessors in the system – one inthe mobile beltpack and the other in thebasestation. Each processor can be usedfor tasks specific to the role of the unit inwhich it is found – for the beltpack thismeans data pre-processing. For thebasestation this means data formattingfor the various output ports. The data isdelivered to the host system in a formatwhere further signal processing can becarried out at the host or network level,completing the distributed processingarchitecture.

The choice of microprocessor does not,however, preclude realtime operation,something typically associated with DSPsystems. The microprocessors weselected have multitasking libraries, andcan handle tasks at interrupt level,allowing a real-time scheduler to bewritten. The data formats and data rateswere chosen to at once fit withinstandards norms yet still allow real timetransmission of data from multipleelectrodes. Computer based RS-232ports are typically limited by theoperating system to 115.2kbps

Post-processing on the basestationallows simultaneous transmission ofRuntime parameter modifications (ofmultiplexer or amplifier, MIDI ordestination IP address) can be userconfigured by commands over theRS232, MIDI, or Ethernet ports. Thismakes possible parallel multiple musicalusage of a single control signal.

Conversely, the inputs are generalized,allowing other sensor types to be utilizedalongside the EMG signal. This openspossibilities for exploiting the biosignalin contexts of multimodal interation[Knapp-Tanaka].

The wireless and network orientation ofthe device bring a new context to sensor-based musical instruments. A user couldchange spaces while wearing the deviceand have his signal be picked up by anew local area network. The ethernetoutput of the system makes the biosignalavailable to whole networks, to be usedby various host machines or to sharedata with similar peers on the samenetwork.

On the user side, EMG is a signal that isvery close to the body, as it is electricity

of the body itself. There are issues inuser-user variation, as well as contextand environment specific variation, thatwe address in the architecture of oursystem. The software controlledamplifier allows the system to adapt tothese different situations – whether theybe at the muscle level, user level, orcontext level. Different muscle groupshave widely ranging output amplitudes.At the other extreme, the same personmight send quite different biosignalsdepending on whether they are in arelaxed situation or in a stressfulsituation (such as stage performance).

There are safety considerations not to beneglected. As the user’s body becomespart of the electrical system, properelectrical isolation is critical. For this, itis not envisioned to use Unit 1 in directserial connection with a host computer.The Bluetooth subsystem allows notonly the performance benefits ofwireless, but the safety benefits ofelectrical isolation of the user from therest of the system – host computer ornetwork.

The system has been successfullyinterfaced with host side signalprocessing environments such as Matlab,and music performance software such asMaxMSP.

References

Cram JR, Clinical EMG for SurfaceRecordings: Volume 1, J&J Engineering,Poulsbo, WA, 1986.

R. Benjamin Knapp and Hugh S. Lusted,"A Bioelectric Controller for ComputerMusic Applications,” Computer MusicJournal, MIT Press, Vol. 14, No. 1, pp.42-47, Spring 1990.

Knapp, R.B. and Tanaka, A. MultimodalInteraction in Music Using theElectromyogram and Relative PositionSensing. In press.

Marrin, T. and Picard, R. 1998. “TheCouductor’s Jacket: a Device forRecording Expressive MusicalGestures.” In Proceesings of the ICMC,San Francisco: ICMA, pp. 215-219.

Nishimoto, K., et al. 2001. NetworkedWearable Musical Instruments WillBring a New Musical Culture. InProceedings of the 5th InternationalSymposium on Wearable Computers.Los Alamitos, California: IEEEComputer Society.

William L. Putnam and R. BenjaminKnapp, “Real-Time Computer ControlUsing Pattern Recognition of theElectromyogram,” Proc. of the IEEEInternational Conf. on Biomedical Eng.,San Diego, CA, pp. 1236-1237, October27-29, 1993.

Rosenboom, D. 1990. Extended MusicalInterface Human Nervous System:Assessment and Prospectus. Cambridge,Massachusetts: MIT Press.

Tanaka, A. 1993. “Musical TechnicalIssues in Using Interactive InstrumentTechnology.” In Proceedings of theICMC, San Francisco: ICMA, pp. 124-126.

Atau Tanaka, “Musical PerformancePractice on Sensor-based Instruments,”In M. Wanderley and M. Battier (eds.)Trends in Gestural Control of Music.IRCAM, p. 389-405, 2000.