jo ur nal home p age: www.elsev ier .com/ locate / jneumeth
linical Neurosciencehort communication
timSync: Open-source hardware for behavioral and MRI experiments
hristopher Rordena,b,∗, Taylor Hanayika,b
Department of Psychology, University of South Carolina, Columbia, SC 290208, USAMcCausland Center for Brain Imaging, University of South Carolina, Columbia, SC 290208, USA
r t i c l e i n f o
rticle history:eceived 9 July 2013eceived in revised form 7 August 2013ccepted 17 November 2013
eywords:uman and animal responsesotor-evoked potentialagnetic resonance imaging
Background: Many neuroscience experiments rely on presenting stimuli and measuring participants’responses to these events. Often computer screens, speakers and keyboards are sufficient. However, thesedevices are not appropriate for some situations. For example, some studies present tactile or olfactorystimuli or brain stimulation. Likewise, keyboard buttons are not appropriate for use with vocal responses,small animals or individuals with motor impairments.New method: We describe StimSync, which simulates USB keyboard inputs, allowing use with most exper-imental software. StimSync can measure button presses, optical signals from magnetic resonance imagingsystems, changes in ambient light (e.g. synchronizing intracranial electrography), and auditory events(a voice key). In addition to the USB keyboard mode (necessarily millisecond precision), StimSync canalso be set to provide higher precision timing. This feature can be used to validate timing, ensuring eventsynchronization (e.g. auditory events, visual events, brain stimulation). In addition to recording inputs,StimSync provides seven digital outputs for controlling external devices. Finally, StimSync can recordanalog inputs; we illustrate how this can be used to evaluate the rise time for computer displays.Results: We observed outputs with a mean latency of 2.1 ms (sd = 0.17 ms) and USB inputs with a mean
latency of 2 ms (sd = 0.54 ms).Comparison with existing method(s): StimSync statistically outperforms two professional solutions andnumerically outperforms other devices described in the literature.Conclusions: StimSync (http://www.mccauslandcenter.sc.edu/CRNL/tools/stimsync) provides an open-source solution for controlling and validating neuroscience experiments. In addition to sharing the design,we have produced a batch of devices to demonstrate the market for professional implementations.
. Introduction
Commodity desktop computers have become popular tools forcquiring behavioral responses. For many experiments, it is suffi-ient to ask participants to observe a computer and make responsessing a keyboard. However, there are situations where specialized
nput and output devices are required. For example, we may wisho present olfactory or tactile stimuli, or trigger brain stimulation.ikewise, keyboard buttons may not be appropriate for individualsith poor motor control or for many species of animals. Further, it is
ften desirable to acquire latency information for another modality such as a ‘voice key’ that detects the onset of speech. Finally, these
lectrical buttons are not appropriate for some environments, suchs functional magnetic resonance imaging for two reasons: first thelectrical signals in the wires cause interference in the MRI signal
∗ Corresponding author at: Department of Psychology, University of South Car-lina, Columbia, SC 290208, USA. Tel.: +1 803 404 2573.
(leading to image artifacts, James et al., 2005), and second gradi-ent switching used to generate images could cause heating in thewires (Kugel et al., 2003). Therefore, there is a need for a flexibleinput device – and ideally one that can appear as a simple keyboardto the host computer (so it can be used without modification withmany popular stimulus presentation packages). A second, relatedproblem is to develop hardware that can help validate the precisetiming of different events – for example, ensuring that a stimuli thatappears on a computer screen always occurs at a consistent timerelative to an external trigger signal, such as brief brain stimulationwith transcranial magnetic stimulation.
A historical solution to detecting the closure of custom buttons isto use the parallel (Dalrymple-Alford, 1992) or RS232 serial (Morris,1992) ports of a computer. However, these ports are no longercommon, with the features supplanted by the universal serial bus(USB) port. Further, the stimulus presentation software must be
customized to detect these inputs. Likewise, there are many pro-fessional digital input devices that can record responses, but thesegenerally also require customization of the software. Ideally, onewould want a device that provides accurate timing, appears as a
tandard keyboard to the host computer, and allows virtually anyorm of button to be connected.
One elegant solution for recording button inputs is to emulate aSB Human Interface Device (HID) – this is a standard protocol thatescribes how keyboards, computer mice, joysticks, etc. communi-ate with a computer. Therefore, any HID device will be recognizedn any popular computer (for example, those running modernersions of the Windows, OSX and Linux operating systems). Poten-ially driven by the popularity of computer-based kiosks, photoooths and other enclosures, there are now a number of profes-ional HID devices (for example, we evaluate the inexpensive UHIDano from www.u-hid.com). Similarly, Schubert and colleagues
2013) describe how the open-source Leonardo Arduino hardwarean act as a HID device for mapping an analog input to keystrokesith low variability (∼1 ms), outperforming many commodity key-
oards (Plant and Turner, 2009). They also describe software thatllows the Arduino to and act as a high precision timer when usedith E-Prime. A benefit of the Arduino is that both the software
nd hardware are open source, empowering users to customizehe hardware for their own applications. Here we also present anrduino-based solution to this problem. Our solution extends theork of Schubert et al. (2013) by incorporating additional features
ncluding the optical inputs, digital inputs, digital outputs, a graph-cal user interface for modifying key mappings without requiringrmware to be recompiled. Further, we remedy minor limitationsf the design of Schubert et al. (2013): our design works reliablyith timing functions such as the KbCheck function in Psychtool-
ox and implements keyboard debouncing independently for eachnput.
In addition to detecting the closure of mechanical switches, therduino inputs can also be used to detect electrical inputs fromther devices. Here we describe how these inputs can be adaptedor recording button presses made inside a magnetic resonancemaging (MRI) scanner. Conventional electrical switches can beroblematic as the wires can generate radio interference (Jamest al., 2005) and the metal parts could potentially heat up dueo induction caused by the rapidly switching magnetic gradientsKugel et al., 2003). Here we describe an optical fiber button for usen this situation.
While button inputs allow scientists to measure manualesponses, it is often useful to measure the onset of analog events.or example, a voice key is a popular tool for measuring the onsetf speech and numerous such devices have been described. Forxample, Abrams and Jennings (2004) describe a software solu-ion compatible with the Visual Basic language. The work of Tylert al. (2005) and the proprietary Serial Response Box (Psychologyoftware Tools, Sharpsburg PA USA) are examples of hardware solu-ions, though both were developed for the now legacy RS422/RS232erial ports. More comparable to our design, is the proprietary SV-
USB voice key (Cedrus Corporation, San Pedro, CA, USA). In ouresign, we provide a pair of analog inputs. This can be used as aoice key – when a microphone is plugged into an analog port
keystroke is emulated when the volume of a sound exceeds ahreshold (with thresholds manually adjusted by a knob). Analognputs can be useful for a wide arrange of uses, beyond that of aoice key. For example, one can plug in an ambient light sensorhat will emulate a keystroke when a light exceeds a given thresh-ld. An important use for such inputs is detecting when an imageccurs on a computer screen. Below we exploit this to validate theccuracy of our stimulus presentation. Further, these analog inputsan be recorded and plotted. For example, we evaluated the perfor-ance of three different computer screens when switching from a
lack to a white image.In addition to recording inputs (e.g. button presses or voice
ey responses), it is often useful to control digital outputs. Whileany behavioral studies find that the screens and speakers built
cience Methods 227 (2014) 90–99 91
into modern computers are sufficient for output, some studiesrequire precise control of additional devices. For example, generat-ing a tactile stimulation, controlling the onset of brain stimulationor dispensing juice for an animal all require some form of digi-tal output. There are proprietary solutions to these problems. Forexample, Psychtoolbox provides support for the proprietary Mea-surement Computing 1208FS (Norton, MA, USA) that includes 16digital outputs which we evaluate here (we also created code tocontrol this device with EPrime). EPrime allows commands to besent via a parallel port allowing simple control for digital outputs.Markham (1993) describes simple hardware for controlling out-puts using the parallel port. Unfortunately, legacy parallel ports arenot included on many modern computers. Our StimSync includes7 digital outputs, and we provide EPrime and Psychtoolbox exam-ples for controlling this device. A limitation of EPrime is that it isproprietary. While Psychtoolbox itself is free, it is typically run withthe proprietary Matlab (though it is possible to run it using the freeOctave language on some operating systems). In contrast, PsychoPy(Peirce, 2007) and OpenSesame (Mathôt et al., 2012) are based onthe Python language and completely open source. Therefore, wealso developed and evaluated Python code for controlling digitaloutputs using the StimSync.
Previous validation studies have suggested that commodityinput devices exhibit both latency and variability in logging keypresses. For example, Segalowitz and Graves (1990) report key-boards and PS/2 mice have about a 10 ms delay ± 7.5 ms (that theyinferred was due to polling rate, and therefore regardless of man-ufacture), while serial connection mice buttons had a delay of31 ms with an error of ±2 ms. Using more modern USB devices,Plant and Turner (2009) found that different keyboards varied inlatency from 18 to 34 ms with a standard deviation of 0.8–3.7 ms,whereas mice had latencies of 10–49 ms with standard deviationsof 0.7–4.3 ms. However, one limitation of these studies is that theyemploy proprietary tools in order to establish these values. Con-sider the USB HID device we describe here – how accurate arethe response times recorded using this device? This question isparticularly challenging as the polling rate could in theory differbetween operating systems (e.g. Linux, Windows or OSX), betweencomputers, or even versions of operating systems (e.g. the hard-ware may not perform well on some future, as yet unreleasedversion of Windows). Therefore, an important goal is to providea robust method to validate timing across systems. To achieve this,we added special firmware routines to the StimSync that allowusers to directly measure the accuracy of these devices on theirown system.
Minimal latency and variability are clearly desirable attributesfor input devices. However, we wish to emphasize that for manybehavioral studies any commodity device will provide reasonabletiming. First of all, a delay that is constant across all conditions willhave no bearing on the subsequent statistics. Further, variability onthe scale reported for computer input devices will have little effecton statistical power for detecting relevant behavioral differences(Ulrich and Giray, 1989). However, there are situations whereprecise timing may be important, especially for outputs. For exam-ple, perceptual stimuli generate time-locked electrophysiologicalresponses with temporal precision ensuring that signal averagingis accurately able to discriminate different components. Specif-ically, Intracranial Electroencephalography (iEEG) can provideexquisite temporal information, so a device like a StimSync canhelp timestamp when a stimulus appeared on a computer screen.Likewise, transcranial magnetic stimulation (TMS) pulses cancause very brief behavioral changes, with precise timing allowing
scientists to calculate when a brain region is required. A niceexample of these types of effects is a TMS study by Chambers et al.(2004), where stimulation of parietal cortex impaired perceptionwhen the pulses occurred at 90 ms and 225 ms after the stimuli
ut not in the intervening period. In these situations, temporalrecision is a very important feature for output devices.
Therefore, our goal was to develop a simple open source hard-are device for recording behavioral inputs and outputs. Next, we
valuated this device to compare its performance to proprietaryolutions. Specifically, we evaluated the UHID Nano, as an inputevice, and Measurement Computing 1208 fs as a digital outputevice. We chose these two proprietary solutions based on their
ow price and software support. The Nano is an inexpensive andiny device that can mimic a USB keyboard with up to 8 digitalnputs. For digital outputs, we tested the Measurement Computing208 fs. The 1208 fs is supported by Psychtoolbox and includes 16igital outputs. We also created software that allows the 1208 fso be controlled by EPrime. Our aim was to demonstrate that thetimSync performs similarly to these professional solutions.
. Methods
.1. Basic design
We developed our code to work with Arduino-compatibleicrocontrollers that can mimic a USB keyboard. These include
he Arduino Leonardo and PJRC Teensy 2 that use the AVR 5 V 8-it central processing unit (CPU) as well as the Arduino Due andJRC Teensy 3 that use 3.3 V 32-bit ARM CPUs. While our firmwares compatible with all these devices, our reference design uses theJRC Teensy 3 (http://www.pjrc.com), due to its small size, low cost,nd high USB bandwidth. We note that the Teensy does use propri-tary BIOS, and therefore is not as open as other Arduino designs.aving firmware compatible with all these designs prevents vendor
ock-in. Note that not all of these devices support all of our features,pecifically the Arduino Due does not include Electrically Erasablerogrammable Read-Only Memory (EEPROM), so this device will
evert to the default keyboard mappings when it is unplugged from
computer.By default, our design communicates and receives power via
he USB port. However, our design reserves pins 0 and 1 of the
ig. 1. Schematic for StimSync. The design includes eight digital inputs (i0–i7), here sholectrical signals from other devices. These switches use 20,000 � pull-up resistors built inhows seven digital outputs (o0–o6), each shown here with a light emitting diode and restate relays, etc. The design also includes two Analog sensors (A0, A1) that provide voltnd 5 V for AVR-based microcontrollers), each paired with a threshold knob (A2, A3: 10lluminates whenever the sensor voltage is greater than the threshold voltage. Also show
icrosecond timing mode and oscilloscope mode (though not USB keyboard inputs). Nottatus: on continuously during USB keyboard mode, flashing during oscilloscope mode an
cience Methods 227 (2014) 90–99
microcontroller for a Bluetooth serial device. This allows inexpen-sive Bluetooth modules to be used for wireless digital outputs,microsecond recording and streaming of analog data. For example,for our reference design we use JY-MCU modules that can be pur-chased for around $8 and can be programmed (with firmware weinclude) to communicate at 460,800 bps. These wireless modulesemulate a legacy serial port, allowing the StimSync to control dig-ital outputs, work in ‘oscilloscope’ mode to record analog inputsand work in microsecond timing mode. However, the Bluetoothmodules do not allow normal USB HID keyboard emulation (as theconnection mimics a standard serial port rather than a USB device).
A schematic of our design is shown in the left panel of Fig. 1.The schematic can be used to build a StimSync using an electri-cal breadboard and a Leonardo, Due, Teensy 2 or Teensy 3. Wealso created and distribute a reference printed circuit board (Fig. 2)based on the Teensy 3 using Fritzing. The resulting Fritzing andGerber files can be mass produced, for example using open sourcefabricators such as Fritzing (http://fab.fritzing.org/) and Oshpark(http://oshpark.com). Our design allows eight digital inputs con-nected to pins 2.9. These pins utilize the microcontroller’s built-inpull-up resistors, so they are driven to the microcontrollers’ posi-tive voltage when open. Connecting any one of these pins to ground(0 V) generates a key press signal, and subsequent disconnectionfrom ground generates a key release signal. One option is to havethe Arduino communicate with the computer using the Arduino’s“print” or “write” command (Schubert et al., 2013). However, thisapproach sends a very brief keyboard signal to the computer, whichare not detected by some routines such as the “KbWait” commandin Psychtoolbox (requiring the usage of the “KbQueue” command).In contrast, we suggest using the Arduino’s “press” and (after adelay) the “release” command, emulating the way an actual key-board functions. This latter solution works with all software wehave evaluated.
We provide graphical software that allows each pin to beassigned to any number or letter. For example, pin 2 could beassigned to generate a ‘2’ character when a button is pressed and a‘b’ character when the button is released. Our software reprograms
wn connected to momentary switch push buttons though note these could also beto the Arduino to ensure that they read high when the switch is open. The schematicistor, though note these could be a variety of components such as transistors, solid
age from the range of 0–Vdd (where Vdd is 3.3 V for ARM based-microcontrollers K potentiometers that can be dialed to 0–Vdd) and a detection light (A4, A5) that
n is an optional Bluetooth module that allows wireless control of digital outputs,e that there is also a built in light (pin 13 on most devices) that is used to indicated blinking twice when in microsecond timing mode.
Fig. 2. A photograph of our reference StimSync design with a light sensor (red) anda fMRI optical pulse sensor. The right panel shows a photograph of our referencedesign (center) with buttons (top), LED light (left), analog input sensors (bottom,showing sound, knob, temperature and light sensors) and fMRI pulse trigger (right).(r
tw
etAnWesbbgsAtdSrmbaSmaclomadXt
For interpretation of the references to color in this figure legend, the reader iseferred to the web version of the article.)
he EEPROM firmware in the Arduino and Teensy, so these changesill be reflected when the StimSync is connected to any computer.
The StimSync has two analog input channels (pins A0 and A1)ach linked to a wheel potentiometer (A2 and A3) and a light emit-ing diode (A4 and A5). For example, if the voltage of analog input0 is greater than the threshold voltage set with the knob con-ected to A2, then the light on A4 will be on, else it will be off.henever the voltage dips below the threshold a key press is gen-
rated, and when the voltage rises above the threshold a key releaseignal is generated. For example, for a voice key the threshold cane adjusted so that a keyboard signal is sent when the participantegins speaking, yet the background noise is not sufficient to trig-er a signal. For our reference design, these analog channels use theame connectors as the Seeed Studio Electronic Brick and Phidgetnalog sensors. These jacks plug into the socket in only one orienta-
ion, ensuring correct polarity. This design also allows any sensorsesigned for the Electronic Brick and Phidget to plug in to the Stim-ync (as long as they work at the voltage of the CPU). A voice keyequires a microphone input. We found that commodity computericrophones with the microphone mounted on a boom work well,
ut these require a pre-amplifier (e.g. a transistor or operationalmplifier to boost the signal). Our reference design is based on theparkfun Breakout Board for Electret Microphone (BOB-09964). Foreasuring screen intensity we used the Sparkfun TEMT6000-based
mbient light sensor (BOB-08688). Both inputs use open sourceircuit designs. In addition to emulating a binary switch, these ana-og inputs can also be tracked and recorded across time by usingur computer software that puts the StimSync into ‘oscilloscopeode’. In this mode the StimSync streams data to the computer at
regular rate (using interrupts to ensure regular intervals), and isisplayed graphically using software that we provide in the Lazarus,Code, Matlab and Processing languages. The computer can select
he sampling rate as well as the number of channels desired, and
cience Methods 227 (2014) 90–99 93
the StimSync then begins streaming binary data for both the ana-log and digital channels. We demonstrate how this can be used toobserve and record the rise time for a computer screen switchingfrom a dark to bright screen.
One issue with both digital and analog inputs is de-bouncing. Amechanical button works by closing an electrical circuit, but duringthe initial vibration the circuit often opens and closes rapidly. Like-wise, with a voice key we typically log the initial vocal response butnot the onsets of the subsequent phonemes. Our software allowsthe user to select a desired debounce time. For example, if thedebounce time is set to 100 ms, an input will send an initial sig-nal when the circuit is closed but disregard changes for the nexttenth of a second. Our debouncing algorithm works independentlyfor each input, as it is often desirable to detect response times tomultiple keys simultaneously.
The StimSync also includes pins for digital outputs. The Teensy3 supports 7 outputs (pins 10.12, 20.23), the Teensy 2 supports 5outputs (pins 10, 12.15), the Arduino Due and Leonardo support 3outputs (pins 10.12). For ARM CPUs, these provide up to 9.4 mA at3.3 V, and the AVR CPUs provide up to 20 mA at 5 V. This is sufficientto power light emitting diodes (LEDs). For devices that require morecurrent, these outputs can be used to drive optically isolated tran-sistors (e.g. 4N35) or solid-state relays (e.g. AQZ207). The StimSyncweb page includes examples for each of these circuits. Controllingthe StimSync outputs is done by connecting to the device as anemulated serial port and writing a single byte value to the port.For example in Matlab one could “fwrite(ser,data)” – where ser isthe open serial port and data is a value 0–127 that specifies thebinary values for the outputs. For example, writing the value 21would turn on the first (20̂= 1), third (22̂= 4) and fifth (24̂= 16) out-puts while switching the others off. For Matlab we developed andprovide the wrappers SerDeviceIndex.m and SerDOut.m that mimicPsychtoolboxs’ DaqDeviceIndex.m and DaqDOut.m for the Mea-surement Computing 1208 fs. We also developed EPrime examplesthat illustrate the usage of the StimSyc as well as the 1208 fs.
Arduinos provide an in-built light emitting diode (pin 13 onmost devices, pin 11 on the Teensy 2). We use this to signal the sta-tus of the microcontroller. When acting as a keyboard this light is oncontinuously, when streaming analog data it blinks on and off, andwhen sending microsecond data it briefly flashes twice followed bya longer pause.
We developed custom software for using the StimSync. Our key-board mapping software allows the user to define which inputcorresponds to which keyboard press (Fig. 3). This software alsologs the microsecond onsets of any inputs, creating a tab-delimitedtext file that can be analyzed to measure the onset of any event. Wehave also created simple Psychtoolbox, OpenSesame, PsychoPy andEPrime scripts that demonstrate how to use the inputs and outputs.Finally, we provide software for plotting and recording the ana-log channels written in the Matlab, Processing, Lazarus/FreePascaland XCode languages. These tools can run on the most popu-lar operating systems. We also provide a full online descriptionof the communication between a computer and the StimSyncthat include examples of the commands in the Python language(http://www.mccauslandcenter.sc.edu/CRNL/stimsync-api).
2.2. MRI compatible extensions
Functional Magnetic Resonance Imaging (fMRI) is a popularmethod for inferring brain activation. This method uses a powerfulstatic magnetic field, rapidly switching field gradients and radiofrequency signals to acquire brain images. This environment is
unsuitable for many electrical and metal devices. Potential prob-lems include heating, projectile movement and image artifacts.Therefore, most participant response devices are not compati-ble for this environment. Several commercial companies have
Fig. 3. Software to support the StimSync. StimSync simulates a keyboard, and cantherefore be used with any software that supports a keyboard. We also include akey mapping program that allows the user to configure the actions generated whena button is pressed or released, this software also generates a text log file for inputs,allowing users to validate onset timing. Further, we include oscilloscope softwarewr
dupkeaawruru
gram the 220,000 � resistor and 0.1 �F capacitor yields a delay of
FHrb3
ritten in Matlab, XCode, Processing, and Lazarus that allow the user to display andecord inputs, for example, screen intensity.
eveloped response boxes especially for fMRI, with most designssing optical fiber cabling inside the magnet hall. However, theseroprietary designs may not be suitable for all situations (e.g. wenow of no commercial foot pedal), and tend to be expensive. Jamest al. (2005) demonstrate that a specific commodity USB keyboardppears safe and generates little artifact during fMRI scanning on
3T MRI system. One concern is that the same device may notork as well with a different MRI sequence or if the cabling were
outed differently (Nakamura et al., 2001). Sommer et al. (2004)
se commodity audio optical fiber and adapt a computer mouse forecording responses. Unfortunately, the optical components theyse are no longer widely available. Hollinger et al. (2007) describe
ig. 4. Schematic for an MRI compatible response button. The analog inputs of an ArduFBR1524 is used to illuminate a fiber optic cable. Closure of a momentary switch button r
ead by the Arduino Analog input (e.g. A0). Note that there are no electronic componentsuttons are closed, and therefore digital keyboard responses are signaled when the volta3 mA, so a large number of buttons will require more power than is available on from th
cience Methods 227 (2014) 90–99
an optical piano creating a fiber-optic MIDI device. Our own designis very similar to that of Sommer et al. (2004), Hollinger et al.(2007) and the commercial alternatives: an optical fiber is illumi-nated, when the participant presses a button this attenuates lighttransmission to a sensor. For our design we use the HFBR1524 lightsource and HFBR2524 light sensor which both use the VersatileLink optical connections. MRI manufacturers commonly use thisoptical connector, so the optical cabling and crimping tools areeasy to acquire. Our design is very simple: one end of the fiber isilluminated, and a button placed half way along the fiber acts as aguillotine such that the sensor at the other end can detect the but-ton state. Our implementation uses constant illumination, and setsthe button-press threshold by measuring light transmission whenthe system starts up. We note that other methods are possible, suchas using a potentiometer to manually adjust the threshold or puls-ing the light to both conserve power and to measure the change inillumination. The electronics of this design are very simple, but themanufacturing of the button itself requires great care (especiallyif one wants a positive tactile response on button closure): in ourimplementation we sliced the optical fiber and had the buttonbreak a 1 mm air gap. We created a foot pedal using an AquilineLinemaster 971-S plastic enclosure where the internal metal springwas replaced with a plastic spring. This design is shown in Fig. 4.
Another fMRI specific requirement is synchronizing the onsetof MRI acquisition with the events that occur during an experi-ment. Most manufacturers provide an optical signal that designatesthe start of acquisition for each volume. Detecting these pulsescan help time lock the start of a behavioral experiment with thestart of the MRI acquisition and can also help detect any driftbetween the clocks of the computer running the MRI from thecomputer acquiring behavioral data. Some vendors of MRI scan-ners provide devices to convert there optical signals to USB inputs.However, their devices are expensive and therefore are not avail-able at all centers, particularly sites that are predominantly focusedon clinical work. Some vendors of MRI-compatible response but-tons include inputs for these optical triggers, but this feature is notuniversal. In Fig. 5 we illustrate a simple trigger that converts a briefoptical pulse into a longer electrical signal. This electrical signal canbe reliably detected by a StimSync or UHID Nano. The circuit uses a555-series integrated circuit to convert the very brief optical signalto a longer (with duration in seconds equal to 1.1*R*C, so in the dia-
24 ms) electrical signal. This pulse duration is long enough to allowthe user to visually detect a flicker from the included light emittingdiode. Further, this pulse length ensures that the Arduino never
ino-compatible device can also be used for MRI compatible input buttons. Here aeduces the amount of this light reaching a HFBR2524 detector, reducing the voltage
or metal inside the magnet hall. On startup, the Arduino assumes that none of thege drops below this baseline voltage. Note that in this design the HFBR1524 drawse Universal Serial Bus (USB) cable, requiring an external power source.
C. Rorden, T. Hanayik / Journal of Neuros
Fig. 5. Simple schematic for converting a brief optical pulse from a MRI scanner intoa keyboard input using either a UHID Nano or StimSync. An HFBR-2524 converts theoptical signal to an electrical signal, and a 555 monostable timer converts this verybrief pulse into a signal with a longer duration (set by the resistor and capacitoraNd
fts
2
pewspphpnS8Tt2cdtF
Fb
ttached to pins 6 and 7, here 24 ms). The output if the 555 can either be a UHIDano (shown) or an Arduino digital input. The diagram also shows a light emittingiode attached to the output, which dims with each pulse.
ails to detect a pulse (reading each input takes a finite amount ofime, so even when an Arduino is tasked with continuously pollingeveral inputs it may fail to detect a very brief signal).
.3. Validation
To evaluate the accuracy of digital inputs we examined theerformance of both the proprietary UHID Nano and StimSyncmulating USB keyboards. We conducted a simple experimenthere a dark screen switched to a bright image. An ambient light
ensor detected the brightness change and sent a keystroke to theresentation software, emulating an individual performing a sim-le visual detection task. We used a cathode ray tube monitor thatas a very rapid onset (as we demonstrate below). We tested threeresentation software tools on two computers: EPrime 1.2.6 run-ing on the Windows XP operating system (using a Sony VAIOZ791N with 2.5 GHz Intel Core 2 CPU, using the NVidia GeForce400GS video card while running from wall current), and Psych-oolbox 3.0.10 revision 3405, as well as PsychoPy 1.77 running onhe OSX 10.8 operating system (using a MacBook Pro 13” with a.4 Ghz Intel Core 2 Duo CPU, and the NVIDIA GeForce 9400 M video
ard, running off wall current). Note that the electron gun of a CRTraws an image over the screen in a finite time, so the placement ofhe optical sensor influences the apparent time of a screen change.or both the EPrime and Psychtoolbox testing the light sensor was
ig. 6. Evaluation of digital inputs in Psychtoolbox. A key press was generated when theoth the StimSync and UHID Nano USB devices across one thousand trials. Note that the S
cience Methods 227 (2014) 90–99 95
mounted on the same location of the CRT, with the StimSync set tothe same threshold (we tested PsychoPy during a separate session).One thousand trials of data were acquired, with the data fromthe first trial discarded. Note that EPrime’s data logging recordsevents in integer milliseconds. Therefore, assuming random syn-chronization between the screen refresh and the timer, even aperfect device would appear to have a standard deviation of around0.5 ms. To allow other users to evaluate their own equipment,we provide a timing validation diagram and scripts for PsychoPy(http://www.mccauslandcenter.sc.edu/CRNL/tools/stimsync-api).
We conducted a similar study to test the performance of thedigital outputs. We sent an electrical signal that was intended tocoincide with the image presentation. We measured these onsetsusing a separate StimSync attached to a different computer thatlogged both the screen onset time and the electrical signal. Wetested EPrime on Windows and PsychToolbox and PsychoPy onOSX. We tested two output devices: the StimSync as well as theMeasurement Computing 1208 fs. For all devices, we acquired 1000trials, discarding the first sample (to avoid variability due to thesystem loading files, system priority changes, or changing powerstates).
We also wanted to evaluate the precision of the StimSync in the‘microsecond’-timing mode. The StimSync sequentially measureseach of the inputs, with the sampling of each input (particularlyanalog inputs) requiring a finite amount of time. In addition, thedevice may be regularly interrupted from this sampling to do othertasks such as USB communication that could also impact the samp-ling rate. Therefore, we measured the variability of measuring thetemporal jitter observed in a periodic square wave sent from awaveform generator.
Finally, we used the oscilloscope mode of the StimSync to eval-uate the response time of three different computer displays. Weused the ambient light sensor to record the intensity of the screensas they transitioned from a black to white image. Our aim was todemonstrate that the StimSync is an effective alternative to theproprietary BlackBoxToolkit (Plant and Turner, 2009) for evaluat-ing devices. We chose a CRT display, a commodity laptop, and a LCDdisplay that is marketed as being suitable for flicker shutter glasses.
3. Results
The analysis revealed that the StimSync had significantlylower latency and variability than the commercial UHID Nanofor digital inputs in both EPrime and Psychtoolbox for thetested hardware. For Psychtoolbox the mean input latency for the
computer screen flashed, and the latency between these two events is plotted fortimSync exhibits lower latency and less variability.
96 C. Rorden, T. Hanayik / Journal of Neuroscience Methods 227 (2014) 90–99
nd UH
Sp(tfdPd
lCc4wi4w
Ff
Fig. 7. Histogram showing typical latencies of StimSync a
timSync was 1.99 ms vs. 6.95 ms for the UHID Nano (t(1996) = 49.2; < 0.001), and the standard deviation was 0.54 ms vs. 2.46 msf(998,998) > 11.7; p < 0.001), as shown in Figs. 6 and 7. For EPrimehe mean input latency for the StimSync was 8.83 ms vs. 12.39 msor the UHID Nano (t(1996) = 47.06; p < 0.001), and the standardeviation was 0.46 ms vs. 2.34 ms (f(998,998) > 11.7; p < 0.001). ForsychoPy the StimSync provides a mean of 2.03 ms with a standardeviation of 0.33 ms (with a range of 1.2–2.94 ms).
Our analysis also revealed that the StimSync had significantlyower latency and variability than the commercial Measurementomputing 1208 fs for digital outputs in both EPrime and Psy-htoolbox. The mean latency in Psychtoolbox was 2.06 ms vs..56 ms (t(1996) = 127.26; p < 0.001), and the standard deviationas 0.17 ms vs. 0.59 ms (f(998,998) > 11.7; p < 0.001), as shown
n Figs. 8 and 9. The mean latency in EPrime was 42.72 ms vs.4.25 ms (t(1996) = 161.85; p < 0.001), and the standard deviationas 0.05 ms vs. 0.3 ms (f(998,998) > 11.7; p < 0.001). For PsychoPy
ig. 8. Evaluation of digital outputs in Psychtoolbox. A digital output was generated whenor both the StimSync and Measurement Computing 1208 fs across one thousand trials. N
ID Nano when used as digital inputs with Psychtoolbox.
the StimSync delivered a mean latency of 1.65 ms with a standarddeviation of 0.20 ms (with a range of 1.1–2.14 ms).
The microsecond timer function was evaluated by using a 20 Hzsignal generated by a DSO Nano Oscilloscope (Seeed Studio, Shen-zhen China) as a digital input. Across 1000 trials we measured amean duration between pulses of 49.996973 ms and a standarddeviation of 0.08 ms (80.49 �s). The range between the shortest andlongest duration was 0.2 ms (209 �s). Note that these errors can beexpected to reflect the combined error of the measuring device andthe signal generator.
We recorded the rise and fall time of screen switching betweenblack and white with three computer monitors. The results areshown in Fig. 10. The Cathode Ray Tube exhibited a rapid pulse witheach refresh, the commodity laptop liquid crystal display showed
a relatively slow switching speed, and the high performance desk-top liquid crystal display showed more rapid switching. We didnot conduct any statistics, as these properties are likely to vary
the computer screen flashed, and the latency between these two events is plottedote that the StimSync exhibits lower latency and less variability.
C. Rorden, T. Hanayik / Journal of Neuroscience Methods 227 (2014) 90–99 97
ment
bc
4
dttkpStp
Fsls
Fig. 9. Histogram showing typical latencies of StimSync and Measure
etween labs. However, the figure illustrates the variability thatan be observed with different devices.
. Discussion
Our primary aim was to design and validate a simple open sourceevice for recording responses, presenting stimuli and validatinghe performance of behavioral experiments. This is useful for situa-ions where commodity components such as computer screens andeyboards are not sufficient. We found that our StimSync device
erformed at least as well as two professional solutions. In addition,timSync includes additional features including a microsecond-iming mode, analog inputs (e.g. voice key) and an oscilloscoperoviding a useful tool for validating behavioral experiments.
ig. 10. StimSync can record and log analog inputs. For example, here is the performancwitching from a black to a white screen. The Cathode Ray Tube (CRT) pulses with each 6iquid crystal display (that is marketed for its fast switching performance) shows a muchcientists are able to accurately measure the performance of their own equipment.
Computing 1208 fs when used as digital outputs with Psychtoolbox.
The StimSync showed statistically less variability than theUHID Nano, and numerically outperformed the commodity USBinput devices described by Plant and Turner (2009). We note thatSchubert et al. (2013) also examined the performance of an ArduinoLeonardo emulating a USB keyboard device (their study 6) andfound EPrime had a standard deviation of 0.53 ms, very similar tothe 0.47 ms we report here. However, we note that in both casestiming was recorded using EPrime where times are recorded onlywith millisecond precision. The theoretical limit for timing withthis resolution is a standard deviation of around 0.5 ms (unless thestimulus and response are phase locked to the timer’s clock). Inter-
estingly, while Psychtoolbox records data with higher resolution(storing timing information as number of seconds with double pre-cision), we still observed a standard deviation of around 0.5 ms. Wespeculate that this reflects the polling rate of the USB device. In
e of three different computer monitors that exhibit very different properties when0 Hz refresh, the laptop screen showed a very sluggish rise time, and the desktop
more rapid rise profile. By simply attaching an ambient light sensor to a StimSync,
9 euros
otai1tehrtb1cbtd
tstbabstdtnrt
tcsntbPqs
dptsbaiitmusrcddl
iAwtTve
8 C. Rorden, T. Hanayik / Journal of N
ther words, our data suggest that our device was sampled nearhe 1 ms rate requested by the Arduino/Teensy (the fastest ratellowed by the UHID standard). This suggests that the StimSyncs typically performing near the theoretically optimal level for a
ms polling rate, though careful inspection of Fig. 6 suggests thathere are occasional outliers that we discuss later. Both Schubertt al. (2013) and our own design provide methods for timing withigher precision, but we argue that this precision is probably notequired by most studies in our field. Specifically, we emphasizehat due to the central limit theorem all of these devices are proba-ly suitable for most psychological experiments (Ulrich and Giray,989). We feel the main benefit of the StimSync USB inputs overommodity devices is the flexibility of choosing different types ofuttons (e.g. much smaller or larger than standard keyboard but-ons), and the ability to use thresholded analog inputs to triggerigital inputs.
Likewise, when turning to digital outputs, the StimSync sta-istically outperformed the Measurement Computing 1208 fs. Weuspect this is due to the fact that controlling the seven digi-al outputs of the StimSync requires transmitting only a singleyte of data over the serial port, whereas most devices require
sequence of bytes (and calling an external library). Regardless,oth devices probably provide suitable timing for most neuro-cience studies. It should be noted that the 1208 fs does providerue analog outputs, whereas contemporary Arduino-compatibleevices only provide pulse-width modulation (PWM) outputs (withhe exception of the Due that provides true digital to analog sig-als). Again, our aim is not to criticize proprietary devices, butather to empower scientists with a flexible open source alterna-ive.
For all input and output devices, we did observe a few outlierrials with unusually long latencies. We suspect these effects areommon in real world studies that attempt to present complextimuli using commodity multitasking operating systems that areot designed for real time precision. Furthermore, careful inspec-ion of Figs. 6 and 8 reveals that there is some timing autocorrelationetween trials. Similar effects can be observed in the data oflant and Turner (2009). Again, we suggest that the size and fre-uency of these errors is not significant for most neurosciencetudies.
For situations where the USB human interface device modeoes not provide sufficient temporal precision, our device also sup-orts a ‘microsecond’ precision mode. By attaching a sensor tohe output device (ambient light sensor attached to a computercreen, microphone attached to computer speaker) one can logutton response times or send electrical signals on the stimulusnd response onsets to another device (e.g. time stamps logged forntracranial recording). In addition, in situations where precise tim-ng is required, it is worth noting that the microcontroller insidehe StimSync can be reprogrammed to actually run the experi-
ent, removing the complexity and variability inherent with these of a multitasking computer. For example, the StimSync couldwitch on and off devices connected to the digital outputs andecord responses observed on the digital inputs. While this pre-ision is beyond needs of most users, limits the types of outputevices, and requires a high level of programming proficiency, itoes emphasize the flexibility of Arduino-based hardware in the
aboratory.The StimSync provides an open source alternative for measur-
ng these errors, complementing the proprietary BlackBoxToolkit.s a validation tool, these devices can help evaluate future hard-are and software. The sheer variety of commodity devices means
hat any study can only evaluate a small number (e.g. Plant andurner, 2009) on a limited set of computers. The StimSync pro-ides an affordable method for a scientist to evaluate their ownquipment and validate the timing of their experimental software.
cience Methods 227 (2014) 90–99
Our evaluation suggests that the microsecond-timing mode offersa precision that is not possible with UHID devices that are limitedto 1 ms polling rates.
We also evaluated three computer monitors. This experimentwas designed to emphasize the variability between differentmonitors. Our device provides a simple tool for scientists toevaluate the screens they have available, regardless of make,model or future technological developments. Further, by send-ing a trigger pulse at the time of an intended screen change onecan observe the delay between the computer sending an imageand the display showing this image. This may be particularlyuseful for monitors that buffer the input data causing displaylatency.
Our Arduino firmware can be programmed onto any Arduino-compatible device that supports USB–keyboard emulation, includ-ing Arduino Leonardo, Arduino Due, Teensy 2, and the Teensy 3.This provides an inexpensive, flexible and easy to use platform forconducting experiments. In addition, we have produced a limitednumber of custom printed circuit boards that we will sell to inter-ested scientists. These boards include snap-fitting sockets that arecompatible with many available analog sensors. Our aim is not tomass market these, but rather establish the market potential forlarger production runs.
Acknowledgments
We are selling a limited number of devices. Our goal is to sellthese near our cost. While not our intention, we may make a smallprofit selling these devices. All our software is distributed under aBSD license, and we provide industry standard Gerber files of ourprinted circuit board. Any user or vendor is free to clone our designand sell them as they wish. This work was supported by the NationalInstitutes of Health (R01 DC009571).
References
Abrams L, Jennings DT. VoiceRelay: voice key operation using visual basic.Behavior Research Methods, Instruments & Computers 2004;36(4):771–7.
Chambers CD, Payne JM, Stokes MG, Mattingley JB. Fast and slow pari-etal pathways mediate spatial attention. Nature Neuroscience 2004;7(3):217–8.
Dalrymple-Alford EC. Response-key input via the IBM PC/XT/AT’s parallel printerport. Behavior Research Methods, Instruments & Computers 1992;24(1):78–9.
Hollinger A, Steele CJ, Penhune VB, Zatorre RJ, Wanderley M. fMRI-compatibleelectronic controllers. In: Proceedings of the 2007 international con-ference on New Interfaces for Musical Expression (NIME07); 2007.p. 246–9.
James GA, He G, Liu Y. A full-size MRI-compatible keyboard response system. Neu-roimage 2005;25(1):328–31.
Kugel H, Bremer C, Püschel M, Fischbach R, Lenzen H, Tombach B, Van Aken H,Heindel W. Hazardous situation in the MR bore: induction in ECG leads causesfire. European Radiology 2003;13(4):690–4.
Markham MR. An interface for controlling external devices via the IBMPC/XT/AT parallel port. Behavior Research Methods, Instruments & Computers1993;25(4):477–8.
Mathôt S, Schreij D, Theeuwes J. OpenSesame: an open-source, graphical experi-ment builder for the social sciences. Behavior Research Methods 2012;44(2):314–24.
Morris CC. Using the IBM-compatible microcomputer’s serial port as aninput–output interface. Behavior Research Methods, Instruments & Computers1992;24(3):456–60.
Nakamura T, Fukuda K, Hayakawa K, Aoki I, Matsumoto K, Sekine T, Ueda H, ShimizuY. Mechanism of burn injury during magnetic resonance imaging (MRI)-simpleloops can induce heat injury. Frontiers of Medical & Biological Engineering2001;11:117–29.
Peirce JW. PsychoPy – psychophysics software in Python. Journal of NeuroscienceMethods 2007;162(1–2):8–13.
Plant RR, Turner G. Millisecond precision psychological research in a world of com-modity computers: new hardware, new problems? Behavior Research Methods2009;41:598–614.
Schubert TW, D’Ausiliob A, Cantob R. Using Arduino microcontroller boards to mea-sure response latencies. Behavior Research Methods 2013 [in press].
egalowitz SJ, Graves RE. Suitability of the IBM XT, AT, and PS/2 keyboard, mouse and
game port as response devices in reaction time paradigms. Behavior ResearchMethods, Instruments & Computers 1990;22(3):283–9.
ommer J, Deppe M, Jansen A, Krach S, Knecht S, Ringelstein EB. A cheapto build and easy to use response box for fMRI-experiments. Neuroimage2004;22(S1):TH337.
cience Methods 227 (2014) 90–99 99
Tyler MD, Tyler L, Burnham DK. The delayed trigger voice key: an improved
analogue voice key for psycholinguistic research. Behavior Research Methods2005;37(February (1)):139–47.
Ulrich R, Giray M. Time resolution of clocks: effects on reaction timemeasurement—good news for bad clocks. British Journal of Mathematical andStatistical Psychology 1989;42(1):1–12.
