Blackwell Science, LtdOxford, UKAORArtificial Organs0160-564X2003 International Society for Artificial Organs271110051015Original Article
MODEL FOR INTRACORTICAL VISUAL PROSTHESIS RESEARCHP. TROYK Et al.
Received July 2003.Address correspondence and reprint requests to Dr. Phillip R.
Troyk, Pritzker Institute of Biomedical Science and Engineering,Illinois Institute of Technology, 10 West 32nd Street, Engineering1-116, Chicago, IL 60616-3793, U.S.A. E-mail: [email protected]
1
Drs. Kufta and Schmidt are now retired from the NIH.
A Model for Intracortical Visual Prosthesis Research
*Pritzker Institute of Biomedical Science and Engineering, Illinois Institute of Technology, Chicago, IL; †Laboratory of Neural Control, National Institutes of Health, Bethesda, MD; ‡The University of Chicago, Chicago, IL; §EIC Laboratories, Norwood,
MA; ¶Huntington Medical Research Institute, Pasadena, CA, U.S.A.
Abstract:
In the field of visual prosthesis research, it hasgenerally been held that animal models are limited to test-ing the safety of implantable hardware due to the inabilityof the animal to provide a linguistic report of perceptions.In contrast, vision scientists make extensive use of trainedanimal models to investigate the links between visual stim-uli, neural activities, and perception. We describe an animalmodel for cortical visual prosthesis research in which novelanimal psychophysical testing has been employed to com-pensate for the lack of a linguistic report. One hundred andfifty-two intracortical microelectrodes were chronically
implanted in area V1 of a male macaque. Receptive fieldmapping was combined with eye-tracking to develop areward-based training procedure. The animal was trainedto use electrically induced point-flash percepts, called phos-phenes, in performing a memory saccade task. It is ourlong-term goal to use this animal model to investigate stim-ulation strategies in developing a multichannel sensorycortical interface.
Implementation of a prosthetic device to substi-tute for normal vision in humans has been a goal ofneural prosthetic researchers for over 30 years. Dur-ing that time, fundamental studies of implantabledevices, electrode materials, and human psychophys-ics have demonstrated that it is feasible to producevisual percepts (phosphenes) through stimulation ofprimary visual cortex (V1) by electrical currents.
Our approach to a cortical visual prosthesis,depicted in Fig. 1, consists of implanted arrays ofpenetrating intracortical microelectrodes whosesuperstructures “tile” the surface of the cortex, withelectrode lead wires connected to fully implantedelectronic stimulator modules. Power for, and com-munication with, the stimulator modules will beaccomplished through a transcutaneous inductivelink. A transmitter coil on the surface of the scalp isdriven by an extracorporeal transmitter connectedto a video processing system whose real-time video
camera provides visual input. Movement of the cam-era, or the image, can be tied to eye movements, thusavoiding the problem of an image that is stationaryin the visual field. Although somewhat futuristic, it isconceivable that camera technology will develop soas to permit implantation of the camera directly intothe eye.
As early as 1918, Löwenstein and Borchardt (1)reported that while performing an operation toremove bone fragments caused by a bullet wound,the patient’s left occipital lobe was electricallystimulated, and the patient perceived flickering inthe right visual field. Foerster (2) and Krause (3)reported similar cases of visual perception caused byelectrical stimulation of the visual cortex duringremoval an occipital epileptic focus. The significanceof these studies was that they demonstrated that theposition of electrically induced visual percepts withinthe visual field was systematically related to the areaof the occipital lobe that received the electrical stim-ulation. Urban (4) inserted electrodes through anoccipital burr hole 3 cm above the inion and 3 cmfrom the mid-line for the purpose of ventriculogra-phy in six patients, one of whom one was blind. Allpatients, including the blind one, perceived spatial
1006 P. TROYK ET AL.
Artif Organs, Vol. 27, No. 11, 2003
visual sensations of various colors and shapes. Stim-ulation of the human visual cortex by Penfield andRasmussen (5) and Penfield and Jasper (6) revealedthat phosphenes could be produced by electricalstimulation. Their subjects described the visual sen-sations as stars, wheels, discs, spots, streaks, or lines.In a 1962 study performed by Button and Putnam(7), four occipital lobe electrodes with percutaneouswires were implanted into each of three human sub-jects, with the wires connected to an apparatus thatvaried the electrical stimulus amplitude and fre-quency based upon the output of a cadmium-sulfidephotocell. The subjects were able to scan an area,holding the photocell in their hand, and grosslydetermine the location of illuminated objects.
Following the elucidation of cortical visual pro-cessing by Hubel and Wiesel (8), the first opportunityto investigate chronic stimulation of the visual cortexresulted from a study by Brindley and Lewin (9), inwhich a 52-year-old woman received an implantedstimulation system consisting of 80 platinum elec-trode discs, placed on the surface of the occipitalpole. Eighty associated transcutaneously poweredimplanted stimulators were placed over most ofthe surface of the right cortical hemisphere.
Approximately 32 independent visual percepts wereobtained. Brindley performed mapping studies andthreshold measurements. Although some attemptwas made to combine the phosphenes into crudeletters and shapes, the implant did not prove to beof any practical use to the subject. Another subjectreceived a second 80-channel implant in 1972 (10–13). Of the 80 implanted electrodes, and stimulators,79 of them produced visual percepts of varied sizeand shape. These were meticulously mapped over3 years. Dobelle (14–16), Pollen (17) and others con-tinued to investigate stimulation of the visual cortexthrough surface electrodes, using relatively largeelectrodes placed on the pia-arachnoid surface inindividuals who were totally blind, following lesionsof the eyes and optic nerves. Dobelle et al. implantedat least three subjects with cortical surface arrays.They also tested the ability of the implanted subjectsto use the perceptions produced by the electrodes to“read visual Braille” (16). Reading rates were con-siderably less than what could be obtained by tactileBraille. One of these subjects, who had retained theelectrode array implanted much earlier, received animproved computer-controlled image processing sys-tem to convert images into stimulus sequences (18).
FIG. 1.
Left: Concept of an intracortical visual prosthesis. A camera captures the image and transmits stimulation patterns through atranscutaneous link. Stimulation of microelectrodes implanted in the visual cortex results in a perception of the image. In a practicalsystem, the camera would be integrated into eyeglass frames and tied to eye movements. Right: Depiction of electrode tiles across thesurface of area V1.
MODEL FOR INTRACORTICAL VISUAL PROSTHESIS RESEARCH 1007
Artif Organs, Vol. 27, No. 11, 2003
All of the reports cited above provided littleinsight as to how electrical stimulation of the visualcortex might be used to communicate complex visualinformation to the brain. Rather, these studies simplyconfirmed what had been known earlier: that spatialvisual percepts can be produced by electrical stimu-lation of the visual cortex.
As an alternative to surface stimulation of thevisual cortex, intramural and extramural studies wereinitiated in the early 1970s at the National Institutesof Health (NIH), for the systematic design, develop-ment, and evaluation of safe and effective meansof
microstimulating
cortical tissue. By implantingfloating fine-wire microelectrodes within the visualcortex, with exposed tip sizes of the same order ofmagnitude as the neurons to be excited, more selec-tive stimulation could, in principle, be achieved,resulting in potentially more precise control of neu-ronal function. Studies of intracortical stimulationwere initiated at Huntington Medical Research Insti-tute (HMRI; Pasadena, CA, U.S.A.) in 1979 in whichthe feasibility of chronic intracortical stimulation ofthe sensorimotor cortex was established (19,20).These studies sought to establish margins of safetyfor intracortical stimulating microelectrodes. Brum-mer’s (21) pioneering electrochemical work aimed atunderstanding stimulating electrode charge transfereventually resulted in microelectrodes made fromactivated iridium.
Based upon studies by Bartlett and Doty (22) inmacaques, as well as acute intracortical microstimu-lation studies that were performed in normal-sightedpatients undergoing occipital craniotomy (23),researchers at the NIH performed a short-termimplantation of intracortical microelectrodes in ahuman volunteer. The initial questions to beanswered by the short-term implant were: (i) doesthe visual cortex of a person, blind for a long periodof time, remain responsive to intracortical microstim-ulation? and (ii) are the visual percepts inducedthrough intracortical stimulation stable over monthsof stimulation?
Thirty-eight microelectrodes were implanted in theright visual cortex of a 42-year-old volunteer who hadbeen totally blind for 22 years secondary to glaucoma(24), near the occipital pole, for a period of 4 monthswith electrical access through percutaneous leadsexiting the scalp. Thirty-four of the 38 microelectrodesinitially produced spatial percepts with threshold cur-rents in the range of 1.9–25
m
A. Phosphene brightnesscould be modulated by varying stimulus amplitude,frequency, and pulse duration. At levels of stimulationnear the threshold, the phosphenes were oftenreported as having colors. As the stimulation level
was increased, the phosphenes generally becamewhite, grayish, or yellowish. When two phospheneswere simultaneously generated, they appeared at dif-ferent depths. When three or more phosphenes weresimultaneously generated, they became coplanar.Intracortical microelectrodes spaced 500
m
m apartgenerated separate phosphenes, but microelectrodesspaced 250
m
m apart typically did not. This two-point resolution was about five times greater than hadpreviously been achieved with electrodes on the sur-face of the cortex. With some individual microelec-trodes, a second, closely spaced phosphene wassometimes produced by increasing the stimulationcurrent. As had been seen with cortical surfacestimulation, phosphenes moved with eye movements.The 4-month study was concluded according to theinformed consent protocol, by removing all extradu-ral components.
The findings from these human studies have thefollowing significance: it appears feasible to invokepoint topographic visual percepts using both surfaceand intracortical microelectrodes. Using intracorticalmicroelectrodes, visual percepts are typically smallerthan those invoked by surface electrodes, the ampli-tude thresholds are up to two orders of magnitudelower than those of surface electrodes, they are stableover weeks to months, and these percepts can becrudely modulated by varying the stimulation param-eters. However, these studies did not demonstratethat a multitude of the observed visual percepts becombined to form meaningful spatial patterns thatmimic natural visual perception. More specifically,the assumption that a phosphene-based visual sensa-tion can substitute for normal vision in the perfor-mance of daily tasks remains at best conceptual.Commonly referred to as a “score-board” approach,the functional utility of phosphenes assembled in atype of bit-mapped image has not been demon-strated, although the idea remains conceptually sim-ple and attractive. Much of what is known about thefunction of the primate visual cortex comes fromneural recordings studies, in non-human primates,with either single, or relatively small numbers ofmicroelectrodes.
From the viewpoint of visual science, years ofcortical physiology in animals suggest that the “bit-mapped approach” may not be the most effectivemanner to produce artificial vision. Since the earlystudy by Hubel and Wiesel (8), it has been clear thatV1 neurons are selective for spatial, temporal, chro-matic, and binocular cues
in addition to
the locationof a stimulus in the visual field. While a phospheneis some form of visual sensation derived from stimu-lation of a cortical location, it must at some level be
1008 P. TROYK ET AL.
Artif Organs, Vol. 27, No. 11, 2003
composed of component sensations (conscious ornot) involving orientation, direction, depth, color,and texture, because these are the signals carried byV1 neurons, regardless of how those neurons areactivated. The location of a phosphene is only a par-tial description of a visual event. These, in additionto
x
and
y
(the location of the stimulus), are thetuning parameters of V1 neurons. To suggest that thelocation of a stimulus is available to the perceptualsystem, but not the features associated with it, is toimply that
x
and
y
represent a different type of infor-mation altogether.
These concepts are not new; in fact, variousphysiologists have succeeded in conveying specificnon-topographical kinds of information to trainedresearch animals by stimulating appropriately tunedcortical neurons. Stunning examples include thesense of self-motion (heading) (25) and the abilityto judge tactile stimulation frequency (26), both inmacaques. However, perhaps for technologicalreasons, this physiologically motivated stimulationapproach has existed somewhat independently of thephosphene idea—the combination of point stimuli—that is pervasive in the visual prosthesis and engi-neering communities. It is not yet known whichapproach will prove more effective in practice, andcertainly it is possible that a combined approach willbe best.
It is the long-term goal of our work to develop animplantable intracortical system for restoration ofvision in a large user population. To accomplish this,improvements are needed in implantable hardwarefor control and communication with large numbersof intracortical microelectrodes, long-term studies of
the stability of intracortical microelectrodes relativeto their electrochemistry and biocompatibility, anda more comprehensive understanding of how animplant comprised of hundreds, or thousands, ofintracortical microelectrodes might manipulate thetuning properties, that is, the neural machinery, ofthe primate visual system. It is a strategy for, andprogress on, this last need that is the subject of thisstudy.
METHODS
Intracortical microelectrodes
We planned to implant 192 microelectrodes in 24groups of eight, in area V1 of a male macaque. Acti-vated iridium-oxide microelectrodes were fabricatedfrom 30-
m
m diameter iridium wires and configured ineight-electrode arrays at HMRI and as individual“hat-pin” microelectrodes (24) at the Laboratoryof Neural Control (NIH). Sixteen eight-electrodearrays (HMRI), as shown in Fig. 2, comprised onegroup of 128 microelectrodes, and eight groups ofeight NIH “hat-pin” microelectrodes comprised theremaining 64, totaling 192 microelectrodes. Exposedelectrode areas were 500
m
m
2
(HMRI) and 200
m
m
2
(hat-pins), chosen as a compromise between theneed for selective neural recording and maintainingsafe stimulation charge density.
Surgical planning
Preoperative CT and MRI scans of the animal’shead were taken in order to plan the surgicalapproach, as well as to provide data files for thefabrication of scale-replica plastic 3-D models of its
FIG. 2.
Left: Configuration of HMRI arrays used for implantation. The long stabilizer pins help to maintain the position of the array inthe cortex. Right: Scanning electron micrograph of a typical microelectrode tip showing the Parylene insulation and the exposed iridiumtip.
Stabilizer pin 2.5mm (L)
2 mm diam array
MultiwireCable
Electrode Lengths: 0.7, 0.9, 1.3 mm
Stabilizer pin 2.5mm (L)
2 mm diam array
MultiwireCable
MODEL FOR INTRACORTICAL VISUAL PROSTHESIS RESEARCH 1009
Artif Organs, Vol. 27, No. 11, 2003
skull and brain. Using the plastic skull and brainmodels, and electrode/lead/connector mock-ups, amock surgery was performed. At that time, we eval-uated the approach for lead routing, and the place-ment of custom-fabricated polycarbonate connectorholders.
Surgery
The animal was anesthetized and placed in a proneposition on the operating room table with the headfixed in a standard primate frame. Using appropriatesterile techniques, a craniotomy was performed,exposing the lateral surface of the right occipitalcortex, from the superior sagittal sinus medially tothe transverse sinus laterally. A sterile template withdimensions determined during preoperative practicesessions with scale models was used to preciselyguide placement of the craniotomy and connectorsites. The dura was opened and microelectrode place-ment sites were identified on the cortical surface. TheHMRI connector bank was attached to the skull withtitanium screws. After routing each eight-electrodecable through an appropriate slot in a custom-designed comb alignment guide attached to the skulladjacent to the edge of the craniotomy, the HMRImicroelectrode arrays were inserted into the occipi-tal cortex using a high-speed insertion tool (Fig. 3).Next, the hat-pin connector bank assembly wasmounted on the skull anterior to the craniotomy siteand the attached microelectrode lead bundles (eightwires each) were routed through a second custom-designed skull-mounted comb alignment guide andinserted freehand into the occipital cortex using spe-cially machined forceps, generally being placed inmore lateral positions than the HMRI arrays (Fig. 3).Once all microelectrodes were inserted and their
cortical locations photo-documented, the dura wasclosed and the bone flap was replaced. The entireoperative site was covered with acrylic, incorporatingmost of the cables and the bases of the connectorhousings. Although we fabricated 192 microelec-trodes, attrition of microelectrodes and connectorcontacts during fabrication of the arrays and connec-tor housings, as well as during surgery, resulted in 152microelectrodes being implanted.
Physical mapping
A 2-week postsurgical recovery period wasallowed. During this time, wound healing was moni-tored, and minor mechanical problems related tothe acrylic skullcap and connector housings wereresolved. The connector housings were designed withan O-ring gasket on the covers to prevent any exter-nal fluid from leaking into the housings and corrod-ing the connectors. Some initial modifications to thecovers and gaskets were required to ensure integrityof the seals. Following the recovery period, each con-tact of the connectors was tested for continuity to amicroelectrode. Each contact was connected to a cus-tom-designed biphasic constant-current driver con-trolled by a computer interface. The battery-poweredstimulator was optically isolated from the computer.The microelectrodes were driven with a continuousconstant-current, biphasic, cathodal-first, 200
m
s/phase, 10
m
A, 30 Hz pulse train. The microelectrodevoltage waveform was captured on a virtual instru-ment computer-based oscilloscope and stored. Acontact was judged as being connected to a micro-electrode if the voltage waveform showed the typicalfeatures of an access resistance and a capacitiveinterface. Open contacts were clearly identifiable. Weused high-resolution digital photographs, taken dur-ing surgery, to identify the physical location of eachelectrode.
Electrophysiology
We obtained neural recordings from the micro-electrodes to determine whether the implantedmicroelectrodes, which were optimized for micro-stimulation, could be used in the reverse direction tomap response properties of neurons surroundingeach electrode. The animal was trained to visuallyfixate on a stationary point presented on a computermonitor placed 57 cm directly in front of it. To trackthe monkey’s eye position, an eye coil, comprised of50
m
m Teflon-coated stainless wires, had been pre-viously implanted in the eye, between the scleraand conjuctiva, with the lead wires brought out to aconnector mounted on the skullcap. By seating themonkey at the intersection of orthogonal, oscillating
FIG. 3.
Left: HMRI arrays following implantation. Note theabsence of bleeding despite penetration of blood vessels. Right:Hat-pin microelectrodes being implanted. A temporary lifterscrew holds microelectrodes above the brain. The HMRI cablesand hat-pin wire bundles are routed through plastic alignmentcombs mounted to the skull.
1010 P. TROYK ET AL.
Artif Organs, Vol. 27, No. 11, 2003
magnetic fields, one vertical, the other horizontal,with frequencies of 60 kHz and 90 kHz, respectively,the gaze angle could be electronically determined.
Recording data were collected in trials. While theanimal fixated, a 1
∞
diameter flickering stimulusappeared for 1 s at one of 400 predetermined loca-tions covering a 10
∞
¥
10
∞
area centered 4
∞
left of, and3
∞
below, the fixation point. Note that when the ani-mal fixates, the coordinate frame of the retina alignswith the coordinates of the screen, centered on thefixation point. Thus, coordinates specified in degreesof visual angle on the screen correspond to degreesof visual angle on the retina.
Neural recordings were made through 96 micro-electrodes simultaneously with a Plexon 96-channelMultichannel Acquisition Processor capable of sort-ing and time-stamping spikes (action potentials) inreal-time. For some channels, spikes could be iso-lated for a single neuron, but for most, the datacollected represented input from approximately 5–10 neurons. Because neurons with similar responseproperties tend to form clusters in V1, multiunit sam-ples are widely studied and generally reflect resultsobtained from single-neuron samples, usually withlower signal-to-noise ratios.
Psychophysical training
The animal was trained to perform a memory sac-cade task. Using operant conditioning, the monkeywas trained to fixate on a stationary spot for approx-imately 1 s. During this time, a small spot was illumi-nated at a random location within the area mappedby the collected V1 receptive fields. The spot was onlyvisible for 100 ms and thus constituted, essentially, aflash. After it disappeared, the animal was requiredto continue to fixate for another 1,000 ms, at whichpoint the fixation point also disappeared. The mon-key was trained to look (saccade) to the location ofthe target flash. If the animal’s eye position waswithin a predefined spatial window, 500 ms after theoffset of the fixation point, it received a liquidreward. The use of a memory rather than direct sac-cade task was crucial. In a direct saccade task, a spotappears and remains visible, and the monkey sac-cades to it. However, if that perception was producedby cortical stimulation, then it would appear to moveas the eyes move, a consequence of cortical compen-sation for eye position. In the memory saccade task,only the memory of the flash persists. Because spatialmemories do not shift with eye position, there isno apparent shift of the target. A computer-basedinstrumentation system presented the visual stimulito the monkey while recording both the eye move-ment and the neural signals from each of the micro-
electrodes. The animal acquired the task over a 2-week period.
Electrical stimulation testing
Using the results from the spatial mapping studies,the receptive field coordinates for each of theimplanted microelectrodes were estimated. Usingthe assumption that stimulation at a given corticalsite produces the sensation of a visual event, a phos-phene, in the part of visual space corresponding tothe receptive fields of the stimulated neurons, thegoal of this task was to train the monkey to look tothis location. The computer-controlled stimulationsystem was used to drive the implanted microelec-trodes with stimulus trials that were in synchroniza-tion with the eye coil data recording, and thepresentation of the central fixation point on the com-puter monitor. The charge-balanced stimulator pro-duced cathodic-first stimulation currents of 20
m
A forthe HMRI electrodes, and 10
m
A for the NIH hat-pinelectrodes. The stimulus duration for each phase ofthe biphasic waveform was 400
m
s, with an interpulseinterval of 5 ms. Each 1 s stimulus trial was com-prised of three 200 ms pulse trains, with an intertraininterval of 200 ms, totaling three trains in 1 s. Differ-ent current amplitudes were used for the HMRI andthe NIH electrodes to allow for the difference inelectrode surface areas for the two types. The differ-ence between the electrical stimulation trials and thepsychophysical training was that for the latter boththe fixation point and the stimulus were visual. Incontrast to the visual trials, used for psychophysicaltraining, for the electrical stimulation trials only thefixation point was visually presented, with the elec-trical stimulation substituting for the normal off-center visual target stimulus. Our expectation wasthat the animal would treat the percept induced bythe electrical stimulation in a manner similar to thatof the visual stimulus, thus performing a saccade tothe location of the stimulus in the receptive field. Ifthe animal’s saccade placed its measured eye positionwithin the reward box, typically 2
∞
on a side, centeredaround the known receptive field center for that par-ticular electrode, it was given a fluid reward.
RESULTS
Electrode identification and testing
We identified 114 microelectrodes with intact elec-trical connections. The remaining 38 microelectrodeswere ones that had known connection problems priorto, or during the surgery. These open connectionsoccurred during manufacturing of the implantable
MODEL FOR INTRACORTICAL VISUAL PROSTHESIS RESEARCH 1011
Artif Organs, Vol. 27, No. 11, 2003
hardware or due to damage of some of the microelec-trodes during surgery. In no case did a connection toa microelectrode deteriorate following the surgery. Atypical microelectrode voltage waveform is shown inFig. 4. Note the characteristic microelectrode voltagewaveshape showing an access resistance and anelectrode–tissue capacitive interface typical of thatobserved with activated-iridium films.
Receptive field mapping
Data from the receptive field measurements werestored in a large database and postprocessed withoutput in a graphical as well as a numerical format.Results showed clearly defined receptive fields for 48microelectrodes, which were fit with 2-D Gaussianfunctions to obtain the central coordinates and sizefor each electrode. Receptive field results for fourtypical microelectrodes are shown in Fig. 5. Using thereceptive field coordinates in combination with theknown physical location of each microelectrode, andpublished macaque maps, a retinotopic map of areaV1 was constructed. This map allowed a cross-checkof the field coordinates for the individual microelec-trodes for consistency. There should be a strong rela-tionship between the location of the receptive fieldsand the physical location of the microelectrodes inthe cortex. As an initial test of this relationship, weregressed the eccentricity of the receptive fields fromthe fovea (measured in degrees of visual angle) andcompared those to the measured eccentricity (fromhigh-resolution digital photographs) of the micro-electrodes from the cortical foveal region (measuredin mm). Results showed close relationship (
r
=
0.92,
P
<
0.001) between these two parameters.
Electrical stimulation
A subset of 33 electrodes was randomly chosen forelectrical stimulation, the remainder being reservedfor unspecified use in the future. At the start of theelectrical training, the animal was probably confusedinitially, and did not obviously look toward the stim-ulated locations. Within approximately 2 weeks, theanimal’s gaze was demonstrably drawn toward thetarget locations, as illustrated in Fig. 6. Each graphplots a region of the visual field roughly correspond-
FIG. 4.
Typical voltage waveform mea-sured across an intracortical microelec-trode during constant current stimulation.Charge density was 1.5 mC/cm
Four receptive field maps illustrating the range of preci-sion obtained. The two panels on the right localize receptivefields with high (approximately 0.5
∞
) precision. The upper leftpanel localizes the field well but the width is uncertain. The lowerleft panel gives little indication of where the receptive field is.Yellow indicates strong activity, black is weak. Horizontal andvertical axes represent the coordinates of visual space relativeto the fixation point (i.e. fovea). All responses were
z
-scored; thusany value above 2 is a significant departure from baseline activity.
1012 P. TROYK ET AL.
Artif Organs, Vol. 27, No. 11, 2003
ing to the collective receptive field locations. The(0, 0) coordinate is the fixation point, and the redcircles represent the locations of the receptive fieldsmapped for the corresponding channel. The size ofthe red circle is arbitrary; receptive fields were typi-cally 1–2
∞
in diameter. The blue crosses show themean saccade endpoint for that channel, with verticaland horizontal bars showing the standard deviationof endpoints in the vertical and horizontal dimen-sions, respectively. The small circles show the individ-ual saccade endpoints, one per trial. Plotting themeans of the endpoints (cross-intersections), oneper channel (Fig. 7), we see a strong correlation inthe horizontal dimension (
r
=
0.87,
P
<
0.001) anda weaker correlation in the vertical dimension(
r
=
0.50,
P
<
0.001), between the receptive field andthe mean saccade endpoint, indicating a tendency forsaccades to go toward the receptive fields. Becausethis was a correlation, any global bias in the eyemovements does not factor in; that is, the correlationexpresses only the relationship between the
scatter
ofthe receptive fields and the scatter of the saccadeendpoints. We do not know why the vertical correla-tion was weaker. One possibility is that a subtle, per-sistent upward drift in the animal’s eye position,
common in fixating monkeys, tended to smear thevertical component of its phosphene percepts.
The individual saccade endpoints, as seen in Fig. 6,are fairly scattered; in some cases they are closer tothe fixation point than the receptive field. However,all trials are plotted here, not just the ones which fellwithin the reward rectangles and thus rewarded.Many of the stray circles were probably the result ofthe monkey not making a saccade at all, perhapsmaintaining straight-ahead fixation [thus a “saccade”with end coordinates near (0, 0)] in anticipation ofthe next trial. In any case, we emphasize that ourtraining strategy was not designed to maximize sac-cade precision, but to maximize the speed at whichthe monkey acquired the basic task. This is becauseno monkey had been previously trained to performa visual-related task based solely on direct corticalstimulation. With the certainty that this could beachieved, we would have trained with smaller rewardwindows and persisted until the animal caught on. Asis typical of complex behavioral training, this couldhave taken many months, possibly more than a year.However, in this case, it was necessary only to dem-onstrate some correlation between stimulation siteand saccade endpoint. The indicated strategy was
FIG. 6.
Sample of data from the electrical stimulation memory saccade task. Graphs show 10 of 33 channels tested (i.e. stimulated).In each panel, cross-hairs denote the fixation point, circles denote the saccade endpoints. The crosses are centered on the meansaccade endpoint corresponding to that channel, and its vertical and horizontal bars show standard deviation of saccade endpoints inthe two dimensions. The circles are centered on the receptive field locations. For perfect performance, the monkey would have to lookto the center of this circle on every saccade (note that he does not really see the circle). The size of the circle is meaningless here.
MODEL FOR INTRACORTICAL VISUAL PROSTHESIS RESEARCH 1013
Artif Organs, Vol. 27, No. 11, 2003
therefore to use large reward windows, typically 6
∞
on a side, thus maximizing the fraction of rewardedtrials, which would in turn accelerate the rate ofexperience gained (feedback obtained) and thushasten the onset of behavioral correlates. However,large reward windows allow the monkey to obtain asufficient fraction of rewarded trials without needingto look at the precise location of his percept—to theextent that that percept is precise. Therefore, the sac-cade results here demonstrate only the feasibility ofbehavioral control through a cortical interface, anddo not comment on the potential precision withwhich that control might be achieved.
DISCUSSION
Although humans report visual sensations whentheir visual cortex is electrically stimulated, thedescription of these percepts varies from subject tosubject. It could be argued that the specific nature ofelectrically induced percepts is irrelevant as long asthe information conveyed through stimulation isuseful to the subject. Certainly, some experienceswith cochlear implants have defied understanding bysome auditory scientists, and there is a temptation toassume that any information in any form applied toa large intracortical array might be suitably inter-preted by the cortex. Despite some remarkable suc-cesses with cochlear implants, previous attempts tocommunicate visual information to the cortex, evenusing a multitude of phosphenes, have not producedcomparable results. Rather, there exists no experi-mental evidence that a technologically feasible arrayof cortical microelectrodes could produce usefulvision. Many human psychophysical studies of simu-lated artificial vision in normal-sighted subjects haveused either regularly spaced, or regularly sizedsimulated phosphenes, neither of which are likelyto occur as the result of cortical stimulation usingmicroelectrodes that are within our current techno-logical grasp. It is our premise that in order to under-
stand how an array of intracortical microelectrodesmight be electrically manipulated to produce artifi-cial vision, an animal model can yield important fun-damental data about how the primate visual systeminterprets the artificial activation of its neuronalcircuitry.
While animal experiments cannot replace humantesting, there is still a great deal to be learned fromnon-human primate experiments, especially duringthe development phase of safe fully implantablehardware. Assuming a 900-electrode array could beimplanted in a human today, we might expect to beable to simulate useful, if not realistic, visual inputs.However, even if the microelectrodes were optimallyspaced, we have no evidence that the subject wouldbe able to perceive an image any better than thatshown in the extremely simplistic representation inFig. 8(A). The face is not clearly recognizable—noteven as being a face—in spite of the relatively large30
¥
30 grid it is mapped upon. It is a common mis-conception that the perception would necessarilylook like Fig. 8(B)—that is, gray scale. Despite somesuggestion of phosphene brightness modulation inearlier human subjects, we do not know how tosystematically simulate the perception of brightness.Simplistically, we might assume that large stimulationcurrents and consequently large neural activity, fol-lowing some strength-duration curve, corresponds toperceptual brightness, but even then, we do not knowhow to create graded responses. Because V1 neuronsnaturally encode such things as luminance, contrast,orientation, spatial frequency (texture), and depth atvarious image locations, restricting our stimulationstrategy to a bit-map approach may severely limit thepossibilities for artificial visual reconstruction. Usingan animal model with large numbers of intracorticalmicroelectrodes allows for the simultaneous devel-opment of safe implantable hardware with a dis-covery of how electrical stimulation can be used todirectly communicate with the cortex. We have cho-sen the rhesus monkey due to its similarity to humans
FIG. 7.
Correlations showing therelationship between saccadeendpoints and receptive field loca-tion. Note that there is a strongercorrelation between the horizontal(
x
) endpoint positions and thereceptive field centers than thereis for the vertical (
y
) positions,although for both
x
and
y
the sac-cade endpoints are clearly influ-enced by the position of electrodeson the retinotopic map.
r = 0.87041rr <0.001
r = 0.5038r <0.001
r = 0.87041P<0.001
r = 0.5038P<0.001
x y
x
y
1014 P. TROYK ET AL.
Artif Organs, Vol. 27, No. 11, 2003
from the standpoint of visual cortex anatomy andfunction.
In this study, the implantation of the microelec-trodes tested our ability to use an animal model fromthe following standpoints: (i) although considerableeffort was needed to design the head-mounted con-nectors and other associated hardware, the feasibilityof fabricating hardware suitable for implantation oflarge numbers of intracortical microelectrodes wasestablished; (ii) the feasibility of performing thecomplex surgical procedure with little to no post-surgical complications was demonstrated; (iii) wedemonstrated neural recording and stimulation for alarge number of intracortical microelectrodes over aperiod of many months. At the time of sacrifice, 18months postsurgery, we were still able to recordneural signals from most of the microelectrodes, thusindicating the mechanical and physiological stabilityof the electrode sites.
While we were confident that the animal couldlearn the visual memory saccade task, we wereimpressed by the ease with which it quickly adapted.Often the monkey was able to perform a series oftrials with only 2–3 s between trials. When weswitched to the electrical stimulation, the monkeyapparently did not initially treat them as equivalentto visual stimuli, because it took about 2 weeks forthe animal to adapt to using the electrically inducedpercepts in performing the memory saccade task.Although not entirely unexpected, it suggests thatthere is some, although perhaps subtle, differencebetween the visual stimuli and the electrical ones.What is significant is that the animal learned to usethe electrically induced percepts to perform a taskthat formerly had been accomplished using visualstimuli. This would seem to be a first step in estab-lishing a communication strategy with the cortextoward the goal of replacing vision.
While we define the memory saccade task as look-ing to the remembered location of a phosphene, it is
conceivable that the animal never really saw a phos-phene; instead, he might have sensed the electrodecurrent in a non-visual way, presumably throughsomatosensory innervation of the vasculature, andfigured out where to look, upon feeling something atthat site, so as to get the reward. We cannot excludethis possibility, but we make the following points.This scenario would require the mapping of 33inputs to 33 outputs, a relatively complex problemcompared to the simplicity of making use of theexisting mapping corresponding to the retinotopy ofV1. To undertake such a
de novo
mapping wouldthus imply the absence of effective visual stimula-tion. Stated another way, it would be easiest to sim-ply look at the perceived flash, if any such perceptwere available. Humans stimulated with similar cur-rents consistently perceive phosphenes, which theyare able to localize, though one cannot assume thatmonkeys necessarily perceive them as well. Still, theimplication here would be that the sparse soma-tosensory innervation of cortex is more easily acti-vated than the visual neurons in immediate contactwith the electrodes. Also, it is important to note thatthe
de novo
somatosensory mapping would requirethat the identity of the inputs be resolvable. Thisimplies that current at one site is detectable asseparate from current at another site based solely onsomatosensory (or at least non-visual) innervation.This itself cannot be excluded; however, it should beunderstood that this would amount to being able to
feel
the difference between the 33 inputs, if theanimal was able to feel them at all. Moreover, whilea somatosensory-to-retinal position mapping is notimpossible at the subconscious level, it would seemunlikely in view of the highly precise, already exist-ing mapping between cortical location and retinalposition.
The data presented here can only be considered aspreliminary, and our animal model needs to be sub-jected to more rigorous testing and analysis. However,
FIG. 8.
A face shown at different resolutionsand gray levels. (A) 900 pixel black andwhite; (B) 900 pixel gray scale; (C)824
¥
924 color.
A B C
MODEL FOR INTRACORTICAL VISUAL PROSTHESIS RESEARCH 1015
Artif Organs, Vol. 27, No. 11, 2003
we feel that we have demonstrated the essential ele-ments of using an animal model for visual prosthesisresearch. Ultimately, the ability of visual prosthesisto restore visual function to a large user populationwill be judged against other rehabilitation devices andtechniques, including the simple cane or guide dog.In this regard, navigation or reading may not be themost compelling function to focus on for visualprosthesis development because for these functionsthere exist other, non-implant-related alternatives.However, important tasks such as face recognition orscene identification may be more significant and lesslikely to be provided for by competing technologies.
CONCLUSIONS
The feasibility of using an animal model for visualprosthesis research, compensating for absence of alanguage report, has been demonstrated. The modelneeds further development and testing in order tounderstand the appropriate interpretation of psycho-physical results and the ultimate limit of the model.In considering a future human implant, with practicalmicroelectrode technologies, there will emerge atrade-off between functionality and implant com-plexity. To optimize the use of a limited number ofintracortical microelectrodes for restoring significantvisual function, it would be advantageous to studystrategies for communicating with the cortex, throughthe artificial interface, prior to the involvement ofhuman volunteers so that the configuration of theimplantable hardware does not limit the functionalpossibilities. As such, use of an animal model forsafety and efficacy testing may enhance the scientificbasis for a multimodal ethical-based decision processfor testing implantable systems in human volunteers.
Acknowledgments:
This work was funded by theBrain Research Foundation, the Prizker-GalvinChallenge Fund, and private donations.
REFERENCES
1. Löwenstein K, Borchardt M. Dtsch.
Z Nervenheilk
1918;58:264–92.
2. Foerster O. Beiträge zur pathophysiologie der sehspäre.
J Psy-chol Neurol (Ppz
) 1929;39:463–85.3. Krause F. Die sehbahnen in chirurgischer beziehung und die
faradische reizung des sehzentrums.
Klin Wschr
1924;3:1260–5.
4. Urban H. Zur physiologie der occipitalregion des menschen.
Z Ges Neurol Psychiat
1937;158:257–61.5. Penfield W, Rasmussen T.
The Cerebral Cortex in Man.
NewYork: Macmillan, 1950.
6. Penfield W, Jasper H.
Epilepsy and the Functional Anatomyof the Human Brain.
London: Churchill, 1954;116–26, 404–6.7. Button J, Putnam T. Visual responses to cortical stimulation
in the blind.
J Iowa Med Soc LII
1962;1:17–21.8. Hubel DH, Wiesel TN. Receptive fields and functional archi-
tecture of monkey striate cortex.
J Physiol
1968;195:215–43.9. Brindley GS, Lewin WS. The sensations produced by electri-
cal stimulation of the visual cortex.
J Physiol (London)
1968;196:479–93.10. Brindley GS, Donaldson PE, Falconer MA, Rushton DN. The
extent of the region of occipital cortex that when stimulatedgives phosphenes fixed in the visual field.
J Physiol (London)
1972;225:57–8P.11. Brindley GS. The variability of the human striate cortex.
J Physiol (London)
1972;225:1–3P.12. Brindley GS. Sensory effects of electrical stimulation of the
visual and paravisual cortex in man. In: Jung R, ed.
13. Rushton DN, Brindley GS. Short- and long-term stability ofcortical electrical phosphenes. In: Rose FC, ed.
PhysiologicalAspects of Clinical Neurology
. Oxford: Blackwell, 1977;123–53.
14. Dobelle WH, Mladejovsky MG. Phosphenes produced byelectrical stimulation of human occipital cortex, and theirapplication to the development of a prosthesis for the blind.
J Physiol (London)
1974;243:553–76.15. Dobelle WH, Mladejovsky MG, Girvin JP. Artificial vision for
the blind: electrical stimulation of visual cortex offers hope fora functional prosthesis.
Science
1974;183:440–4.16. Dobelle WH, Mladejovsky MG, Evans JR, Roberts TS, Girvin
JP. “Braille” reading by a blind volunteer by visual cortexstimulation.
Nature
1976;259:111–12.17. Pollen DA. Some perceptual effects of electrical stimulation
of the visual cortex in man. In: Tower DB, ed.
The NervousSystem
, Vol. 2
: the Clinical Neurosciences.
New York: RavenPress, 1975;519–28.
18. Dobelle WH. Artificial vision for the blind by connecting atelevision camera to the visual cortex.
