. INTRODUCTIONhysical limitations and consequences due to the variabil-
ty of the packing arrangement of retinal cones were firsteported by Yellott1 in 1983. Since then, the packingtructure of retinal cones has been studied both anatomi-ally and psychophysically.2–6 The advent of retinal imag-ng systems with adaptive optics (AO) has made it pos-ible to image the living human retina at the microscopiccale.7 Now that noninvasive studies of the anatomy andhysiology of human cones are possible,8,9 it is reasonableo expect future studies to be done on a much greatercale. Automated methods for data analysis have foundpplication in many medical and scientific fields and havehe potential to become a useful tool in the field of retinalmaging. The quantity of data included in any study is ofreat importance. Reliable automated routines are natu-ally preferred when large quantities of data need to benalyzed. We want to facilitate the process of determiningone density and cone packing arrangement in AO imageso encourage the inclusion of larger data sets in futuretudies. Manually labeling cones in an image is usuallyeliable, but it becomes an impractical solution when mul-iple images are involved. In this paper, we automate theone labeling process with an algorithm implemented inATLAB (Mathworks, Inc., Natick, Massachusetts) and
unctions from the MATLAB Image Processing ToolboxIPT). Code can be accessed from our research group’s
eb page.10 We demonstrate the algorithm’s effectivenessn analyzing actual AO retinal images. Algorithm perfor-
ance is assessed by comparisons with manually labeledesults for six different AO retinal images.
Studies that have addressed the sampling limitationsf retinal cones usually model the cone mosaic as a hex-gonal array of sampling points.2,6,11 Maximum cone den-ity is achieved with this packing arrangement. It is
ikely that this is the main reason why retinal cones tendo develop into a hexagonal array in areas where few oro rods are present. A hexagonal sampling grid is the op-imal solution for most signal processing applications.12
evertheless, the sampling performance of cones is moreommonly associated with cone density rather than ar-angement geometry. Knowing that the packing arrange-ent of cones affects the sampling properties of the
etina, why are cones hexagonally arranged only in local-zed regions and what advantages do such variations inacking arrangement offer?3,5,6,11 Furthermore, analysesf cones across a variety of eccentricities show that theiracking arrangement is also highly variable. Using ourlgorithm, we analyzed the cones from seven montagesonstructed from images acquired at various eccentrici-ies using AO. The computed cone locations are then usedo generate density contour maps and to make quantifiedeasurements of their packing structure. The consistency
f our results indicates the reliability of an automatedethod for analyzing AO images. Our analysis demon-
trates how the process of extracting quantitative infor-ation about the density and packing arrangement of
ones from images of living retinas can be accomplishedfficiently.
. METHODmage formation for an optical system can be described byhe convolution of the object with the system’s impulse re-ponse or point-spread function (PSF):
I��x1,x2� = I�x1,x2� � ��x1,x2�, �1�
here I��x1 ,x2� is the observed image, I�x1 ,x2� is the ac-ual cone mosaic under observation � is the convolutionperator, and ��x ,x � is the PSF. AO corrects the aberra-
1 2
007 Optical Society of America
tbtrtciooicq
wpatarstapt
tdvdesfitst
tmmmafintteohiia
wFtTp3s1spe1MTpf
vmvqshItdfnchsDDab
wssawwq
F(fnT1
K. Y. Li and A. Roorda Vol. 24, No. 5 /May 2007/J. Opt. Soc. Am. A 1359
ions of the eye to minimize the distribution of ��x1 ,x2�,ut residual wavefront error after correction is still oftenoo high for us to see cones near or at the fovea. Imageestoration techniques such as deconvolution can enhancehe appearance of the image even further,13 but if retinalones are not optically resolved, no method can reliablydentify them. Under the condition that the cones underbservation are optically resolvable, we show that it isnly necessary to understand the role of noise in the conedentification process. The observed image I��x1 ,x2� isonverted by the detector into a finite two-dimensional se-uence:
I��n1,n2� = I��x1T1,x2T2� + ��n1,n2�, �2�
here T1 and T2 are the horizontal and vertical samplingeriods determined by the finite pixel size of the detectornd ��n1 ,n2� is a generalized noise term. The power spec-rum of an AO retinal image shown in Fig. 1 contains anpproximate hexagonal band region produced by theegular arrangement of cones. Signals that do not corre-pond to cones, which may originate either from noise inhe detection channel or from other features in the retina,re effectively noise. Filtering and morphological imagerocessing are used to isolate signals corresponding onlyo cone photoreceptors.14,15
The intensity of light incident on each pixel of the de-ector is encoded as integral values that make up the two-imensional sequence I��n1 ,n2�. This sequence can beiewed as a surface topography of intensity counts over aiscrete plane. Cones are single-mode optical fibers ori-nted toward the direction of the pupil center with verymall disarray.9,16 As a result, reflected light from thesebers can be considered as an array of point sources athe retinal plane. Therefore, the light distributions ob-erved in an AO image are PSFs whose peaks correspondo the actual cone locations. Surface topography features
ig. 1. (Color online) Power spectrum of an AO retinal imageenhanced by log scale) generated using the fast Fourier trans-orm. The shape and size of the band region indicate the hexago-al arrangement and sampling limits of the cones in the image.he periodicity of the tightest packed cones is about45 cycles per degree (cpd).
hat correspond to these PSF peaks are points of localaxima. When cones are optically resolved, we can deter-ine the locations of all the cones by identifying all localaxima in the original image. In an image sampled by an
rray of finite detector elements, a local maximum is de-ned to be any pixel whose value exceeds each of itseighboring pixels. IPT function imregionalmax effec-ively accomplishes this task. However, direct applicationo the observed sequence I��n1 ,n2� produces a result thatxceeds the actual number of cones in the image. Theseveridentifications are primarily due to the presence ofigh-frequency noise and can be greatly reduced by apply-
ng the appropriate low-pass filter. A low-pass finite-mpulse-response (FIR) filter is designed for this purposend applied to I��n1 ,n2� prior to further image processing:
f�n1,n2� = h�n1,n2� � I��n1,n2�, �3�
here h�n1 ,n2� is the impulse response (15 by 15 pixels).ilter parameters are selected based on the field size of
he system and the estimated minimum cone spacing.he minimum cone spacing in the central fovea is ap-roximately 2 �m.4 Using a conversion factor of00 �m/deg eccentricity, the frequencies of retinal coneshould range from nearly zero to approximately45 cycles per degree (cpd). The FIR filter h�n1 ,n2� is de-igned to implement the desired ideal low-pass filter thatasses only spatial frequencies within 145 cpd. The dens-st cones that can be imaged using AO reside within45 cpd, so filter parameters can be adjusted accordingly.any FIR filter types exist and can be rather extensive.he most popular filter designs can be found in signalrocessing texts,17,18 so we will not state further detailsor this step.
There is more than one method to implement the con-olution operation in Eq. (3), but an appropriate methodust provide a justifiable solution for computing the pixel
alues toward the boundary of f�n1 ,n2�. Suppose the se-uences I��n1 ,n2� and h�n1 ,n2� have N�N and M�Mupport, respectively. The convolution of I��n1 ,n2� with�n1 ,n2� initially requires extending the bounds of��n1 ,n2� to achieve M+N−1�M+N−1 support. This isypically done by zero padding I��n1 ,n2�, which introducesiscontinuities along its boundaries. The resulting�n1 ,n2� would contain unnecessary blurring and noiseear the boundaries. A second method is to take the dis-rete Fourier transform (DFT) of both I��n1 ,n2� and�n1 ,n2� and multiply them in frequency space. The re-ultant f�n1 ,n2� is then restored by taking the inverseFT. Due to the nonperiodicity of I��n1 ,n2�, taking theFT in this manner introduces aliasing along the bound-ries of f�n1 ,n2�. A practical solution to this problem coulde to remove the corrupted pixels by applying a window:
f�n1,n2� = W�n1,n2��h�n1,n2� � I��n1,n2��, �4�
here W�n1 ,n2� is the windowing function. The resultantequence f�n1 ,n2� is reduced to have only N−M�N−Mupport. A reliable alternative exists to avoid this loss innalyzable data. We can extend the bounds of I��n1 ,n2�ith values equal to its nearest border values rather thanith zeros. As a result, N�N support of the input se-uence I �n ,n � is preserved along with continuity at its
� 1 2
bf
qcptcmitzds2e
pdcpdofiaccttactTusm
mqUtplmstcsfivasse
3TraWctAt
Fqisba
Fa1p
1360 J. Opt. Soc. Am. A/Vol. 24, No. 5 /May 2007 K. Y. Li and A. Roorda
oundaries. IPT functions fwind2 and imfilter are usefulor the implementation of these operations.
IPT function imregionalmax generates a binary se-uence, as shown in Fig. 2(a), where each nonzero pixelorresponds to a local maximum. Missidentifications ap-ear as multiple nonzero pixels that lie within a vicinityhat is not physically realizable by the same number ofones. These pixels are grouped into a single object usingorphological dilation (IPT function imdilate). As shown
n Fig. 2(b), this operation translates a binary disk acrosshe domain of the binary sequence, replacing each non-ero element with the disk similar to the convolution. Theisk is set at 2 �m in diameter (i.e., the minimum pos-ible cone spacing). The final cone locations, shown in Fig.(c), are determined by computing the center of mass ofach object after dilation.
ig. 2. Output of IPT function imregionalmax is a (a) binary se-uence whose values are nonzero at all identified local maximan the input sequence. (b) This sequence is dilated with a disk-haped structuring element, and (c) cone locations are computedy computing the center of mass of each object in the sequencefter dilation.
ig. 3. (Color online) Every marker in these six grayscale im-ges is manually placed by the authors. Each image is 8 bits and28�128 pixels, and each marker is accurate to the nearestixel.
We took six retinal images, all with 128�128 pixel sup-ort, and labeled them manually as shown in Fig. 3. Thisiverse set of images varies considerably in cone density,ontrast, and, in some cases, biological structure. Com-arisons are made with automated results on the sameata set to assess the agreement between the two meth-ds. Comparisons are also made between outcomes fromve experienced human observers for two of the six im-ges. An agreement is made when a pair of correspondingones is located within 2 �m of each other. This value washosen because cone diameters at the observed eccentrici-ies are at least 4 �m. Identified cones that do not satisfyhis criterion are defined as disagreements. The level ofgreement between the labeling methods is quantified byomputing the mean displacement along both the horizon-al and the vertical directions and is reported in Table 2.he cone packing arrangement is analyzed graphicallysing Voronoi diagrams.19 Voronoi analyses for the twoelected images are done on the results from the five hu-an observers and the algorithm.The image montages analyzed are acquired from oneonkey and six human retinas. The images were ac-
uired by both the flood-illuminated AO system at theniversity of Rochester and an AO scanning laser oph-
halmoscope (SLO) (see Table 1).7,20 Cone density is com-uted at each cone by counting the number of cones thatie within a defined circular sampling window (approxi-
ate radius of 0.07 deg or 20 �m). Cones whose corre-ponding sampling window extends beyond the bounds ofhe image are excluded. We generated contour maps ofone density values to observe the variability of cone den-ity across each montage. Linear interpolation is used toll the spaces between cones with the estimated densityalues. Since packing structure tends to vary greatlycross the retina, each montage is divided into 0.125 degections, and Voronoi analysis is done on each individualection. Voronoi regions that extend beyond the bounds ofach image section are excluded from all analysis.
. RESULTShe number of cones identified in every image of Fig. 3 iseported in Table 2. For each image, the algorithm foundpproximately equal numbers of cones as the authors did.hen analyzing the packing arrangement of retinal
ones, we are more concerned with the agreement be-ween the automated and the manual labeling methods.s outlined in Table 2, a 93% to 96% agreement between
he two methods is achieved, and the physical locations of
Table 1. Sources for Cone Photoreceptor Imagesa
Image System Eccentricity (deg)
1 Monkey AO flood illuminated 1.10 to 1.862 Subject 1 AO flood illuminated 0.30 to 1.683 Subject 2 AO SLO 0.60 to 2.604 Subject 3 AO SLO 1.15 to 4.275 Subject 4 AO SLO 1.35 to 4.976 Subject 5 AO SLO 0.74 to 2.617 Subject 6 AO SLO 1.55 to 5.06
aRefs. 7, 20, 21.
efoeip(srimeFstmgFsVbFcetVasfdnta
Fmtobservers.
Fw
K. Y. Li and A. Roorda Vol. 24, No. 5 /May 2007/J. Opt. Soc. Am. A 1361
ach corresponding cone pair deviated within 0.52 �mrom each other. Disagreements between the two meth-ds, ranging from 1% to 9%, are due to human observerrrors and/or unrelated signals that are not removed dur-ng the preprocessing step of the algorithm. The highestercentages of disagreement are seen for images (a) andc). The cone mosaic in (a) clearly has some unusualtructures,22 and the cones present in (c) are borderlineesolvable in many areas. In contrast to (a) and (c), othermages resulted in better agreement between the two
ethods. Agreement between outcomes from five experi-nced human observers spanned from 74.1% to 90.5% forig. 3(c) and 92.1% to 96.8% for Fig. 3(d). Analysis of thehape of each Voronoi region allows one to predict the spa-ial sampling capabilities offered by the observed coneosaic. Voronoi regions are divided into hexagonal (light
ray) and nonhexagonal (dark gray) categories as seen inigs. 4 and 5. Voronoi diagrams appear to be rather sen-itive to modest differences in cone labeling results.oronoi diagrams of Fig. 4 are of the same image labeledy five human observers and the automated algorithm.eature variability in these diagrams indicates that un-ertainty due to questionable image quality and/or inad-quate experience of the observer is not a negligible fac-or. Among the observers, the percentage of hexagonaloronoi regions spanned from 39.4% to 52.5% for Fig. 4(c)nd 55.36% to 62.71% for Fig. 4(d). The corresponding re-ults for the automated method were at 40.8% and 57.7%or Figs. 4(c) and 4(d), respectively. These observations in-icate that decisions made by human observers can sig-ificantly influence the outcome of a packing analysis andhat the consistency of an automated method may actu-lly be more appropriate for certain images.
ig. 4. (Color online) Voronoi diagrams corresponding to theosaic of Fig. 3(d) computed from cone locations acquired by (a)
he automated method and (b)–(f) the five experienced human
O flood-illuminated system at the University of Rochester and labeled
mwkmadrgsc
dtt11tptcar7rdmfaant(aem
4TmpthsshtT1autgmsEp
Fg
Ft
Fe
1362 J. Opt. Soc. Am. A/Vol. 24, No. 5 /May 2007 K. Y. Li and A. Roorda
We computed cone densities and Voronois using theethods described above for seven large montages withell-resolved cones. The montage acquired from the mon-ey is shown in Fig. 5. The Voronoi diagram for this coneosaic and its corresponding cone density contour map
re given in Figs. 6 and 7. Voronoi diagrams and coneensity plots illustrate the variability in cone packing ar-angement and density across the retina. Voronoi regionsradually increase in size at higher eccentricities, whichignifies how cone density decreases monotonically as ec-entricity increases. This can also be seen from the cone
ig. 6. Voronoi diagrams generated for the montage given in Fions are shaded in light gray.
ig. 7. (Color online) Topography map of retinal cone density forhe montage given in Fig. 4.
ig. 8. Percentages of hexagonal Voronoi regions plotted overccentricity for all seven montages analyzed in this study.
ensity contour map. The retinal montages cover eccen-ricities from 0.17 to 5.00 deg. Cone density values withinhis range varied from approximately040 to 6500 cones/deg2. This converts to approximately2,300 to 72,200 cones/mm2, which agrees with therends reported by Curcio et al.4 In a large montage, it isossible to identify localized patches in the mosaic wherehere is prominent hexagonal-packing structure. The per-entages of hexagonal regions across 0.125 deg sectionsre plotted in Fig. 8. In general, at least 40% to 50% ofetinal cones are hexagonally arranged, but more than0% hexagonal packing may appear in certain localizedegions. In a study done by Curcio and Sloan,3 an analysisone on one prepared human retina suggested that conesay be more hexagonally arranged near the edge of the
ovea (between 0.20 and 0.25 deg eccentricity) rather thant the fovea center. This suggests that, even though conesre more densely packed toward the foveal center, they doot necessarily become more hexagonally arranged. Thisendency may also be present in our data for the monkey71% at 1.17 deg and 46% at 0.17 deg) and subject 1 (60%t 0.50 deg and 42% at 0.37 deg) as shown in Fig. 8. How-ver, better-quality images of the foveal center and its im-ediate periphery are needed to confirm this hypothesis.
. DISCUSSIONhe abundance of hexagonal Voronoi regions reveals thatany localized patches of hexagonally arranged cones ap-
ear throughout each mosaic. Hexagonal arrays have in-eresting sampling properties. Natural scenes generallyave circular symmetric power spectrums, and hexagonalampling arrays provide the most efficient solution forampling such signals.12 The Nyquist limit for a normalexagonal array is ��3s�−1 along one axis and �2s�−1 alonghe other, where s is the center-to-center spacing of cones.his means that a computation savings of approximately3% over standard rectangular sampling is achieved. Ournalysis indicated that nonhexagonal Voronois can makep less than 30% of a mosaic region near the fovea, buthis percentage increases sharply to 50% to 60% atreater eccentricities. Cones at higher eccentricities areore randomly arranged and offer the visual system
ome protection from perceiving an aliased signal.1,5,23
arlier work by Yellott1 showed from data taken from theeripheral retina of a monkey that the power spectrum
t the eccentricities specified above each diagram. Hexagonal re-
ig. 4 a
ridtpsesaSrs
ptouAstccdodmmlwtepqqimeom
ATsNvtCsc
a
R
1
1
1
1
1
1
1
1
1
1
2
2
2
2
K. Y. Li and A. Roorda Vol. 24, No. 5 /May 2007/J. Opt. Soc. Am. A 1363
esembled that of a Poisson (random) distribution. Alias-ng does not occur at the fovea because the Nyquist limit,ue to higher cone densities, extends beyond frequencieshat can pass through the optics of the eye. The coneacking arrangement decreases at higher eccentricities,o, even though cone densities are lower, aliasing is gen-rally replaced by noise. We hope that the methods pre-ented in this paper will encourage further quantitativenalysis on the packing arrangement of rods and cones.uch analyses are essential for understanding how photo-eceptors migrate into their permanent arrangementtructure during development.
Cones are optical fibers oriented in the direction of theupil center.9 Reflected light emitted from the cone aper-ures are effectively point sources that generate an arrayf PSFs in AO images. For this reason, it is important tonderstand that not all PSFs correspond to cones in anO image with questionable quality. When AO fails to re-olve individual cones or when interference is present inhe system, a single PSF-like intensity distribution mayorrespond to multiple cones or noise. When individualones are resolvable, the cone labeling process can beone reliably using the proposed algorithm. This is dem-nstrated by evaluating the algorithm performance on aiverse set of AO images and comparing the outcomes toanually labeled results. Comparisons between the auto-ated and the manual labeling methods resulted in simi-
ar outcomes. The extent to which the two methods asell as different human observers agreed is influenced by
he appearance or quality of the analyzed image. This isspecially true when performing Voronoi analyses, soacking arrangement studies should be done only withuality images where cones are well resolved. A low-uality image or an image with unique features resultedn lower levels of agreement between manual and auto-
ated methods. Equivalent comparisons between severalxperienced human observers often resulted in greaterutcome variability, suggesting that a consistent auto-ated solution may be more reliable in many cases.
CKNOWLEDGMENTShis work was supported by NIH Bioengineering Re-earch Partnership EY014375 to Austin Roorda and byIH T32 EY 07043 to Kaccie Li. The authors thank Pa-an Tiruveedhula for programming assistance. The au-hors also acknowledge Joseph Carroll, Yuhua Zhang, andurtis Vogel for generous data contributions and helpfuluggestions concerning the algorithm and analysis pro-ess.
Corresponding author K. Y. Li can be reached by e-mail
EFERENCES1. J. I. Yellott, “Spectral consequences of photoreceptor
sampling in the rhesus retina,” Science 221, 382–385(1983).
2. D. R. Williams, “Topography of the foveal cone mosaic inthe living human eye,” Vision Res. 28, 433–454 (1988).
3. C. A. Curcio and K. R. Sloan, “Packing geometry of humancone photoreceptors—variation with eccentricity andevidence for local anisotropy,” Visual Neurosci. 9, 169–180(1992).
4. C. A. Curcio, K. R. Sloan, R. E. Kalina, and A. E.Hendrickson, “Human photoreceptor topography,” J. Comp.Neurol. 292, 497–523 (1990).
5. D. R. Williams and R. Collier, “Consequences of spatialsampling by a human photoreceptor mosaic,” Science 221,385–387 (1983).
6. D. R. Williams and N. J. Coletta, “Cone spacing and thevisual resolution limit,” J. Opt. Soc. Am. A 4, 1514–1523(1987).
7. J. Z. Liang, D. R. Williams, and D. T. Miller, “Supernormalvision and high-resolution retinal imaging throughadaptive optics,” J. Opt. Soc. Am. A 14, 2884–2892 (1997).
8. A. Roorda, A. B. Metha, P. Lennie, and D. R. Williams,“Packing arrangement of the three cone classes in primateretina,” Vision Res. 41, 1291–1306 (2001).
9. A. Roorda and D. R. Williams, “Optical fiber properties ofindividual human cones,” J. Vision 2, 404–412 (2002).
0. A. Roorda and K. Y. Li, “AO image processing,” (2006),retrieved 2006, vision.berkeley.edu/roordalab/Kaccie/KaccieResearch.htm.
1. D. R. Williams, “Aliasing in human foveal vision,” VisionRes. 25, 195–205 (1985).
2. R. M. Mersereau, “Processing of hexagonally sampled two-dimensional signals,” Proc. IEEE 67, 930–949 (1979).
3. J. C. Christou, A. Roorda, and D. R. Williams,“Deconvolution of adaptive optics retinal images,” J. Opt.Soc. Am. A 21, 1393–1401 (2004).
4. R. C. Gonzalez, R. E. Woods, and S. L. Eddins, DigitalImage Processing Using Matlab (Pearson Education, 2004),pp. 65–193.
6. J. M. Enoch, “Optical properties of the retinal receptors,” J.Opt. Soc. Am. 53, 71–85 (1963).
7. J. S. Lim, “Finite impulse response filters,” in Two-Dimensional Signal and Image Processing, A. V.Oppenheim, ed. (Prentice Hall, 1990), pp. 195–263.
8. R. C. Gonzalez and R. E. Woods, “Image enhancement inthe frequency domain,” in Digital Image Processing, 2nded. (Addison-Wesley, 2001), pp. 147–215.
9. M. B. Shapiro, S. J. Schein, and F. M. Demonasterio,“Regularity and structure of the spatial pattern of bluecones of macaque retina,” J. Am. Stat. Assoc. 80, 803–812(1985).
0. A. Roorda, F. Romero-Borja, W. J. Donnelly, H. Queener, T.J. Hebert, and M. C. W. Campbell, “Adaptive opticsscanning laser ophthalmoscopy,” Opt. Express 10, 405–412(2002).
1. Y. H. Zhang, S. Poonja, and A. Roorda, “MEMS-basedadaptive optics scanning laser ophthalmoscopy,” Opt. Lett.31, 1268–1270 (2006).
2. J. Carroll, M. Neitz, H. Hofer, J. Neitz, and D. R. Williams,“Functional photoreceptor loss revealed with adaptiveoptics: an alternate cause of color blindness,” Proc. Natl.Acad. Sci. U.S.A. 101, 8461–8466 (2004).
3. R. L. Cook, “Stochastic sampling in computer graphics,”ACM Trans. Graphics 5, 51–72 (1986).
