A Computerized Method of Visual AcuityTesting: Adaptation of the Early Treatment ofDiabetic Retinopathy Study Testing Protocol
ROY W. BECK, MD, PHD, PAMELA S. MOKE, MSPH, ANDREW H. TURPIN, PHD,FREDERICK L. FERRIS III, MD, JOHN PAUL SANGIOVANNI, SCD,
CHRIS A. JOHNSON, PHD, EILEEN E. BIRCH, PHD, DANIELLE L. CHANDLER, MSPH,TERRY A. COX, MD, PHD, R. CLIFFORD BLAIR, PHD, AND RAYMOND T. KRAKER, MSPH
● PURPOSE: To develop a computerized method of visualacuity testing for clinical research as an alternative to thestandard Early Treatment for Diabetic RetinopathyStudy (ETDRS) testing protocol, and to evaluate itstest-retest reliability and concordance with standardETDRS testing.● DESIGN: Test-retest reliability study.● METHODS: Multicenter setting of a study population of265 patients at three clinical sites. Visual acuity wasmeasured with both the electronic visual acuity testingalgorithm (E-ETDRS) and standard ETDRS protocol(S-ETDRS) twice on one eye of each patient. E-ETDRStesting was conducted using the electronic visual acuitytester (EVA), which utilizes a programmed Palm (Palm,Inc, Santa Clara, California, USA) hand-held devicecommunicating with a personal computer and 17-inchmonitor at a test distance of 3 meters.● RESULTS: For the E-ETDRS protocol, test-retest reli-ability was high (r � 0.99; with 89% and 98% of retestswithin 0.1 logMAR and 0.2 logMAR of initial tests,respectively) and comparable with that of S-ETDRStesting (r � 0.99; with 87% and 98% of retests within0.1 logMAR and 0.2 logMAR of initial test, respective-ly). The E-ETDRS and S-ETDRS scores were highlycorrelated (r � 0.96 for initial tests and r � 0.97 forrepeat tests). Based on estimates of 95% confidence
measure in clinical research of eye diseases. Inmulticenter clinical trials, considerable effort is
placed on the standardization of acuity testing acrosssites.1–3 To more easily standardize measurement of visualacuity in clinical trials and to provide a method to directlycapture acuity data electronically, we have developed acomputerized vision tester called the electronic visualacuity tester (EVA).
Previously we reported on the adaptation of the Ambly-opia Treatment Study (ATS) visual acuity testing protocolfor the EVA.4 This testing protocol was developed tofacilitate the standardization of visual acuity testing inclinical trials of pediatric eye disease involving childrenfrom 3 to 6 years old.5
We now report on the development and evaluation of asecond testing protocol for the EVA, one for the testing ofolder children and adults. This protocol is based in part onthe testing protocol developed for Early Treatment forDiabetic Retinopathy Study (ETDRS),1,6 a protocol whichhas been the standard for visual acuity testing in mostclinical research for more than 15 years. We conducted astudy to assess test-retest reliability for both the electronic
Accepted for publication Aug 21, 2002.InternetAdvance publication at ajo.com Nov 14, 2002.From the Jaeb Center for Health Research (R.W.B., P.S.M., R.C.B.,
R.T.K.), Tampa, Florida; Curtin University (A.H.T.), Bentley, WesternAustralia; the National Eye Institute, Division of Epidemiology andClinical Research, the National Institutes of Health (F.L.F., J.P.S.G.,T.A.C.), Bethesda, Maryland; Discoveries in Sight Research Labs, DeversEye Institute (C.A.J.), Portland, Oregon; and the Retina Foundation ofthe Southwest (E.E.B.), Dallas, Texas.
This study was supported by the National Eye Institute grant #EY13095.Inquiries to Roy W. Beck, MD, PhD, Jaeb Center for Health Research,
3010 E. 138th Ave., Suite 9, Tampa, FL 33613; fax: (813) 975-8761;e-mail: [email protected]
(E-ETDRS) and standard ETDRS (S-ETDRS) protocolsand to compare visual acuity scores obtained with the twoprotocols.
METHODS
● DESCRIPTION OF THE EVA: The EVA utilizes a pro-grammed Palm hand-held device (Palm, Inc., Santa Clara,California, USA) that communicates with a personalcomputer running a Linux operating system (IBM Corp.,Armonk, New York, USA) (Figure 1). Stimuli are highcontrast, black-and-white letters with luminance of 85 to105 candelas/meter2 and contrast of 98%. Both Sloan andHOTV letter sets are available. Single letters are presentedframed with crowding bars that are spaced a letter widtharound the letter. With a high-resolution (1600 � 1200)17-inch monitor, letters can be displayed from 20/800 (1.6logarithm of the minimum angle of resolution [logMAR])to 20/12 (�0.2 logMAR) at a test distance of 3 meters.Letters are rendered and presented on the monitor bymanipulating the individual points in a graphical image,known as pixels (picture elements). Letter sizes are deter-mined by translating octave steps (3 logMAR lines) to thenumber of pixels for a given stroke width, beginning with3 pixels for a 20/12 letter. Letter size is a close, but notexact, approximation of the logMAR progression of theETDRS charts (within about 2% of the letter size at eachlogMAR level). The Palm hand-held device, which isconnected to the personal computer through a serial cable,provides instructions for the technician, displays the letterthat is being illustrated on the monitor, records the responses,and sends instructions to the personal computer with regardto the sequence of letter presentations. The size of each letterpresentation can be either controlled by the technician ordetermined from a computer program based on the subject’sresponses. Both the letter graphics and the Palm applicationsare written in the C programming language.
● ELECTRONIC ETDRS VISUAL ACUITY TESTING PRO-
TOCOL: The E-ETDRS testing protocol is described inFigure 2. Testing begins with a screening phase to deter-mine an approximate visual acuity threshold, using theletters V, R, K, and D (which have been reported to be ofintermediate and comparable identification difficulty6).This phase is followed by threshold testing to determine anupper logMAR level at which 5 of 5 letters are correctlyidentified and a lower logMAR level at which 0 of 5 lettersare correctly identified. The specific letters tested at eachlogMAR level are the same letters as those on theoriginal-series ETDRS charts for the right and left eyes.6 Aletter score to approximate the S-ETDRS score is com-puted as the number of letters correctly identified duringthreshold testing, plus 5 letters for each logMAR lineabove the upper boundary through 20/800.
FIGURE 1. Electronic visual acuity tester. The electronic visualacuity tester utilizes a programmed Palm (Palm, Inc., Santa Clara,California, USA) hand-held device (bottom) that communicateswith a personal computer running a Linux operating system (IBM,Armonk, New York, USA). With a high-resolution (1600 �1200) 17-inch monitor (top), single letters can be displayed from20/800 to 20/12 at a test distance of 3 meters.
A COMPUTERIZED METHOD OF VISUAL ACUITY TESTINGVOL. 135, NO. 2 195
● STUDY PROTOCOL: The test-retest reliability of bothE-ETDRS and S-ETDRS testing were evaluated at threesites. The study population consisted of individuals age 7
years old and older who were being seen as patients at thethree sites and who agreed to undergo the testing. Thestudy was approved by the respective Institutional Review
FIGURE 2. Electronic Early Treatment for Diabetic Retinopathy Study (E-ETDRS) visual acuity testing strategy. Testing begins witha screening phase to determine an approximate visual acuity threshold. This phase is followed by threshold testing to determine an upperlogMAR level at which 5 of 5 letters are correctly identified and a lower logMAR level at which 0 of 5 are correctly identified.
AMERICAN JOURNAL OF OPHTHALMOLOGY196 FEBRUARY 2003
Boards, and was in accordance with the Declaration ofHelsinki; informed consent was obtained from patients.
Visual acuity was measured with both the E-ETDRS andS-ETDRS protocols twice on one eye of each patient. TheE-ETDRS testing protocol, which uses single-letter presen-tations at a 3-meter test distance, is described above. TheS-ETDRS testing protocol uses a 5-letter per line chart ata 4-meter test distance. The patient is asked to read eachletter starting at the top of the chart (20/200). If fewerthan 20 letters are correctly identified at 4 meters, testingis also done at 1 meter. Testing ends when no letters arecorrectly identified on a line.
For the first 151 patients, the right eye was the study eye;thereafter, the eye with the worse acuity was tested so as toincrease the number of eyes with reduced acuity in thestudy sample. At two sites (at which 128 and 60 patientsparticipated) the testing order was S-ETDRS, E-ETDRS,repeat E-ETDRS, and repeat S-ETDRS. This testing orderpermitted completion of the initial S-ETDRS testingbefore any other testing to avoid interfering with otherstudies for which the S-ETDRS data were being used. Atone site at which 77 patients participated, patients werenot participating in any other studies using the acuity data;therefore the testing order of the S-ETDRS and E-ETDRSprotocols was randomly determined for the initial tests,followed by repeat testing in the same order. The testingwas conducted in sequence without a break unless thepatient was tired, in which case a short break was given. In
most cases, a patient’s repeat testing was conducted by thesame technician as the one who conducted the initialtesting. The S-ETDRS testing was conducted with chartsplaced in a retroilluminated light box at a test distance of4 meters (as per the protocol, the test distance was reducedto 1 meter when the letter score at 4 meters was less than20).6 The E-ETDRS testing was conducted at a testdistance of 3 meters. The S-ETDRS letter score wascalculated as the number of letters correctly identified at
TABLE 1. Conversions Between Letter, LogMAR, andSnellen Visual Acuity Scores
Letter Score LogMAR Value Snellen Equivalent
5 1.6 20/800
10 1.5 20/640
15 1.4 20/500
20 1.3 20/400
25 1.2 20/320
30 1.1 20/250
35 1.0 20/200
40 0.9 20/160
45 0.8 20/125
50 0.7 20/100
55 0.6 20/80
60 0.5 20/63
65 0.4 20/50
70 0.3 20/40
75 0.2 20/32
80 0.1 20/25
85 0.0 20/20
90 �0.1 20/15
95 �0.2 20/12
LogMAR � logarithm of the minimal angle of resolution.
TABLE 2. Baseline Characteristics of Subjects
Baseline Characteristics* Total Number � 265
Age n (%)
7–17 years 24 (9)
18–50 years 108 (41)
� 50 years 130 (50)
mean (� SD), years 50 � 22
Female sex, number (%) 155 (59)
Ethnicity, number (%)
Caucasian 201 (77)
African American 40 (15)
Hispanic 11 (4)
Asian 9 (3)
Other 1 (.4)
Education, number (%)
Less than high school 30 (15)
High school 56 (28)
College 80 (40)
Post-graduate 35 (17)
Clinical diagnosis, number (%)
Normal 53 (20)
Uncorrected refractive error 21 (8)
Age-related macular
degeneration
25 (10)
Diabetic retinopathy 15 (6)
Other retinal disease 58 (22)
Cataract 11 (4)
Optic nerve 3 (1)
Cornea 5 (2)
Uveitis 30 (11)
Glaucoma 29 (11)
Amblyopia 2 (1)
Other 11 (4)
Visual acuity on initial S-ETDRS
test, number (%)
� 20/20 (logMAR � 0.00) 55 (21)
� 20/20 to � 20/40 (logMAR
0.02 to 0.28)
77 (29)
20/40 to 20/100 (logMAR
0.30 to 0.70)
80 (30)
� 20/100 (logMAR � 0.70) 53 (20)
S-ETDRS � standard Early Treatment for Diabetic Retinopa-
thy Study.
*Missing data: age (3), gender (3), ethnicity (3), education (64),
and diagnosis (2).
A COMPUTERIZED METHOD OF VISUAL ACUITY TESTINGVOL. 135, NO. 2 197
4 meters plus 30 when the 4-meter score was � 20 and plusthe number of letters correctly identified at 1 meter whenthe 4-meter score was � 20. All testing was conductedeither with the patient’s current spectacles or withoutcorrection. A refraction was not performed for purposes ofthe study; therefore the measured acuities did not neces-sarily represent best-corrected visual acuity. Study enroll-ment continued until there were at least 50 subjects withstudy eyes with acuity worse than 20/100.
The primary diagnosis/cause of visual loss of each eyewas recorded as one of the following: normal, uncorrectedrefractive error, macular disease (age-related macular de-generation, diabetic retinopathy, or other retinal diseasesor dystrophies), cataract, optic nerve disease, uveitis,glaucoma, amblyopia, or other.
● DATA ANALYSIS: Only patients who had scores forall four tests were included in the analysis (data from 14patients with incomplete testing were not included). Forboth the E-ETDRS and S-ETDRS tests, the letter scoreswere converted to logMAR equivalents using the formulalogMAR � 1.7 � (.02)(letter score). Conversions be-tween letter scores and Snellen scores are given in Table 1.With this conversion, a 5-letter difference in visual acuityis equivalent to a difference of 0.1 logMAR and to oneSnellen line.
To assess test-retest reliability, frequency distributions ofthe differences in visual acuity scores between the initialand repeat test score were evaluated and intraclass corre-lation coefficients were computed. Ninety-five percentconfidence intervals (95% CI) for an acuity score weredetermined based on the standard error of measurement,7
and for a change in an acuity score from a baseline levelwere determined based on the standard error of thedifferences. Analyses were replicated in subgroups basedon visual acuity and age. McNemar test was used tocompare proportions in paired data; Fisher exact test wasused to compare proportions between subgroups. Separateassessments were made using identical methods for theS-ETDRS and E-ETDRS tests.
To compare E-ETDRS scores with the S-ETDRSscores, frequency distributions of the differences betweenE-ETDRS and S-ETDRS visual acuity scores for eachsubject were constructed and intraclass correlation coeffi-cients were computed. Analyses were replicated in sub-groups based on visual acuity and age. Similar methodswere used to compare E-ETDRS and S-ETDRS scores in asubset that included only eyes with high test-retest reli-ability (both E-ETDRS and S-ETDRS test-retest scoreswithin five letters).
Bland-Altman plots8 were constructed to further assesstest-retest reliability and the concordance of S-ETDRSand E-ETDRS according to level of visual acuity. Allanalyses were conducted using SAS software version 8(Cary, North Carolina, USA).9
RESULTS
● DESCRIPTION OF SUBJECTS: The mean age of 265patients was 50 � 22 years old; 155 were female (59%),and 201 were Caucasian (77%). Fifty-three (20%) studyeyes were considered normal (including corrected refrac-tive error), 21 had an uncorrected refractive error (8%),and 189 had eye disease (72%). Visual acuity was betterthan 20/40 in 132 eyes (50%), 20/40 to 20/100 in 80 eyes(30%), and worse than 20/100 in 53 eyes (20%) (Table 2).
● TEST-RETEST RELIABILITY ASSESSMENTS: The distri-butions of the test-retest differences for both the E-ETDRSand S-ETDRS testing appeared similar (Table 3). For bothmethods, the correlation between the initial and the retestvisual acuity scores was 0.99. For the E-ETDRS testing,89% of the retest scores were within 0.1 logMAR of theinitial test score and 98% were within 0.2 logMAR;whereas for the S-ETDRS testing, 87% of the retest scoreswere within 0.1 logMAR of the initial test score and 98%were within 0.2 logMAR (Table 4 and Figure 3).
Test-retest reliability was high across the range of visualacuity. With both testing methods, more than 90% of
TABLE 3. Test-Retest Differences With the ElectronicETDRS and Standard ETDRS Protocols
Absolute Value of
Difference in Letters*
E-ETDRS
(Number � 265)
S-ETDRS
(Number � 265)
n (%)†Cumulative
Percent† n (%)†Cumulative
Percent†
0 34 (13) 13% 44 (17) 17%
1 63 (24) 37% 71 (27) 43%
2 53 (20) 57% 40 (15) 58%
3 40 (15) 72% 27 (10) 69%
4 26 (10) 82% 28 (11) 79%
5 19 (7) 89% 21 (8) 87%
6 12 (5) 93% 12 (5) 92%
7 5 (2) 95% 12 (5) 96%
8 5 (2) 97% 2 (1) 97%
9 3 (1) 98% 2 (1) 98%
10 1 (.4) 98% 1 (.4) 98%
11 1 (.4) 99% 2 (1) 99%
12 1 (.4) 99% 2 (1) 100%
13 0 (0) 99% 1 (.4) 100%
14 1 (.4) 100% 0 (0) 100%
15 0 (0) 100% 0 (0) 100%
16 0 (0) 100% 0 (0) 100%
17 0 (0) 100% 0 (0) 100%
18 1 (.4) 100% 0 (0) 100%
E-ETDRS � electronic Early Treatment for Diabetic Retinop-
logMAR � 15 letters.†Due to rounding, percentages may not sum to 100 and/or
their respective cumulative percentages.
AMERICAN JOURNAL OF OPHTHALMOLOGY198 FEBRUARY 2003
retests were within 0.2 logMAR of the initial test evenwhen acuity was worse than 20/100. When acuity wasbetter than 20/40, the proportion of retests within 0.1logMAR of the initial test was higher with the E-ETDRSmethod than with the S-ETDRS method (P � .02; Table4 and Figure 4). Table 4 also provides the 95% CI for bothmethods for an individual acuity score as well as for achange in acuity.
Test-retest reliability was high irrespective of age (Table5). With both testing methods, more than 95% of retestswere within 0.2 logMAR of the initial test even in patients65 years old or older.
● COMPARISON OF E-ETDRS AND S-ETDRS SCORES:
Comparing each patient’s E-ETDRS score and S-ETDRSscore, there was no tendency for scores on one test to behigher than on the other (Table 6). The correlation
between the E-ETDRS and S-ETDRS acuity scores was0.96 for the initial tests and 0.97 for the retests. TheE-ETDRS and S-ETDRS scores differed by � 0.1 logMARon 74% of the initial tests and 79% of the repeat tests anddiffered by � 0.2 logMAR on 94% and 96% of tests,respectively (Table 6 and Figure 5).
Agreement between the E-ETDRS and S-ETDRS scoreswas lower in patients with acuity worse than 20/100compared with patients with acuity of 20/100 or better (foragreement within 0.2 logMAR, P � .05 for initial testsand P � .01 for repeat tests; Table 6 and Figure 6), butagreement was similar when comparing patients 65 yearsold or older with patients younger than 65 years old(Table 7).
With the analysis limited to the 207 patients with hightest-retest reliability (test-retest scores within 0.1 logMARwith both the E-ETDRS and S-ETDRS tests), the
TABLE 4. Test-Retest Differences With the Electronic ETDRS and Standard ETDRS Protocols According to Visual Acuity*
(95% CI) (0.33 to 1.21) (0.68 to 1.53) (�0.19 to 0.77) (0.18 to 1.35) (0.34 to 1.96) (0.76 to 2.24) (0.01 to 2.82) (0.24 to 2.44)
Intraclass
correlation
coefficient
.99 .99 .94 .86 .91 .87 .93 .95
95% CI (two-sided)
for acuity score,
logMAR: half-
width
.10 .10 .08 .09 .09 .08 .14 .11
95% CI for change
between two
acuity scores,
logMAR
Half-width of
two-sided CI
.14 .14 .11 .13 .13 .11 .20 .15
Width of one-
sided CI
.12 .12 .09 .11 .11 .10 .17 .13
CI � confidence interval; E-ETDRS � electronic Early Treatment for Diabetic Retinopathy Study; logMAR � logarithm of the minimal angle
of resolution; S-ETDRS � standard ETDRS; SD � standard deviation.
*Visual acuity based on letter score from initial standard ETDRS test. Visual acuity � 20/40 � logMAR � 0.28; 20/40–20/100 � logMAR
0.30 to 0.70; and � 20/100 � logMAR � 0.70.†0.1 logMAR � 5 letters; 0.2 logMAR � 10 letters; 0.3 logMAR � 15 letters; 0.4 logMAR � 20 letters.‡Positive value means repeat score was higher (better).
A COMPUTERIZED METHOD OF VISUAL ACUITY TESTINGVOL. 135, NO. 2 199
E-ETDRS and S-ETDRS scores differed by � 0.1 logMARon 81% of the initial tests and 84% of the repeat tests, anddiffered by � 0.2 logMAR on 95% and 99% of tests,respectively.
● DIFFERENCES AMONG CENTERS: Although there weresome differences in the characteristics of the patients whencomparing the three clinical sites (as was expected, due tothe different types of patients each had access to forinclusion in the study), there were no meaningful differ-ences in the results comparing the centers (data notshown).
At the one site at which the testing order of S-ETDRSand E-ETDRS was determined at random, the test-retestreliability results and the results of the comparison of the
scores on the two tests were similar to the overall results(data not shown).
DISCUSSION
WE FOUND THAT OUR COMPUTERIZED VISUAL ACUITY
testing protocol has test-retest reliability comparable to thestandard ETDRS chart testing protocol on which it wasbased. With both testing methods, close to 90% of patientstested within 0.1 logMAR (5 letters) of the initial score onthe retest, and more than 95% tested within 0.2 logMAR(10 letters) on the retest. Test-retest reliability did notvary with age, but, not unexpectedly, variability wasslightly greater in patients with poor acuity. Still, about
FIGURE 3. Distribution of test-retest differences in acuityscores (number � 265). A positive difference indicates therepeat score was better than the initial score.
FIGURE 4. Bland Altman8 plot of test-retest difference versusaverage visual acuity score (number � 265). A positive differ-ence indicates the repeat score was better than the initial score.The dotted lines separate the test-retest scores that were within10 letters of each other from those in which the difference wasgreater than 10 letters.
AMERICAN JOURNAL OF OPHTHALMOLOGY200 FEBRUARY 2003
80% of patients with poor acuity (worse than 20/100) hadretest scores within 0.1 logMAR (5 letters) of the initialtest scores, and about 95% retested within 0.2 logMAR ofthe initial score. Based on our estimates of 95% CI, wefound that a change in acuity of 0.2 logMAR (10 letters)
from a baseline level is unlikely to be related to measure-ment variability using either the E-ETDRS or theS-ETDRS visual acuity testing protocol.
We found agreement between the E-ETDRS andS-ETDRS scores to be high, suggesting strong concurrent
TABLE 5. Test-Retest Differences With the E-ETDRS and S-ETDRS Protocols According to Age*
CI � confidence interval; E-ETDRS � electronic Early Treatment for Diabetic Retinopathy Study; logMAR � logarithm of the minimum angle
of resolution; SD � standard deviation; S-ETDRS � standard ETDRS; SD � standard deviation.
*Visual acuity based on letter score from initial standard ETDRS test: � 20/40 � logMAR � 0.28; 20/40–20/100 � logMAR 0.30 to 0.70;
and � 20/100 � logMAR � 0.70.†0.1 logMAR � 5 letters; 0.2 logMAR � 10 letters; 0.3 logMAR � 15 letters; 0.4 logMAR � 20 letters.‡Negative value means S-ETDRS letter score was higher (better).
A COMPUTERIZED METHOD OF VISUAL ACUITY TESTINGVOL. 135, NO. 2 201
validity. The differences in visual acuity scores, however,were greater in comparing the two testing methods than inthe repeating of each test. This is to be expected becausethe two testing strategies and their methods of computingthe visual acuity score differ, thus adding a second source
of variation in addition to the test-retest variability. This isparticularly true for the testing of patients with low vision,for whom the S-ETDRS score combines the results oftesting at 4 meters and 1 meter, whereas the E-ETDRStesting is at a single distance. The E-ETDRS score was
FIGURE 5. Distribution of differences between acuity scores on electronic Early Treatment for Diabetic Retinopathy Study(ETDRS) and standard ETDRS testing (number � 265). A positive difference indicates the electronic ETDRS score was betterthan the standard ETDRS score.
AMERICAN JOURNAL OF OPHTHALMOLOGY202 FEBRUARY 2003
developed to be similar to the S-ETDRS letter score and tomaintain the advantages of letter scoring over line scor-ing.6,10–13 However, some variation undoubtedly is due todifferences in the scoring methods and to the effect ofusing single-letter presentations with the E-ETDRSprocedure and line presentations with the S-ETDRS
procedure.14 –16 There was no tendency for the E-ETDRS and S-ETDRS scores to be higher or lower thanthe other.
For both procedures, retest scores on average were aboutone letter (or less) higher than the initial test scores,indicative of an inconsequential learning effect. Most prior
FIGURE 6. Bland Altman8 plot of difference between acuity scores on electronic Early Treatment for Diabetic Retinopathy Study(ETDRS) and standard ETDRS vs average visual acuity score (number � 265). A positive difference indicates the electronicETDRS score was better than the standard ETDRS score. The dotted lines separate the E-ETDRS: S-ETDRS scores that werewithin 10 letters of each other from those in which the difference was greater than 10 letters.
A COMPUTERIZED METHOD OF VISUAL ACUITY TESTINGVOL. 135, NO. 2 203
studies2,3,6,11 have also reported that the learning effect isminimal using S-ETDRS or similar testing.
Camparini and colleagues17 reported the results of anadaptive strategy for visual acuity chart testing calledETDRS-fast. The procedure involves a determination ofapproximate threshold followed by the testing of all 5letters on a line. The approximate threshold is determinedby showing one letter per line on the chart starting at20/200 until a letter is missed and then showing multipleletters per line until the lowest line with 4 or 5 correct isidentified. Smaller lines are then tested as per the standardETDRS testing protocol until 0 or 1 letter is correct on aline. A letter score is determined, similar to our scoringmethod, by assuming that all letters larger than the largesttested line would have been correctly identified and noneof the letters smaller than the smallest tested line wouldhave been correctly identified. Both test-retest reliability(r � 0.96; 97% of retests within 0.1 logMAR of initial test)and the correlation of the ETDRS-fast score with thestandard ETDRS score (r � 0.95) were found to be high ina study of 57 patients with acuities in the range of 20/100to 20/10, most of whom had acuity of 20/40 or better.
With the E-ETDRS testing method, the number ofletter presentations in our study averaged about 25 letters(for the screening phase and threshold testing combined)when visual acuity was better than 20/40 and about 30letters when acuity was 20/40 or worse. Compared withS-ETDRS testing, the number of letter presentations withthe E-ETDRS method usually will be fewer for acuitiesbetter than 20/63. The better the acuity, the larger thedifference in number of letters tested will be. For instance,with acuity of 20/20, about 55 letters will be tested withS-ETDRS testing compared with about 25 letters withE-ETDRS testing. For very poor acuity, however, theE-ETDRS procedure may test more letters than will theS-ETDRS procedure, depending on whether S-ETDRStesting is also done at 1 meter.
Several other studies have demonstrated that theETDRS testing and similar optotype testing with letterscoring have high test-retest reliability.2,6,11,13 Most of thestudies have predominately included eyes with normal or
near-normal vision. Only the study of Blackhurst andassociates2 included a substantial number of eyes withvisual impairment. In that study, visual acuity was testedtwice using the ETDRS charts in 89 eyes with acuity of20/100 or better and in 75 eyes with acuity worse than20/100. Test-retest reliability was higher in the eyes in thebetter vision group. The percentages of eyes with retestscores within 0.1 logMAR and 0.2 logMAR of the initialscores were 92% and 98%, respectively, in the better visiongroup and 80% and 89%, respectively, in the worse visiongroup. The intraclass correlation coefficient was 0.95 ineach group.
In summary, the E-ETDRS protocol has high test-retestreliability and good concordance with S-ETDRS testing.Although we expect that this testing method will be usefulfor pediatric patients as young as 6 or 7 years of age, we hadtoo few children in the study to be able to determine thisand, therefore, will be conducting a separate test-retestreliability study in a pediatric population. For a clinicaltrial, the potential advantages of using a computerizedmethod of testing over manual testing include betterstandardization of the testing procedure across multiplesites, less training required for the technicians administer-ing the test, reduction of potential bias by limiting the roleof the technician in the testing procedure, the ability totest visual acuity from 20/800 to 20/12 at a single testdistance, the ability to directly capture the testing dataelectronically without the need to manually record everyresponse on a score sheet and to calculate the scoreautomatically, and the need for fewer optotype presenta-tions when visual acuity is good. Disadvantages include thecost of purchasing and maintaining the equipment, theinability to measure 20/10 visual acuity at the 3-meter testdistance (although this would be possible if the testdistance were increased to 4 meters), and the fact thatletter size is a close approximation, but not an exactlogMAR progression, due to the method by which theletters are rendered on the personal computer. Despite thepotential advantages of the E-ETDRS protocol, incorpo-rating this new procedure into ongoing clinical studies thatare using the S-ETDRS protocol to assess change in visual
TABLE 7. Differences Between E-ETDRS and S-ETDRS According to Age
Absolute Value of
Difference*
Age � 40 Years Old (Number � 86) Age 40–� 65 Years Old (Number � 94) Age � 65 Years Old (Number � 82)
Initial Test n (%) Repeat Test n (%) Initial Test n (%) Repeat Test n (%) Initial Test n (%) Repeat Test n (%)
AMERICAN JOURNAL OF OPHTHALMOLOGY204 FEBRUARY 2003
acuity seems inappropriate. However, for new clinicalstudies, the E-ETDRS method is a viable alternative thatshould be considered. The E-ETDRS protocol can beadapted for use on commercially-available computerizedvisual acuity testers using different operating systems.However, for use with standard office personal computers,considerable modifications would be needed to provide thenecessary pixel resolution. Information on the requiredtechnical specifications can be obtained by contacting theauthors. Future work is indicated to determine whether theE-ETDRS testing algorithm can be modified to furtherreduce the number of letter presentations by altering thestopping rules and/or reducing the number of presentationsat each logMAR level.
ACKNOWLEDGMENTS
The following research staff participated in the study at theclinical sites: R. Mercer, L. Goodman, D. Koutsandras, T.LaReau, R. Nashwinter, J. P. Rowan, G. Foster, W. R.O’Donnell (National Eye Institute, Division of Epidemi-ology and Clinical Research, National Institutes of Health,Bethesda, MD); K. G. Locke, C. E. Wilson (RetinaFoundation of the Southwest, Dallas, TX); and J. Thomp-son, T. Smith, C. Blachly, K. Novitsky (Discoveries inSight Research Labs, Devers Eye Institute, Portland, OR).
REFERENCES
1. Early Treatment Diabetic Retinopathy Study ResearchGroup. Early treatment diabetic retinopathy study design andbaseline patient characteristics. ETDRS report number 7.Ophthalmology 1991;98:741–756.
2. Blackhurst DW, Maguire MG, Macular PhotocoagulationStudy Group. Reproducibility of refraction and visual acuitymeasurement under a standard protocol. Retina 1989;9:163–169.
3. Elliott DB, Sheridan M. The use of accurate visual acuity
measurements in clinical anti-cataract formulation trials.Ophthalmic Physiol Opt 1988;8:397–401.
4. Moke PS, Turpin AH, Beck RW, et al. Computerizedmethod of visual acuity testing: adaptation of the amblyopiatreatment study visual acuity testing protocol. Am J Oph-thamol 2001;132:903–909.
5. Holmes JM, Beck RW, Repka MX, et al. The amblyopiatreatment study visual acuity testing protocol. Arch Oph-thalmol 2001;119:1345–1353.
6. Ferris FL, Kassoff A, Bresnick GH, Bailey I. New visualacuity charts for clinical research. Am J Opthalmol 1982;94:91–96.
7. Crocker L, Algina J. Introduction to classical and moderntest theory. New York: Holt, Rinehart and Winston, 1986:150–151.
8. Bland JM, Altman DG. Statistical methods for assessingagreement between two methods of clinical measurement.Lancet 1986;1:307–310.
10. Vanden Bosch ME, Wall M. Visual acuity scored by theletter-by-letter or probit methods has lower retest variabilitythan the line assignment method. Eye 1997;11:411–417.
11. Lovie-Kitchin JE. Validity and reliability of visual acuitymeasurements. Ophthalmic Physiol Opt 1988;8:363–370.
12. Bailey IL, Bullimore MA, Raasch TW, Taylor HR. Clinicalgrading and the effects of scaling. Invest Ophthalmol Vis Sci1991;32:422–432.
13. Arditi A, Cagenello R. On the statistical reliability ofletter-chart visual acuity measurements. Invest OphthalmolVis Sci 1993;34:120–129.
14. Stager DR, Everett ME, Birch EE. Comparison of crowdingbar and linear optotype acuity in amblyopia. Am Orthopt J1990;40:51–56.
15. Flom MC, Weymouth FW, Kahneman D. Visual resolutionand contour interaction. J Opt Soc Am 1963;53:1026–1032.
16. Stuart JA, Burian HM. A study of separation difficulty: itsrelationship to visual acuity in normal and amblyopic eyes.Am J Ophthalmol 1962;53:471–477.
17. Camparini M, Cassinari P, Ferrigno L, Macaluso C. ETDRS-fast: implementing psychophysical adaptive methods to stan-dardized visual acuity measurement with ETDRS charts.Invest Ophthalmol Vis Sci 2001;42:1226–1231.
A COMPUTERIZED METHOD OF VISUAL ACUITY TESTINGVOL. 135, NO. 2 205
