A Prospective Three-year Studyof Response Properties ofNormal Subjects and Patientsduring Automated Perimetry

Chris A. Johnson, PhD, Jacqueline M. Nelson-Quigg, BS

Purpose: The purpose of this study is to evaluate prospectively the reliability char­acteristics of patients undergoing automated perimetry over a 3-year period and comparethese results with the results of previous investigations.

Methods: The subjects included 48 normal observers, 32 ocular hypertensive sub­jects, and 19 patients with early glaucomatous visual field loss. Both eyes were testedannually for 3 years with automated perimetry, using the standard procedures for theHumphrey Field Analyzer. Fixation losses, false-positive errors, false-negative errors,and short-term variability (double determinations) were evaluated.

Results: Short-term variability was slightly higher for the early glaucoma group thanfor the normal observer and ocular hypertensive groups, but there were no meaningfulchanges in short-term variability over 3 years. False-positive errors were very low in allthree groups throughout the investigation. False-negative errors were slightly higher inthe early glaucoma group, but all three groups had relatively low false-negative errorrates throughout the study. Fixation losses were the most common source of unreliableresults. The number of fixation losses decreased for the second and third years of thestudy.

Conclusion: Contrary to a previous report, a relatively low number of unreliabletests were found for both initial and follow-up visits. The majority of unreliable visualfield tests were sporadic events. Only a few subjects repeatedly produced unreliabletest results. The authors conclude that automated perimetry can provide a reliablemeans of following patients over extended periods of time.Ophthalmology 1993;100:269-274

Originally received: June 29, 1992.Revision accepted: September 22, 1992.

From the Optics and Visual Assessment Laboratory (OVAL), Departmentof Ophthalmology, University of California at Davis, Davis.

Presented in part at the Association for Research in Vision and Oph­thalmology Annual Meeting, Sarasota, May 1991.

Supported in part by National Eye Institute Research Grant #EY03424,Bethesda, Maryland (Dr. Johnson), a Research to Prevent Blindness Se­nior Scientific Investigator Award, New York, New York (Dr. Johnson),and an unrestricted research grant from Research to Prevent Blindness,Inc, New York, New York.

The authors have no financial interest in the instrumentation used inthis study.

Reprint requests to Chris A. Johnson, PhD, Optics and Visual AssessmentLaboratory (OVAL), Department of Ophthalmology, University of Cal­ifornia at Davis, Davis, CA 95616.

Automated perimetry has become the standard procedurefor visual field examination of glaucoma patients andglaucoma suspects. One of the important aspects pertain­ing to the clinical efficacy of automated perimetry is itsability to provide reliable and reproducible test results.Although stimulus conditions and test strategieshave beenoptimized to provide the most accurate visual field mea­sures, the patient's ability and/or willingness to complywith the demands of the test procedure also are criticalfactors in the quality and clinical use of automated pe­rimetry test results. It is for this reason that manufacturersof many automated perimeters provide indicators of re­sponse reliability as a means of evaluating the validity ofvisual field test results.

The Humphrey Field Analyzer (Humphrey Instru-

269

Ophthalmology Volume 100, Number 2, February 1993

ments, San Leandro, CA) incorporates several proceduresthat are designed to provide information about a patient'sresponse reliability and fixation behavior during test­ing.':" Fixation losses are denoted by periodically pre­senting a stimulus at the location of the patient's blindspot and determining whether the patient responds to thestimulus. If the blind spot stimulus is seen, it is recordedas a fixation loss. False-negative errors are evaluated bypresenting stimuli that are 9 decibels (dB)above previouslydetermined thresholds and recording the number of in­stances that the patient fails to respond. Similarly, false­positive errors are determined by recording the numberof responses that occur when no stimulus is presented.The reproducibility of visual field sensitivity measures isevaluated by performing double threshold determinationsat 10 preselected visual field locations. A short-term fluc­tuation value is then calculated from these 10 double de­terminations. The reliability indices (fixation losses, false­positive errors, false-negative errors) and short-term fluc­tuation are compared with age-related normal populationcharacteristics. If one or more of these values are beyondacceptable limits for the normal population, a messageindicating low patient reliability is printed on the test re­sults.

Several investigators'r!? have studied the performancecharacteristics of the reliability indices of the HumphreyField Analyzer. To date, most studies'"!? have found alarger than expected number ofcases in which the rate offixation lossesexceeded the normal criterion (greater than20%). However, a recent investigation? indicates that ad­ditional instruction and training of technicians adminis­tering automated perimetry can substantially reduce therate of excessive fixation losses. The number of cases inwhich false-positive and/or false-negative responses haveexceeded the normal criterion (greater than 33%) has var­ied from one study to another. Some studies have reportedonly a small number ofcases exceeding false-positive andfalse-negative criteria." 10 some investigators have reporteda high number of unreliable tests,6,8 whereas others havereported intermediate results.' The basis for these differ­ences is probably due in part to differences in the popu­lations tested and in part to the training and ongoingfeedback and instruction given to the technicians. In ad­dition, it also appears that excessive false-negative ratesare related to the amount of the patient's visual field loss.6,8

In addition to determining the reliability of test resultsfor a single examination, it also is important for patientmanagement to assess response reliability over repeatedtesting. Katz et al8 recently reported that 4% of visuallynormal subjects and 8% to 9% ofocular hypertensive andglaucoma patients had unreliable test results every timethey were tested (3 or more tests). They also reported thatapproximately 36% of visually normal subjects, 51% ofocular hypertensive subjects, and 68% of glaucoma pa­tients had at least one unreliable visual field out of 3 to 4visual fields performed over a period of several years. Be­cause automated perimetry is an important clinical pro­cedure for following ocular hypertensive and glaucomapatients over many years, these findings are quite dis­turbing.

270

In contrast to the findings of Katz et al, patients en­rolled in the Optic Neuritis Treatment Trial have dem­onstrated a very low rate of unreliable automated perim­etry results, both for initial and for follow-up tests (pre­sented as posters at the 1990 and 1991 Annual Meetingsof the Association for Research in Vision and Ophthal­mology, Sarasota, Florida). However, the optic neuritispatient population is considerably different from the sub­jects in the study by Katz et al. Therefore, the purpose ofthe current study is to examine the reliability of normalobservers (n = 48, both eyes), ocular hypertensives (n= 32, both eyes), and early glaucoma patients (n = 19,both eyes) enrolled in our prospective study of early glau­coma detection11-13 for a period of 3 years, and compareit with the results obtained by Katz et al." We were par­ticularly interested in determining whether unreliable vi­sual field tests were sporadic events or whether they wereconsistently present over repeated years of testing for thesame observer. In addition, we were interested in char­acterizing the short-term fluctuation characteristics (dou­ble determinations) of each of the groups, and in deter­mining whether reliability improved as a function of timeand additional experience with automated perimetry.

Methods

For all participants, visual field examinations were per­formed on both eyes using program 30-2 (size III target)and the standard background luminance of 31.5 apostilbson the Humphrey Field Analyzer. Details of the test strat­egy used by the Humphrey Field Analyzer have been de­scribed previously.3,4-6,11-13 All visual fields were per­formed with an optimal refractive correction in place(distance correction plus an appropriate near add), whichwas refined by subjective refraction at the perimeter bowl.In addition to static threshold measurements, the Hum­phrey Field Analyzer also evaluates false-positive errors(responses when no stimuli are presented), false-negativeerrors (no response to a target that has been previouslyestablished to be highly visible), fixation losses (a responseto a stimulus presented in the blind spot), and doubledeterminations (static thresholds obtained twice for thesame location).

Forty-eight visually normal observers (ages 20 to 72),32 ocular hypertensive patients, and 19 patients with earlyglaucomatous visual field loss participated in the study.Informed consent was obtained from all participants ineach year of the study, in accordance with National In­stitutes ofHealth guidelines for protection of human sub­jects, by means of an approved Human Subjects ConsentForm (University of California, Davis Institutional Re­view Board protocols #88-152, #89-171, and #90-200).Sixteen visually normal subjects were younger than 40years, 17 were between 40 and 60 years, and 15 were olderthan 60 years of age. In the ocular hypertensive group, 1individual was younger than 40 years, 8 were between 40and 60 years, and 23 were older than 60 years of age. Theearly glaucoma group had 1 individual younger than 40

Johnson and Nelson-Quigg· Response Properties during Automated Perimetry

~ ~ .threshold measures for double determination points used to assess short­term fluctuation on the Humphrey Field Analyzer. Results for normalobservers (dotted bars), ocular hypertensive subjects (hatched bars), andpatients with early glaucomatous visual field loss (solid bars) are presentedfor 3 successive years of testing.

32

YEAR

o

• EarlyGlaucomas

m OCular Hypenenatv81

iii" Cl Nonnals:!!. '----------'

~ 3zWIZ:WILILQ

~ 2s;::iiizw.,w~IZ:w>«

the study. In general, these findings indicate a relativelylow rate of false-positive results among all groups for all3 years. Figure 2B presents the percentage of normal ob­server, ocular hypertensive, and early glaucoma partici­pants that exceeded the 33% false-positive reliability cri­terion for years 1 to 3 of the study. None of the visuallynormal observers or early glaucoma patients exceeded thefalse-positive criterion during any part of the study. Twoof the 64 ocular hypertensive eyes (approximately 3.1%)exceeded the 33% false-positive criterion. Since the 33%false-positive criterion is intended to represent the normal95% confidence limits, these findings are within expectedvalues.

The average false-negative rates for the normal ob­server, ocular hypertensive, and early glaucoma groupsare presented in Figure 3A for years 1 to 3. Both the nor­mal observer and ocular hypertensive groups showed anaverage false-negative rate of approximately 3% to 4%,which remained constant through all 3 years. The earlyglaucoma group with visual field loss demonstrated ahigher average false-negative rate that was between 7%and 11 % for years 1 through 3. As indicated in Figure 3B,there were a small number of cases in which the 33%false-negative criterion was exceeded. None of the ocularhypertensive eyes demonstrated excessive false-negativeerrors in any of the 3 years. One normal observer eyeexceeded the false-negative criterion in years 2 and 3; allnormal observer eyes were within normal limits for false­negativesin year 1.Between 3%and 5% of early glaucomaeyes were outside normal limits for false-negative errorsin years 1 and 3, but all were within normal limits inyear 2.

years, 5 between 40 and 60 years, and 13 older than 60years of age.

Inclusion criteria for visually normal observers con­sisted of a corrected visual acuity of 20/40 or better inboth eyes, a refractive error of no greater than 5 dioptersspherical equivalent and 3 diopters of cylinder in botheyes, intraocular pressure of less than 20 mmHg in botheyes, no history of ophthalmic or neurologic disease orsurgery, no history of diabetes or other systemic diseases,and no use of medications known to affect visual fieldsensitivity. Ocular hypertensive patients had the same in­clusion criteria as visually normal observers, except thattheir intraocular pressures were greater than 21 mmHgin both eyes.Their visual fields(Humphrey 30-2) at entryto the study had to be within normal limits on the threevisual field indices available in the 1987 version of Statpak(MD, PSD, CPSD). Early glaucoma patients had the samecriteria as the ocular hypertensive patients, except thatthey had to exhibit early visual field loss (P < 5% prob­ability level for one or more of the visual field indices[MD, PSD, CPSD]) in one or both eyes.

All participants were tested with automated perimetryat yearly intervals for 3 years. Since previous studies'showed no age-related differences in reliability character­istics for visually normal subjects or patients, the normalobserver, ocular hypertensive, and early glaucoma groupswere not stratified by age; rather, data from all age groupswere combined. In all cases, the visual fields performedat entry to the study (for purposes of eligibility and clas­sification) also were used as the visual field data for year1 of the study. Before this investigation, all ocular hyper­tensive and early glaucoma patients had previously un­dergone automated or manual visual field testing. Mostof the visually normal observers had not previously un­dergone visual field examinations.

Results

Figure 1 presents the average difference in sensitivity (indB) between the first and second threshold measures often double determination points. This value provides anindication of the consistency of sensitivitymeasures withina single examination. The visually normal observer andocular hypertensive groups showed an average differenceof approximately 1.5 dB, which remains constantthroughout all 3 years of the study. The early glaucomagroup had a slightly higher average difference of approx­imately 2.3 dB for years 1 and 3, and approximately 3 dBfor year 2. In general, there were no meaningful changesin short-term variability for any of the groups over the 3years of the study.

The average false-positive rates for all three groups inyears 1 to 3 are presented in Figure 2A. Early glaucomapatients had the lowest average false-positive rate of be­tween 2% and 3% for all 3 years of the study. Visuallynormal observers had an average false-positive rate thatdeclined slightly from 4% in year 1 to approximately 3%in year 3. Ocular hypertensive patients had slightly higheraverage false-positive rates of 5% to 7% for the 3 years of

271

Ophthalmology Volume 100, Number 2, February 1993

14

~w 12I-<tII:

W 10>i=<t 8ewzw 6(J)...J<t

4u..wo<t 2II:W><t 0

A

Early GlaucomasOCular HyperlenslvesNorma ls

2 3

Year

pected for a criterion based on the normal 95% confidencelimits .

One of the critical factors associated with monitoringthe visual fields of patients over time is whether they areable to provide reliable test results. If unreliable test resultsare obtained, it is also important to know the likelihoodthat future visual field examinations will be unreliable aswell. In the current study, there were no cases in whichexcessive false-positive errors or excessive false-negativeerrors were obtained for more than I year. All of theseinstances of unreliable test results were thus isolated spo­radic events.

Excessive fixation losses (20% criterion) also weremostly isolated incidents, although there were some in­dividuals who demonstrated repeatedly high fixationlosses. Approximately two thirds of the unreliable visual

32

• Early Glaucomas~ Ocular Hypertensiveso Normals

Year

14>l!~

w 12I-<a:w 10>i=-e 8t'wZ

w 6en..J< 4u..we< 2II:W>< 0

A32

• Early GlaucomasE3 OCular Hypenens lveso Normals

Yea r

z0a: 7wt::II:U

w5>

i=<towz

3w(J)...J<tu..

oZawwUxw;!. B

32

• Early GlaucomasE!3 Ocular Hypertensiveso Normals

zQa: 7wl-e:U

w5>

i=<owz

3wen..J<U.

ozi5wwUxW

>l! B0

Year

Figure 3. Average false-negative rate (A) and percentage of tests exceedingthe false-negative reliability criterion of 33% (B) for normal controls(dott ed bars), ocular hypertensive subjects (hatched bars), and patientswith early glaucomatous visual field loss (solid bars) for the 3 years ofthis study.

Figure 2. Average false-positive rate (A) and percentage of tests exceedingthe false-positive reliability criterion of33% (B) for normal controls (dottedbars), ocular hypertensive subjects (hatched bars), and patients with earlyglaucomatous visual field loss (solid bars) for the 3 years of this study.

The average percentage of fixation losses for normalobserver, ocular hypertensive, and early glaucoma groupsis presented in Figure 4A for all 3 years. All three groupsdemonstrated a reduction in fixation losses from year 1to years 2 and 3. The normal observer group had an av­erage percentage of fixation losses of approximately 11%for year 1, which decreased to approximately 6% for years2 and 3. The patient groups (ocular hypertensive and earlyglaucoma) had average fixation losses of 13% to 15% foryear 1, which decreased to approximately 10% for years2 and 3. The percentage of tests exceeding the 20% fixationloss criterion is shown in Figure 4B for the three groups.If the fixation loss criterion is changed from 20% to 33%,as advocated in a previous investigation.i then the per­centage of tests with excessive fixation losses is between1% and 8% for all groups over all 3 years (see Fig 4C).These values are more consistent with what might be ex-

272

32o

321o

32o

Johnson and Nelson-Quigg . Response Properties during Automated Perimetry

20 25 25

• Early Glaucomas • Early Glaucomas • Early Glaucomas

EI Ocular Hypertens ives EI Ocular Hypertensives a Ocular Hypertensive s

0 NormalsZ

0 Z0 Normals 0 0 Normals

~ ll: 20 a: 20t.. 15 w ww 1= l-

I- ll: a:et U UIt CIl CIlrtJ CIl 15 CIlrtJ 0 0 150 -I -I.J

Z Zz 10 0 00 i= i=i= <I: <I:et X X)( u:: 10 u:: 10ii: e ew z zC) C CetIt 5 w ww w w> o 5 oet X X 5w w

::l1 ::l10 0

A Year B Year c Year

Figure 4. Average rate of fixation losses (A) , percentage of tests exceeding the fixation loss reliability criterion of 20% (B), and percentage of testsexceeding the revised 33% fixation loss criterion (C) for normal subjects (dotted bars), ocular hypertensive subjects (hatched bars), and patients withearly glaucomatous visual field loss (solid bars) for the 3 years of this study.

fields due to excessive fixation losses were single events,whereas one third of the cases had two or three successiveunreliable visual fields due to high fixation losses. For thevisually normal observer group , 5.2% had excessive fix­ation lossesfor 2 or more years, and 1.0%had high fixationlosses for all 3 years. For the ocular hypertensive group,7.8% had excessive fixation losses for 2 or more years,and 3.1% were beyond the 20% fixation loss criterion forall 3 years. For the early glaucoma group, 13.1% had ex­cessive fixation losses for 2 or more years, and 5.3% hadexcessive fixation losses for all 3 years.

If the fixation loss criterion is increased to 33%, theresults are considerably better. Only 1.0% of the normalobserver group exceeds this value for 2 years, and noneexceed it for all 3 years; 4.7% of the ocular hypertensivegroup exceed the 33% fixation loss criterion for 2 years,and 3.1% are beyond it for all 3 years; 5.3% of the earlyglaucoma group exceed the 33% criterion for 2 years, andnone are beyond it for all 3 years.

Discussion

Our results may be summarized as follows: (I) the vastmajority of automated perimetry tests in visually normalsubjects, ocular hypertensive subjects, and glaucoma pa-

dents are reliable; (2) short-term fluctuations (double de­terminations) are relatively constant over several years,although they are moderately higher in patients withglaucomatous visual field loss than in patients with normalvisual fields; (3) excessive false-positive errors are veryinfrequent; (4) excessive false-negative errors are uncom­mon in visually normal and ocular hypertensive subjectsand are moderately higher in glaucoma patients with vi­sual field loss; (5) excessive fixation losses are the primarysource of unreliable automated perimetry results; (6) in­creasing the abnormal fixation loss criterion to 33% pro­duces results that are more in keeping with what wouldbe expected on the basis ofa 5% probability level; and (7)the majority of unreliable visual field test results are iso­lated sporadic events.

Our findings reveal better reliability of automated vi­sual field tests over time than the results previously re­ported by Katz and associates.v" There are several factorsthat may be responsible for these differences. First, thepopulation evaluated by Katz et al was part of a largescreening study for identifying individuals with glaucoma,whereas our subject population was recruited from ourEye Clinic at the University ofCalifornia, Davis, and fromreferrals from community ophthalmologists and patientsfrom the Sacramento Veterans Administration. Second,the inclusion criteria used by the two studies were some-

273

Ophthalmology Volume 100, Number 2, February 1993

what different, especially for the glaucoma patients. Allof the glaucoma patients in our study had early visualfieldloss,whereas the study by Katz et al included patientswith a spectrum of severity of visual field loss. Finally,the testing protocol and training procedure for technicianswas somewhat different between the two studies. The testprocedures, training, and continuous quality control as­sessment of visual fields in our study were nearly identicalto the procedures used in the Optic Neuritis TreatmentTrial. 14 These procedures were developed in our labora­tory, and include a training and certification procedure,a standardized testing protocol, ongoing quality controlassessment of each visual field test performed, and con­tinuous feedback to technicians regarding performance.The results of the Optic Neuritis Treatment Trial (pre­sented as posters at the 1990 and 1991 Annual Meetingsof the Association for Research in Vision and Ophthal­mology, Sarasota, Florida) suggest that these proceduresare effective in maintaining reliable visual field results.Furthermore, Sanabria et al? have directly demonstrateda significant reduction in visual fields with excessive fix­ation losses following a specific technician training regi­men. To summarize, there appear to be several factorsthat contribute to unreliable automated visual field tests,including the patient's cooperation or ability to complywith instructions, the population being tested, the degreeof visual field sensitivity loss, and the extent of technicianinvolvement and training related to administration of theautomated visual field test.

In addition to the differences between the two studies,there are also several common findings. First, nearly allreports of patient reliability during automated perimetryreport an excessive number of patients demonstrating highfixation losses.Although, a recent study indicates that therate of fixation losses can be reduced by proper trainingof technicians,9 there are still a greater number of patientswith high fixation losses than expected (based on a 5%probability level) even after a concerted training program.When the fixation loss criterion is changed from 20% to33%, the results are more in accord with an expected 5%frequency of unreliable visual fields. Secondly, there isgeneral agreement that false-negative errors becomegreater with increasing visual field loss. Finally, all studiesreport a low percentage of excessive false-positive errors.

The resultsof the present study indicate that automatedperimetry can provide a reliable means of following pa­tients over time, provided that technicians are properlytrained, standardized test protocols are employed, andtechnicians are given continuing feedback concerningtheir performance in administering the test. However, newmethods need to be developed for monitoring patient re­liability during automated perimetry testing. Currently,excessivefixation losses mayor may not accurately reflectthe patient's viewing behavior during the test, and highfalse-negative rates in a patient with visual field loss may

274

or may not reflect lapses of attention. Clearly, the task ofseparating response reliability characteristics from pa­thology-related (i.e., visual field loss) and technician-re­lated influences requires a more complex and sophisti­cated approach for automated perimetry.

References

1. Haley MJ, ed. The Field Analyzer Primer. San Leandro,CA: Allergan Humphrey, 1986.

2. STATPAK User's Guide. San Leandro, CA: AllerganHumphrey, 1986.

3. Heij1 A. The Humphrey Field Analyzer, construction andconcepts. In: Heij1A, Greve EL, eds. Sixth Int'I Visual FieldSymposium. Dordrecht: Dr W. Junk, 1985; 77-84. (DocOphthalmol Proc Ser; 42).

4. Heijl A, Lindgren G, Olsson J. Reliability parameters incomputerized perimetry. In: Greve EL, Heijl A, eds. SeventhInt'I Visual Field Symposium. Dordrecht: Martinus Nijhoff/Dr W. Junk, 1987; 593-600. (Doc Ophthalmo1 Proc Ser;49).

5. Nelson-Quigg JM, Twelker JD, Johnson CA. Responseproperties of normal observers and patients during auto­mated perimetry. Arch Ophthalmol 1989;107:1612-15.

6. Katz J, Sommer A. Reliability indexes of automated peri­metric tests. Arch Ophthalmol 1988;106:1252-4.

7. Bickler-Bluth M, Trick GL, Kolker AE, Cooper DG.Assessing the utility of reliability indices for automatedvisual fields. Testing ocular hypertensives. Ophthalmology1989;96:616-19.

8. Katz J, Sommer A, Witt K. Reliability ofvisual field resultsover repeated testing. Ophthalmology 1991;98:70-5.

9. Sanabria 0, Feuer WJ, Anderson DR. Pseudo-lossof fixationin automated perimetry. Ophthalmology 1991;98:76-8.

10. Hardage L, Stamper RL. Reliability indices for automatedvisual fields [letter]. Ophthalmology 1989;96:1810.

11. Johnson CA, Adams AJ, Lewis RA. Automated perimetryofshort-wavelength-sensitive mechanisms in glaucoma andocular hypertension. Preliminary findings. In: Heijl A, ed.Perimetry Update 1988/89: Proc VIII Int'! Perimetric So­ciety Meeting. Berkeley: Kugler & Ghedini, 1989;31-7.

12. Adams AJ, Johnson CA, Lewis RA. S cone pathway sen­sitivity loss in ocular hypertension and early glaucoma hasnerve fiber bundle pattern. In: Drum B, Moreland JD, SerraA, eds. Colour Vision Deficiencies X: Proc Tenth Sympo­sium of the Int'I Research Group on Colour Deficien­cies. Dordrecht: Kluwer Academic, 1991;535-42. (DocOphthalmol Proc Ser: 54).

13. Johnson CA, Adams AJ, Casson EJ, Nelson-Quigg JM. Canshort wavelength sensitivity losses predict the developmentof glaucomatous visual field defects? In: Optical Society ofAmerica. Noninvasive Assessment ofthe Visual Field: Top­ical Meeting, Feb 1991. Washington, DC: The Society,1991;216-9. (1991 technical digest series; v.1).

14. Keltner JL, Johnson CA, Beck RW, et al. Quality controlfunctions of the Visual Field Reading Center (VFRC) forthe Optic Neuritis Treatment Trial (ONTI). Controlled ClinTrials (In press).