7/28/2019 Field Brady 1997

1/17

Pergamon PII: S0042-6989(97)00181-8Vision Res., Vol. 37, No. 23, pp. 3367-3383, 1997 1997 Elsevier Science Ltd. All rights reservedPrinted in Great Britain0042-6989/97 $17.00 + 0.00

V i s u a l S e n s i t i v i t y , B l u r a n d t h e S o u r c e s o fV a r i a b i l i t y i n t h e A m p l i t u d e S p e c t r a o f N a t u r a lS c e n e sD A V I D J . FI EL D ,* :~ N U A L A B R A D Y ?Received 16 July 1996; in revised form 6 June 1997

A n u m b e r o f r e s e a r c h e r s h a v e s u g g e s te d t h a t i n o r d e r t o u n d e r s t a n d t h e r e s p o n s e p r o p e r t i e s o f c e ll si n t h e v i su a l p a t h w a y , w e m u s t c o n s i d e r t h e s t a ti s ti c a l s t r u c t u r e o f t h e n a t u r a l e n v i r o n m e n t . I n t h i sp a p e r , w e f o c u s o n o n e a s p e c t o f t h a t s t r u c t u r e , n a m e l y , t h e c o r r e l a t i o n a l s t r u c t u r e w h i c h i sd e s c r i b e d b y t h e a m p l i t u d e o r p o w e r s p e c t r a o f n a t u r a l s c en e s . W e p r o p o s e t h a t t h e p r i n c ip l ei n s i g h t o n e g a i n s f r o m c o n s i d e r i n g t h e i m a g e s p e c t r a i s i n u n d e r s t a n d i n g t h e r e l a t i v e s e n s i t i v i t y o fc e ll s t u n e d t o d i f f e r e n t s p a t i a l f r e q u e n c i e s . T h i s s t u d y e m p l o y s a m o d e l i n w h i c h t h e p e a k s e n s i t iv i t yi s c o n s t a n t a s a f u n c t i o n o f f r e q u e n c y w i t h l i n e a r b a n d w i t h i n c r e a s in g ( i. e. , a p p r o x i m a t e l y c o n s t a n ti n o c t a v e s ) . I n s u c h a m o d e l , t h e " r e s p o n s e m a g n i t u d e " ( i. e. , v e c t o r le n g t h ) o f c el ls i n c r e a s e s a s af u n c t i o n o f t h e i r o p t i m a l ( o r c e n t r a l ) s p a ti a l f r e q u e n c y o u t t o a b o u t 2 0 c y c / d eg . T h e r e s u l t is a c o d ei n w h i c h t h e r e s p o n s e t o n a t u r a l s c e n e s , w h o s e a m p l i t u d e s p e c t r a t y p i c a l l y f a l l a s 1/ f , i s r o u g h l yc o n s t a n t o u t t o 2 0 c y c / d e g . A n i m p o r t a n t c o n s i d e r a t i o n i n e v a l u a t i n g t h i s m o d e l o f s e n s i t iv i t y i s t h ef a c t t h a t n a t u r a l s c e n es s h o w c o n s i d e r a b l e v a r i a b i l i ty i n t h e i r a m p l i t u d e s p e c t r a , w i t h i n d i v i d u a ls c e n e s s h o w i n g f a l lo f f s w h i c h a r e o f t e n s t e e p e r o r s h a l l o w e r t h a n 1/ f . U s i ng a n e w m e a s u r e o f i m a g es t r u c t u r e ( th e " r e c t if i e d c o n t r a s t s p e c t r u m " o r " R C S " ) o n a s e t o f c a li b r a t e d n a t u r a l i m a g e s, i t i ss h o w n t h a t a l a r g e p a r t o f t h e v a r i a b i l i t y i n t h e s p e c t r a i s d u e t o d i f f e r e n c e s i n t h e s p a r s e n e s s o f l o c a ls t r u c t u r e a t d i f f e r e n t s c al e s. T h a t i s, a n i m a g e w h i c h i s " i n f o c u s " w i ll h a v e s t r u c t u r e ( e. g ., e d g e s )w h i c h h a s r o u g h l y t h e s a m e m a g n i t u d e a c r o s s s c al e. T h a t i s, th e l o ss o f h i g h f r e q u e n c y e n e r g y i ns o m e i m a g e s is d u e t o t h e r e d u c t i o n o f t h e n u m b e r o f r e g i o n s t h a t c o n t a i n s t r u c t u r e r a t h e r t h a n t h ea m p l i t u d e o f t h a t s t r u c t u r e . A n " i n f o c u s " i m a g e w i ll h a v e s t r u c t u r e ( e. g ., e d g e s ) a c r o s s sc a l e t h a th a v e r o u g h l y e q u a l m a g n i t u d e b u t m a y v a r y i n t h e a r e a c o v e r e d b y s t r u c t u r e . T h e s l o pe o f t h e R C Sw a s f o u n d t o p r o v i d e a r e a s o n a b l e p r e d i c t i o n o f p h y s i c a l b l u r a c r o s s a v a r i e t y o f s c e n e s i n s p i te o ft h e v a r i a b i l i t y i n t h e i r a m p l i t u d e s p e c t r a . I t w a s a l s o f o u n d t o p r o d u c e a g o o d p r e d i c t i o n o fp e r c e i v e d b l u r a s j u d g e d b y h u m a n s u b j ec t s . 1 9 97 E l se v i e r S c i en c e L t dNa tura l scenes Wav e le t Image process ing Redund ancy Vi sua l sys t em Blur

I N T R O D U C T I O NT h e p s y c h o p h y s i c a l a n d n e u r o p h y s i o l o g i c a l m e t h o d s o ft h i s l a s t c e n t u r y h a v e p r o v i d e d u s w i t h m a n y i m p o r t a n ti n s ig h t s i n t o t h e b e h a v i o r a n d f u n c t i o n o f s i n g le c e l l s i nt h e v i s u a l p a t h w a y . H o w e v e r , o v e r th e l a s t f e w y e a r s , an u m b e r o f re s e a r c h e r s h a v e t a k e n t h e p o s i t i o n t h a t ac o m p l e t e u n d e r s ta n d i n g o f v i s u al i n f o r m a t i o n p r o c e s s i n ga l s o r e q u i r e s a b e t t e r u n d e r s t a n d i n g o f t h e n a t u r e o f t h ei n f o r m a t i o n a v a i l a b l e i n o u r n a t u r a l e n v i r o n m e n t ( e . g ,B a r l o w , 1 9 6 1 ; S r i n i v a s a n , L a u g h l i n , & D u b s , 1 9 8 2 ;F i e l d , 1 9 8 7 , 1 9 9 3 , 1 9 9 4 ; A t i c k , 1 9 9 2 ; A t i c k & R e d l i c h ,1 9 9 2 ; R u d e r m a n , 1 9 9 4 ) . T h i s a p p r o a c h c o n s i d e r s t h e*Departmen t of Psychology, Cornell Universit y, Ithaca, NY 14853,U.S.A.tDepartmen t o f Psychology, Manchester University, Manchester, U.K.~:To whom all corresp ondenc e should be add ressed [Fax: +1 607 2558433; Email djf3@cornell.edu].

s t a t is t i c a l s t r u c t u r e o f th e e n v i r o n m e n t ( e . g . , n a t u r a ls c e n e s ) i n r e l a t i o n t o t h e k n o w n p r o p e r t i e s o f s e n s o r yc o d i n g . T h e p r i n c i p l e a s s u m p t i o n o f th i s a p p r o a c h i s t h a tt h e v i s u a l s y s t e m h a s e v o l v e d a n d / o r d e v e l o p e d t op r o d u c e a n e f f i c i e n t r e p r e s e n t a t i o n o f i ts n a t u r a l e n v i r o n -m e n t . C o n s i d e r i n g t h e g r e a t n u m b e r o f s tu d i e s t h at h a v ee x p l o r e d t h e i n f o r m a t i o n p r o c e s s i n g c a p a b i l i t i e s o f t h em a m m a l i a n v i s u a l s y s t e m , i t i s s u rp r i s in g t h a t o n l y a f e wh a v e t a k e n a c l o s e l o o k a t t h e s o r ts o f i n f o r m a t i o n a n dr e d u n d a n c y t h a t i s a v a i l a b l e a r o u n d u s . T h i s m a y b e d u ep a r t l y to t h e b i as t h a t m a n y r e s e a r c h e r s h a v e i n a s s u m i n gt h a t o u r e n v i r o n m e n t i s r e l a t i v e l y r a n d o m a n d d o e s n o tp r o d u c e s t a t i s t i c s t h a t a r e c o n s i s t e n t a c r o s s s c e n e s . T h ew o r k o f t h e l a s t s e v e r a l y e a r s h a s s h o w n t h a t t h is i s f a rf r o m t h e c a s e .

I n c h a r a c t e r i z i n g t h e s t r u c t u r e o f n a t u r a l s c e n e s , am a j o r i t y o f r e s e a r c h e r s h a v e c o n c e n t r a t e d o n t h e r e d u n -d a n c y d e s c r i b e d b y t h e p a i r w i s e c o r r e l a t i o n b e t w e e n

3367

7/28/2019 Field Brady 1997

2/17

3 3 6 8 D . J . F I E L D e t a l .

luminance values at different points in an image. Onesuch measure o f spatial correlation is the autocorrelationfunction (acf), which describes pairwise correlations as afunction of the distance between pixels. If the imagestatistics are "stationary" (i.e., the correlation as a func-tion of distance is the same at all image locations for thepopulation of images) then the acf provides a completedescription of all pairwise correlations, as does the imagepower spectrum, which is given by the Fourier transformof the acf (Field, 1987). A number of studies havedemonstrated that the amplitude spectrum falls withfrequency (f) by a factor of about 1 / f as would beexpected from a scale-invariant environment (hence, thepower spectrum falls by about f- 2) . However, the falloffacross scenes shows significant variability ranging fromapproximately f-0.6 to f-1.6 with averages from differentstudies in the ra ng ef -9 t o f 1.2. (Burton & Moorehead,1987; Field, 1987, Field, 1993; Tolhurst, Tadmor, &Cha t, 1992; Ruderman & Bialek, 1994; van der Schaaf &van Hateren, 1996). However, it should be noted thatCarlson (1978) noticed the falloff with amplitude as afunction of frequency with his images and even Kretzmer(1952) noted that television signals might be efficientlycompressed because of the predictable falloff in thecorrelations in images as a function of distance (Fig. 1).There have been a number of attempts to account forproperties of visual neurons in terms of this form ofredundancy (Srinivisan e t a l . , 1982; Atick & Redlich,1992; van Hateren, 1992; van der Schaaf & van Hateren,1996; Switkes e t a l . , 1978; Hancock, Baddeley, & Smith,1992). However, we argue here that the amplitudespectrum provides us primarily with insights into theoverall sensitivity of visual neurons (Field, 1987). Toaccount for why cells in the early visual pathway have

their particular bandwidths and spatial profiles, it wasproposed that one needs to consider higher-orderstatistics as represented by the phase spectra (Field,1987, 19 93, 1994; Olshausen & Field, 1996). Theargument presented in this work is that the particularparameters found in the primary visual cortex are near tooptimal, if optimal is defined in terms of "sparseness".That is, these properties provide a means o f representingany particular natural scene with a minimal number ofactive neurons (i.e., a minimal description length).However, the subset that is active will change fromimage to image resulting in a low probability that anyparticular cell is active. This sparse representation istheorized to be the most independent representation thatis possible, given a semi-linear representation like thatfound with simple cells. Olshausen & Field (1996) haverecently reported that a neural network designed toincrease statistical independence by finding the mostsparse representation produces "receptive fields" likethose found in primary visual cortex. These receptivefields form even when the spectrum is "whitened" toremove all correlations. The amplitude spectra of theimages is largely irrelevant to whether such a sparserepresentation is possible. In Fourier terms, it is the phasespectra that is relevant to the question of sparseness(Field, 1987, 1994) and therefore it follows that theamplitude spectra of natural scenes provides little insightinto understanding why visual neurons have theirparticular bandwidths and spatial profiles.

If this is the case, then does one gain any insight intovisual coding by understanding the statistical regularityrevealed by the amplitude spectra of natural scenes? Inthe following sections, we propose that the amplitudespectra provide insights into the absolute senstivity of

(A) (B) Am plitude Spectrum