arX

iv:1

005.

2825

v1 [

astr

o-ph

.IM

] 1

7 M

ay 2

010

EZ: A TOOL FOR AUTOMATIC REDSHIFT MEASUREMENT

B.Garilli, M.Fumana, P.Franzetti, L.Paioro, M.ScodeggioIASF-Milano, INAF, Via Bassini 15, I-20133, Milano, Italy

O. Le FevreLaboratoire d’Astrophysique de Marseille, UMR 6110 CNRS-Universite de Provence, 38 rue Frederic

Joliot-Curie, F-13388 Marseille Cedex 13, France

S.PaltaniISDC, Geneva Observatory, University of Geneva, ch. d’ Ecogia 16, CH-1290 Versoix, Switzerland

R.ScaramellaINAF, Osservatorio Astronomico di Roma, via Frascati 33, 0040 Monte Porzio Catone (RM), Italy

ABSTRACT

We present EZ (Easy redshift), a tool we have developed within the VVDS project to helpin redshift measurement from otpical spectra. EZ has been designed with large spectroscopicsurveys in mind, and in its development particular care has been given to the reliability of theresults obtained in an automatic and unsupervised mode. Nevertheless, the possibility of runningit interactively has been preserved, and a graphical user interface for results inspection has beendesigned. EZ has been successfully used within the VVDS project, as well as the zCosmos one.In this paper we describe its architecture and the algorithms used, and evaluate its performancesboth on simulated and real data. EZ is an open source program, freely downloadable fromhttp://cosmos.iasf-milano.inaf.it/pandora.

Subject headings: Data Analysis and Techniques, Galaxies, Astronomical Techniques

1. Introduction

Thanks to larger telescopes and more pow-erful instruments, during the last decade wehave witnessed an explosion in the size of spec-troscopic surveys both in the nearby and inthe more distant Universe: from the hundredsof objects of the surveys carried out in theearly nineties (ESP, Vettolani et al. (1997),CFRS,Le Fevre et al. (1995)) to the thousands of galax-ies of present days deep spectroscopic programs(zCosmos, Lilly et al. (2007), VVDS, Le Fevre et al.(2005); Garilli et al. (2008) , DEEP2, Coil et al.(2004)) and to the hundred thousands of morenearby surveys (e.g. 2dF, Colless et al. (2001),SDSS, Abazajian et al. (2009), 6dF, Jones et al.

(2009)) or to the hundred thousand or millionobjects of the just started programs (VIPERS,Guzzo (2009), WiggleZ, Drinkwater et al. (2009),BOSS, Schlegel et al. (2007)) and the few mil-lions of galaxies of the future (e.g. EUCLID,Cimatti et al. (2009)).Such an evolution in survey size requires an analo-gous step forward in the way the data are reducedand analyzed, making compulsory the use of reli-able automatic tools. Among the various tasks tobe done when carrying out a redshift survey, red-shift measurement is one of the most demandingin terms of human resources and skills required.Historically, the rvsao (Kurtz & Mink 1998) pack-age within IRAF has been the first widely used

1

program which has been adopted for redshift mea-surement. It makes use of emission lines (emsaotask) and of the continuum shape (xcsao task),on the basis of a correlation of the spectral con-tinuum against galaxy or stellar templates, im-plementing the algorithm originally proposed byTonry & Davis (1979). Although very powerful,it does not foresee to weigh spectral features byan error spectrum. When the VIMOS-VLT DeepSurvey (from here on, VVDS) started, it was im-mediately understood that this could be a majordrawback in using rvsao within that project: VI-MOS Low Resolution Red (LR-Red) spectra, likethose obtained within the VVDS project, are af-fected by heavy fringing redwards of ∼ 8000A. Asa consequence, spectra show features which arepurely due to fringing but which can easily bemistaken with emission lines by any automatism,as well as by humans, unless they are properlyweighted. Furthermore, if both emission and ab-sorption lines are to be used, some scripting isrequired to couple together results from the purecross correlation algorithm and the emission linessearch one. Last, but not least, the IRAF imple-mentation of rvsao did not allow a direct interfacewith the VIPGI package used for VVDS Data re-duction (Scodeggio et al. 2005).For all these reasons, we have decided to developEZ (Easy redshift, pronounced “easy”), a new toolwhich inherits from the rvsao experience and thecross correlation technique, but allows to be easilyplugged within VIPGI and could also be used as aVO-compliant tool via SSAP (Tody et al. 2008) orother communication protocols. From the user’spoint of view, the most important requirementwas to be able to run the program both in a com-pletely unsupervised and in fully interactive mode,with a user friendly graphical interface to inspectthe results obtained using different parameters ortemplates. Furthermore, it must be possible touse templates which do not necessarily cover thefull spectral range for any redshift value. Finally,it has to be easily usable for different projects, i.e.all parameters must be defined either through aproject dependent parameter file or via commandline. EZ has been initially based upon the proto-types KBred (Scaramella 1999, unpublished) andYAZ (Scaramella 2004, VVDS team Internal re-port), with which the first 10000 VVDS spectrawere measured, and which contained the correla-

tion and fitting algorithm described below.In section 2 we illustrate the rationale of EZ. Sec-tion 3 gives a short illustration of the softwarearchitecture and the most important algorithmsused are illustrated in section 4. Section 5 de-scribes in full how a redshift can be measured byEZ and section 6 illustrates how a reliability flagis attached to the best solution proposed. In sec-tions 7 and 8 we evaluate the performances onsimulated and real data respectively. Finally sec-tion 9 shows how much time and efforts can bespared using a tool like EZ when carrying out alarge redshift survey.

2. EZ basic concepts

There can be several approaches to performredshift measurement: from the most simple ones(like emission line finding or cross correlation) tomore sophisticated ones, like bayesian approach orprincipal component analysis. Within EZ, we havechosen to develop a kind of expert system: the coreof EZ is a decisional tree which tries to mimick thedecisional pattern followed by astronomers whenmeasuring redshifts from spectra. When an as-tronomer manually measures a redshift, his brainalmost unnoticeably performs several functions:he looks for the existence of emission lines, andif there are some, looks whether they match on asingle redshift solution; he looks at the shape ofthe continuum, to find out whether the object isan early type galaxy, and in that case searches fora D4000 break and Ca H and K absorption lines;discards parts of the spectrum affected by heavynoise and spurious features, like zero orders or skysubtraction residuals; possibly, looks at the raw 2-dimensional spectrum to decide whether emissionfeatures are real lines, or sky subtraction residu-als. The basic idea we have tried to implementin EZ is to allow the user to combine the avail-able functions in the most appropriate way forthe data at hand, thus building new user definedfunctions and methods. At the upmost level, aredshift measurement decisional tree can be built,which mimicks the decisional path followed by anastronomer to get to the measure of the redshift.Such decisional path can differ from data set todata set, and for this reason several decisionaltrees can coexist in EZ, exactly as several imple-mentations (methods) of the same function cancoexist.

2

EZ implements several functions to perform thedifferent tasks required: read data files and tem-plates, subtract the continuum from a spectrum,find emission lines, measure the redshift, measurespectral features. Each of these functions, in turn,supports different methods, i.e. can use differentalgorithms to carry out that particular task: forexample, fitting can use a chi-square minimizationor some more sophisticated algorithm, correlationcan be performed weighting data by a noise spec-trum or without any weight, data can be readfrom fits or ascii files.Before coming to a detailed description, it isworth mentioning the importance that spectraltemplates have during the redshift measurementprocess. Whenever the solution relies on correla-tion or fitting methods (i.e. is not based solely onemission lines matching), the reference templatesused in the process should reproduce as much aspossible the spectra to be analyzed. It is not al-ways straightforward to have at hands templatesfor all possible galaxy types and covering a wave-length range as large as to cover from the UVto the NIR rest frame. Nor the same template li-brary is necessarily the best suited for all projects.EZ does not provide templates itself, but allowsthe user to have different template suites and usethose he thinks are more suitable for that particu-lar data set. Furthermore, it is possible to specifya minimum range of overlap between the templateand the spectrum to be used. This feature easesa bit the problem of providing templates coveringan extremely large wavelength range when dealingwith data sets sampling galaxies from the nearbyuniverse up to z > 4.

3. Software architecture

EZ is implemented in Python in the form of acommand-line interpreter. It consists of a mainPython class which imports other classes dedi-cated to redshift estimation, service functional-ities, file I/O, line finding, line matching, andothers. A complete description of the EZ classesis provided in the downloadable user’s manual. Ingeneral, only the higher level methods are used(e.g. the decisional tree) from within the EZ en-vironment, but experienced users can directly usethe Python shell to import the various classes andcombine lower level methods in different ways,

thus exploiting at full the flexibility of the pack-age. Even at the higher level, several sessions (i.e.instances of the same classes) can coexist, andusers can directly compare the results obtainedusing different methods or functions.

Algorithms are implemented in Python, or in Cfor the most CPU intensive tasks: all algorithmsimplemented in C are called through a unique Cinterface, so that all of them can easily be usedfrom a Python environment. All the classes arebuilt in such a way that they can be imported asmodules from any other Python code as well asfrom the Python shell. The main class containsmethods that return the best solution obtained aswell as a ”theoretical” spectrum for a given tem-plate and a given redshift normalized to the inputspectrum. Any other Python code can simply im-port the main class and then directly handle theproposed solution according to needs.

While EZ is primarily developed as a command-line tool, a gtk based graphical interface is avail-able to the user. By design, it is ”merely” an in-terface to the command-line interpreter: it showsthe spectrum and the associated noise spectrum,allows superposition of the best fitting solution,lists the other (less probable) solutions and allowsto overplot them onto the spectrum. An exampleof the Graphical User Interface is shown in Fig.1. In this figure we show a low signal to noisespectrum with the purpose of demonstrating EZcapabilities also on low quality data.

4. Algorithms

4.1. Emission line finding

One way to measure a redshift is to look foremission lines on to which to anchor a solution.The basic concept of an emission line search algo-rithm is to look for sharp peaks in the spectrum,as candidate emission lines, and then see whethersuch peaks can be matched with a single redshiftsolution. In presence of fringe patterns whichhighly resemble emission lines, the main problemis to discard those peaks which are very likelyfake features, without loosing true lines. The im-plemented procedure first builds a rough peak listcontaining the position of all pixels showing a flux

3

above a user defined significance threshold. Agaussian is then fitted to each of such positions,and only those peaks for which the width of thefitting gaussian is within some user defined limitsare retained: the minimum and maximum widthof a line depends on the resolution of the spectrum,and as such are configurable. This check discardspeaks due to non or badly cleaned cosmic rays,which are too narrow to be a real line. Amongthe remaining line candidates, a further check ismade whether the real peak flux is within a factorof two from the fitted gaussian peak. This allowsto discard most of the fake peaks due to fringingwhich often have an irregular shape. Dependingon spectral resolution, partially resolved lines canalso be poorly represented by a gaussian. For thisreason such check can be tailored or altogethersuppressed through user parameters. Finally, thesignificance of the peaks is computed, subtract-ing a local continuum and computing the ratiobetween the peak height and the noise weightedlocal continuum: only peaks showing a signifi-cance above a pre defined σ (usually 4 or 5) areretained. As the noise is stronger in fringing af-fected regions, spurious peaks with a gaussianshape get discarded by this check. The lines thusidentified are further subdivided into strong andnormal lines, according to whether the peak valueis higher than strongcut ∗ σ where strongcut is auser defined parameter. Strong lines are treateddifferently within the decisional tree (see section5).Finally, the algorithm tries to match all or someof such peaks to known combinations of emissionlines at different redshifts. The lines to be usedfor the matching are defined in a configurationfile, and can be changed at wish, according to theredshift range or the type of object explored bythe survey.

4.2. Correlation

Before applying the correlation both the spec-trum and the template are continuum subtractedso that what drives the results are the local fea-tures (such as weak emission lines, absorptionlines, spectral breaks). Each available spectraltemplate is redshifted to a given redshift and thecorrelation function is computed. During the com-putation, each pixel can be optionally weighted

by its associated noise. In the noise weighted ap-proach, the correlation function has the form:

c(z) =

∑

j∈Λ(sj−s)(tj−t)

nj

∑

j∈Λ

√∑j∈Λ

(sj−s)2√∑

j∈Λ(tj−t)2

nj

, z ∈ Θ

where sj and nj are the spectrum flux and noiseat pixel j respectively; s is the spectrum meancomputed as

s =∑

j∈Λsj/nj∑

j∈Λ1/nj

tj(z) is the interpolated flux of the template atpixel j, once put at redshift z; t is the templatemean; Λ is the wavelength range in use and Θis the redshift range to explore. Each templatedoes not necessarily cover the full spectral range,in other words for different templates and redshiftsthe correlation function can be computed using adifferent number of points. In order to comparethe results obtained for different templates, thevalue of the correlation function is normalized tothe number of points used (N).

c(z)red =c(z)

N

Once the correlation function is produced for aparticular template, the highest peak is returnedas the best correlation solution for that template.

4.3. Fitting

The fitting of a spectrum against a templateis performed using a non-continuum subtractedspectrum, so that the results are affected also bythe overall shape of the underlying continuum. Ituses a standard least squared metric: each spectraltemplate available is redshifted to a given redshiftand the mean square deviation between spectrumand template is computed as

χ2red =

∑

j∈Λ

(

fj−Atjσj

)2

N, z ∈ Θ

where fj and σj are the spectrum flux and noiseat pixel j respectively; tj(z) is the interpolatedflux of the template at pixel j, once put at red-shift z; Λ is wavelength range in use and Θ isthe redshift range to explore; N is the number of

4

data points used in the computation. Each tem-plate does not necessarily cover the full spectralrange, in other words for different templates andredshifts the χ2 can be computed using a differentnumber of points. In order to compare the resultsobtained for different templates, the χ2 is dividedby the number of data points used The normaliza-tion constant A is computed as:

A(Λ, z) =

∑j∈Λ

fj

σ2j

∑j∈Λ

tj

σ2j

.

The minimum reduced χ2 for each template andredshift range is returned.

4.4. The solve method

When no emission lines have been found in aspectrum, or when the lines found do not point toone single solution, redshift determination can bedone using a correlation, or a fitting procedure,as described above. The solve function within EZis meant to combine cross correlation, fitting orany other elementary method which can lead toa redshift solution. As all other functions, it canhave different methods, i.e. the various elemen-tary algorithms can be combined in different waysaccording to wishes.The current implementation of the solve methodforesees the use of cross correlation first, and afurther fitting step. We have noticed that in theVVDS and zCosmos data sets, the highest peak ofthe correlation function as described above is notalways the best solution, and this depends mainlyon the fact that the noise spectrum does not (neg-atively) weigh enough the spurious features of thefringing. On the other hand, also the simple fit-ting does not give satisfactory results in terms ofpicking up the correct redshift. A sequential usageof the two methods, instead, has proven to be theone giving the best results for our data. First acorrelation of the spectrum against each templateis performed: as described above, this step allowsto properly take into account the local features ofthe spectrum, like spectral breaks, absorption linesand weak emission features. At the end of the cor-relation, for each template the n redshifts (wheren is a user defined parameter) corresponding tothe highest correlation peaks are retained. At thispoint, we are faced with nxm solutions (where mis the number of available templates). To discrim-inate among them, we use the fitting procedure:

as fitting is performed using the spectrum withoutsubtracting the continuum, the overall spectrumshape plays a role into getting a lower reducedχ2, and can help to pin point the correct redshift.Finally, the solution giving the minimum reducedχ2 is chosen.

5. EZ decisional tree

The core of EZ is the “decisional tree”, whichtries to mimick the human decisional process ap-plied during redshift measurement. This is whereone defines the actions to be performed, and inwhich sequence they must be carried out in or-der to measure a redshift. Each logical block ofa decisional tree makes use of the lower level al-gorithms available. Several decisional trees cancoexist in EZ: each one acts differently in differentsituations, and this can be useful to better tunethe measurement process according to the kind ofdata at hand. For example, in a survey wherestars have been a priori removed, it may be usefulnot to check for M stars, thus reducing the degreesof freedom of EZ, and consequently the possibil-ity that it takes a false track. As an example, weillustrate here the decisional tree we have imple-mented for the VVDS and zCosmos surveys. Theinput consists of one or several observed spectrawith a noise spectrum associated to each of them.Following fig 2, the steps performed are as follows

1. Check if the object is an M type star: thecharacteristical wavy shape of the spectra ofthis kind of stars can be recognized by look-ing if around the expected position a verylarge gaussian can be fitted, and if the spec-trum is steadily becoming redder and redderwith increasing wavelength.

2. If such check is positive, in order to assigna best-fitting template to the object a fit ismade using only M-star templates, and theredshift is set to zero.

3. If the M star check has failed, search foremission lines in the spectrum; if lines arefound go to step 4, otherwise go to step 8

4. Search for a match between lines. If one ormore matches with at least 2 lines have beenfound go to step 5, otherwise go to step 6.

5

5. Fit each of the possible solutions given bythe matches above with emission lines tem-plates. Choose the solution giving the mini-mum reduced χ2.

6. Check if strong lines have been found, (see4.1). If this is the case go to step 7, otherwisego to step 8.

7. Using only the redshifts satisfying matcheswith the one strong line, and only the emis-sion line templates, compute the redshiftusing the algorithm defined by the solvemethod (4.4)

8. Using all selected templates and the wholeredshift range indicated by the user, com-pute the redshift using the algorithm definedby the solve method (4.4)

6. Redshift reliability

Complete automation of the redshift measure-ment process can be tricky when spectra are noisy(as they always are at the faint limit of a survey)or in presence of artifacts such as fringing correc-tion residuals, so that it is by no means guaran-teed, a priori, that the best solution proposed byEZ is also a correct solution. For this reason wehave added the computation of an integer relia-bility flag which summarizes the goodness of thesolution proposed. Such a flag is computed mim-icking the kind of logical reasoning applied by anastronomer when trying to evaluate if a redshift isreliable or not. The flow is as follows

• each template is described by a “reliability”file. An illustration of such file for an Ellip-tical galaxy and for a StarBurst galaxy, asused with the VVDS and zCosmos projects,is given in Table 1: in the first column, thefeatures which are tested are given, togetherwith their wavelength (column 2). Each fea-ture has a weight associated (column 5), ac-cording to the prominence the feature usu-ally has in standard spectra. For example,in elliptical galaxies the D4000 break is themost prominent feature, thus its weight ishigher than all other lines. On the otherhand, if the feature is not found, then theredshift becomes suspect, thus a negativeweight is associated in this case (column 6).

Some features are commonly found togetherwith other features (e.g., the [OIII] doublet,or the D4000 which usually comes with CaH and Ca K absorption lines). The thirdcolumn lists such secondary features, if ap-plicable. If these features are found togetherwith the main line, then the weight of themain line is increased to the value listed incolumn 4. Note that what is important is notthe absolute value of the weight itself, but itsrelative value with respect to the other ex-pected fetaures. The reliability files shouldbe created according to user’s needs and tothe templates used.

• the “best fitting” template is redshifted tothe measured “best redshift”, and a correla-tion around each expected spectral featureis performed, taking into account the noisespectrum. The correlation value must beabove a user defined threshold for the featurebeing considered as “found”. In this step,observational constraints are taken into ac-count: e.g. if one of the expected lines fallsoutside the observed spectrum (or too closeto the border) it is ignored.

• if the “best solution” is a star also the“color” is computed, as the difference be-tween the mean value in the bluer and redderpart of the spectrum, and used as a “feature”(blue color for earlier star types). If the startype is M, a dedicated algorithm searchesfor the characteristic “wavy” shape and usesthem as features found (or not found).

• if a feature has been found, then its cor-relation value is weighted according to theweight given to that feature in the reliabil-ity file for that template (column 4 or 5 inTable 1). Features listed in the reliabilityfile, but not found, are negatively weighted(column 6 in Table 1).

• the weighted correlation values are summedup, and normalized by the number of lineswhich have been found, giving a rate

• finally, the rate is converted into a flag, ac-cording to the ratio of found features withrespect to expected features and to the rateitself: the higher the rate and the number

6

of features found, the higher the flag. Theconversion is made in such a way that EZreliability flags resemble as much as possi-ble the VVDS reliability flags as defined inLe Fevre et al. (2005) in terms of confidencelevel.

Flags thus computed range from 0 (solution isnot reliable) to 4 (highly reliable solution). In Ta-ble 2 we summarize the confidence level of the dif-ferent flags as defined in the VVDS project, andthe criteria used by EZ to assign each flag.

7. Performances on simulated data sets

For any given spectrum, EZ gives in output thebest redshift and the best fitting template, whilethe output of the reliability process is the flag andthe number of features found. When evaluatingperformances, we should make a distinction be-tween the performances of the redshift measure-ment methods, i.e. the capability of finding thecorrect redshift, and the performances of the reli-ability method, i.e. the capability of assessing thegoodness of the solution.To perform such evaluations, we have carried outtwo different sets of tests: a first set, using simu-lated spectra, and a second set, making use of realspectra from the VVDS and the zCosmos survey.

7.1. Simulated test set

Using simulated spectra to evaluate perfor-mances of an algorithm has the advantage thatthe answer is known beforehand, and thus resultscan be evaluated with no error margin. On theother hand, even if simulations are carried outas carefully as possible, such set can only givean upper limit to the performances, as simulatedspectra never take into account all possible noisesources and defects existing on real data.The procedure we used to simulate spectra beginswith an input catalog of objects: in this case wehave used the Cosmos Mock Catalog (from hereon CMC), described in Jouvel et al. (2009). Inbrief, CMC is a simulated catalog built directlyfrom the observed COSMOS (Scoville et al. 2007)catalog of Capak et al. (2007) and the COSMOSphotometric-redshift catalog (Ilbert et al. 2009).In CMC, a redshift and a spectrophotometric typeare associated to each galaxy of the COSMOS cat-

alog, using a model fitting procedure of the pho-tometric data. The resulting catalog contains amix of galaxy populations which by constructionis representative of a real galaxy survey. Emis-sion line fluxes are also computed and magnitudesin a number of filters are made available. De-tails on the simulation of observed spectra willbe given elsewhere (Franzetti et al., in prepara-tion). To summarize the procedure, a rest framespectrum, as can be obtained from galaxy modellibraries, has been associated to each galaxy typeas provided in the catalog (elliptical, early spiral,late spiral and starburst galaxy). In this step,line broadening due to galaxy velocity dispersionhas been neglected, as at the resolution of theVVDS and zCosmos data this it is irrelevant. Themonodimensional incident spectrum has been ob-tained by redshifting the rest frame spectrum andnormalizing it so as to give the object apparentmagnitude in the chosen selection band, IAB . Theincident spectrum has then been degraded for theVIMOS efficiency curve, as can be obtained fromthe ESO Exposure Time Calculator. In the noisecalculation, we have taken into account the Pois-sonian noise (from both sky and object), the flatfielding accuracy and the electronic noise (for adetailed explanation of the different contributionsee e.g. Newberry (1991)). The sky spectrum usedhas also been derived from ESO VIMOS exposuretime calculator. We have compared our simulatedspectra with those which are obtained from ESOexposure time calculator, once the same exposuretime, galaxy type and apparent magnitude areused, and the results are extremely similar bothin terms of signal to noise as a function of wave-length, and in terms of sky subtracted spectrum.It is important to note that our simulated data setincludes the electronic noise, the poissonian noise,the flat fielding accuracy but does not include theeffect of fringing.We have simulated ∼ 11000 galaxy spectra, in themagnitude range 17.5 ≤ IAB ≤ 22.5 using thesame exposure time as for the VVDS wide sur-vey and the zCosmos bright survey (Garilli et al.(2008) and Lilly et al. (2007)), and the VIMOSLR red grism. CMC does not contain stars, butwe are interested to check EZ performances also ondifferent types of stars, as the selection function ofgalaxies for surveys never completely succeeds inexcluding stars on the basis of photometry. Thus

7

we have simulated 11000 stars, with magnituderanging from 17.5 to 22.5, of different spectraltypes. The Pickles stellar library has been usedfor their model spectra. On this simulated dataset, we have run EZ in totally blind unsupervisedmode and obtained for each object a redshift anda redshift flag. The template set we have used inEZ is the one obtained within the VVDS projectand built from the VVDS data themselves: itcomprises two templates for early type galaxies,one template for Sbc galaxies, one for Scd galaxiesand 3 for starburst galaxies, with different inten-sities of emission lines and line ratios. These tem-plates extend from ∼ 3500A to about ∼ 8000Arest frame and are particularly suited to searchfor a redshift solution within the redshift range0 to 2.0, a range which well matches the redshiftrange of the simulated data given the magnitudecut we have imposed, in spectra covering the ob-served range of the VIMOS LR-Red grism. Themeasured redshift has then been compared to thereal input value, and classified as correct when themeasured value is within 10−3 of the real value,this limit corresponding to the theoretical redshifterror given by the grism resolution.

7.2. Global performances on simulated

spectra

In Table 3 the results of such comparison aresummarized: for galaxies and stars separately, aswell as for the whole sample(which has comparablenumbers of galaxies and stars), we give the successrate (computed as the ratio between correctly re-trieved redshifts and total number of objects), thenumber of correct redshifts, and the total numberof objects to which EZ has assigned the given flag.Results for each object type are splitted per EZredshift flag.Table 3 shows that on simulated data, EZ is ex-ceptionally good at retrieving the correct redshift,the success rate being 97% on the whole sample.In spite of the large number of simulated spectra,very few objects are classified with a reliabilityflag of 1 or 2, so that the average success rate forthese flags has not the same statistical significanceas the other flags.Going deeper in the analysis of results, we notethat stars are practically always recognized assuch, even if the flag associated to their measure-

ment is extremely low in half the occurrences.This is intrinsic to the way we associate flags: fortypes younger than K, stellar spectra are poor inspectral features in the wavelength range explored.At the resolution we used, only NaD and Hα ab-sorption lines are clearly visible. Furthermore,NaD falls very near to a strong sky line, and it isoften not detected due to the higher noise. Thusonly one out of two features is clearly detectable.This explains the frequency of zero flag objects inthis category. However, only 191 galaxies (< 2%)are erroneously mistaken for stars, while amongthe 10762 objects classified as galaxies, only 19were actually stars.

7.3. Dependence on magnitude and red-

shift

We have shown that on the simulated test set,the global success rate is very high (97%). Stillwe expect it to show a trend with object magni-tude, which can be considered a proxy for signalto noise ratio for a fixed exposure time. This isshown in figure 3, bottom panel, where we plot thesuccess rate obtained as a function of magnitude.Black circles show the success rate obtained con-sidering all flags above 1, while red crosses showthe success rate obtained considering very secureredshifts (flags 3 or 4) only. Even using all flagsabove 1 (secure redshifts), it is evident that thefainter objects have an increasingly lower successrate, even if it stays always well above 80%.The top panel of figure 3 shows the success rate asa function of the true redshift of the object. Thedrop in success rate for z > 1.4 for secure redshiftflags is not very surprising: between z=1.4 andz=1.5, both the [OII] line and the D4000 breakstart to fall out of the observed wavelength range,and the redshift can be secured only for very fewgalaxies. In other words, we are entering in theredshift desert regime, were observations in the redvisible part of the spectrum are known to be in-efficient. Possibly more surprising, at first glance,can be the drop in success rate observed abovez=0.9 when only very secure redshifts are consid-ered. This is mainly due to the way we computethe reliability of the redshift: at z=0.9, [OIII]aand Hβ lines progressively fall out of the observedwavelength range, so that the only strong and eas-ily detectable line for late objects types remains

8

[OII]. For this reason, many objects get a flag 9(only one strong line). On the other hand, goinghigher in redshift objects become fainter, and theircontinuum gets noisier: therefore also early typeobjects are more difficult to measure and to get avery secure flag is even harder.

8. Performances on real data sets

We have tested EZ performances on the threedata sets of the VVDS Deep survey (Le Fevre et al.2005), the VVDS Wide survey (Garilli et al. 2008)and the zCosmos survey (Lilly et al. 2007). Thethree data sets have different characteristics, andare complementary to evaluate EZ performances:the VVDS Deep sample is cut to a deeper ap-parent magnitude limit (IAB ≤ 24.0), thus itsredshift distribution extends beyond z=1.5 withsignificant numbers. On the contrary, the VVDSWide is cut at a brighter limit (IAB ≤ 22.5), andhas a stronger star contamination. The zCosmossample, finally, has the same depth as the VVDSWide sample, but it has been observed with ahigher resolution grism and stars have been apriori discarded on the basis of their morpholo-gies and spectral energy distributions: the criteriaused were intentionally quite conservative and asmall fraction of stars is expected in the sample(Lilly et al. 2007). All redshifts for these threesamples have been manually measured by two dif-ferent persons (also using beta versions of EZ ininteractive way), discrepant measures have beenreconciled, and a flag has been assigned to eachredshift, corresponding to a confidence level as de-scribed in table 2, column 2, a long and time con-suming procedure which has requested big effortson the part of the people involved. A posteriori,we have run EZ in blind and unsupervised modeon the data sets, using only objects with a mea-sured redshift between 0 and 2.0. By comparingthe results found by EZ with those published bythe VVDS and zCosmos consortia, we can eval-uate EZ performances on real data. We expectsuch performances to be worse than what ob-tained on simulated data sets, as now all possibleerror sources are present, last but not least thefringing above ∼ 8000A.Table 4 and Figure 4 show the success rate of theEZ blind measurement in the three different sam-ples. As expected, the global performances arenot as exceptional as for the simulated data, rang-

ing from ∼ 68% to ∼ 76% in the best case of thezCosmos data. However, a straight comparison isnot totally fair: as declared in the surveys them-selves and reported in Table 2, measured redshiftsare never claimed to be 100% correct. The lastcolumn of Table 4 reports the weighted successrate: for each EZ flag, the non concordant red-shifts have been weighted in number according tothe humanly given flag, so that discordant objectshaving a human flag 1 count by half, if they havehuman flag 2 or 9 they are weighted 75%, if theyhave human flag 3 and 4 they are weighted 90%and 95% respectively. Using such weights, the suc-cess rate obviously increases,as the uncertainty ofthe humanly measured redshifts is properly takeninto account.Still, the success rate remains lower than what wehad for simulated data, especially in the VVDSsamples. The main reason for this are the fringingfeatures, which if not properly weighted are easilymistaken for real features by any automatic proce-dure. As explained in the previous section, in thesimulated data set we have not included the effectof fringing, and for this reason simulated spec-tra are much less noisy redwards of 8000A andEZ is not fooled by spurious emission features.The noise spectrum associated to real data shouldhelp in weighting considerably less such features,but when fringing features are particularly strong,the noise spectrum, as currently computed, is notenough to completely neutralize their effect onthe emission line finding algorithm. As a test forthis hypothesis, we have run EZ on these samedata sets without using the noise spectrum andthe resulting success rate is considerably lower. Inthe zCosmos sample, thanks to the larger dither-ing width the fringing pattern is better removedfrom the data and the associated noise spectrumbetter allows to take into account any residual.This is one of the reasons why in the zCosmossample the success rate for the most secure EZflags is the same as for simulated data. A secondimportant effect is the presence of the zero orderin the extracted spectra: if this is not removed, itis mistaken for an emission line and the redshiftis obviously wrong. Again, zCosmos data are lessaffected because the multiplexing of those obser-vations is much lower than for the VVDS data.Finally, also the observational conditions have animpact on the data quality and the redshift mea-

9

surement: clouds and bad seeing diminish the S/Nratio, a lower exposure time as well as a too highairmass decrease the signal to noise, bad centeringof the object in the slit increases slit losses. Allthese factors contribute to lower the global red-shift measurement success rate with respect to theidealized case of simulations.Looking deeper at the concordant measurements,it can be noted that the flag assigned by EZ isnot always identical to that assigned by the as-tronomers. This is summarized in Table 5, wherewe compare EZ flag with the human flag for con-cordant measurements. In this table, we use onlymeasurements obtained for galaxies, as we havealready shown that when the object is a star theflagging system we have implemented gives lowreliability values. We have grouped results forflags 2 and 9, because experience has taught thatastronomers’ ideas on the two flags differ fromperson to person: when only one emission lineis clearly visible, some people look at the con-tinuum slope and assign a flag 2, others don’t.Table 5 shows that the similarity of the two flag-ging systems is confined to the highest confidencebin: the vast majority of the most reliable mea-surements by EZ are considered very secure alsoby astronomers. Going to lower EZ flags, the as-tronomer’s flag is usually higher than what judgedby the automatic algorithm. This effect is not un-expected, since the EZ flagging system has beenset up to be as reliable as possible, at the ex-penses of being too pessimistic in many cases. Wemust also consider that the criteria used by EZand by an astronomer are not exactly the same:astronomers only grossly take into account thecorrelation with the continuum slope, while in ab-sence of emission lines this is very important inan automatic tool. Also the lines to be searchedmust be very significant for EZ to take them intoaccount, and the significance is weighted with thecorresponding noise spectrum, while an eye mea-surement is more elastic in judgment, also becauseof the possibility of double-checking 1D with 2Dspectra. Finally, the astronomer’s judgment issubjective and changes not only from person toperson, but also with the tiredness of the personperforming measurements. An automatic evalua-tion, on the contrary, although more pessimistic,is always based on the same criteria.As reported in the papers presenting the surveys

(Le Fevre et al. (2005), Garilli et al. (2008) andLilly et al. (2007)), a redshift has not been mea-sured for all objects. These measurement failuresare indicated in the catalogs with a conventionalhuman flag of zero and no redshift indication, thusthey are not included in table 4. Nevertheless, itis interesting to see which flag EZ gives to thesecritical data. In all the three samples we have con-sidered, more than 80% of the spectra for whichastronomers did not measure a redshift get a flag0 or 1 from EZ, a clear indication of a highly un-reliable solution. In the remaining few cases, EZgets fooled by fake features, indicating that thenoise spectrum is not accurate enough.

9. Application on large extragalactic red-

shift surveys

The original purpose for which EZ has beendeveloped is to ease and speed up the long andpainful phase of measuring redshifts in large sur-veys, reducing human intervention as far as pos-sible while keeping the redshift measurement suc-cess rate acceptable for the scientific purpose ofthe project. On the basis of the results shown inTable 3, and keeping in mind such purpose, we canexplore which would be the limitations of adopt-ing straightforwardly EZ results in measuring red-shifts in a generic redshift survey performed in op-timal conditions, as simulated data are, which ex-plores the redshift range we have considered here,i.e. 0 < z < 2.We define as completeness the number of mea-sured redshifts over the total number of spectra,while purity is the number of correctly recoveredredshifts with respect to the measured ones. Theresults are given in table 6: accepting blindly EZresults for all objects with a measured redshift> 0,irrespective of flag, the survey would be > 98%complete, with < 6% of wrong redshifts. Accord-ing to the degree of completeness which can be sac-rificed to purity, it is possible to retain only galax-ies with very high reliability flag (3 or 4): as fromtable 6, in this case the resulting galaxy samplewould be slightly less than 90% complete, but itspurity would be extremely high (> 95%). If we ap-ply these percentages to next generation surveys,where the number of spectra foreseen is of the or-der of few hundred thousands, the gain (meant asnumber of spectra for which human measurement

10

can be avoided) in using a reliable automated red-shift measurement tool is evident.One may argue that real surveys are not alwayscarried out in optimal conditions, real data areusually more difficult to treat than simulated dataand as such the percentages given in table 6 areonly upper limits. In section 8, we have shownthat EZ is extremely reliable to find the correctsolution for spectra classified as very secure (thesuccess rate being above 90% for EZ flag 3 and4) even on real data, affected by different sourcesof noise. Using these encouraging results, we canextend the previous considerations to more realis-tic sets and estimate what would be the gain, interms of number of spectra which can skip the hu-man check, if we would fully trust the redshifts towhich EZ assigns a flag 3 or 4. This is summarizedin table 7, where for each of the real samples wehave analyzed we give the total number of objectsobserved, the number of objects for which EZ mea-sured a highly reliable redshift, the purity of thishighly reliable sample and the gain meant as thefraction of measurements which could be avoided.Such gain ranges from 40 to more than 50%, andit translates into a factor of almost 2 sparing ofhuman effort and time. Given the difficult natureof the data used for our tests (see the discussionon fringing effects in the previous section) such anumber can be considered as a lower limit to theeffective gain obtainable by using EZ in a largeextragalactic redshift survey.

10. Summary

We have presented, EZ, an automated tool de-voted to redshift measurement. The concept atthe basis of the tool is the decisional tree, i.e. thesequence of operations to be performed to obtain aredshift. Such sequence of operations can be cus-tomized according to the kind of sample at hand:e.g. by discarding any check on stars, or using onlyemission lines without performing the correlationson the continua in the case of spectra obtainedwith an instrument operating in slitless mode.EZ can be used both interactively, with the help ofa graphical user interface, or totally blindly in un-supervised mode. It is developed in Python, withthe bulk of computations performed in C to in-crease its computational speed. Its Python classescan be directly imported in any other Pythonbased program, thus making it fully embeddable

in any application.Within EZ we have developed a method to assigna reliability flag on the measurement obtained, inorder to mimic the kind of reasoning done by as-tronomers when assessing the goodness of the so-lution found. The implemented flagging system,though, is rather conservative.We have tested EZ on VIMOS-like simulated data,and have shown that its performances are excep-tionally good, the redshift measurement successrate being above 95%.We have blindly applied EZ to the VVDS-Deep,VVDS-Wide and zCosmos bright samples, andhave demonstrated how this tool behaves very wellalso on real data. The success rate obtained isaround 70%, and rises above 90% for redshifts clas-sified as very secure by astronomers.Finally, we have shown that the adoption of a simi-lar tool can save from a minimum of 50% to a max-imum of 95% of the redshift measurement load,according to the quality of the data and to the de-gree of completeness and purity of results one iswilling to sacrifice in favor of a fast output.EZ is now being routinely and blindly run as partof the reduction process within the VIPERS sur-vey, while a customized version has been set upto perform simulations of the E-NIS spectrograph(Cimatti et al. 2009).EZ is an open source program, freely downloadablefrom http://cosmos.iasf-milano.inaf.it/pandora

We wish to thank the whole VVDS and zCos-mos collaboration for their testing of the earlierversions of EZ and their helpful feedback. A spe-cial thank to Dario Maccagni, for his invaluablehelp in preparing the manuscript. This work hasbeen fully supported by INAF, through fundingby the Project Department via the InformationSystems unit.

REFERENCES

Abazajian, K. N., Adelman-McCarthy, J. K.,Agueros, M. A., et al. 2009, ApJS, 182, 543

Capak, P., Aussel, H., Ajiki, M., et al. 2007, ApJS,172, 99

Cimatti, A., Robberto, M., Baugh, C., et al. 2009,Experimental Astronomy, 23, 39

11

Coil, A. L., Davis, M., Madgwick, D. S., et al.2004, ApJ, 609, 525

Colless, M., Dalton, G., Maddox, S., et al. 2001,MNRAS, 328, 1039

Drinkwater, M. J., Jurek, R. J., Blake, C., et al.2009, ArXiv e-prints

Garilli, B., Le Fevre, O., Guzzo, L., et al. 2008,A&A, 486, 683

Guzzo, L. e. a. 2009, http://vipers.inaf.it

Ilbert, O., Capak, P., Salvato, M., et al. 2009, ApJ,690, 1236

Jones, D. H., Read, M. A., Saunders, W., et al.2009, MNRAS, 399, 683

Jouvel, S., Kneib, J., Ilbert, O., et al. 2009, A&A,504, 359

Kurtz, M. J. & Mink, D. J. 1998, PASP, 110, 934

Le Fevre, O., Crampton, D., Lilly, S. J., Hammer,F., & Tresse, L. 1995, ApJ, 455, 60

Le Fevre, O., Vettolani, G., Garilli, B., et al. 2005,A&A, 439, 845

Lilly, S. J., Le Fevre, O., Renzini, A., et al. 2007,ApJS, 172, 70

Newberry, M. V. 1991, PASP, 103, 122

Schlegel, D. J., Blanton, M., Eisenstein, D., et al.2007, in Bulletin of the American Astronomi-cal Society, Vol. 38, Bulletin of the AmericanAstronomical Society, 966–+

Scodeggio, M., Franzetti, P., Garilli, B., et al.2005, PASP, 117, 1284

Scoville, N., Aussel, H., Brusa, M., et al. 2007,ApJS, 172, 1

Tody, D., Dolensky, M., & Mc Dowell, J., e. a.2008, Simple Spectra Access Protocol, Version1.04 (IVOA Recommendation)

Tonry, J. & Davis, M. 1979, AJ, 84, 1511

Vettolani, G., Zucca, E., Zamorani, G., et al. 1997,A&A, 325, 954

This 2-column preprint was prepared with the AAS LATEX

macros v5.2.

12

Table 1

Examples of reliability weighting scheme

Feature wavelength secondary weight if weight if secondary weight ifname feature all found not found not found

StarBurst galaxy

[OII] 3727.5 10 -20Hβ 4861.3 Hγ 25 10 -20

[OIII]a 5006.8 [OIII]b 30 10 -20Hα 6562.8 SII 25 10 -20

Elliptical galaxy

D4000 4000 CaH CaK 50 20 -10Gband 4304.4 10 0Hγ 4340.4 10 0MgI 5175.4 10 0

Ca+Fe 5269.0 10 0NaD 5892.5 10 0

Note.—The weighting scheme presented is the one adopted for the VVDS andzCosmos bright surveys

Table 2

Significance of VVDS flags and criteria for EZ flags

Flag VVDS EZ lines rateconfidence

4 >95% Ndet/Nexp > 0.5 high3 90% Ndet/Nexp > 0.5 low2 75% Ndet/Nexp < 0.5 high9 75% Ndet = 1 high1 50% Ndet/Nexp < 0.5 low0 0% Ndet = 0 low

Note.—Nexp and Ndet indicate the num-ber of expected and detected spectral featuresrespectively. See section 6 for a detailed expla-nation.

13

Table 3

EZ performances on simulated data

EZ flag SR Ncorrect Ntotal

Galaxies

any 92% 10092 109343-4 95% 9307 97092 35% 14 409 98% 708 7171 29% 43 1440 6% 20 324

Stars

any 99% 8103 81223-4 99% 4320 43212 99% 129 1301 100% 3 30 99% 3651 3668

Galaxies and stars

any 97% 18196 219553-4 97% 13627 140302 91% 143 1709 98% 708 7171 42% 46 1470 93% 3671 3992

Note.—The success rate (SR) is de-fined as the percentage of simulatedspectra (Ntotal) that are assigned a cor-rect redshift (Ncorrect). These are tab-ulated as a function of the EZ flag de-scribed in the text.

14

Table 4

EZ performances on real data

EZ flag SR Ncorrect Ntotal SR weighted

VVDS Deep

any 68% 5833 8514 76%3-4 91% 3845 4191 94%2 51% 220 427 61%9 75% 730 962 82%1 48% 372 762 58%0 30% 666 2172 40%

VVDS Wide

any 70% 12242 17436 78%3-4 89% 9091 10126 93%2 62% 810 1288 72%9 51% 263 509 59%1 50% 698 1381 56%0 33% 1381 4132 43%

zCosmos Bright

any 76% 6403 8404 80%3-4 97% 3735 3822 98%2 87% 445 510 90%9 57% 194 336 62%1 53% 677 1260 59%0 54% 1352 2476 62%

15

Table 5

EZ and humanly assigned flags comparison

human flagEZ flag 3-4 2-9 1

VVDS Deep

3-4 84 % 15 % 1 %2-9 42 % 54 % 4 %1 55 % 41 % 4 %0 39 % 52 % 9 %

VVDS Wide

3-4 72 % 24 % 3%2-9 26 % 61 % 12 %1 39 % 52 % 9 %0 24 % 56 % 20 %

zCosmos Bright

3-4 89 % 9% 2 %2-9 58 % 36 % 6 %1 74 % 20 % 5 %0 40 % 44 % 15 %

Table 6

Completeness and purity of the simulated galaxy sample

flag Ninput Nmeasured Ncorrect Completeness Purity

any 10934 10762 10092 98.4% 93.8%3-4 10934 9709 9307 88.8% 95.8%

2;3;4;9 10934 10466 10029 95.7% 91.7%

Note.—Completeness is defined as the number of measured red-shifts over the total number of input spectra, purity is defined thefraction of correctly recovered redshifts with respect to the measuredones

16

Table 7

Purity of the real data sample

sample Ntotal EZ flag 3-4 purity gain

VVDS-Deep 9742 3845 91% 40%VVDS Wide 18984 10126 89% 53%

zCosmos Bright 8404 3822 97% 45%

Note.—Gain is defined as the percentage of spectra forwhich human measurement can be spared

17

L���������

�����������

� �������������������

������� ������������������

���

�����

��������������������� ���

��������� ����������������

����

��������������������� ���

��������� ����������������

L��

���

�����

���� �������� ���

���� ������������

� ���������

������� ������������

�����

!�

�"

�"

!�

!�

�"

�"

!�

#���$�

��� �

������������ ������� ���

%��

L��

��

&

'

(

)

*

+

!�,

Fig. 2.— Example of EZ decisional tree

Fig. 3.— EZ success rate as a function of magni-tude (lower panel) and of redshift (upper panel),considering flags 2 and above (black open circles)and only flags 3 and 4 (red crosses). Horizontal er-ror bars indicate the bin width, vertical error barsare poissonian errors

Fig. 4.— EZ success rate as a function of reliabil-ity flag. Black dots for VVDS Deep sample, redcrosses for VVDS Wide sample , green squares forzCosmos bright sample

18

Fig. 1.— EZ Graphical User Interface. In this figure we show a low signal to noise spectrum with the purposeof demonstrating EZ capabilities also on low quality data. Top panel: observed spectrum with best fittingtemplate superimposed (red line). Most important emission or absorption lines are shown as vertical dottedred lines, emission lines found by EZ are marked with vertical blue dashed-dotted line. Middle panel: thenoise spectrum. Bottom panel: the best reduced χ2 found for each template as a function of redshift. Leftpanel: top, pull down menu with the list of currently loaded spectra; middle, the list of available templates,the one currently shown is highlighted; bottom, the redshift of the currently displayed solution (default tothe best solution found) and the corresponding reduced χ2

19