S. Peter Martyr (Rieti, Italy): a study case for 3D color laser scanner (RGB-ITR)

Massimiliano Guarneri1a, Giorgio Fornetti a, Mario Ferri De Collibus a, Massimo Francucci a, Roberto Ricci a, Marcello Nuvoli a

aC.R. ENEA Frascati, Via E. Fermi 45, 00044 Frascati (Rome), Italy

ABSTRACT

Since several years our laboratory in ENEA Frascati Research Center is involved in development of laser scanners for Cultural Heritage investigation problems. Actually the best result obtained in this field by our laboratory is a 3D Red Green Blue Laser scanner, called RGB-ITR: the main feature of this scanner, further then measuring distances (up to 20m with a sub-millimetric resolution), is the ability to capture remotely color information by three calibrated laser sources: this information is collected for each point sampled by the instrument and is not affected by external light sources’ influence. Moreover the ability to acquire color and distance information at the same time and for each point decrease drastically the post-production pipeline of a complete mesh. In this work the results of a complete scan of S. Peter Martyr in Rieti are shown, highlighting the efficiency and robustness of color calibration algorithms introduced for a correct color representation. Keywords: laser scanner, color, remote monitoring, cultural heritage, 3D model, colorimetry, ITR, RGB, S. Peter Martyr, Rieti

1. INTRODUCTION Since several years our labs are involved in development of opto-electronic sensors, mainly for solving Cultural Heritage investigation problems. One of our convictions is that the commercial instruments are able to solve a large variety of problems, but a lot of time they are insufficient to satisfy specific requests of Cultural Heritage environments, like no-invasive diagnosis, very high quality analysis, no time consuming pre- and post-data collection. Another constraint, which cannot be ignored mainly in this field, is the economic factor, which involves not only the effective cost of diagnosis, but also possible money loss due to public interdiction of area under investigation: for this reason the opportunity of executing multiple analysis with the same instrument assumes a central rule. The Red, Green, Blue Imaging Topological Radar (RGB-ITR) scanner was developed in respect of all these constraints, so to guarantee both high-quality and no-invasive analysis, fast and cheap data post-production. Using amplitude modulation technique of three different laser sources, the system is able to acquire five informations for each collected point – two distinct distance measurements (short and long distance), three distinct amplitudes of back-reflected signals by the target (color information). The combination of all of these informations permits to execute remote colorimetry of investigated scene. In this work the results obtained during the most recent RGB-ITR’s campaign are presented. The campaign was organized on invitation of “Scuola Interforze Per La Difesa Nbc”, where the chapel is placed in. The scan had the aim to

1 Email: massimiliano.guarneri@enea.it Phone: +39-06-9400-5553

O3A: Optics for Arts, Architecture, and Archaeology III, edited by Luca Pezzati, Renzo Salimbeni, Proc. of SPIE Vol. 8084, 80840N · © 2011 SPIE · CCC code: 0277-786X/11/$18 · doi: 10.1117/12.888965

Proc. of SPIE Vol. 8084 80840N-1

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms

monitoring the healthiness of the structures and frescos, but also valorizing the place through a 3D model, adapted for multi-disciplinary and educational purposes. The small church build on the half of XIII sec. near Velino river was called S. Peter Martyr in 1576 and assigned at Merchant's Confraternity, who provided to enrich with new frescos [7]. The most important is the Judgement Day painted by Bartolomeo and Lorenzo Torresani: the fresco covers a big part of Oratory, three walls plus the vault, where Judge Christ with Passion's instruments is represented. In the frontal wall the central scene of Judgement Day is represented, in the center the Angels playing trumpets divide Elects from Damneds. With their masterpiece, Torresani brothers demonstrate they know and imitate great painters like Beato Angelico, Signorelli and Michelangelo, representing some elements from Orvieto's Cathedral and Sistina Chappel in Vatican [8].

2. THE RGB-ITR SCANNER 2.1 ITR’s characteristics and functioning principle

The RGB-ITR scanner is essentially composed by two main modules: a so-called passive module, which coincides with the system’s optical head, basically including the transmitting and receiving optics; a so-called active module, which is composed of laser sources, modulators, detectors and all necessary electronics for collecting and processing data. The two modules are physically separated and optically connected by means of optimized optical fibers. This enables the use of the system in hardly accessible or even hostile environments [1]. The Figure 1 shows the entire system assembled as a tower so to guarantee the minimum size and movement facility; the modular configuration permits to adapt the system in the operating environment.

Figure 1: RGB-ITR modules assembled all together on a mobile platform. From bottom, the first two boxes are composed

by lock-in, for laser modulation and signals detection, and a motion controller; inside the next box, laser sources and detectors; on the top, the optical head with the scanning mirror.

The core of electronic system is composed by three lock-in: a lock-in amplifier is designed for extracting a signal with a known carrier wave from an extremely noisy environment. The model used in ITR system is a Stanford SR844: this

Proc. of SPIE Vol. 8084 80840N-2

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms

lock-in has two ports, one output for the modulation of laser source and one input connected to avalanche photodiode (APD) detector. Informations coming out from this instrument are the amplitude of the detected signal, expressed in voltage, and the phase-shift between modulating and reflected signals, expressed in degrees. The main formula for detecting distances by amplitude modulated radar systems are:

d =v ⋅ Δϕ4π fm

v → light speed in the transmitting mediumd → measured distancesΔϕ → phase shift between reference and back-reflected signalsfm → modulation frequency

(1)

σ R ∝1

m ⋅ fm ⋅SNRi

where SNRi =P ⋅η ⋅τh ⋅ f ⋅Γ

SNRi → signal-to-noise ratiofm → modulation frequencym → modulation depthP → collected powerη → detector's quantum efficiencyτ → integration timef → laser optical frequencyΓ → overall optics merit factorh → Planck's constant

(2)

The formula (1) expresses the distance as function of the phase shift, between reference and detected signals, and modulation frequency; the equations (2) represent the accuracy of the instrument as function of signal-to-noise ratio [6]. The AM range finding technique is affected by the folding ambiguity due to phase periodicity. The instrument returns the same measurement for two different points of the scene separated by half of the modulation wavelength. For this reason, two modulation frequency are adopted in ITR system: the high modulation frequency, 190MHz (about 0.78m), is used for high precision distance measurement, while the low frequency, 5MHz (about 30m), permits to solve the ambiguity of the shortest wavelength. Actually, because of technological limits, the high frequency modulation is employed on red channel, while the low frequency on blue channel. Efficient noise rejection is obtained by using narrow field-of-view, interferential filtering and low noise detection electronics. Self-occlusions, as well as off-axis aberrations, are avoided by the monostatic configuration of the launch and receiving optics [10]. 2.2 ITR hardware core

RGB-ITR takes advantage of two optical mixers, one for the launching and one for the receiving of the three laser beams, and of amplitude modulation technique for color and distance detection. Optical mixers (Figure 2) are essentially passive elements, internally composed by mirrors transparent at particular wavelengths and lenses able to focalize laser beams in optical fibers, used for connection with external components. Four fiber optics come out from each optical mixers: the launching one is composed by three input monomodal fibers, connected to the three laser sources, and one output monomodal fiber, connected on the other side to the optical head, from whom a coherent white laser light, obtained by the superimposition of the three beams, comes out; the receiving

Proc. of SPIE Vol. 8084 80840N-3

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms

mixer gets the back-reflected light in the input multimodal fiber optic by a 50.6mm-diameter doublet lens (Newport P086) and then splits it into three multimodal fibers, connected to APD detectors for opto-electronic conversion. This hardware configuration and the amplitude modulation technique can be considered the core of ITR system, like charged couple device (CCD) for digital cameras. Differently by CCDs, the ITR hardware uses an active technique for collecting signals, thus only lasers light is responsible of color detection rejecting any other external sources.

Figure 2: One of the two optical mixers. On the left side, the three laser sources input are visible; on the right side, the

“white” laser beam output.

2.3 ITR software modules

All software is assembled and customized in ENEA and it is composed by two main interfaces: the first one, called ScanSystem, controls all the aspects of scanning; the second one, called ITRAnalyzer, is able to build, analyze and export all ITR data (Figure 3). ScanSystem permits to change a lot of ITR scanner’s parameters, so to guarantee the best solution for the acquisition of the scene under investigation. Another interesting feature of ScanSystem resides in SCRIPT programming by means of the acquisition process can be divided in several sub-scans, permitting a scanning-parameters customization based on scene complexity. In the last period a color calibration section was added at the software. ITRAnalyzer manages all data collected by RGB-ITR system, display color and structure profiles of the investigated target and permits the registration of meshes obtained by different point-of-view scans. This software can be considered the link between RGB-ITR raw data and commercial software for 3D modeling and photo manipulation.

Figure 3: on left, ScanSystem; on the right, ITRAnalyzer

Proc. of SPIE Vol. 8084 80840N-4

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms

3. COLOR AND DISTANCE INFORMATION 3.1 RGB-ITR vs. digital camera color information

As reported in paragraph 2.2, the introduction of optical mixers in ITR systems permits the production of color photos, obtained by three-laser stimulus. The use of active light and lock-in technique for color acquisition removes typical problems of digital cameras, like no-influence by external light sources and white calibration at different distances [2]. Furthermore typical working operation for photo cameras devices is limited to 8-12 bits, so the effective color depth is strongly depending on the distance and the illumination, and it can be only increased with multi-exposition technique and dedicated post-production software. ITR system has a resolution of 16 bits for each channel and the distance information is used for color correction, as explained in the next section. Table 1 shows some other consistent differences between the two technologies.

Table 1: main differences by RGB-ITR scanner and digital cameras

3.2 Color calibration method

In this section the various factors that influence color-measuring features of RGB-ITR are examined. The main effort of color measurement research regards the univocal characterization and reproducibility of colorimetric information. For any sampled surface point, raw color data returned by the RGB-ITR consists of a triplet of voltage values. Each value represents the (amplitude of the oscillating) red, green or blue light power reflected by a particular surface point-like area, as collected by the receiving optics and revealed by the detector. If considering a real surface, neither fully specular nor fully Lambertian, the reflection of light on it can be expressed as the weighted sum of three main terms: specular, Lambertian and retro-reflected radiance. Since the contribution of the retro-reflected component is usually small – apart from a minor category of very special surfaces (retro-reflectors) – only the Lambertian component is considered [5]. Lambertian reflection is isotropic, that is, exiting radiance (I) is constant over the whole hemisphere. At large distances z, the power falling on the receiver is roughly equal to:

RGB-ITR DIGITAL CAMERA No-influence by external light sources Influence by external light sources Possibility to acquire during 24/24h Impossible to acquire without the presence of external lightPossibility to acquire without the presence of supervisors No possibility to acquire without operator White color can be calibrated as function of distance White calibration works in a small range (some meters) No lenses aberrations Lenses aberrations RGB-ITR is a prototype Digital cameras are commercialized since several years Pixel resolution determined by laser spot focalization and motor precision

Pixel resolution determined by CCD technology

It is portable, but require more then one person for transport

It is portable and generally is sufficient one person for transport

Working range: 2.50-30m Working range: depends on mounted optics Working time: several hours up to one day for the maximum resolution and field-of-view 80°x310°

Working time: generally it’s a one-shot capture, with times depending on external light conditions

Proc. of SPIE Vol. 8084 80840N-5

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms

P ≅I ⋅R ⋅S ⋅cosθ

z2 (3)

where R is the area of the receiver and S is the area illuminated by the laser spot on the target, which varies with the

angle θ formed by the receiver axis with the normal to the target surface. Ideally, since S = S0/cosθ, the dependence on θ should cancel out [10]. As shown in (3), the distance influences the color measurement: the received power roughly depends on the inverse of squared distance. Since distance is recorded contextually at color information, a correction of raw color data can be made: actually this correction is made by collecting three calibration curves, obtained by illuminating with the ITR lasers a white target (Spectralon STR-99-020, diffuse reflectance), placed at different distances.

4. RGB-ITR CAMPAIGN RESULTS Figure 4 shows the three curves obtained during an acquisition in S. Peter Martyr (Rieti). At the moment only a qualitative interpretation of the shape can be done, a more complete theoretical model is under study. Considering the three lasers with a good approximation as Gaussians, the maximum of the curves corresponds at the beam waist. It’s also true, looking mainly at the blue curve (Figure 4c), that the bell-shape is obtained by contribution of other factors, like possible misalignments of the launching and receiving optical axes and the combined effect of the receiving optics efficiency with the varying dimension of the spot on the target.

(a) (b)

(c)

Figure 4: Single channel calibration curves, obtained striking a white target with the three lasers. Each curve represents back-reflected signal amplitude (V) function of distance, expressed as phase shift (degrees) between reference and back reflected signals. (a) red, (b) green, (c) blue curves.

These curves can be considered as a sort of ITR’s fingerprint: once optical parameters are fixed, like spot and optical receiving focalization, color information can be univocally represented. The color correction by calibration curves is expressed by the following pseudo-code:

Proc. of SPIE Vol. 8084 80840N-6

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms

Table 2: Pseudo-code for color correction

1. Extrapolation of the three curves by fitting points with Hermite polynomials W = r φ( ), g φ( ), b φ( )( ) 2. For each ITR value

rCi = Ri,Gi, Bi,φi( ) the corresponding white value is selected wi = ri φi( ), gi φi( ), bi φi( )( )

3. The normalized ITR color is obtained by expression: rCi,norm = Ri / ri φi( ),Gi / gi φi( ), Bi / bi φi( ),φi( )

Analyzing data collected in the chapel, the effects of distance on color information are evident (Figure 5). The left side of scene, closest at the ITR sensor then the right one, presents a sort of shadow (Figure 5a). It cannot be considered really a shadow, because in this case there is not a lack of information, rather it is an unweight information. Figure 5b shows the same image corrected by calibration curves. This approach permits to characterize quantitatively the color information and keep trace in time of the pigment modifications.

(a) (b) Figure 5: on the left, RGB-ITR raw data; on the right, RGB-ITR data after color calibration

Figure 6 shows different points of view of the chapel 3D model. The scanner acquired data at a working range of 3-8 meters in three days (H24). The acquisition was divided in several slices, corresponding at one hour of acquisition process, reducing drastically the data loss due to electric power instability, especially during no-supervised period. The model is composed by 12000x23000 pixels corresponding to a spatial resolution of 0.6 mm. Figure 7 shows the high quality of laser texture on a vault’s detail, highlighting the good zoom capability reachable by these images.

Figure 6: Two view of the S. Peter Martyr's chapel 3D-model

Proc. of SPIE Vol. 8084 80840N-7

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms

Figure 7: A detail of the chapel. The colors are captured directly by lasers

5. CONCLUSIONS The RGB-ITR is the first AM 3D laser scanner that enables the remote simultaneous recording of sub-millimetric range and self-registered color (RGB) information. This feature, combined with other characteristics such as non-invasiveness, versatility, transportability and possibility to operate at long distances without compromising performances, make the new system unique in the - quite crowded - panorama of 3D digitizers. The combination of color and distance informations opens new way for diagnosis, cataloguing and educational purposes in Cultural Heritage environment. Moreover the combination of these informations for each point reduces drastically post-production time and costs [4]. A more compact system is under investigation: the main idea is the replacement of the three lock-in with a single electronic system equipped with several input/output ports. The possibility to add other channels for multi-sensorial analysis (i.e. fluorescence) is the next challenging.

REFERENCES

[1] Bartolini, L. Bordone, A. Coletti, A. Ferri De Collibus, M. Fornetti, G. Lupini, S. Neri, C. Riva, M. Semeraro, L. & Talarico C. 2000. “Laser In Vessel Viewing System for Nuclear Fusion Reactors”, International Symposium on Optical Science and Technology, Proc. SPIE 4124: 201-211.

[2] Ferri De Collibus, M. Bartolini, L. Fornetti, G. Francucci, M. Guarneri, M. Nuvoli, M. Paglia, E. Ricci, R. 2007. “Color (RGB) imaging laser radar”, Proceedings of SPIE Vol. 6622, 662201, 662201-1 – 662201-11.

[3] Ferri De Collibus, M. Fornetti, G. Guarneri, M. Paglia, E. Poggi, C. & Ricci, R. 2005. “ITR: an AM laser range finding system for 3D imaging and multi-sensor data integration.”, Proc. of ICST 2005 (International Conference on Sensing Technology, Palmerston North, New Zealand, 21-23 November 2005: 641-646.

[4] Levkowitz, H. 1997. “Color Theory and Modeling For Computer Graphics, Visualization and Multimedia Applications”, Library of Congress Cataloging in Publication Data, Kluwer Academic Publishers.

Proc. of SPIE Vol. 8084 80840N-8

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms

[5] Nicodemus, F. 1965. “Directional reflectance and emissivity of an opaque surface.”, Applied Optics 4 7: 767–775

[6] Nitzan, D. Brain, A. E. Duda, R. O. 1977. “The Measurement and Use of Registered Reflectance and Range Data in Scene Analysis.”, Proc. IEEE 65: 206

[7] Online information on S. Peter Martyr in Rieti: http://www.rietinvetrina.it/index.php?option=com_content&view=article&id=83&Itemid=305.

[8] Online information on S. Peter Martyr in Rieti: http://www.rieti2000.it/r2k/dove/lasabina/7.htm. [9] Poujouly, S. Journet, B. 2002. “A twofold modulation frequency laser range finder”, Journal of Optics, S356-

S363. [10] Ricci, R. De Dominicis, L. Ferri De Collibus, M. Fornetti, G. Guarneri, M. Nuvoli, M. Francucci M. 2009.

“RGB-ITR: an amplitude-modulated 3D colour laser scanner for cultural heritage applications.”, Proceedings of the International Conference LACONA VIII − Lasers in the Conservation of Artworks.

Proc. of SPIE Vol. 8084 80840N-9

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms