+ All Categories
Home > Documents > Inspection of complex surfaces by means of structured light patterns

Inspection of complex surfaces by means of structured light patterns

Date post: 01-Oct-2016
Category:
Upload: yannick
View: 215 times
Download: 0 times
Share this document with a friend
19
Inspection of complex surfaces by means of structured light patterns Yannick Caulier Department Process Integrated Inspection Systems Fraunhofer Institute for Integrated Circuits IIS, D-91058 Erlangen, Germany [email protected] Abstract: This paper addresses the generalization of a surface inspection methodology developed within an industrial context for the characterization of specular cylindrical surfaces. The principle relies on the interpretation of a stripe pattern, obtained after projecting a structured light onto the surface to be inspected. The main objective of this paper is to apply this technique to a broader range of surface geometries and types, i.e. to free-form rough and free-form specular shapes. One major purpose of this paper is to propose a general free-form stripe image interpretation approach on the basis of a four step procedure: (i) comparison of different feature-based image content description techniques, (ii) determination of optimal feature sub-groups, (iii) fusion of the most appropriate ones, and (iv) selection of the optimal features. The first part of this paper is dedicated to the general problem statement with the definition of different image data sets that correspond to various types of free-form rough and specular shapes recorded with a structured illumination. The second part deals with the definition and optimization of the most appropriate pattern recognition process. It is shown that this approach leads to an increase in the classification rates of more than 2 % between the initial fused set and the selected one. Then, it is demonstrated that with approximately a fourth of the initial features, similar high classification rates of free-form surfaces can be obtained. © 2010 Optical Society of America OCIS codes: (100.2650) Fringe analysis; (150.3040) Industrial inspection; (999.9999) Struc- tured light; (999.9999) Fringe analysis. References and links 1. Aceris-3D, “Fc substrate bump inspection system,” Clark Graham 300, Baie D’Urfe, Qu´ ebec, Canada, (2005). 2. Comet-AG, “Feinfocus fox, high resolution 2d/3d,” Herrengasse 10, 31775 Flamatt, Switzerland, (2005). 3. Solvision, “Precis 3d, wafer bump inspection solution,” 50 De Lauzon, Suite 100, Boucherville, Qu´ ebec, Canada, (2007). 4. Y. Caulier, K. Spinnler, S. Bourennane, and T. Wittenberg, “New structured illumination technique for the in- spection of high reflective surfaces,” EURASIP Journal on Image and Video Processing, 2008, 14 pages, (2007). 5. Y. Caulier, K. Spinnler, T. Wittenberg, and S. Bourennane, “Specific features for the analysis of fringe images,” J. Opt. Eng. 47, (2008). 6. S. Kammel, “Deflektometrische Untersuchung spiegelnd reflektierender Freiformfl¨ achen,” Ph.D. dissertation, University of Karlsruhe (TH), Germany, (2004). 7. S.J. Raudys and A.K. Jain, “Small sample size effects in statistical pattern recognition: Recommendations for practitioners,” IEEE Trans. Pattern. Anal. Mach. Intell. 13, 252–264 (1991). 8. W.B. Li and T.J. Cui and X. Yin and Z.G. Qian and W. Hong, “Fast algorithms for large-scale periodic structures using subentire domain basis functions”, IEEE Trans. Antennas Propag. 53, 1154–1162 (2005). #120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010 (C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6642
Transcript
Page 1: Inspection of complex surfaces by means of structured light patterns

Inspection of complex surfaces by meansof structured light patterns

Yannick CaulierDepartment Process Integrated Inspection Systems

Fraunhofer Institute for Integrated Circuits IIS,D-91058 Erlangen, Germany

[email protected]

Abstract: This paper addresses the generalization of a surface inspectionmethodology developed within an industrial context for the characterizationof specular cylindrical surfaces. The principle relies on the interpretation ofa stripe pattern, obtained after projecting a structured light onto the surfaceto be inspected. The main objective of this paper is to apply this technique toa broader range of surface geometries and types, i.e. to free-form rough andfree-form specular shapes. One major purpose of this paper is to proposea general free-form stripe image interpretation approach on the basis of afour step procedure: (i) comparison of different feature-based image contentdescription techniques, (ii) determination of optimal feature sub-groups,(iii) fusion of the most appropriate ones, and (iv) selection of the optimalfeatures. The first part of this paper is dedicated to the general problemstatement with the definition of different image data sets that correspondto various types of free-form rough and specular shapes recorded witha structured illumination. The second part deals with the definition andoptimization of the most appropriate pattern recognition process. It isshown that this approach leads to an increase in the classification rates ofmore than 2 % between the initial fused set and the selected one. Then, it isdemonstrated that with approximately a fourth of the initial features, similarhigh classification rates of free-form surfaces can be obtained.

© 2010 Optical Society of America

OCIS codes: (100.2650) Fringe analysis; (150.3040) Industrial inspection; (999.9999) Struc-tured light; (999.9999) Fringe analysis.

References and links1. Aceris-3D, “Fc substrate bump inspection system,” Clark Graham 300, Baie D’Urfe, Quebec, Canada, (2005).2. Comet-AG, “Feinfocus fox, high resolution 2d/3d,” Herrengasse 10, 31775 Flamatt, Switzerland, (2005).3. Solvision, “Precis 3d, wafer bump inspection solution,” 50 De Lauzon, Suite 100, Boucherville, Quebec, Canada,

(2007).4. Y. Caulier, K. Spinnler, S. Bourennane, and T. Wittenberg, “New structured illumination technique for the in-

spection of high reflective surfaces,” EURASIP Journal on Image and Video Processing, 2008, 14 pages, (2007).5. Y. Caulier, K. Spinnler, T. Wittenberg, and S. Bourennane, “Specific features for the analysis of fringe images,”

J. Opt. Eng. 47, (2008).6. S. Kammel, “Deflektometrische Untersuchung spiegelnd reflektierender Freiformflachen,” Ph.D. dissertation,

University of Karlsruhe (TH), Germany, (2004).7. S.J. Raudys and A.K. Jain, “Small sample size effects in statistical pattern recognition: Recommendations for

practitioners,” IEEE Trans. Pattern. Anal. Mach. Intell. 13, 252–264 (1991).8. W.B. Li and T.J. Cui and X. Yin and Z.G. Qian and W. Hong, “Fast algorithms for large-scale periodic structures

using subentire domain basis functions”, IEEE Trans. Antennas Propag. 53, 1154–1162 (2005).

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6642

Page 2: Inspection of complex surfaces by means of structured light patterns

9. J.P. Besl and J. Ramesh, “Three-dimensional object recognition,” ACM Comput. Surv. 17, 75–145 (1985).10. Gerd Haulser, “Verfahren und Vorrichtung zur Ermittlung der Form oder der Abbildungseigenschaften von

spiegelnden oder transparenter Objekten,” Patent, (1999).11. A. Williams, “Streifenmuster im spiegelbild,” Inspect Magazine, GIT Verlag GmbH & Co. KG, Darmstadt

(2008).12. P. Marino, M.A. Dominguez, and M. Alonso, “Machine-vision based detection for sheet metal industries,” in The

25th Annual Conf. of the IEEE Industrial Electronics Society (IECON’1999), 3, 1330–1335 (1999).13. I. Reindl, and P. O’Leary., “Instrumentation and measurement method for the inspection of peeled steel rods,” in

IEEE Conf. on Instrumentation and Measurement (IMTC’2007), (2007).14. F. Pernkopf., “3d surface inspection using coupled hmms,” in Proc. of the 17th Int. Conf. on Pattern Recognition

(ICPR’2004), (2004).15. M. Petz, and R. Tutsch, “Optical 3d measurement of reflecting free form surfaces,” (2002).16. G. Delcroix, R. Seulin, B. Laalle, P. Gorria, and F. Merienne., “Study of the imaging conditions and processing

for the aspect control of specular surfaces,” Int. Society for Electronic Imaging 10, 196–202 (2001).17. R. Seulin, F. Merienne, and P. Gorria, “Machine vision system for specular surface inspection: use of simula-

tion process as a tool for design and optimization,” in 5th Int. Conf. on Quality Control by Artificial Vision(QCAV’2001), (2001).

18. S.K. Nayar, A.C. Sanderson, L.E. Weiss, and D.A. Simon, “Specular surface inspection using structured highlightand gaussian images,” IEEE Trans. Rob. Autom. 6, 208–218 (1990).

19. F. Puente Leon, and J. Beyerer, “Active vision and sensor fusion for inspection of metallic surfaces,” in Intel-ligent Robots and Computer Vision XVI: Algorithms, Techniques, Active Vision, and Materials Handling, D.P.Casasent (ed.), Proc. SPIE 3208, 394–405, (1997).

20. R. Woodham, Y. Iwahori, and R. Barman, “Photometric stereo: Lambertian reflectance and light sources withunknown direction and strength,” University of British Columbia, Vancouver, BC, Canada, 1991, (1991).

21. J.S. Weska, “A survey of threshold selection techniques,” Comput. Graph. Image Process. 7, 259–265 (1978).22. Huang Zhi, and Rolf B. Johansson, “Interpretation and classification of fringe patterns,” in 11th Int. Conf. on

Image, Speech and Signal Analysis (IAPR’1992) 3, 105–108 (1992).23. L. Lepisto, J. Rauhamaa, I. Kunttu, and A. Visa, “Fourier-based object description in defect image retrieval,”

Machine Vision Applications 17, 211–218 (2006).24. Cem Unsalan, “Pattern recognition methods for texture analysis case study: Steel surface classification,” Ph.D.

dissertation, University of Hacettepe, Turkey, (1998).25. D.M. Tsai, and T.Y. Huang, “Automated surface inspection for statistical textures,” Image Vision Comput. 21,

307–323 (2003).26. H. S. Soon, K. Qian, and A. Asundi, Fringe 2005: Fault detection from temporal unusualness in fringe patterns.

Stuttgart, Germany, (2005).27. T.M. Cover, and P.E. Hart, “Nearest neighbor pattern classification,” IEEE Trans. Inf. Theory 13, 21–27 (1967).28. R. Gutierrez-Osuna, “Pattern analysis for machine olfaction: A review,” IEEE Sens. J. 2, 189–202 (2002).29. R. Kohavi, “A study of cross-validation and bootstrap for accuracy estimation and model selection,” in IJCAI,

1137–1145 (1995).30. I. H. Witten, and E. Frank, Data mining: Practical machine learning tools and techniques, 2nd ed., ser. The

Morgan Kaufmann series in data management systems. Amsterdam: Morgan Kaufmann/Elsevier, (2008).

1. Introduction

One major goal of computer vision processes is the characterization of industrial (quality con-trol) or medical (diagnosis) objects using automatic surface inspection methods. In other words,this research field tackles the processing of different surfaces to be characterized, by means ofdifferent types of illuminations and/or recording techniques, for the visual enhancement of de-fective surface parts. Common and main requirements of proposed vision methods are theirability to characterize the surface to be inspected and their rapidity in terms of inspection time.Thus, effective lightings, and algorithms, but also efficient handling and software frameworkshave to be involved, in order to address more and more challenging tasks, like the recognitionof different types of surface defects in real-time.

In general, the visual enhancement of a certain type of defective surfaces is directly depen-dent on the lighting and the recording technology. Typically, depth defects related to geometricdeformations of the surface, or textural defects synonymous of different surface roughness haveto be visually enhanced by means of a structural and a diffuse lighting. Different automatic in-spection systems have been recently proposed, as the measurement of electronic devices by the

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6643

Page 3: Inspection of complex surfaces by means of structured light patterns

companies Aceris [1] or Comet [2], or the wafer inspection by the company Solvision [3].With the same objective of increasing different defect types with one system, an alterna-

tive surface inspection procedure has been proposed in [4]. It has been demonstrated that thecharacterization of a projected structured pattern serves the direct surfaces interpretation. Theadapted and textural feature-based content description method of the corresponding stripe im-ages rely on the characterization of the depicted bright/dark structures [5]. The advantages ofsuch a method are manifold, especially in terms of real-time processing, “simple” algorithmicprocedure, and process simplification, enhancement of different defect types in one camerashot.

However, in order to simplify the algorithmic processing of the stripe images, periodical andvertical bright/dark structures have been considered. Hence, the described inspection task in [5]can be applied to the same approaches, i.e. the inspection of specular and cylindrical surfacesby means of an adapted illumination technique, or to further inspection tasks necessitating theinterpretation of similar vertical bright/dark structures. Such results are therefore only applica-ble in case of the characterization of vertical patterns. There are two possible steps toward thegeneralization of the proposed inspection method to further non-cylindrical free-form surfaces.

The first possible alternative could be to adapt the structured illumination to the inspectedsurface shape so that a periodical vertical pattern is depicted in the recording sensor. This ap-proach is difficult or even impossible to implement in case of free-formed surfaces, in particularif these are highly specular which are more difficult to record: the camera does not observe thesurface itself, but the reflection of the light on it. This problem is addressed in detail in [6].The second possible solution could be to consider the characterization of non-vertical and non-periodical bright/dark structures which are produced when a light pattern is projected ontofree-form surfaces. This approach is tackled in this paper.

Hence, one major purpose of this paper is to define and to optimize a free-form stripe pat-tern recognition process, in terms of retrieving the most relevant set of features that accuratelyclassifies the reference image sets. According to Raudys and Jain [7], the main steps defining atypical pattern recognition system are the data collection, the pattern class formations, the char-acteristic feature selections, and the classification algorithm specifications. With the proposedfree-form surface inspection task a successive optimization of these stepwise procedures willbe addressed.

At first, various reference stripe image sets defining the free-form surfaces to be character-ized will be introduced. Each considered set of patterns will be classified in two formations,corresponding to two distortion types. Then, two different stripe feature-based image contentdescription methods will be considered: a method specially adapted for the characterizationof such stripe patterns, and a general textural Fourier-based approach. The optimization of thefeature selection will be addressed by means of specific feature fusion and one optimal fea-ture selection method. Finally, the determination of the optimal pattern recognition process isachieved by means of the classification rate.

Hence, on the basis on the previous experiments related to the characterization of cylindricalspecular surfaces, the purposes of this paper are:

• to generalize a surface inspection method based on stripe illumination, initially definedfor cylindrical specular objects, for the characterization of free-form specular and roughsurfaces,

• to define a new feature selection procedure based on a four steps approach, by (i) com-paring different approaches, (ii) determining optimal sub-groups, (iii) combining those,and (iv) selecting optimal feature sub-sets using known feature selection methods,

• and to apply this approach to the case of non-vertical and non-periodical stripe structure

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6644

Page 4: Inspection of complex surfaces by means of structured light patterns

characterization.

The rest of this paper is organized as follows: The use of structured illumination for surfacequality control is introduced in Sec. 2. The two considered image content description methods,namely the general Fourier and the adapted stripe approaches are presented in Sec. 4. Section 5describes the involved proposed four steps procedure for the determination of optimal featuresubsets in case of free-form bright/dark structure characterization. Finally, a summary is givenin Sec. 6.

2. Structured Light for Surface Characterization

2.1. General approach

The general purpose of structured light is the shape recovery of objects or scenes to be in-spected, see [9]. Concerning object reconstruction principles, the adequate method is relatedto surface reflectivity, i.e. wether the surface is rough (diffuse reflection) or reflective (specularreflection).

An approved method for specular surface inspection is the deflectometric-based approach[10], which is used in many industrial inline inspection processes [11–13]. According to lightsource intensity and surface diffuse reflection proportion, 3D triangulation-based methods weredefined for rough surface reconstruction [14]. However, all cited 3D-shape recovery methodsnecessitate a preliminary recording set-up calibration [6, 15]. Complete 3D reconstruction canbe avoided by adapting the set-up elements (light, surface, sensor) to visually enhance defec-tive geometrical surface parts [16–18]. Depending on the involved lighting, geometrical and/ortextural surface information can be directly recovered: light-sectioning [13], image fusion [19], or photometric stereo [20] for example.

However, all these methods do not address the inline inspection (complex calibration proce-dure) for the simultaneous detection of geometrical and textural defects (non-adapted lighting)in real-time environments (complex handling or recording processes).

2.2. Adapted Approach for Limited Surface Geometries

In addition to previously cited conventional approaches, and as stated in the introduction, anew cylindrical surface interpretation principle, based on the projection of a structured lightpattern, has been defined [4]. This surface inspection task is a 21

2D approach, no 3D depth in-formation is required, as all the relevant information is contained in the image. Indeed, patternscorresponding to a geometrical deformation of the surface can be discriminated by “only” in-terpreting the stripe disturbance degree, their real depth is not retrieved. Figure 1 depicts thepattern arrangement for the three considered classes ΩA, ΩR,3D, and ΩR,2D.

Such regular patterns depicted in Fig. 1 are only a part of the non-vertical and non-periodicalbright/dark stripe structures that would be depicted in case of free-form surfaces, i.e. when thelight cannot be adapted to the surface geometry.

2.3. Adapted Approach for Free-Form Surfaces

The aim of this paper is to apply the inspection principle to a broader range of surface typesand geometries. Figure 2 depicts two examples of free-form surfaces, a rough and a specularone, being illuminated with a “non-adapted” structured light pattern.

Although the recording principle of rough and specular objects are different [6], the camerafocusses on the surface in case of the former and on the lighting screen for the latter, bothdefective 3D depth defects depicted in Fig. 2(b) can be visually enhanced by means of thedepicted bright/dark structures in the images.

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6645

Page 5: Inspection of complex surfaces by means of structured light patterns

ΩA (Acceptable) ΩR,3D (Rejected) ΩR,2D (Rejected)

Geometry Grey level

changes changes

no / small medium / large medium / large

Fig. 1. Six examples taken from the reference initial set of images Φ00 made of 252 patterns.

This set has been used for the qualification of the industrial system for the cylindrical spec-ular surfaces inspection. All the patterns have been classified into distinct classes: Accept-able ΩA, rejected (non-acceptable) ΩR,3D, and ΩR,2D. All other 246 patterns depict similarstructures, i.e. correspond to similar geometry and/or grey level changes/perturbations.

Fig. 2. (a) Surface inspection principle: A camera C records the object surfaces to beinspected S, illuminated by a lighting L. (b) Recordings of one free-form rough surface andone free-form specular surface, both are illuminated by a structured light pattern.

The major difference with the considered images in [4], where regular periodical stripe struc-tures are observed, is that, as both objects have a non-planar surface, and a conventional planarstructured lighting is used, the bright and dark stripes in the images of Fig. 2(b) are neithervertical nor periodical. Rather, their geometries depend on the shape of the inspected objects.

As a consequence, with the purpose of generalizing the inspection principle to free-formrough and specular surfaces, an extensive range of stripe structures have to be considered.Such stripe geometries can be obtained by means of different surface shapes, structured lightpositions or sensor types. Our task is not to enumerate all possible combinations of these com-ponents and to compute the corresponding stripe geometries. This would hardly be possible.Hence, it is preferable to focus our investigations on a restricted and predefined number ofnon-vertical and non-periodical bright/dark stripe deformations.

Figure 3 depicts two examples of bright/dark geometries. Each geometry can be obtainedwhen a rough or a specular surface is inspected. The case of a linear moving object with speed�V recorded with a static line-scan camera is considered. One application example for a sphericalobject is also shown.

For the purpose of clarity, only the case of surfaces recorded by means of line-scan sensors

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6646

Page 6: Inspection of complex surfaces by means of structured light patterns

Fig. 3. Left and middle: Two different examples of stripe deformations arising when (a1)and (b1) free-formed rough or (a2) and (b2) free-formed specular objects are recorded.These two examples show that depending on the surface geometry, similar stripe deforma-tions can be observed for rough or specular objects. These two examples show the inspec-tion of a surface S illuminated by a lighting L and recorded by a line-scan camera C duringits linear movement along V. Right: One possible bright/dark structure example in case ofa round (spherical) object. Upper image shows a sphere with 3 surface portions of differentsizes. Lower images show the corresponding image structures if a light pattern is projected.

has been considered in Fig. 3. However, this does not restrict the application of the proposedsurface inspection principle, as similar patterns can be obtained when matrix cameras are used.

On the basis of these two examples, it can be demonstrated that it is possible to obtain thesame bright/dark structures for specular and rough surfaces.

The underlying assumption is that the disturbances induced by the depicted bright/dark im-age patterns are always distinguishable from the undisturbed pattern. Thus, as stripe structuregeometry and/or gray level is used for the defect localization and characterization, this meansthat in the vicinity of a defective region, the “background” variations, i.e. of the surface geome-try and/or the surface texture, are below a certain level. The illumination is considered as idealand projects a homogeneous bright/dark light structure on all the surfaces to be inspected.

Hence, the same reference stripe patterns can be used for the characterization of rough andspecular surfaces. The considered reference patterns used for retrieving the ideal free-formstripe pattern recognition process, and the influence of such different bright/dark structures onthe classification rate, will be tackled in the next sections.

As it is not possible to consider all possible stripe disturbance induced by object geometry, forthe rest of the paper, the investigations will be “restricted” to the two types of stripe geometricaldeformations depicted in Fig. 3(a) and 3(b), the perspective and the geometrical ones. Indeed,Fig. 3(c) shows an example of bright/dark stripe geometry obtained for a round object. It canbe seen how these disturbances encompass, i.e. can be described by the two considered.

Thus, perspective and geometrical disturbances will serve for the generalization of the pro-posed inspection method based on the direct interpretation of “almost” free-form stripe pat-terns, i.e. non-vertical and non-periodical ones, for the characterization of free-form surfaces.

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6647

Page 7: Inspection of complex surfaces by means of structured light patterns

The terminology “almost” is used, as it is assumed that the complex bright/dark structures to becharacterized, must permit the localization and description of all the defects situated completelyon the surface to be inspected.

The limiting factors for this statement is the bright/dark structures disturbance degree. Figure3(c) clearly shows by means of a concrete example, that the recorded surface size must beadapted to object geometry. Content description with the proposed method will be more robustif it is applied to image (1) than to image (3). Thus, in case of spherical objects, it would bepreferable to increase the number of recordings in order to permit a robust inspection of thewhole object.

2.4. Defining the Reference Image Sets

The primary condition to evaluate the proposed inspection method for the characterization offree-form surfaces, is to define a set of reference stripe image patterns, where number andtype of reference stripe patterns depend on the considered inspection task. For example, thereference Brodatz database is used for the evaluation of various textural analysis approaches,so that it encompasses different textural gray level pattern types, as regular and stochastic ones.A good alternative therefore is to consider the reference stripe patterns that have been involvedfor the qualification of the industrial system [4].

Each stripe pattern, which depicts one type of surface to be characterized, has been recordedby an adapted stripe illumination, producing vertical and periodical stripe structures. The wholeset of reference patterns, named Φ0

0 , is made of 252 elements manually annotated and classi-fied into three distinct classes ΩA, ΩR,3D, and ΩR,2D. These classes correspond to acceptablesurfaces, rejected 3D geometrical, and rejected 2D textural surfaces.

Further pattern structures have therefore to be defined. The easiest and simplest way consistsof using the patterns of Φ0

0 and to “transform” or “adapt” them, so that these can be used forthe characterization of free-form surfaces.

Thus, the stripe-illumination-based free-form surface inspection task will be addressed bymeans of different image sets: The reference initial set Φ0

0 , previously introduced, and eightfurther derived sets. The four sets Φ1

1-Φ41 correspond to the warping of all patterns of Φ0

0 withincreasing projective transformations. The four sets Φ1

2-Φ42 correspond to the warping of all

patterns of Φ00 with increasing cylindrical transformations. Both projective -1- and cylindrical

-2- transformations correspond to the types of stripe pattern geometries depicted in Figs. 3(a1),3(a2) and 3(b1), 3(b2). All sets are made of 252 patterns.

Figure 4 shows 3 of the 9 considered stripe image data sets. Φ00 is the reference set where the

stripe structures are periodical and vertical, Φ41 is the set corresponding to the warped patterns

of set Φ00 with a maximum perspective distortion -1- and Φ4

2 is the set corresponding to thewarped patterns of set Φ4

2 with a maximum cylindrical distortion -2-.Such nine sets of reference patterns can be used to address the proposed inspection task

within a general approach, if following conditions are fulfilled:

• all the defective surfaces to be characterized induce stripe geometrical and textural de-formations that are always distinguishable from the non-defective surfaces,

• the position, the geometry and the period of the light structure allow the enhancementof the whole surface, typical recording set-up problems such as occlusions are not ad-dressed.

The first condition addresses the necessary minimal size and intensity of the defect to bedetected, whereas the second condition is related to the capacity of the illumination to enhanceall the surface to be inspected. In the following, the lighting is considered to be ideal, i.e. pro-duces a homogeneous bright/dark structure, for all possible surface geometries and reflectance.

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6648

Page 8: Inspection of complex surfaces by means of structured light patterns

Fig. 4. Left: Reference patterns for the classification of free-form rough and specular sur-faces. These image patterns correspond to three different surface shapes illuminated witha regular periodical structured illumination: -0- for surfaces inducing no deformations, and-1- and -2- for surfaces inducing perspective and cylindrical distortions. Φ0

0 correspondsto patterns without distortion related to the shape of the object. These patterns have beenmeasured. Φ4

1 and Φ42 corresponds to patterns with a maximal perspective distortion of type

-1- and a maximal cylindrical distortion of type -2-. These patterns have been simulated bytransforming patterns Φ0

0 with perspective and cylindrical distortions. All the patterns havea size of 64× 64 pixel. Right: Bright/dark geometry and reflectance characteristics: Sur-face: Period dL,P and coefficient ρS ; Defect: Size [dD,u ×dD,v] and coefficient ρD.

Defect minimal size and intensity is directly linked to the bright/dark structure deformationsand sensor sensitivity.

In order to be detected, each defect D must be at least as huge as the minimal depictedbright/dark structure period and have a significant reflectance coefficient. Hence, if dD,u anddD,v are the defect width and height, rC,u and rC,u the sensor resolution in u− and v− directions,dL,P the projected light stripe period, ρD and ρS the reflectance coefficients of the defect D andthe neighboring surface S, following equation holds:

dD,u > dL,P ≥ 4.rC,u and dD,v ≥ 2.rC,u

ρD/ρS > α (1)

Factor 2 comes from the Shannon theorem, linking the sampling frequency, r−1C,u and r−1

C,u,

with the signal frequency in u−, (dP/2)−1, and in v−, dv. Factor α depends on the sensorsensitivity, the lighting intensity, and surface reflectance.

Hence, for the further stripe image content description, Eq. (1) must be verified for all theconsidered reference patterns. For the measured Φ0

0 patterns, used for the qualification of the in-dustrial system [4], above conditions are fulfilled, as these correspond to the customer’s require-ments. Concerning the other reference artificial patterns, Φ4

1 and Φ42 , obtained after simulating

a perspective and a cylindrical transformation, the constrain was that all depicted transformeddefects are still characterized by above Eq. (1).

Thus, the necessary assumption in case of real images obtained with a surface inspection sys-tem based on the proposed researches, is that the requirements defined by Eq. (1), are fulfilled.It is therefore assumed, that the considered lighting technique, described in [4], can alwaysbe adapted and applied for the characterization of free-form surfaces by means of almost free-form, i.e. non-vertical and non-periodical bright/dark patterns. Necessary set-up optimizations

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6649

Page 9: Inspection of complex surfaces by means of structured light patterns

for optimal components spatial arrangements are not tackled here.

3. Stripe Image Content Description

The next important step now consists of defining the most appropriate algorithmic proceduresthat best characterize non-vertical and non-periodical stripe patterns, i.e. to search for adequatefeature sets and groups that best describe such bright/dark structures. As stated in the intro-duction, a hierarchical method is proposed to optimize the retrieval of the most appropriatenon-vertical and non-periodical stripe pattern characterization features. The steps are: (i) eval-uation of two different methodologies, (ii) individual evaluation of different feature groups,and determination of the most appropriate feature subsets by means of appropriate feature (iii)fusion and (iv) selection approaches.

Concerning the considered feature families/methodologies, two approaches will be evalu-ated. An adapted one, previously defined in [5], and a general one, based on Fourier analysis,see [21]. The reasons for involving these two procedures are described hereafter.

Concerning the adapted features, a set of 14 stripe features has been evaluated in a previouspaper. It has been shown that these characteristics outperform in terms of classification ratesfive further different types of textural feature families. Moreover, 8 of these 14 features wereadapted from previously defined features describing fringe patterns [22], which are also usedfor non-destructive inspection purposes, based on interferometric approaches. A particularityof such fringe patterns is that they have a more complex geometrical structure, see [5]. Thisis also the case of the depicted bright/dark stripes that will be considered in case of the free-form surface inspection, see Fig. 4. In addition, the classification results in [5] showed that theadapted stripe characterization approach mostly outperforms the general textural methods. It istherefore strongly assumed that such adapted features are particularly suited for the free-formsurface inspection task considered in this paper.

Fourier-based approaches are very attractive methods in case of real-time applications, wherelow computation costs are demanded. [23] proposes a Fourier-based approach for the descrip-tion of industrial surface defects, and demonstrates that such a technique is accurate and com-putationally light. Unsalan [24] uses Fourier-based features for the description of steel surfacesand the Fast Fourier Transform (FFT) to increase the speed of the transformation. [8] et al.propose two efficient algorithms to analyze large scale periodic structures by means of FFT-based methods. Then, the Fourier transform has the property of periodic features description,which makes such an approach very attractive in terms of stripe pattern characterization de-picting periodic or almost periodic structures. Several authors use this property to describeimages depicting periodic structures. Within the field of surface inspection, [25] uses the in-verse Fourier transform to remove the repetitive periodic patterns of statistical features. Qian etal. [26] propose a fault detection method by means of interferometric fringe patterns based ona windowed Fourier transform approach. As such fringes can be considered as non-vertical andnon-periodical stripe structures, it is strongly believed that such an approach will also be suitedfor this paper’s purposes.

All these facts concerning the adapted and the Fourier-based transformation, are strong ar-guments in favor of using adapted stripe features and textural Fourier-based features for thecharacterization of non-vertical and non-periodical stripe images.

3.1. Textural Analysis with Fourier Features

The characterization of the stripe structures by means of the textural Fourier analysis will bebased on the approach of Weska [21]. The author uses the power spectrum P as an imagesignature for the discrimination of different types of image patterns F. P , defined as the squareof the spectral’s magnitude, is a matrix of same size as the matrix F. The major goal of [21]

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6650

Page 10: Inspection of complex surfaces by means of structured light patterns

was to use the particularity of the spectral domain by selecting different frequency subbands,which is equivalent to retaining certain levels of details and directions in the patterns to beanalyzed. Weska considers the radial and the angular spectral distributions, saying that theformer is sensitive to texture coarseness and the latter to texture directionality. He also uses thedistributions corresponding to the principal spectral image axes, the u- and v-directions.

The features are directly computed from amounts of values in the Fourier spectrum for differ-ent spectral regions. [21] defines various radial, directional, horizontal and vertical frequencyregions. The assumption is here that the use of different spectral regions characterizing differentimage frequencies, would be more appropriate for the description of almost free-form patterns,corresponding to spatial frequency variations of bright/dark structures.

3.2. Defining Fourier Feature Groups

The assumption in using different parts of the power spectrum for Fourier-based image contentdescription is that some regions may be more discriminative or representative of certain classesof stripe images.

A major part of the considered bright/dark stripes are characterized by a vertical patternwhose disturbances are synonymous of defective surfaces. Hence, a first hypothesis could bethat filtering out the power spectrum regions which correspond to the vertical stripe pattern,could lead to an increase of the signal (defective surface) to noise (vertical patterns) ratio, andtherefore lead to higher classification rates. Such a filtering could be obtained by consideringfor example only the horizontal or the directional frequency.

The considered disturbed stripe patterns are also characterized by local variations of thepattern, corresponding to high frequency changes (geometric disturbances) or low frequencychanges (grey-level disturbances), see Fig. 1. Thus, the signal (disturbance) to noise (verticalpattern) ratio could be increased using different pass-band radial frequency filters.

Next Fig. 5 illustrates the relation of the frequency distribution and the image contents withthree examples, and shows the mathematical expression of the considered spectral regions.

The images show that disturbances in the spatial domain have typical signatures in the fre-quencies representation. The above figure illustrates three different cases.

As depicted in the left image in Fig. 5, a regular non-disturbed pattern is represented withtwo peaks related to the pattern frequency. If the pattern intensity changes along a certain direc-tion in the spatial domain, its transformed counterpart in the Fourier domain is orthogonal, asdepicted in the middle image in Fig. 5. Then, if pattern geometrical disturbance is characterizedby a local variation of pattern frequency, a broader energy distribution is observed at two peakpositions, see right image in Fig. 5 .

For these three examples, it is highly probable that the directional or the horizontal compo-nents in the frequency domain may be strongly discriminative in terms of stripe pattern char-acterization. Hence, with a generalization purpose, this approach can be applied for all theconsidered stripe disturbances. In case of the stripe pattern analysis, feature vectors integratingdifferent subbands of the frequency domain were taken into consideration. The following fivedifferent feature vectors of lengths Nc will be used:

cFr,θ ,v,u = {cF

r ;cFθ ;cF

v ;cFu}: Nc = 33

cFr : Nc = 8

cFθ : Nc = 10

cFv : Nc = 5

cFu: Nc = 10

(2)

The length of each feature vector depends on the considered frequency regions. The vectorcF,r,θ ,v,u, which considers all possible regions has a maximal length of Nc = 33.

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6651

Page 11: Inspection of complex surfaces by means of structured light patterns

Pr1,r2 = ∑r21≤κ2+λ2<r22

0<κ< Mu2 ;0<λ< Mv

2

| f (κ,λ )|2 Pv1,v2 = ∑0<κ<Mu;v1<λ<v2

| f (κ,λ )|2

Pθ1θ2 = ∑θ1≤tan−1(λ/κ)<θ10<κ< Mu

2 ;0<λ< Mv2

| f (κ,λ )|2 Pu1,u2 = ∑u1<λ<u2;0<κ<Mv

| f (κ,λ )|2

Fig. 5. Image characterization from amounts of values in the Fourier spectrum, accordingto [21]. Four different spectral regions are considered: The radial Pr1,r2 , directional Pθ1θ2,horizontal Pv1,v2 and vertical Pu1,u2 spectral ones.

3.3. Use of Adapted Stripe Features

In [5], 14 features for the characterization of the bright and dark stripes have been introduced.Six features were specially developed for the purpose of vertical bright and dark structure char-acterization, eight features were adapted from fringe features originally defined by Zhi [22] forthe purpose of free-form bright fringes description. In order to propose a homogeneous adaptedapproach, we also applied these eight features to the dark stripes of the considered non-verticaland non-periodical patterns.

The notations and names of the 20 considered stripe features are listed in the table below:

Table 1. Notation and names of the 20 considered adapted features of the stripe featurevector cS. These features characterize the bright and the dark stripe depicted in a pattern.

cS(00), cS(01) Bright and dark stripes horizontal deviationscS(02), cS(03) Bright and dark horizontal minimum distancescS(04), cS(05) Bright and dark horizontal maximum distancescS(06), cS(07) Bright and dark intensitiescS(08), cS(09) Bright and dark tangentscS(10), cS(11) Bright and dark curvaturescS(12), cS(13) Bright and dark lengthscS(14), cS(15) Bright and dark shapescS(16), cS(17) Bright and dark straightnesscS(18), cS(19) Bright and dark number of elements

Hence, a total of 20 adapted features are computed for each stripe pattern F. The algorithmic

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6652

Page 12: Inspection of complex surfaces by means of structured light patterns

procedure to retrieve these feature images is described in [5]. The mathematical expression ofrelated feature vectors of lengths Nc are described in [5].

3.4. Defining Adapted Feature Groups

Within the context of defining optimal adapted feature groups, the above described features canbe classified in two main groups: The 6 first features specially developed for vertical stripe de-scription, and the remaining 14 features defined for fringe structure description. Figure 6 illus-trates these two groups, depicting three “projected” stripe patterns and three “interferometric”fringe patterns examples, and showing the computation of “adapted stripe” feature deviationand “adapted fringe” feature shape.

cm = 1Nw

∑Nwn=1 Om m ∈ {02;08}

O02 = min[dist1−→H ((B)sic);dist2−→H ((B)si

c)]

O08 = tan((B)sic)

(B)sic, central bright pixel element ; Nw, number of (B)si

c in F

Fig. 6. Typical “projected” and “interferometric” bright/dark structures. The three upperimages correspond to ideal “projected” structures, the three lower ones to more complex“interferometric” fringe structures. The first “vertical stripes” feature group was defined forthe characterization of the former, whereas the second “free-form” feature group was de-veloped for description of the latter. For illustration purpose, the equations of one adapted“vertical stripes” “minimum distance” feature c02 and one adapted “free-form stripe” “tan-gent” feature c08 are listed. The results of corresponding operators are written for one cen-tral blue marked pixel (B)si

c. Both are the average results of operators O02 and O08 appliedto all bright stripes central pixel elements (B)si

c of the considered image F.

The major difference between the two feature groups is that for the former group the as-sumption is made that the stripe structures are vertical, i.e. that the result depends on the maindirection of the structures. For the latter group, the feature value is independent of the stripedirection, with the assumption that it is more adapted for the characterization of non-verticaland non-periodical structures.

The following three different feature vectors of lengths Nc will be used:

cS = {cS06;cS

14} Nc = 20cS

06 = cS([00 : 05]): Nc = 06cS

14 = cS([06 : 19]): Nc = 14(3)

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6653

Page 13: Inspection of complex surfaces by means of structured light patterns

The considered feature vector cS encompasses two different feature groups or types, cS06, cS

14,each defined for similar bright/dark image structure characterization tasks. Former group con-sists of vertical bright/dark structures, whereas the latter of more free-form bright/dark struc-tures. It therefore seems appropriate, to fuse these two feature groups into one describing vec-tor, as the considered inspection task consists of describing and characterizing disturbed “pro-jected” stripe patterns, whose disturbance degrees are in between the disturbance degrees of theconsidered bright/dark image groups, see Fig. 4. Thus, we are convinced that such a fused fea-ture vector should lead to optimal image classification rates, as non-vertical and non-periodicalbright/dark structures must be interpreted.

4. Feature Selection and Pattern Classification

This section addresses the involved feature subset selection (FSS) and classification proceduresapplied for the general inspection problem stated in this paper, i.e. the inspection of free-formobjects using structured illumination. As stated in Sec. 2.4 it is assumed that for all consideredfree-form surfaces, it is possible to define an adapted lighting which produces “almost” free-form bright/dark structures, so that the requirements defined by Eq. (1), are fulfilled.

Testing the influence of various feature selection and pattern classification approaches wouldbe beyond the scope of this paper. Previous investigations will therefore be considered.

Concerning the feature-based interpretation of stripe structures, three different rules wereconsidered in [5]: the Naive Bayes, the One-Nearest-Neighbor and the Three-Nearest-Neighbor.Their influence on the classification of vertical periodical bright/dark structures was evaluated.The comparison of these three classificators showed that, in general better classification rateswere obtained using the One-Nearest-Neighbor approach. This is a strong argument in terms ofusing “only” this approach for our purposes. Furthermore, Cover [27] and Guttierez [28] showthat the k-NN method approaches the results of the Naive Bayes classifier in case of a large dataset as we have here.

With the methodology, a 10-fold stratified validation, which is certainly the mostly used ap-proach within the pattern classification community, was addressed in [5]. Various n-fold cross-validation approaches for different values of n have been evaluated and compared with thebootstrap technique by Kohavi [29]. He shows that a stratified 10-fold cross-validation is themore appropriate model in terms of classification accuracy. Moreover, Witten [30], referred thata ten times sampling is the right number of folds to get the best estimation error.

Thus, as our aim is to evaluate the two involved feature families, Fourier and adapted stripefeatures, and not a certain stripe pattern classification, in the following considered feature se-lection will be a 1-NN-wrapper-based procedure, whereas a 1-NN classification rule will becombined with a stratified 10-fold cross-validation for supervised pattern classification.

5. Proposed Four Steps Procedure: Results for Free-Form Surfaces

This section addresses the involved proposed four steps procedure for the determination ofoptimal feature subsets using feature evaluation, grouping, fusing, and selection in case of thegeneral inspection problem stated in this paper, i.e. the inspection of free-form objects usingstructured illumination.

As stated before this evaluation is based on two feature families: 33 Fourier and 20 adaptedstripe features. It has been demonstrated that these two sets of 33 and 20 features can be dividedinto four and two groups, see Eqs. (2) and (6). The evaluation criteria for each feature groupis the rate R of correctly classified non-vertical and non-periodical bright/dark stripe patterns.This rate is expressed in percent.

The optimal feature groups are determined in the first subsection by means of the referenceimage data set Φ0

0 . Then, the second section addresses evaluation of these considered fea-

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6654

Page 14: Inspection of complex surfaces by means of structured light patterns

ture groups in case of problem generalization, i.e. free-form surface quality control. The thirdsection addresses the evaluation of FSS methods by considering the further eight referencedatabases defined for problem generalization to free-form surfaces. Finally, the last and fourthsection is dedicated to the evaluation, in terms of types and number, of the previously selectedfeatures.

Evaluation criterion for all these investigations is the classification rate R , expressed in per-cent, which corresponds to the amount of correctly classified patterns for the 3 consideredclasses, ΩA, ΩR,3D, and ΩR,2D. These 3 distinct pattern classes and the annotated data setΦ0

0 were considered, as these correspond to the qualification requirements of the referenceindustrial system [4].

Thus, changing this predefined pattern distinction and/or the reference data set, would havea direct impact on the results. In case of other applications, the number n of distinct patternclasses, but also the number ni, i = {0, ..,n− 1} of reference pattern for each considered classi, must be determined in accordance. Typical industrial applications for example consider 2-classes problems n = 2 with the same proportion of reference pattern ni

∼= n j∀i �= j,{i, j} ={0, ..,n−1}.

However, in case of the considered inspection task, other different approaches would havebeen possible, such as the consideration of two consecutive 2-classes procedures. It would havebeen possible to first classify all good and all bad patterns, ΩA, {ΩR,3D;ΩR,2D}, and then toclassify all 3D- and 2D-bad ones, ΩR,3D, ΩR,2D.

5.1. Fourier and Adapted Feature Groups Evaluation

Concerning the determination of optimal feature groups, Table 2 shows the classification resultsof image set Φ0

0 by means of vector cS made of the 20 adapted stripe features, vector cFr,θ ,v,u

made of the 33 Fourier features, and the four and two feature group vectors described in Eqs.(2) and (6). Classification rates CP and corresponding root mean square errors Erms are listed.

Table 2. Rates R of correctly classified patterns for image set Φ00 with Fourier’s textural

features and stripe adapted features by means of a 1-NN classifier.

Feature vector CP Erms

cFr,θ ,v,u 87.7 0.28

cFr 79.3 0.36

cFθ 92.4 0.22

cFv 86.5 0.30

cFu 86.5 0.30

cS 89.3 0.26

cS06 72.6 0.42

cS14 86.5 0.30

In Sec. 3.1 the assumption was made that some frequency subbands could be more represen-tative of the stripe patterns to be characterized. This is clearly observable in case of the results

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6655

Page 15: Inspection of complex surfaces by means of structured light patterns

Rates for increasing distortions of types -1- and -2- for feature vectors cFr,θ ,v,u cF

θ and cS.

Fig. 7. The detection rates were computed for different image sets and correspond to in-creasing distortions of type -1- and of type -2-. Left to right values: detection rates forimage set Φ0

0 to image sets Φ41 and Φ4

2 .

listed in Table 2. A high discrepancy in the classification results is observable concerning theFourier-based approach. Best classification rates of 92.4% are obtained when only the 10 direc-tional Fourier features cF

θ are used. With the adapted features, the best rate could be obtainedwhen all the 20 features are used. It is however noticeable that the 14 “free-form” featuresoutperform the 6 “adapted” ones.

These first results show that from the 33 Fourier features and the 20 stripe features, the featuregroup made of the 10 directional Fourier features is particulary relevant in terms of stripe patterncharacterization. Thus, further investigations are dedicated to the characterization of free-formsurfaces using the 33 Fourier, the 10 directional Fourier, and the 20 adapted features sets andgroups.

5.2. Feature Groups Evaluation for Free-Form Surfaces

The previously determined optimal feature groups for the reference patterns are now evaluatedon all considered pattern sets, in order to tackle the inspection of free-form objects. These ninepattern sets were introduced in Sec. 2.4.

Figure 7 shows the classification rates for image distortions of type -1- and of type -2- bymeans of the three feature vectors cF

r,θ ,v,u, cFθ , cS.

For both types of distortions the 20 adapted stripe features lead to higher classification ratesthan when the complete 33 Fourier features are considered. The directional Fourier featuresare more characteristic of certain types of distortions. In case of the results depicted in Fig. 7,stripe distortions of type -2- (cylindrical) are better characterized using the directional Fourierfeatures.

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6656

Page 16: Inspection of complex surfaces by means of structured light patterns

Rates for increasing distortions of type -1- and -2- for feature vectors cFr,θ ,v,u

S, cFθ

S, and1−NNcF

θS.

Fig. 8. The detection rates were computed for different image sets and correspond to in-creasing distortions of type -1- and of type -2-. Left to right values: detection rates forimage set Φ0

0 to image sets Φ41 and Φ4

2 .

The next step now consists of attempting to improve these results in terms of increasing theclassification rates and decreasing the number of necessary features, by fusing and selecting theinvolved features.

5.3. Feature Groups Fusion and Selection for Free-Form Surfaces

This section investigates to what extend an appropriate fusion and selection of the Fourier andthe adapted stripe feature sets can lead to a better quality control of the free-form surfaces. Forthis purpose, three different feature vectors will be considered. cF

r,θ ,v,uS is a vector combining the

33 Fourier and the 20 adapted features, and cFθ

S is a vector made of the 10 directional Fourierand the 20 adapted features. Vector 1−NNcF

θS is made of the selected features of vector cF

θS using

a 1-NN-wrapper-based feature subset selection (FSS) method.Figure 8 shows the classification rates for image distortions of type -1- and of type -2- by

means of these three feature vectors.On the whole, the reported classification rates in Fig. 8 are higher than those depicted in

Fig. 7. Indeed, in the first case, more features or relevant selected features by means of a wrapperapproach are considered.

These results show that fusing optimal feature groups leads to higher classification rates,which are in case of the considered problem, of approximately 2 % (difference between themaximal detection rates of both considered graphics).

Concerning the use of a feature selection method, both graphics of Fig. 8 show that for theconsidered sets of patterns, the considered FSS method does not improve the classification

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6657

Page 17: Inspection of complex surfaces by means of structured light patterns

rates, but leads to similar classification results when the combined 10 directional Fourier and20 adapted stripes are used.

Thus, the last investigation is dedicated to a more detailed depiction of the considered FSSmethod, in order to determine the relevant features.

5.4. Evaluation of Selected Features

The influence of increasing distortions of type -1- and of type -2- in the number and types ofselected features using a wrapper 1-NN approach with a 10-Fold cross-validation are depictedin Tables 3 and 4.

Table 3. Selected features when a wrapper 1-NN approach is used, for increasing distortionof type -1-. The maximum number of times a feature can be selected is 10. The variablesNc,sub on the left give the total number of selected features after the 10 runs. The 10 time,9 time and 8 time selected features are marked with ∗∗∗, ∗∗ and ∗. Results for all relevantfeatures are marked in bold.

Feature set Φ00 Φ4

1 Φ42 Φ4

3 Φ44

Nc,sub 90 90 95 107 108cS(00) 0 0 0 0 0cS(01) 1 1 0 0 3cS(02) 0 0 0 0 0cS(03) 0 0 1 2 3cS(04) 3 3 2 8∗ 2cS(05) 4 4 1 3 3cS(06) 2 2 4 1 0cS(07) 9∗∗ 9∗∗ 3 8∗ 3cS(08) 6 6 6 3 3cS(09) 7 7 4 6 4cS(10) 7 7 6 7 5cS(11) 0 0 3 5 4cS(12) 6 6 3 3 5cS(13) 2 2 6 8∗ 10∗∗∗cS(14) 0 0 1 0 7cS(15) 0 0 3 2 7cS(16) 9∗∗ 9∗∗ 6 6 9∗∗cS(17) 2 2 5 10∗∗∗ 6cS(18) 1 1 5 4 6cS(19) 6 6 8∗ 10∗∗∗ 9∗∗cF

θ (0) 0 0 2 1 0cF

θ (1) 0 0 2 1 1cF

θ (2) 5 5 3 4 1cF

θ (3) 4 4 5 2 3cF

θ (4) 4 4 4 1 3cF

θ (5) 10∗∗∗ 10∗∗∗ 10∗∗∗ 10∗∗∗ 10∗∗∗cF

θ (6) 0 0 0 1 0cF

θ (7) 1 1 2 1 0cF

θ (8) 1 1 0 0 1cF

θ (9) 0 0 0 0 0

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6658

Page 18: Inspection of complex surfaces by means of structured light patterns

Table 4. Selected features when a wrapper 1-NN approach is used, for increasing distortionof type -2-. The maximum number of times a feature can be selected is 10. The variablesNc,sub on the left give the total number of selected features after the 10 runs. The 10 time,9 time and 8 time selected features are marked with ∗∗∗, ∗∗ and ∗. Results for all relevantfeatures are marked in bold.

Feature set Φ00 Φ4

1 Φ42 Φ4

3 Φ44

Nc,sub 90 99 107 116 107cS(00) 0 0 1 0 0cS(01) 1 1 0 1 0cS(02) 0 0 2 0 0cS(03) 0 0 3 2 3cS(04) 3 3 4 5 3cS(05) 4 2 1 3 5cS(06) 2 4 1 5 8∗cS(07) 9∗∗ 8∗ 7 6 5cS(08) 6 6 6 6 6cS(09) 7 5 4 4 2cS(10) 7 4 6 6 1cS(11) 0 0 1 1 0cS(12) 6 6 2 6 1cS(13) 2 5 5 8∗ 4cS(14) 0 4 4 4 3cS(15) 0 5 0 2 6cS(16) 9∗∗ 4 6 3 4cS(17) 2 1 4 5 8∗cS(18) 1 3 0 4 6cS(19) 6 7 7 7 4cF

θ (0) 0 0 0 0 1cF

θ (1) 0 1 3 3 2cF

θ (2) 5 7 4 5 5cF

θ (3) 4 2 5 2 3cF

θ (4) 4 5 9∗∗ 6 6cF

θ (5) 10∗∗∗ 10∗∗∗ 10∗∗∗ 10∗∗∗ 10∗∗∗cF

θ (6) 0 2 4 0 0cF

θ (7) 1 2 2 5 4cF

θ (8) 1 1 5 3 1cF

θ (9) 0 1 1 4 6

An important parameter is the variable Nc,sub, which is the total number of selected featuresafter the 10 runs of the 10-Fold cross-validation. As 10 is the maximum number of times afeature can be selected, Nc,sub/10 is the average measure of feature relevance. For both tables,increasing the distortion of the bright/dark structures, leads to an increase of the necessaryrelevant features.

A general remark for both tables concerns the types and the number of selected features,which are approximately the same. It appears that approximately seven features, i.e. only afourth of the initial 30 ones, are relevant. Most of the selected features are adapted ones, whereasmainly the directional 90o Fourier features have a strong relevance.

It is also noticeable, that feature relevance is related to the bright/dark structure distortiondegree. As an example, in case of both tables, the importance of feature cS

13 is proportional tothe distortion degree, whereas the contrary is observed for feature cS

07.

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6659

Page 19: Inspection of complex surfaces by means of structured light patterns

6. Summary

In this paper, a general structured-illumination-based method for the characterization and inter-pretation of free-form and rough surfaces is proposed. Such a procedure was initially definedfor the inspection of cylindrical specular industrial objects.

Starting from the reference image set defined for the qualification of the industrial process,eight additional image sets, corresponding to various types of free-form rough and specularshapes recorded with a structured illumination could be defined. All these image sets are usedto search for the most appropriate pattern recognition process, in terms of retrieving the mostadequate subset of features.

In order to address such an inspection task within a general approach, two different imagecontent description methods, a Fourier-based approach and an adapted stripe-based technique,were considered. These methods necessitate the computation of a huge amount of 33 and 20features, which signifies high computational costs. Hence, in order to propose a competitivesolution adapted to real-time processes, extensive investigations were done to retrieve only themost relevant features that accurately classify the reference image sets.

A four steps feature evaluation, grouping, fusion, and selection procedure was taken intoconsideration. At first, each feature group is evaluated and compared individually. Then, theinfluence of various feature combination and selection techniques on the detection accuracywas evaluated. Finally, it has been demonstrated that feature grouping leads to an increaseof at least 2 % of the classification rates, and that on average approximately a fourth of theinitial features are relevant for free-form surfaces characterization by means of a structuredillumination.

Acknowledgment

The author would like to thank the Bavarian Research Foundation BFS (BayerischeForschungsstiftung) for its financial support.

#120418 - $15.00 USD Received 25 Nov 2009; revised 25 Jan 2010; accepted 26 Jan 2010; published 16 Mar 2010

(C) 2010 OSA 29 March 2010 / Vol. 18, No. 7 / OPTICS EXPRESS 6660


Recommended