Automatic visual inspection based upon a variant of the n-tuple technique M.Ouslim K.M.Curtis Indexing terms: Automatic visual inspection, n-tuple technique, Rapid image analysis technique Abstract: The authors present a new rapid image analysis technique ideally suited for both production line and generalised flaw detection. It is a variant of the n-tuple technique and uses nine selected attributes based on five pixels arranged in a cross shape. The technique requires a flaw free image as well as the image under test. The estimation of the flaw size and its shape is given by recording frequencies of occurrence of the nine attributes by scanning both the corrupted image and the flaw free image. The application of this technique to the inspection of simple flaws in printed circuit boards, proved to be successful in detecting defects as small as one pixel in an image of 512 x 512 pixels with the object occupying as little as 3 x 3. Its application to defect identification within a multigrey level image is also shown to be successful. Its ease of implementation makes this technique a very good candidate for implementation as an embedded part within a front end processor for a real time automatic inspection system. It offers an improvement in speed of analysis, simplicity of operation, versatility in the type of problem it can tackle and cost-effectiveness over recently published methods (Newman and Jan, 1995; and Chin, 1989). 1 Introduction Flaw detection is an important process in both indus- trial and nonindustrial inspection systems. The analysis has to be accurate and fast. It has been shown that such requirements can be achieved by vision based inspection systems. In particular, one flexible method which offers these features is the n-tuple technique applied to image analysis. In this case a n-tuple is an ordered group of pixels which is processed as a single entity. The way the n pixels are arranged is called a state or an attribute of the selected n-tuple [3, 41. The n-tuple technique was first introduced in the late 0 IEE, 1996 IEE Proceedings online no. 19960616 Paper frst received 8th November 1995 and in revised form 21st May 1996 The authors are with The Pardel Processing Specialist Group - Electrical Engineering Department, University of Nottingham, University Park, Nottingham NG7 2RD, UK 1950s and was practically used in the early 1980s when I. Aleksander and his group fabricated the pattern recognition device WISARD known also as the n-tuple classifier [5]. Since then, different variants of this technique have been proposed, most of them are implemented using the random access memory (RAM) device as a basic building block or one of its derivatives such as the probabilistic logical node (PLN) [5], the goal seeking neuron (GSN) [6] or the probabilistic RAM @RAM) [7]. These methods rely upon a digital neural network using a weightless RAM based neuron with the input layer being directly connected to the input image. Each n-input RAM decodes the states of a group of n-pixels, with each pixel being mainly considered only once within any n-tuple. In this paper we propose a new strategy which has advantages over recently published methods [l, 21 and is based on the n-tuple technique that offers economical implementation whilst retaining versatility in the type of applications it can be applied to. We restrict the set of all possible n-tuple states, choosing only the most relevant ones, whereas each pixel in the image is sam- pled more than once. These attributes are static geo- metric descriptors independent of the task at hand. This introduces generality when performing feature extraction. This new approach allows for a total over- lap among successive n-tuples making it suitable for the analysis of compact shapes such as simple defects occurring in binary or grey-scale images. We exploit this advantage to tackle the problem of flaw detection within 2-D images, where our main concern is the processing speed enhancement offered by the tech- nique. Within this paper, the difficult process of select- ing the correct attributes is analysed and a n-tuple state extraction algorithm developed. From this a strategy is formulated for the categorisation of defects within two- dimensional images. Then results are presented for the application of the technique to two diverse applica- tions, the first being the visual inspection of printed cir- cuit boards, the second the analysis of both simple and complex faults within a multigrey level image. It is shown how the technique can be adapted to efficient running on a parallel processing system due to the unconnected nature of the attributes, the algorithm used and the nature of the problem. From this it can be conjectured that its implementation on dedicated hardware would achieve real time analysis. 2 For the sake of obtaining good performance while applying the n-tuple method, the size of the n-tuple, the Selection of the n-tuple parameters 30 1 IEE Proc.-Vis. Image Signal Process., Vol. 143, No. 5, October 1996
Transcript
Automatic visual inspection based upon a variant of the n-tuple technique
Abstract: The authors present a new rapid image analysis technique ideally suited for both production line and generalised flaw detection. It is a variant of the n-tuple technique and uses nine selected attributes based on five pixels arranged in a cross shape. The technique requires a flaw free image as well as the image under test. The estimation of the flaw size and its shape is given by recording frequencies of occurrence of the nine attributes by scanning both the corrupted image and the flaw free image. The application of this technique to the inspection of simple flaws in printed circuit boards, proved to be successful in detecting defects as small as one pixel in an image of 512 x 512 pixels with the object occupying as little as 3 x 3. Its application to defect identification within a multigrey level image is also shown to be successful. Its ease of implementation makes this technique a very good candidate for implementation as an embedded part within a front end processor for a real time automatic inspection system. It offers an improvement in speed of analysis, simplicity of operation, versatility in the type of problem it can tackle and cost-effectiveness over recently published methods (Newman and Jan, 1995; and Chin, 1989).
1 Introduction
Flaw detection is an important process in both indus- trial and nonindustrial inspection systems. The analysis has to be accurate and fast. It has been shown that such requirements can be achieved by vision based inspection systems. In particular, one flexible method which offers these features is the n-tuple technique applied to image analysis. In this case a n-tuple is an ordered group of pixels which is processed as a single entity. The way the n pixels are arranged is called a state or an attribute of the selected n-tuple [3, 41.
The n-tuple technique was first introduced in the late 0 IEE, 1996 IEE Proceedings online no. 19960616 Paper frst received 8th November 1995 and in revised form 21st May 1996 The authors are with The Pardel Processing Specialist Group - Electrical Engineering Department, University of Nottingham, University Park, Nottingham NG7 2RD, UK
1950s and was practically used in the early 1980s when I. Aleksander and his group fabricated the pattern recognition device WISARD known also as the n-tuple classifier [5]. Since then, different variants of this technique have been proposed, most of them are implemented using the random access memory (RAM) device as a basic building block or one of its derivatives such as the probabilistic logical node (PLN) [5], the goal seeking neuron (GSN) [6] or the probabilistic RAM @RAM) [7]. These methods rely upon a digital neural network using a weightless RAM based neuron with the input layer being directly connected to the input image. Each n-input RAM decodes the states of a group of n-pixels, with each pixel being mainly considered only once within any n-tuple.
In this paper we propose a new strategy which has advantages over recently published methods [l , 21 and is based on the n-tuple technique that offers economical implementation whilst retaining versatility in the type of applications it can be applied to. We restrict the set of all possible n-tuple states, choosing only the most relevant ones, whereas each pixel in the image is sam- pled more than once. These attributes are static geo- metric descriptors independent of the task at hand. This introduces generality when performing feature extraction. This new approach allows for a total over- lap among successive n-tuples making it suitable for the analysis of compact shapes such as simple defects occurring in binary or grey-scale images. We exploit this advantage to tackle the problem of flaw detection within 2-D images, where our main concern is the processing speed enhancement offered by the tech- nique. Within this paper, the difficult process of select- ing the correct attributes is analysed and a n-tuple state extraction algorithm developed. From this a strategy is formulated for the categorisation of defects within two- dimensional images. Then results are presented for the application of the technique to two diverse applica- tions, the first being the visual inspection of printed cir- cuit boards, the second the analysis of both simple and complex faults within a multigrey level image. It is shown how the technique can be adapted to efficient running on a parallel processing system due to the unconnected nature of the attributes, the algorithm used and the nature of the problem. From this it can be conjectured that its implementation on dedicated hardware would achieve real time analysis.
2
For the sake of obtaining good performance while applying the n-tuple method, the size of the n-tuple, the
Selection of the n-tuple parameters
30 1 IEE Proc.-Vis. Image Signal Process., Vol. 143, No. 5, October 1996
grouping of the n pixels (input mapping or sampling) and the form of the attributes (i.e. different permutations of the n pixels) all have to be adequately selected. In the case of a binary image, in which each pixel can have only two values either 0 (background) or 1 (object), the number of states for each n-tuple is 2n. Normally, any size and grouping can potentially be used. However, such a range of choice is difficult and costly to implement. Previous work showed that the optimum size of IZ and the grouping of pixels is dependent on the data handled as well as on its diversity [8]. However, in many industrial applications we are concerned with compact shapes where a defect is considered as a localised irregularity of an undesirable kind. We chose the size of the template by testing different sizes (12 = 4, 5, 6, 7, 8, 9) using different arrangements as shown in Fig. 1 and have selected the one that gave the best performance in terms of simplicity and speed of operation [9]. Due to the fact that the compact combination of several pixels of the same luminance determines the shape of an object or pattern in a binary image [4], the following criteria were used to guide our selection.
n=4
3 x 2 2 x 3 n =7 n=6
n =9 n =8
Different values of n
1 2 3
7 8 9
can be either 0 or 1
1. Fig. 2 Different values of selected attributes
(i) When n increases, the number of attributes will increase as 2n. Thus, a subset of these attributes is selected. (ii) The relevant attributes should be able to detect the
302
four local diagonal borders and the four local horizontal and vertical borders, as in many inspection problems only border regions contain relevant information [lo, 111. (iii) The features should have a regular grouping and be of a symmetric form in the plane, so that the hard- ware implementation will be cost-effective. This led us to choose an odd value of n. We also noticed that values of IZ higher than five require a bigger set of attributes to be selected resulting in two main limitations, their decoding takes more time and their redundancy is difficult to avoid (i.e. a feature is extracted by more than one attribute) [12]. As a consequence, we select n = 5 mapped in a 3 x 3 kernel and arranged in a cross shape around a central pixel allowing a 4-connectedness with the other pixels. Therefore, we restrict the number of attributes to nine as shown in Fig. 2. Attribute 1 identifies the interior of the object, attributes 2 and 3 extract the local vertical orientation, attributes 4 and 5 extract the local horizontal orientation and the other four attributes (6, 7, 8 and 9) carry the information of the edges (four edge slopes of nl4, 3d4, 4 4 and -3nl4). A given image will be scanned searching for these attributes, by placing the central pixel of the 3 x 3 kernel over each pixel. As a consequence, the other 23 attributes (corresponding to the remaining combinations of the five significant pixels) are considered as representing noise (isolated pixel values: pixel on within a neighbourhood of off pixels or the reverse) or redundancy and thus they are irrelevant to many inspection problems [l 11.
The technique’s operating speed is dependent on the searching procedure for the occurrence of the selected attributes. To enhance the search procedure, all attributes have their central pixel on, and a weight is associated to the four significant positions connected to the central pixel within the 3 x 3 kernel. In the case of a binary image, the weights are a power of 2, that is 8, 4, 2 and 1. This representation will allow us to distinguish uniquely among the nine attributes, and thus the recording of their frequencies of occurrence will be easily computed using the basic shift operation rather than the computationally intensive multiplication operation Fig. 3. The resultant algorithm is shown in Fig. 4. As a result, an attribute will be identified by the decimal number 8 * i3 + 4 * i2 + 2 * il + io, where ik = 0 or 1 for all k E (0, 1, 2, 3). Taking into consideration the previous constraints, the star input mapping considered is the most appropriate one for the task to be tackled.
2. I Defect categorisation To illustrate the categorising of defects, based on their shapes and sizes, we performed two tests. In the first, we used simulated flaws of varying sizes and shapes. White faults in the shape of a rectangle, square, trian- gle and circle were introduced one at a time on a dark background. The measurements are expressed in terms of the flaw area A and compactness C = P2/A as derived from the statistical distributions of the attributes. The flaw perimeter P and the flaw size (area) can be, respectively, approximated by eqns. 1 and 2.
P = f 2 + f ig f 2 (1) 2 = 2 2 = 6
IEE Proc -Vis Image Signal Process, Vol 143 No 5, October 1996
Fig, Weis
i=9
i=l
Example of attributes extraction , of pixels T, R, B And L are 1, 2, 4 and 8 respectively
load image 1 M y M 1
shift 3 times L shift twice 6 shift onceR
image add
8L+ 4B+2R+T
p z ! r l and update 1
I print result 1 U
Fig.4 Algorithm of attributes extraction
where J1 is the frequency of occurrence of the ith attribute in a given image. Fig. 5 presents the compactness function of the size of each flaw shape investigated. These results show four distinct traces, corresponding to the four shapes used, this distinction is particularly observed for bigger flaw sizes. However, these regular shapes represent a trivial case since, in real problems, a flaw can have a random shape. Nevertheless, a coarse estimation of the flaw size and shape can still be obtained using such a method. Secondly, we plotted the histograms of the nine attributes for each fault investigated, this is shown in Fig. 6. The rectangle is characterised by different distributions of vertical and horizontal attributes, whereas the square presents equal distributions for such attributes. In the case of the triangle the occurrence of vertical attributes besides attributes 6, 7 and 8 indicates a shape with three sides. This is not the case of the circle presenting similar distributions for symmetric attributes. Several other irregular real fault shapes were also considered, an example is given in Fig. 7. We conclude from these results that the
IEE Proc -Vis Image Signal Process , Vol 143, No 5, October 1996
frequency of occurrence of the selected nine attributes can be used as a rapid means to categorise defects. To assess the effectiveness of this approach, we consider its application to binary images. The inspection of simple flaws within bare 2-D electronic printed circuit boards (PCB), known also as PWB inspection, is used as this problem presents the most active application machine vision [l].
0 100 200 300 4c size,pixels
Defect categorisation: compactness and size Fig. 5 .. .-. circle V---V square 0.- --E rectangle 0 ........ 0 triangle
for
~
rectangle- Fi .6 Defect categorisation: hutogram of selected attuibules for regular defJects
Fig.7 irregular deLcts
De ect categorisatirn: histogram of selected attributes for
3 PCB inspection
Any technique, to be of a practical use for visual PCB inspection, must fulfil two main goals, these being speed of operation and versatility of application. In an attempt to achieve these, various industrial inspection
303
methods have been proposed. They mainly fit in one of the three major categories: image subtraction, design rule based and model based. All these schemes have their own merit, where one goal is improved in detriment of the other [lo, 11, 131. We propose here, the use of the a-tuple technique discussed above as an efficient PCB inspection method, due to its simplicity of operation and its ability to balance between speed and versatility without requiring any adaptation of the selected attributes. The technique will be applied to a limited number of faults these being the most frequently occurring ones: (i) The maximum width of the track not respected (under-etching in the original board can lead to a short circuit defect). (ii) The minimum width of the track not respected (over-etching in the original board can lead to an open circuit defect). (iii) Annular ring inspection (drilled holes bigger or smaller than the normal size or missing pads).
3. I Proposed PCB inspection method As in the case of model based methods [13, 141, we operate on the features extracted by the attributes selected from the part under inspection. These are then compared to those obtained for the fault free image. The 3 x 3 kernel is centred over each pixel of the PCB image to record the rates of occurrence of the nine attributes. We apply the technique to the two images and analyse the difference of the obtained frequencies of occurrence of similar attributes, rather than applying the technique directly to the difference image. Thus we avoid the problem of alignment, which is hard to achieve in practice, while comparing the prototype image and the faulty image. That is, we compute the differences given by eqn. 3 for all i E { 1, ..., 9}:
where i is the attribute number, J1 and f , are the frequencies of occurrence of the ith attribute corresponding, respectively, to the reference and the faulty images. In this case a defect is detected if the histograms of similar attributes extracted from the two images are different, that is if the expression given by eqn. 3 is not equal to 0 for any i E { 1, ..., 9}.
aft = fi - f,' ( 3 )
1 I I pad and drilled holedefects 1 a b
Fig.8 Example of PCB images a PCB prototype b Simulated defects
3.2 Discussion of the results Using appropriate lighting conditions, we can consider the PCB image as a binary one, where conductor tracks and pads are coded as 1s and the board area (sub- strate) as Os. The prototype image shown in Fig. 8 4 which consists of two tracks having circular pads at
304
their end, was used to simulate artefacts. Different fault sizes were introduced within the PCB track correspond- ing to a nick or extraneous copper. The size of the nick fault was limited by the track's width whereas the extra-copper fault was limited by the distance between the consecutive tracks. Various faults occurring in a PCB pad were also simulated as shown in Fig. 8b. Results are given in Tables 1, 2 and 3. The difference in the occurrence rates of the nine attributes are indicative of the fault size and type. Tables 1 and 2, dealing with vertical track defects, show that small faults of one pixel can be extracted for an object size as small as the kernel size used 3 x 3. By using algebraic values it is possible to distinguish between the two track defects. The absolute value of attribute 1 gives an estimate of the fault size. The positive value recorded for attribute 3 shows that the fault occurs within a vertical track. The same values recorded for the symmetric attributes, respectively, 4 and 5 then 6 and 7, give an indication of the fault shape, considered here as circular.
Table 1: Track under-etching defect
Fault size (pixels) Differences in occurrence rates
By consulting Table 1, it can be seen how to distinguish between under-etching and the short circuit defect. The minus sign for attribute 1 indicates extra copper within the compact shape of the track (under- etching). This negative value is interpreted as the absence within the faulty image of an isolation between successive tracks, this isolation shrinks until we get a short. The occurrence of attribute 2, when we increase the fault size, shows the absence of a vertical border
IEE Proc -Vis. Image Signal Pvocess., Vol. 143, No. 5, October 1996
between the two consecutive tracks, indicating a copper contact between them.
In the case of Table 2, an absence of part of the track is recorded by the occurrence of a positive value for attribute 1, reflecting the amount of missing copper within the faulty image. When the fault covers the whole track width (open circuit defect), we notice the occurrence of attribute 2, where its value indicates how large the cut is within the track, giving the possibility of controlling the allowable limit above which such a defect is not acceptable. The fact that attributes 8 and 9 do not occur indicates that the fault is affecting the track in the direction from left to right, recorded, respectively, by their symmetric attributes 6 and 7. If the fault direction is reversed attributes 8 and 9 occur instead of 6 and 7.
The first entry of Table 3 shows the missing pad in the faulty image by recording positive values for all the attributes. Horizontal attributes 4 and 5 are recorded with high values due to their absence within the faulty image, this is not the case for vertical attributes where the presence of the vertical end of the track within the faulty image, relatively compensates for the occurrence of attribute 2 (Fig. 8b). The second entry of Table 3 reflects a missing drilled hole, by recording a negative value for attribute 1. This result confirms the fact that the drilled hole is considered as part of the substrate. Drilled holes less than the normal size are illustrated by negative values of attribute 1 (missing copper) and positive values of the border attributes (extra borders). However, drilled holes greater than the normal size are presented as positive values for attribute 1 (extra copper) and negative values for the other attributes (missing borders). Furthermore, the same occurrence rates obtained respectively for symmetric attributes 2, 3, 4, 5 and 6, 7, 8, 9 indicate a circular shape for the drilled hole.
We note finally, that the differences in the occurrence rates of the nine attributes, shown in Tables 1, 2 and 3, reflect the technique's ability to distinguish between the faults investigated. Other flaw types occurring in various places within the PCB were also considered, the results obtained confirm that such a technique is suitable for PCB inspection.
3.3 Performance analysis According to the algorithm used for the extraction of the n-tuple states, as shown in Fig. 4, the processing
IEE Proc -Vu Image Signal Process, Vol 143, No 5, October 1996
time depends upon the content of the image. An image with a bigger compact object will require more processing time than that with a smaller object. Using the C-language to implement the technique, the imean value of the operating speed was around 0.27s for a standard 512 x 512 PCB image, on an IEIM-PC 486 running at 33MHz, where the copper area represents 1/8 the total size of the image, achieving thus an adequate processing rate of lo6 pixelds.
Assuming board sizes of 100 x 100mm2, each pixel will have a width of 0.2mm, then as the typical conductor (track) width is 2 mm a track will occupy approximately 10 pixels. Since the technique is able to detect compact defects as small as one pixel, the detection accuracy for a track fault is 1/10. We can enhance this factor (i.e. defect definition) by increasing the image resolution, which increases the track resolution, while keeping the smallest detectable fault unchanged. This is particularly advantageous in the case of a finer PCB. This technique can detect drilled holes having a diameter of 0.2mm, which is less than the typical value estimated at 0.35mm [13].
It has been noted that, whilst achieving very good defect definition, the technique is sensitive to noise. However, this only affects scattered pixels (such as dust) affecting the copper area. Such noise occurring within the substrate will not lead to false detection. Since both conductor and nonconductor parts of a PCB are important to the inspection task, we can state that, on average, the technique presents an acceptable immunity to noise, offering thus a good compromise between versatility and speed.
A cost-effective hardware implementation of the technique can be achieved by using off-the-shelf components such as a decoder, counters and random access memories. Therefore, the method described is adequate in terms of speed, cost, accuracy and versatility. Any attempt to increase the kernel size will not achieve such results when using a limited number of attributes.
4 Grey-scale images
A full grey-scale image is often required in many image processing applications [l, 131. However, the use of multigrey-level images complicates the application of the n-tuple technique as the number of attributes used might, if attribute pixel values were allowed to take other values than on and off become excessive. A thresholding method can be used, but this will came a loss in image information content. If a thresholding method was adopted, there would be a need for an intelligent scheme to specify the best threshold values, as these are problem dependent [15]. One approach was proposed to handle the grey-scale n-tuple without using intermediary binary images [16]. It consists of representing the n-tuple state by the order of the n- tuple elements (pixels) sorted according to their grey- values. This has the advantage of reducing the number of the n-tuple states. However, the number required is impractical and any attempt to reduce this number further involves more complexity in the algorithm To keep the n-tuple technique as simple as in the binary case, without loosing the generality in applying it, we decode the grey level of each pixel using N digits. Thus forming N binary planes having the sarne spatial resolution as the original grey-scale image, as shown in Fig. 9. Consequently no information loss will occur.
305
This method, called bit-plane decomposition [ 171, decodes a given pixel value Xv within the original image using the following equation:
5(N-1)23 + . . . + 2 P X P t 3 + . ' ' + XOZ3 x - 2 N - l (4) Z 3 -
where x(N-ljEl, ..., xPv, ..., xoB are either 0 or 1 and represent the pixel values corresponding, respectively, to the N binary planes at position (iJ) for all ij E CO, ..., M - 1} (for an M x M image) and N = log2P, P being the number of grey-levels. However, the main drawback of bit-plane encoding is that small changes in a grey-level can have a significant impact on the complexity of the bit planes. In particular, the n-tuple technique used here is dependent on the transitions from 1 to 0 or the reverse. To avoid such a problem, a grey coding scheme [18] can be used to represent the original image. Here, we present a solution based on the reduction of the planes to be handled. Only the highest order binary planes are used as these are the most representative of the main image characteristics. This is shown in Figs. 10 and 11 for the specific example of the Lenna image, where the 512 x 512 256- grey-scale image is represented by the four highest weighted binary planes. This approximation is particularly good when handling compact shape faults where neighbour pixels have nearly the same grey level over smooth and uniform regions within the image. In this case, low order planes present scattered pixels (high randomness) and neglecting them when reconstructing the image results in little loss in the visual contents of the original image. Eqn. 5 , derived from eqn. 4, is used to approximate the original grey- value for a 256-grey-scale image.
optimum choice, especially when we have a restricted number of grey-values. In such a case we recommend the use of a preprocessing stage, based on the image histogram, to breed the optimum number of binary planes. To assess the effectiveness of the technique, we use the Lenna image and a simulated defect, hidden
IEE Pioc -Vis Image Signal Process, Vol 143, No 5 October 1996
right eye as shown in Figs. 12 and 13, and we apply the n-tuple technique to each binary plane. The results presented in Table 4 are used to extract the information about the ranges of grey-values corresponding respectively to the interior of the fault (attribute 1) and its borders (attributes 2 to 9). Knowing that the weights of the four binary planes are respectively 128, 64, 32 and 16 and by consulting the histogram of the fault given in Fig. 13, we can see that the technique is suitable for the categorisation of multigrey-level faults. To confirm this, the technique was also applied to a standard 512 x 512 256-grey-scale airplane test image and a number of faulty versions of the same image, as shown in Fig. 14. Various defects of different size and shape have been simulated by introducing them into the prototype image. Here, we report upon the results when dealing with two main flaw categories. The first flaw type is a small, regular and simple fault (restricted range of grey-values) which consists of a missing one in the airplane’s identification number. The second flaw is a complex irregular fault (covering a wide range of grey-values) and comprises the hidden star with its surrounding filled circle. The algorithm describing the different steps used to identify these defects is as follows: (i) Get the four-binary planes from the prototype grey- scale image. (ii) Get the four-binary planes from the grey-scale test image. (iii) Record the frequency of occurrence of the nine attributes for the 8 binary images. (iv) Perform the difference of the occurrence rates extracted from corresponding planes.
The results are represented in Tables 5 and 6 from which we derive the following conclusions:
(i) Presence of a fault (loss of information within the prototype image: Ai # 0). (ii) The fault is regular (uniform distriibution of attributes, case of Table 5). (iii) The fault is not regular (random distribution of attributes, case of Table 6). Table 5 shows that planes 1 and 2 are the planes mainly involved in identifying the fault. All pixels representing the interior of the fault are extracted by plane 2 through the value recorded for attribute 1 (A1 entry). Approximately similar frequencies are noticed for symmetric attributes, implying a regular shape of the fault, where the interior of the fault is represented by one homogeneous grey-value. The small value recorded for all attributes indicates a small fault size. These results are confirmed by examination of the fault which is considered as regular and simple. ’The results shown in Table 6 indicate the use of all four planes to categorise the fault. Planes 4, 3 and 1 have large values for attribute 1, indicating that the pixels chairacterising the interior of the fault have different grey-values. Since attributes 2 to 9 extract grey values at transition pixels from background to the fault’s zone, differences in these attributes noticed for all planes indicate different grey-values for pixels at the fault’s borders. Roughly speaking, we can state that the fault which has zones that have grey-levels bigger than those at border pixels is extracted by the planes with higher weights, that is planes 3 and 4. The smooth transitions from the background to the fault’s zone are mostly represented by a small change in grey-values. These will be extracted by the smaller weight binary-planes (i.e. 2 and 1). Therefore, differences in border attributes are approximately all extracted by these planes. This is confirmed by examination of the fault’s type and its histogram. Finally, note that the minus sign, recorded in the previous Tables, symbolises the absence of the corresponding attribute within the original image. Negative values can occur, owing to the falct that the simulated faults have been created by using a rnask with the 0 grey-value (black) hiding the specified part within the fault free image.
4.1 Parallelisation of the proposed technique According to the algorithm used in the case of grey- scale images, the problem has a regular geometrical structure with spatially limited interactions, the calcula- tions involved are uniform and repetitious. Thus the approach offers great potential for parallel processing [19]. Here an implementation of the technique on a net- work of transputers is considered, where a two stage approach is adopted. The first is the teaching phase,
Fig. 14 Multigrey-scale faults (airplane) A complex defect (hidden star) is shown to the right of the main image and a simple defect (missing one) is shown on the far right
IEE Proc.-Vis. Image Signal Process., Vol. 143, No. 5, October 1996 307
dealing with the reference image, which is assumed to be an off-line operation. The second is the testing phase, where the search for the attributes is carried out for the image under inspection and the difference of the corresponding frequencies of occurrence of the nine attributes calculated. This phase is considered as an on- line operation. An optimisation of the algorithm code has led to a reduction in both the operating time and the image size. This improvement has been achieved by handling the pixel value as one bit instead of one byte and operating the technique at the bit level using the basic shift and add operators rather than the multipli- cation and division operators. This has the main bene- fit of reducing the data communication overhead through the transputer links, by a factor of 8, while achieving a mean value of 1.82 for the speed-up factor. Using a T800-20 MHz transputer platform, running the ANSI C toolset, hosted on an IBM PC and assuming that the images are a priori loaded into the host mem- ory, then the operating speed of the extraction of the n- tuple states for different number of transputers is given in Table 7. This Table refers to the case where the four 512 x 512 binary plane images, extracted from the 256- grey-scale airplane image, are used. In the case of one transputer, the processing is performed serially on each image. In the case of three and four transputers, as shown in Figs. 15 and 16, two different methods are used. The first one consists of splitting the binary image, respectively, into three and four subimages each of which is handled by one transputer. The second method consists of handling the four binary planes in parallel by the network of transputers operating on the whole binary image. From Table 7 we can conclude that, by using an array of four transputers, each of which handle 114 of an image, we can obtain a good speed up factor 3.38 with an acceptable efficiency of 84.5%. Better results are expected for a higher speed processing platform. Thus the technique is eminently suitable, using multigrey level images, for carrying out rapid visual inspection tasks.
In this paper we have proposed a new technique for flaw identification within images, based upon the n- tuple approach, where only nine adequately selected attributes were used. It has been shown that, by sliding a 3 x 3 window through the image, the extracted distribution of these attributes can be successfully used as signatures for rapid flaw detection in binary and grey-scale images. In the case of binary images, where the technique was applied to PCB inspection, the results obtained were indicative of the suitability of the technique to such a task. We have also demonstrated the application of the technique to grey-scale images by using the bit-plane encoding scheme and shown how a network of transputers can achieve good efficiency in
308
terms of speed. This proves that the technique is suitable for carrying out rapid visual inspection tasks. One limitation of this technique is its sensitivity to rotations, more attributes are required to overcome this problem. However, this solution degrades the processing speed. As a final conclusion we can state that the proposed technique due to its simplicity, rapidity, versatility and cost effectiveness, when using a restricted number of attributes, can be best utilised as a preprocessing stage for a complete autonomous flaw detection system.
F] 20 MHz
.; 20 MHz 20MHz
transputer motherboard TMB16
.I5 Transputer implementation: tree configuration
IBM-PC U 20MHz 20 MHz
I I
Transputer implementation: array configuration
C@§
1 NEWMAN, T.S., and JAN, A.K.: ‘A survey of automated visual inspection’, Computer Vision Image Understanding, 1995, 61, pp. 231-262
5 ALEKSANDER, I.: ‘Weightless neural models for cognitive design’, J. Intelligent Syst., 1992, 2, pp. 31-52
6 DE CARVALHO, A., FAIRHURST, M.C., and BISSET, D.L.: ‘Classifying images using goal-seeking neural network architec- tures’, IEE Proc. I , 1993, 140, (l), pp. 12-17
7 CLARKSON, T.G., GORSE, D., TAYLOR, J.G., and NG, C.K.: ‘Learning probabilistic RAM nets using VLSI struc- tures’, IEEE Trans., 1992, C 4 1 , (12), pp. 1552-1561
IEE Proc.-Vis. Image Signal Process., Vol. 143, No. 5, October 1996
~
8
9
FAIRHURST, M.C., and KORMITO, S.R.: ‘Some economic 13 PAU. L.F.: ‘Comouter vision for electronics manufacturing’ considerations in the design of an optimal n-tuple pattern classi- fier’, Digital Process., 1977, 3, pp. 321-329 OUSLIM, M., CURTIS, K.M., and BELMEKKI, B.: ‘Recogni- tion method using the n-tuple technique with different input map- pings’. International conference on Electrical and electronic engineering, ICEEE’94, Oran, 1994, pp. 28-33
10 YASUHIKO, H., HIDEAKI, D., and KOICHI, K.: ‘A system for PCB automated inspection using fluorescent light’, ZEEE Trans., 1988, PAMI-10, pp. 69-71
12 OUSLIM, M., and CURTIS, K.M.: ‘Image analysis based upon two variants of the n-tuple technique’. Proceedings of interna- tional AMSE conference on Communication, signals and systems, Rabat, 1995, pp. 83-92
