# Automated Visual Inspection - Part 4: Color in inspection, bringing inspection to the factory

*Receive a 15% discount on "Machine Vision: Theory, Algorithms, Practicalities" by ordering from www.elsevierdirect.com and using offer code 92839.*

*[Part 1 of this article begins with a look at the process of inspection and the various types of objects that visual inspection systems have to cope with. Part 2 looks at a simple but useful means of checking shapes, and then examines the computation and application of the radial histogram. Part 3 considers the inspection of printed circuits and products with high levels of variability, and examines issues related to x-ray inspection.]*

22.11 **The Importance of Color in Inspection**

In many applications of machine vision, it is not necessary to consider color, because almost all that is required can be achieved using gray-scale images. For example, many processes devolve into shape analysis and subsequently into statistical pattern recognition. This situation is exemplified by fingerprint analysis and by handwriting and optical character recognition.

However, there is one area where color has a big part to play. This is in the picking, inspection, and sorting of fruit. For example, color is very important in determining apple quality. Not only is it a prime indicator of ripeness, but it also contributes greatly to physical attractiveness and thus encourages purchase and consumption.

Whereas color cameras digitize color into the usual RGB (red, green, blue) channels, humans perceive color differently. As a result, it is better to convert the RGB representation to the HSI (hue, saturation, intensity) domain before assessing the colors of apples and other products.^{4} Space prevents a detailed study of the question of color; the reader is referred to more specialized texts for detailed information (e.g., Gonzalez and Woods, 1992; Sangwine and Horne, 1998). However, some brief comments will be useful. Intensity *I* refers to the total light intensity and is defined by:

*I* = (*R* + *G* + *B*)/3 (22.24)

Hue *H* is a measure of the underlying color, and saturation *S* is a measure of the degree to which it is not diluted by white light (*S* is zero for white light). *S* is given by the simple formula:

*S* = 1 - min(*R*, *G*, *B*)/*I* = 1 - 3 min(*R*, *G*, *B*)/(*R* + *G* + *B*) (22.25)

which makes it unity along the sides of the color triangle and zero for white light (*R* = *G* = *B* = 1). Note how the equation for *S* favors none of the *R*, *G*, *B* components. It does not express color but a measure of the proportion of color and differentiation from white.

Hue is defined as an angle of rotation about the central white point *W* in the color triangle. It is the angle between the pure red direction (defined by the vector **R - W**) and the direction of the color C in question (defined by the vector **C - W**). The derivation of a formula for *H* is fairly complex and will not be attempted here. Suffice it to say that it may be determined by calculating cos *H*, which depends on the dot product (**C - W**) • (**R - W**). The final result is:

(22.26)

or 2π minus this value if *B* > *G* (Gonzalez and Woods, 1992).

When checking the color of apples, the hue is the important parameter. A rigorous check on the color can be achieved by constructing the hue distribution and comparing it with that for a suitable training set. The most straightforward way to carry out the comparison is to compute the mean and standard deviation of the two distributions to be compared and to perform discriminant analysis assuming Gaussian distribution functions. Standard theory (Section 4.5.3, equations (4.41) to (4.44)) then leads to an optimum hue decision threshold.

In the work of Heinemann et al. (1995), discriminant analysis of color based on this approach gave complete agreement between human inspectors and the computer following training on 80 samples and testing on another 66 samples. However, a warning about maintaining the lighting intensity levels identical to those used for training was given. In any such pattern recognition system, it is crucial that the training set be representative in every way of the eventual test set.

Full color discrimination would require an optimal decision surface to be ascertained in the overall 3-D color space. In general, such decision surfaces are hyperellipses and have to be determined using the Mahalanobis distance measure (see, for example, Webb, 2002). However, in the special case of Gaussian distributions with equal covariance matrices, or more simply with equal isotropic variances, the decision surfaces become hyperplanes.

**Footnotes:**

^{4} Usually, a more important reason for use of HSI is to employ the hue parameter, which is independent of the intensity parameter, as the latter is bound to be particularly sensitive to lighting variations.