A

Colour spaces

This appendix describes the colour spaces that are most commonly used for brightness normalisation in

order to detect skin colour. A large set of skin and background samples is used to show their spread in

the different colour spaces.

A.1 The RGB colour space

Extensive experiments in the human visual system have showed that the cones — sensors in the eye

responsible for colour vision — can be divided into three principal sensing categories, corresponding

roughly to red (R), green (G) and blue (B) [WS00]. Therefore,colours are seen as combinations of these

so-called primary colours [GW00]. For this reason, most of the cameras and emissive colour displays

represent pixels as a triple of intensities of the primary colours in the RGB colour space:[R,G,B] ∈ IR3.

This is also the reason why the RGB space is very commonly usedby the computer graphics and image

processing community.

A disadvantage of the RGB representation is that the channels are very correlated, as all of them

include a representation of brightness. This is illustrated in Figure A.1 and A.2, in which the brightness

information can be recognised from R, G and B channels shown separately.

True colour 24 bits RGB images have the triple[R,G,B] represented by 256 discrete values (ranging

from 0 to 255) [Jac01], thus the range of RGB colour values forms the cube of (28)3 possible values as

shown in Figure A.3. The high correlation between lightnessand RGB channels can be noted by the line

of the grey values, whereR = G = B. In fact, if the corresponding elements in two points,[R1, G1, B1]

A.2 The CIE chromatic space 185

(a) (b)

Figure A.1: Sample colour image (a); and its grey level version (b).

R G B

Figure A.2: RGB channels of image in Figure A.1(a) shown separately.

and[R2, G2, B2], are proportional, i.e.,

R1

R2

=G1

G2

=B1

B2

, (A.1)

they have the same colour, but different brightness [YW96].Differences in brightness are often dis-

regarded by humans, as our visual system is capable of adapting to different brightness and various

illumination sources such that the perception of a colour constancy is maintained within a range of envi-

ronmental lighting conditions [WS00].

A.2 The CIE chromatic space

The CIE chromatic space is a standard proposed in 1931 by the Commission Internationale de l’Eclairage

– the International Commission on Illumination. Some modifications have been proposed later, but this

A.3 The perceptual colour space 186

white

0

black R = G = B

magenta

Blue cyan

yellow

255Green

255

255Red

Figure A.3: The RGB colour cube.

section is restricted to the 1931 standard. It has been used in several colour processing tasks [GW00] and

it is used to define the colour gamut, i.e., the range of possible colour values that a device can represent.

This two dimensional space has thex andy axes respectively defined by the pure chromatic colours

red and green(r, g), defined by this normalisation process:

r = RR+G+B

g = GR+G+B

(A.2)

which is, in fact, a IR3 → IR2 map. Pure blue (b) is redundant after the normalisation becauser+g+b = 1

[WS00].

The use of this colour space for skin detection has became popular specially after the work on face

tracking developed at SCS, Carnegie Mellon University [YW96, YLW98b].

A.3 The perceptual colour space

The perceptual colour spaces were designed by Smith in [Smi78] in order to provide a more “intuitive”

way of describing colours and lightness. Three quantities are used to define them: hue, saturation and

brightness. Brightness embodies the achromatic notion of intensity. Hue is an attribute associated with

the dominant wavelength in a mixture of light waves. It represents colour as perceived by an observer.

Thus, when we call an object blue, yellow or red, we are specifying its hue. Saturation refers to the

relative purity or the amount of white light (or grey of equalintensity) mixed with a hue. Primary

colours (pure red, green and blue) are fully saturated, whereas colours such as pink (red and white) and

lavender (violet and white) are less saturated. The degree of saturation is inversely proportional to the

amount of white light added [GW00].

Basically, there are two distinct perceptual colour spaces: HSL (hue, saturation, lightness); and

A.3 The perceptual colour space 187

V=

H=H=

H=

H= H=

H=1

1

H

V

Red

YellowGreen

Cyan

300

o

o

o

o

180

120o

60

0

o240

Black

Blue

o0

S

White

Magenta

HSV

o

L=Cyan

o180 0.5H=

Grey

H

Red

YellowGreen

300

o

o

o

120o

60

0

240

Black

Blue

o0

L

1

1 S

H=

H=

H=

H= H=

White

Magenta

HSL

Figure A.4: HSV and HSL colour spaces.

HSV (hue, saturation, value). Both are defined with polar coordinate systems. HSV is represented by a

hexcone where Hue is the angle around the vertical axis, S is the distance from the central axis and V

is the distance along the vertical axis. Primary and secondary pure colours are fully saturated (S = 1).

As illustrated in Figure A.4, starting fromH = 0o (which represents pure red), a secondary or primary

colour is located at each60o of hue. Complementary colours are180o opposite one another measured

by H. Colours along the vertical axis have zero saturation, i.e., grey scale values. Note that whenS = 0,

the value of H is irrelevant [Jac01], [Smi78].

HSL colour space is a double hexcone and can be thought of as a deformation of the HSV space. The

distinction between HSV and HSL lays in the representation of brightness information, which determines

the distribution and dynamic range of both the brightness (Lor V) and saturation (S). In practice, the

HSL colour space is best for grey level image processing and also for representing objects in such a

way that colour images can be distinguished even in monochrome images (e.g. to show colour cartoons

on black-and-white TV receivers), whereas the HSV image space is a better representation for colour

processing [Jac01].

A.4 The YUV and YCbCr colour spaces 188

As described in [RMG98], [AP96], and [ZYW00], on performingskin detection, the brightness

channel is discarded and the HS space is used instead. Therefore, there is no significant difference

between HSV and HSL in this application [Bow99].

H S V

Figure A.5: HSV channels from Figure A.1(a) shown separately.

Figure A.5 shows the H, S and V channels obtained from image from Figure A.1(a).

A.4 The YUV and YCbCr colour spaces

The YUV image space was created in order to make colour television broadcasts backwards-compatible

with black and white TV receivers. The colour signal also needed to conserve bandwidth because three

channels of RGB data would not fit into the limited broadcast signal bandwidth. The Y channel de-

scribes Luma, the range of value between dark and light, which is the signal shown in black and white

televisions. The U and V chrominance channels subtract the Luminance values from Red (U) and Blue

(V) to represent the colour only information (without brightness) [Mal02]. Thence, the basic conversion

equation from RGB to YUV is:

Y = 0.3R + 0.6G + 0.1BU = B − Y

V = R − Y

(A.3)

The coefficients used to obtain luma are the same as those usedfor the NTSC standard conversion

from RGB to grey level images [Poy96]. These coefficients arebased on psychovisual experiments that

estimated the proportion of red, green and blue that we perceive. It is shown that approximately 65% of

all the cones in the human eye are sensitive to green light, 33% are sensitive to red light and only about

2% are sensitive to blue, but the blue cones are the most sensitive [WS00].

A.4 The YUV and YCbCr colour spaces 189

The YCbCr colour space was developed as part of ITU-R BT.301 during the development of a world-

wide digital component video standard. This colour space was extensively used in the development of

the JPEG standard, and was used for skin colour detection by several research projects, including the

Pfinder [WADP97].

As shown in equation A.4, YCbCr is a scaled and zero-shifted version of the YUV, so that the

chrominance values are always positive [PM93]:

Cb = U2

+ 0.5

Cr = V1.6

+ 0.5,

(A.4)

for U ranging between[−0.9, 0.9] andV ranging between[−0.7, 0.7], which are the ranges obtained

from the conversion from RGB∈ [0, 1]. So the range ofCb and Cr are (0.05, 0.95) (0.06, 0.94),

respectively. For digital 8-bits values of U and V, a 128 shift is employed, rather than 0.5.

Figure A.6 shows the RGB colour cube in the YCbCr colour space. It shows that not all the possible

values in the triple[Y,Cb,Cr] represent possible RGB colours. Therefore, special care must be taken to

about overflow or underflow in RGB, when converted from YCbCr.Brightness normalisation is done by

discarding the Y channel.

0 50 100 150 200 250 0

100

2000

50

100

150

200

250

Cr

Cb

Y

Y=255Cb=128Cr=128

WhiteYellow

CyanGreen

RedMagenta

BlueBlack

Cb

Figure A.6: The RGB colour cube in the YCbCr colour space, using 8 bit representation of values.

A.5 Visualising the colour spaces 190

A.5 Visualising the colour spaces

To illustrate the effect of brightness normalisation usingthe above colour spaces, each method has been

applied to the image of Figure A.1(a). An intermediate grey level (127) was chosen and the resulting

images are shown in Figure A.7. Note that skin areas appear uniform and that shading information is lost

for all the three methods.

(a) (b) (c)CIE Pure Colours HSV CbCr

Figure A.7: Resulting images after brightness normalisation of the image in Figure A.1(a) using the CIEcolour space (a), the HSV without the V channel (b), and the YCbCr without the Y channel (c).

In order to illustrate the distribution of skin samples in the colour spaces, a database with images

from 141 different people was used. This database is composed by hand images grabbed from seven

volunteers, and the AR face detection database of the University of Purdue [MB98]. The Purdue database

contains 134 people (men and women) from several ethnic groups. Background samples were obtained

from the background regions in the images (e.g. people’s clothing and other objects) and other images

grabbed in the laboratory, as shown by some samples in FigureA.8. The camera used in the acquisition

had the automatic colour and brightness balance.

Skin and background regions of the images in this database were manually selected in order to obtain

the data set. After the training process, more than 0.5 million samples of skin and more than 1.2 million

samples of background were obtained. Figure A.9 shows the skin and background samples in the RGB

colour space.

Figure A.10(a) shows the plot of only skin samples in the RGB colour space. Note that the samples

are more spread in the direction of the brightness variation. The directions of global variation of the sam-

ple data can be evaluated by performing Principal ComponentAnalysis in this space [dCJ01], [Mar02].

A.5 Visualising the colour spaces 191

(a) (b) (c)

Figure A.8: Database samples: (a) hand images from local volunteers; (b) faces from the AR database;(c) background.

The eigenvector of the covariance matrix of the samples which is associated to the largest eigenvalue is

oriented according to the largest variation of the data set.The second eigenvectors points to the direction

that is perpendicular to the first, and has the second largestvariation of the data, and so on. The eigen-

vectors of the skin colour database are shown in Figure A.10(b). The angle between the first eigenvector

and the vector that points to the direction of the brightnessvariation is only 3.35 degrees. This confirms

that it is necessary to use a normalised colour space or remove brightness information in order to get a

more compact cluster of skin samples.

To illustrate the compression of the skin colour cluster in normalised colour spaces, Figures A.11,

A.12, A.13 show the skin and background samples in the CIE, HSV and YUV chromatic spaces, respec-

tively.

In comparison to the plots of the skin and background samplesin the full colour spaces, the plots in

normalised planes illustrate that such projections lead tolower dimensional spaces with more compact

skin colour samples, improving the separability between them and background samples.

A.5 Visualising the colour spaces 192

Figure A.9: Colour samples in the RGB space: skin (red circles) and background (blue crosses) samplesplot together.

0100

2000 50 100 150 200 250

0

50

100

150

200

250

G

R

B

(a) (b)

Figure A.10: Variation of skin samples: (a) Skin colour samples in the RGB space; (b) Eigenvectorsof the skin samples in the RGB space in their respective mean position. The first, the second and thethird eigenvectors are indicated by a star, a square and a circle in its end, respectively. The dashed lineindicates the grey level (brightness) direction.

A.5 Visualising the colour spaces 193

Figure A.11: Skin (red circles) and background (blue crosses) in the CIE chromatic space.

(a) (b)

Figure A.12: Skin (red circles) and background (blue crosses) shown in the HSV colour space (a); andtheir projection into the HS plane (b).

A.5 Visualising the colour spaces 194

(a) (b)

Figure A.13: Skin (red circles) and background (blue crosses) shown in the YCbCr colour space (a); andtheir projection into the CbCr plane (b).

B

Adjoint transformation in DCT

This appendix complements information of Chapter 6 about the adjoint transformation used by Drum-

mond and Cipolla.

Consider two framesa andb where points are related by the homogeneous transformation

Xb = T

baX

a =

(

Rba tab

0⊤ 1

)

Xa . (B.1)

To derive the effect of changing frame on the screw vector,α, consider writing the scene velocity in

frameb in two different ways(

Xb

0

)

=∑

i

αbiGiT

ba

(

Xa

1

)

= Tba

∑

i

αai Gi

(

Xa

1

)

(B.2)

indicating that∑

i

αbiGi = T

ba

∑

i

αai GiT

ab . (B.3)

Recalling thatα = (ω⊤v⊤)⊤, and the earlier expressions forGi, Eq. (B.3) is just

(

[ωb]× vb

0⊤ 0

)

=

(

Rba tab

0⊤ 1

)(

[ωa]× va

0⊤ 0

)(

Rab −R

abtab

0⊤ 1

)

, (B.4)

where[ω]× is the antisymmetric matrix such that[ω]×r = ω × r. Hence

[ωb]× = Rba[ω

a]×Rab (B.5)

which is equivalent to

ωb = R

baω

a . (B.6)

196

Also

vb = −R

ba[ω

a]×Rabtab + R

bav

a = [tab]×Rbaω

a + Rbav

a . (B.7)

The relationship between the screws is defined as

αb = Ad(Tb

a)αa (B.8)

and hence using Eqs. (B.5,B.7) the adjoint is given by

Ad(Tba) =

(

Rba 0

[tab]× Rba R

ba

)

(B.9)

Eq. (B.9) agrees with Drummond and Cipolla, given that they recoverα = (v⊤ω

⊤)⊤. Though a minus

sign appears missing from the definition of their antisymmetric matrix [t]∧.

As the measurement vectord is an invariant,Fbα

b = Faα

a and so

Fb = F

a Ad(Tba)

−1 (B.10)

which gives

Cb = F

b⊤F

b = Ad(Tba)

−⊤C

a Ad(Tba)

−1 . (B.11)

This differs from the equivalent in Drummond and Cipolla, a difference which may arise because

equation (29) in ref [DC02] statesTbaGi(T

ba)

−1 =∑

j Ad(Tba)ijGj, in contradiction with the later (agreed)

statement in equation (32) in ref [DC02] thatαb = Ad(Tb

a)αa.