DXMPsych2012Project

Back to Psych 221 Projects 2012

Introduction

With the proliferation of digital cameras and advancement of processors, software image processing and image perception applications have become increasing important and widely-used. In applications such as face localization, human detection and pose classification, skin detection is usually the first step for segmentation. Skin detection using color information has been long studied. Color based skin detection is fast and simple to implement, however, it usually suffer from high false detection error due to metamerism, i.e. nonskin surface have the same color space value as skin. This project will evaluate performance of skin detection in different color spaces, and answer the question - is there a color space that is the best for skin detection?. In this project, I will build skin classifiers in RGB, YCbCr, HSV, xyY and CIELAB, and evaluate their performance based on the false detection and false dismissal rate. In addition, we construct hybrid skin detectors that combine information of multiple color spaces.

Below is another example of a reinotopic map in a different subject.
Figure 2

Once you upload the images, they look like this. Note that you can control many features of the images, like whether to show a thumbnail, and the display resolution.

Figure 3

Methods

Color Space

We use a pixel based histogram approach to construct skin detectors. For each pixel, we extract the chromatic values in all 5 color spaces, we call these values chromatic features. These chromatic features are used to construct histogram-based skin detectors. There are 10 chromatic values in a pixel, $[rgCbCrHSxyab]$. We ignore brightness and lightness values in the color spaces in order to achieve illuminant invariance in the skin detectors. In the next section, we will explain how the chromatic features are defined.

RGB

$[rg]$ are the chromatic features in RGB color space. They are normalized red and green values that are defined as

$[rg]=\frac{[RG]}{R+G+B}$

YCbCr

$[CbCr]$ are the blue-difference and red-difference values in YCbCr color space. They are normalized to range between 0 and 1 for this project.

HSV

$[HS]$ are the hue and saturation values in HSV color space.

xyY

xyY color space is a normalized presentation of XYZ color space. $[xy]$ are defined as,

$[xy]=\frac{[XY]}{X+Y+Z}$

CIELAB

$[ab]$ are normalized $a^{*}$ and $b^{*}$ values in CIELAB color space. $a$ and $b$ range between 0 and 1. $a=0$ indicates green and $a=1$ indicates red, similarly, $b=0$ indicates blue and $b=1$ indicates yellow.

Classification

Histogram

We used MCG-Skin dataset to train, validate and test our skin detectors. MCG-Skin consist of 1000 images of people collected from the internet, the dataset also contains the ground-truth of human skin for the 1000 images. From the MCG-Skin dataset, We chose 100 images for training, 100 images from validation, and 100 images for testing.

With all the chromatic features define, we constructed a two-dimensional histogram for each color space from a training set images. Let $H_{rg}$ be the histogram of RGB color space, and $H_{rg}(i,j)$ is the value of bin $[i,j]$ of the histogram. Let $N(k)$ be the number of pixels in training image $k$, and $I_{(k,l)}$ be the $l^{th}$ pixel of $k^{th}$ image. Define $r_{(k,l)}$ and $g_{(k,l)}$ to be the $r$ and $g$ values of $I_{(k,l)}$. In addition, we denote the bin size of the histogram as $\tau$. The histogram can be expressed as

$H_{rg}[i,j]=\sum _{k=1}^{100}\sum _{l=1}^{N(k)}\mathrm{𝟏}\{i\tau \le r_{k,l}<(i+1)\tau \}\mathrm{𝟏}\{j\tau \le g_{k,l}<(j+1)\tau \}(\mathrm{𝟏}\{isskin(I_{k,l})\}-\mathrm{𝟏}\{\mathrm{\neg }isskin(I_{k,l})\})$

In other word, $H_{rg}(i,j)$ is the number of skin pixels minus the number of non-skin pixels in the range that is specified by bin $)(i,j)$.

We then normalize the histogram to $[0,1]$ by

$H_{rg}[i,j]=\frac{max(H_{rg}(i,j),0)}{max(H_{rg})}$

The histogram is further smoother by a 2D Gaussian filter $G_{\sigma }$ whose variance is ${\sigma }^2$.

$H_{rg}[i,j]=(H_{rg}\star G_{\sigma })[i,j]$

Histograms for other color spaces $H_{CbCr}$, $H_{HS}$, $H_{xy}$ and $H_{ab}$ are similarly constructed.

Classifier

The classifier for a color space is a binary matrix that has the same size as the corresponding histogram. Let $C_{rg}$ be the classifier of RGB color space, and let $T_{rg}$ be the threshold for $H_{rg}$. Then

$C_{rg}[i,j]=\mathrm{𝟏}\{H_{rg}[i,j]>T_{rg}\}$

classifiers for other color spaces $C_{CbCr}$, $C_{HS}$, $C_{xy}$ and $C_{ab}$ are similarly constructed.

We define 1 to indicate skin, and 0 to indicate non-skin. A pixel $I$ with chromatic features $[rgCbCrHSxyab]$ is classified to be either 1 or 0 with following equations. We only present classification using RGB classifier $C_{rg}$, since classification using classifiers in other color spaces are applied in similar ways.

$I_{classify}=C_{rg}[i,j]$

$i=floor(\frac{r}{\tau })j=ceiling(\frac{g}{\tau })$

Note that $\tau$ is a hyperparameter that is specific to the classifier. $\tau$ of a diffident classifier will take on a different value.

Evaluation

We evaluate the performance of a classifier by computing the false dismissal rate ${ϵ}_{FN}$ and the false detection rate ${ϵ}_{FP}$ of the classifier. The false dismissal rate ${ϵ}_{FN}$ of a classifier is the number of skin pixels that are not classified as skin divided by the total number of skin pixels. The false detection rate ${ϵ}_{FP}$ of a classifier is the number of non-skin pixels that are classified as skin divided by the total number of non-skin pixels.

Hyperparameters

In constructing classifiers, we need to find the value of hyperparameters $\tau$, $\sigma$, and $T$ for each color space. The hyperparameters of a classifier are calculated by minimizing the false detection and false dismissal error rate of the classifier.

Failed to parse (unknown function "\limit"): {\displaystyle [\tau \; \sigma \; T] = argmax \limit_{\tau \; \sigma \; T} }

proposed method

What is the proposed method

Results - What you found

Results in different color spaces

Some text. Some analysis. Some figures. Maybe some equations.

Results with combined method

Some text. Some analysis. Some figures. Maybe some equations.

Conclusions

Here is where you say what your results mean.

References

[1] Cahi, D. and Ngan, K. N., “Face Segmentation Using Skin-Color Map in Videophone Applications”, IEEE Transaction on Circuit and Systems for Video Technology, Vol. 9, pp. 551-564 (1999).

[2] Crowley, J. L. and Coutaz, J., “vision for Man Machine Interaction”, Robotics and Autonomous Systems, Vol. 19, pp. 347-358 (1997).

[3] Kjeldsen, R. and Kender., J., “Finding Skin in Color Images”, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition, pp 312-217 (1996).

[4] Singh, S. K., Chauhan, D. S., Vatsa, M. And Singh, R., “A Robust Skin Color Based Face Detection Algorithm”, Tamkang Journal of Science and Engineering, Vol. 6, No. 4, pp. 227-234 (2003).

Appendix I - Code and Data

Code

File:CodeFile.zip

Data

zip file with my data

Appendix II - Work partition (if a group project)

Brian and Bob gave the lectures. Jon mucked around on the wiki.