Deep Learning for Illuminant Estimation

Authors: Xuerong Xiao, Jennifer Li

Introduction

Our project aims to estimate the illuminant of a scene using various methods of machine learning with a focus on the deep learning method, convolutional neural networks (CNN). Illumination estimation is an area of interest, because it has applications in topics including image reproduction and image retrieval. For image reproduction, an image may be captured under a certain lighting but rendered under a different lighting. In image retrieval and computer vision, the illumination of the scene needs to be estimated so that it can be accounted for when images of objects are obtained under different lighting.

In past research on illuminant estimation, various methods have been used to recover the illuminant. These methods include gamut mapping and random forest, an ensemble machine learning method, and are described in more detail in the Background section.

Background

In related literature, gamut mapping has been used to recover the illuminant of the scene. [1] In this method, the gamut of each illuminant is precomputed. The gamut is the range of colors that are possible to be displayed. And each gamut of an illuminant is precomputed using a database of measured reflectance spectra. Then, the sensor values obtained for each image are compared to the illuminant gamut using a correlation coefficient. This method was able to classify black body radiator illuminants from 2500K to 8500K correctly to within a few hundred degrees Kelvin.

A drawback of this method is the precomputation of all the illuminant gamuts. This can be resolved by directly comparing feature vectors of images through machine learning. The paper, "Illuminant Classification Based on Random Forest" [2] uses the ensemble machine learning method of random forests to estimate the illuminant. In this method, a random feature vector of the image is used to create decision branches of a decision tree used to classify the illuminant of the image. The results of this research were measured in terms of angular errors, which is the difference between a ground truth color vector of the illuminant and an estimated color vector. These angular errors were compared to those of gamut mapping, and the results were similar.

Another literature of work brings up an interesting application of detecting digital fogery using illuminant estimation [3]. In the method used, the image is segmented into regions of similar chromaticity and the differences in these regions are analyzed. The dissimilarity between objects across region boundaries are compared to the dissimilarty between neighboring objects in that region to detect whether the two regions are under different illuminants. If regions in the image are estimated to be lighted under differing illuminants, this could indicate a digital forgery.

In our method, we have chosen to use machine learning algorithms, like in [2] to estimate the illuminant, so that we can focus on feature extraction and the learning algorithm without the steps of precomputing gamuts or separating the image into regions.