Deep Learning for Illuminant Estimation

Authors: Xuerong Xiao, Jennifer Li

Introduction

Our project aims to estimate the illuminant of a scene using various methods of machine learning with a focus on the deep learning method, convolutional neural networks (CNN). Illumination estimation is an area of interest, because it has applications in topics including image reproduction and image retrieval. For image reproduction, an image may be captured under a certain lighting but rendered under a different lighting. In image retrieval and computer vision, the illumination of the scene needs to be estimated so that it can be accounted for when images of objects are obtained under different lighting.

In past research on illuminant estimation, various methods have been used to recover the illuminant. These methods include gamut mapping and random forest, an ensemble machine learning method, and are described in more detail in the Background section.

Background

In related literature, gamut mapping has been used to recover the illuminant of the scene. [1] In this method, the gamut of each illuminant is precomputed. The gamut is the range of colors that are possible to be displayed. And each gamut of an illuminant is precomputed using a database of measured reflectance spectra. Then, the sensor values obtained for each image are compared to the illuminant gamut using a correlation coefficient. This method was able to classify black body radiator illuminants from 2500K to 8500K correctly to within a few hundred degrees Kelvin.

A drawback of this method is the precomputation of all the illuminant gamuts. This can be resolved by directly comparing feature vectors of images through machine learning. The paper, "Illuminant Classification Based on Random Forest" [2] uses the ensemble machine learning method of random forests to estimate the illuminant. In this method, a random feature vector of the image is used to create decision branches of a decision tree used to classify the illuminant of the image. The results of this research were measured in terms of angular errors, which is the difference between a ground truth color vector of the illuminant and an estimated color vector. These angular errors were compared to those of gamut mapping, and the results were similar.

Another literature of work brings up an interesting application of detecting digital fogery using illuminant estimation [3]. In the method used, the image is segmented into regions of similar chromaticity and the differences in these regions are analyzed.