PadmanabanVarma
Introduction
Camera technology has been advancing at a fast pace, from giving everyone access to a high-quality cameras in their smartphones to looking beyond 2D images and being able to capture depth information. However, one of the persistent problems that still remains is being able to capture high quality images in low light conditions. As we often see with the images we capture at night with our smart phones, they are usually poor quality and often corrupted by noise that looks like speckles in the image. Specifically, these images are dominated by Poisson shot noise that is inherent to the distribution of photons that are recorded by camera sensors. This photon noise is usually too small relative to the number of photons captured in daylight/bright-light settings and therefore not noticeable. However as photon count drops in dark settings, this noise begins to dominate image formation.
Image denoising is a popular problem that has been studied extensively in the past as a signal denoising problem. However, one of the major drawbacks of these processes is that they focus almost solely on Gaussian noise in images. Gaussian noise is inherently different from Poisson noise in that it is easier to "average out". As described in the next section, there are some methods that look at removing non-Gaussian noise as well, but these methods are very complex and computationally expensive.
Our objective in this project is to find a simple and inexpensive method to be able to denoise existing and new low-light images. We approach this problem via a machine learning perspective, specifically linear regression, to learn a parameter that maps the noisy values of a patch in the image to its center pixel value. Once this parameter is learned, it is easy to use it to denoise previously captured images. Moreover, since the approach denoises a patch at a time, it can be applied to images of varying sizes. With this formulation, the denoising only relies on vector-vector multiplications, which can can be computed efficiently and quickly.
Background
The first attempts at denoising images began with applying techniques from signal processing such as Wiener filtering. However, this relied on the underlying image being smooth, which is not always true for captured images. A more advanced approach looked at transforming the image to the Fourier or Wavelet domain and removing noise by thresholding the coefficients. The two drawbacks of this approach in context of low light image denoising are that it is computationally expensive and it is targeted towards images with Gaussian, not Poisson, noise. A more recent approach applies independent component analysis (ICA) to images in order to denoise them. This procedure works well with non-Gaussian noise but requires multiple frames of the same scene to be able to denoise the image properly. For previously captured images, this is not a viable option.
- Insert other ML focused denoising paper*
Dataset
For the base images of our dataset, we start with the Berkeley Segmentation Data Set [1]. This dataset consists mostly of outdoor images, but includes people, nature and monuments. This ensured that we had a good variety of images to learn from and the learned parameter was not biased towards certain colors or spatial frequencies.
We processed these daytime images to simulate nighttime low-light image captures of the same scenes. We do this using the Image Systems Engineering Toolbox~\cite{iset}. The overall process involves converting from RGB to the number of photons, adjusting their relative counts as if the image had been taken in a daylight color spectrum, scaling the photon counts down, adding Poisson noise, and saving the now dark and noisy image as an RGB image. Specifically, when doing this, we assumed that the daylight is 6500~K blackbody radiation, which matches a sunny day well. Images taken taken in outdoor night settings can look almost exactly like daylight photos with a long exposure~\cite{moonlight}, which informs our decision to create low light images by simply scaling down the photon count.
The above procedure details how to create the noisy training examples. The noise-free values corresponding to these will be generated in much the same way, but excluding the Poisson noise addition step. These noise-free values are used as ground truth for the linear regression algorithm.
Results
Conclusions
References
Appendix I
Appendix II
