ChristineHitha
Introduction
Images captured under low-light conditions suffer from degradation, such as low visibility, low contrast, and high-level noise. Although they can be alleviated by professional devices and advanced photographic skills to a certain extent, the inherent cause of the noise is unavoidable and cannot be addressed at the hardware level. Without sufficient amount of light, the output of camera sensors is often buried in the intrinsic noise in the system. Longer exposure time can effectively increase the signal-to-noise ratio (SNR) and generate a noise-free image, however it breeds new problems such as motion blur. Thus, low-light image enhancement technique at the software level is highly desired in consumer photography. Moreover, such technique can also benefit many computer vision algorithms (object detection, tracking, etc.) since their performance highly relies on the visibility of the target scene.
Background
Methods
In order to generate a low light image, we utilized ISETCAM which is a matlab toolbox
ISETCAM Simulation
ISETCAM is split into four main modules: scene, optics, sensor, and processing. For each module, there are useful create, set, and get parameters as well as a display (window) function which allows the user to experiment with the simulation and then see the result. In addition, there are extra built-in functions which allow for even more customization. A good start would be in the script t_introduction2ISET.m found in /isetcam/tutorials/introduction. From there, we were able to build our own script to simulate the scene, optics, sensor, and processing pipeline. First, we generated a scene from a jpeg file and then adjusted the mean luminance of the scene in order to simulate low light conditions. From example, in the below image which originally had a mean luminance of 35.4 cd/m, we adjusted it to be less than a third of that to 10 cd/m.
I = im2double(imread('img.jpg'));
scene = sceneFromFile(I, 'rgb');
scene = sceneAdjustLuminance(scene, 'mean', 10);
In the figure, the after illuminance energy has noticeably decreased by a magnitude of 10 due to the nature of the scene being low light. The effects of this will be seen during the sensor stage. There will be lower number of photons hitting the sensor which would yield a low light image as well as increase the shot noise of the captured image compared to a well lit scene. The problem with this noise will be seen and addressed later on.
Then, we had to use oiSet and oiGet routines to adjust the optical image parameters. Using diffraction limited optics, we were able to compute the optical image (irradiance) from the scene (radiance). Again, each module has display features so could easily see changes in each step.
The irradiance is then captured by a simulated sensor which results in an array of output voltages. There are a lot of different sensor parameters as well as pixel structure properties. Here we just use the simple Bayer-gbrg sensor and set a few of its basic properties such as exposure. As you can see the captured irradiance in the second image is darker which is what we were aiming to do. Additionally, it is important to note that the sensor data looks green because in the Bayer pattern, there is more green filter the red and blue channels. The final image will go through the ISET image processing pipeline and will undergo processing such as demosaicing and illuminant correction so that we can see the actual output image.
Denoising after ISET ISP
Image denoising refers to the recovery of a digital image that has been impure by the noise. In case of image denoising methods, the characteristics of the degrading system and the noises are assumed to be known beforehand. With the creation of low-light images with ISETCam the image s(x,y) is blurred by a linear operation and noise n(x,y) is added to form the degraded image w(x,y). The main idea of this project is to have an added step to the ISET ISP to deal with the noise component at low light.
The idea is to convolve the image with the restoration procedure g(x,y) to produce the restored image z(x,y). The "Linear operation" shown in the figure is the addition or multiplication of the noise n(x,y) to the signal s(x,y).Once the corrupted image w(x,y) is obtained, it is subjected to the denoising technique to get the denoised image z(x,y).We shall compare and contrast a few denoising techniques after the discussing the characteristics of the kind of noise that corrupts the image.
While simulating the low light image the ISETCam is used to modelled the two major factors which affect the amount of noise in the image that is sensor temperature and light levels. When there is insufficient amount of light, the image majorly gets corrupted by what is called Shot noise. Photons (and photoelectrons) are quantum particles—this means that they can only come in whole numbers and the camera can never collect fractions of photons. Because of this and random statistical fluctuations, when collecting photons from an unvarying source over a set amount of time—say ten random 30 seconds intervals over the course of twenty minutes—it is highly unlikely that you will collect 20 photons every single time. Therefore this Shot noise is said to have a Poisson distribution. This noise has root mean square value proportional to square root intensity of the image. Different pixels are suffered by independent noise values. The photon noise and other sensor based noise corrupt the signal at different proportions. The PDF of the Poisson is described as $p(x) = e^-𝜆$ 𝑝(𝑥) = 𝑒 −𝜆𝜆 𝑥 𝑥! 𝑓𝑜𝑟 𝜆>0 and x=0,1,2 ...
In common there are two types of image denoising
Results
Conclusions
References
A simulation tool for evaluating digital camera image quality (2004). J. E. Farrell, F. Xiao, P. Catrysse, B. Wandell . Proc. SPIE vol. 5294, p. 124-131, Image Quality and System Performance, Miyake and Rasmussen (Eds). January 2004