ShuangLiu

1.Introduction

The modern digital cameras are far more than simply using lens to focus photons onto the onto an image pickup device. In fact, there are various processing performed by the cameras, including auto-focus, white point adjustment, color interpolation, color balancing, image compressing[1]. This project concentrates on one component of the processing pipeline in camera, the demosaicing algorithm, which is needed in every modern consumer-level camera today to render the output image as close to the actual scene as possible from the image captured by the sensor(CFA image).

To capture incident photons of different wavelengths on the optical sensor, i.e., to discriminate and measure the energy of lights of different colors, professional cameras may employ multiple (Usually, three or four) sensors covered by different selective filters to enable each sensor to detect a particular color channel. But in order to reduce cost and size, most consumer-level digital cameras use a single sensor whose surface is overlaid with color filter arrays(CFA) ,which make pixels at the sensor have different types of spectral sensitivities(generally, three or four different types). Since the reconstruction of the full color image requires each pixel must have R, G and B values, however, typically only one color is sampled at a particular spatial location, a subsequent interpolation step, which is commonly called “demosaicing” because of the mosaic pattern of the CFA, is needed for estimating the missing color values at each pixel location.

Professor Farrell et al. classify the demosaicing algorithms into three categories [2]:

(a).Single-channel interpolation:

These algorithms, such as bilinear/bicubic interpolation and spline interpolation, estimate missing color values for each of the

three color planes separately. They can be easily implemented, but they totally ignore the inter-channel correlation that can be used

to improve the demosaicing performance, for instance, reducing color fringes at sharp edges. As a result, these costeffective methods

usually introduce demosaicing artifacts such as blurring and false colors.

(b).Inter-channel interpolation:

These methods exploit the inter-channel dependency and interpolate all color planes to get the missing values .Generally, these

algorithms, such as template matching, detect edges in the CFA image first, then average the pixels along the edges rather than across

them. They tend to have better performance than those in (a), but their performance varies with the content of image and can have

large computational complexity[3].

(c).Frequency selection interpolation:

These methods exploit the characteristics of CFA image, and isolate the luminance and chrominance in spatial frequency domain by means

of filtering. These methods seem simple in theory, but the performance highly depends on the design of filters used to select spectral

components and the content of input image. As a result, incorporating adaptive filtering[4] in the original method proposed by

Alleysson and his colleagues[3], can significantly improve the performance of the algorithm[4][1]. Besides, the trade-off between

color rendering and spatial resolution is obvious in these method, as shown in [7], preserving high resolution detains indicating poor

color accuracy and accurate color rendering means false luminance patterns.

In general speaking, the demosaicing algorithms require a luminance and chrominance decomposition. Usually, the luminance should contain most of the spatial information in CPA image, so it looks just like a gray-scale rendering of the scene, and should not be applied to the following convolution steps. On other hand, since there’s little spatial information in chrominance, we can average the values of the neighborhoods to get a good color rendering[5].In this project, I first explore the characteristics of CFA image proposed by Alleysson et al. in [3][6], then simulate the general procedure of frequency-selection demosaicing algorithm, in the end briefly discuss several issues encountered in the interpolating step: the choice of CFA pattern, the effect of the filter and the measures used in the comparison of different demosaicing algorithm.

2.Methods

2.1 Luminance and chrominance model

Let I represent the natural full-color image at the spatial position k=(x,y):

I(k)=I(x,y)={Ir(k),Ig(k),Ib(k)}={Ii(k),i∈{r,g,b}}

Let Im represent the CFA mosaic image at the spatial position k=(x,y):

Im(k)=Im(x,y)={Imr(k),Img(k),Imb(k)}={Imi(k),i∈{r,g,b}}

Let mi represent the sampling functions of the CFA pattern:

If we employ Bayer rggb CFA pattern(which is also true for the following simulation), the sampling functions can be expressed simply as follows[4]:

Then the CFA image Im can be expressed by the full-color image I and the sampling functions mi as:

As in [3][4][6], we define lum as the luminance component of the CFA image and chr1+chr2 as the chrominance component of the CFA image. Clearly by this definition, the luminance doesn’t depend on the sampling function, which means it preserves a part of information of the color image I exactly, so we can also call lum the luminance of the color image. But the corresponding chrominance part of the CFA image, chr, is different from that of the original image. Similarly, we can define the chrominance of the original color image as:

chri=chr1i+chr2i=I-lum=0.75 Ir(k)+ 0.5 Ig(k)+ 0.75 Ib(k)= chrir (k)+ chrig (k)+ chrib(k); (3)