WeiHsuChaoTsunHanHuang

Introduction

Sensor noise analysis is one of the most important features that we need to take care of when we are testing our sensors. Today, the CMOS image sensor is widely used by many smart phones. There are some common camera (sensor) noise that exist in such type of image sensor. One is fixed pattern noise (FPN) and the other one is temporal noise. For the former one, there are two major noise we have to further explore. One is PRNU (Photo-Response Non-Uniformity) and the other one is DSNU (Dark Signal Non-Uniformity). For the temporal noise, read noise and dark noise are the two we are going to measure in this project. This project is going to give a high-level view of how to calculate and measure these values with different color channels and ISO speed. We aim to find some relationship or pattern during this project.

Background

Noise Source

Fixed Pattern Noise:
The fixed pattern noise can be referred to a particular noise pattern that we can measure it from a particular sensor during exposure of light. It can vary with different conditions under which sensors are. For example, environment temperature, image integration time and exposure time can affect fixed pattern noise. PRNU and DSNU are fixed pattern noise.

DSNU
DSNU is the variation of the "offset" that is resulted from the normal characteristic of the sensor array due to materials and substrates.
PRNU
PRNU is the variation of the "slope" resulted from the way that pixels respond to light.

Temporal Noise:
Temporal noise is usually stronger than FPN (Fixed Pattern Noise). It varies from different images. Further, it is harder to measure temporal noise than fixed pattern noise.

Read noise
Read noise is resulted from charge conversion when electrons become volts.
Dark noise (dark current rate)
Dark noise is resulted from the flow of electrons that are generated thermally within pixels.

Methods

1. PRNU and DSNU

To Estimate PRNU/DSNU, we take pictures with different ISO Speed 55/99/299. For each ISO Speed, we take pictures with 5 different exposure times at 5 different timestamps. For each ISO Speed, we corp the 25 images at the same location (200 * 200 square) and sample the values from the same color channel :

Each square has side length equaling to 200 pixels, and we sample every 2 pixels in order to get the values from a fixed color channel, so each time we will have 100 * 100 pixels to calculate the mean and standard deviation. (We use the value at the upper-right green pixel as our green channel value.)

First, we average the pixel values at the 5 different timestamp to eliminate out the effects of the temporal noises such as photon noise. Because we have 5 different exposure times, so we will have five (exposure time, average pixel value) points.

Second, we plot the pixel value as a function of exposure time and fit a straight line via regression. And we record the slope and offset.

Third, we perform unit conversion to convert the units of slope and offset (pixel_value/s and pixel_value) back to their original units (V/s and V).

From isetcam sensorGet information, the sensor's output voltage swing is 0.459 V under the analog gain 0.204. And we assume that the analog gain is linear to ISO Speed and the analog gain of ISO Speed 55 is 1. And the camera uses 10 bits to quantize the output voltage swing. So, we can convert the pixel value back to its original voltage value before quantization via the following formula :

pixel value * 0.459 / 0.204 * ISO_Speed / 55 / 1023

Forth, since we sample 100*100 pixels each time, we will have 10000 slopes and 10000 offsets. We calculate the mean and standard deviation of the 10000 slopes and the 10000 offsets respectively. (PRNU percentage = slope std / slope mean * 100 % , DSNU = offset std)

Last, we can compare PRNU/DSNU at different locations/color channels/gains(ISO Speeds), which is shown at the next section - Results.

2. Dark Noise

For dark noise estimation, we follow the same procedure as PRNU and DSNU, but we take the pictures in the dark. And we take the mean of the 10000 slopes to represent the dark current rate of the particular length-200 square.

3. Read Noise

For each ISO Speed, we use very short exposure time (near 0) and take pictures at 5 different timestamps in the dark. So, under the same ISO Speed, for each pixel, we will have 5 values (5 timestamps) and we calculate the standard deviation of the 5 values to represent the read noise of the pixel. Then we do unit conversion to convert the value back to voltage magnitude. Then we use the mean of the 10000 standard deviations to represent the read noise of the particular length-200 square.

Results

1. PRNU Result

a. Comparison of different color channels

Blue Channel has the highest PRNU percentage, while Red Channel and Green have similar PRNU percentages. (usually Blue > Red > Green)

b. Comparison of different position

(1,1) square has higher PRNU percentage than (2001, 1501) square.

c. Comparison of different ISO Speed

Higher ISO Speed leads to larger PRNU percentage.

2. DSNU Result

a. Comparison of different color channels

Green Channel has the highest DSNU, while Blue Channel has the lowest DSNU. (Green > Red > Blue)

b. Comparison of different position

(2001,1501) square has higher DSNU than (1, 1) square.

c. Comparison of different ISO Speed

Higher ISO Speed leads to larger DSNU value.

3. PRNU/DSNU Distribution

Because we are experimenting on the noise model of pixel level noise (Photon Response and Dark Signal, PR/DS), it is interesting to see the distributions after we have collected the PR/DS values of a larger number of pixels (100 * 100 = 10000). We expect that the distributions are almost bell curves and approach Gaussian Distribution with a large number of values.

Fixing ISO Speed to 299, color in green, and location at (1,1) square, we can see that the distribution of Photon Response and that of Dark Signal are similar to Gaussian Distribution :

Fixing ISO Speed to 299, we can see that different color channels have their own distributions of Photon Response. Since green channel has the highest value of photon response slope (V/s), so the rightmost distribution on the histogram is green channel's distribution. The middle distribution is red, and the leftmost distribution is blue channel's distribution, which has lowest value of photon response slope (V/s).

And in (1, 1) square, green and red channel's distributions are close to each other and almost combine together but still have two peaks. In (2001, 1501) square, green and red channel's distributions are more separated :

However, for DSNU, the distributions of the three channels are combined into one big normal distribution together and only has one single peak. There is no separations between R/G/B channels' distributions :

4. Dark Noise Result

a. Comparison of different color channels

For A2 Block (Orange bars), green channel has highest value at ISO 55 and ISO 99, but Green channel has lowest value at ISO 299. So, which color a channel belongs to does not affect the magnitude of the pixel's dark noise.

b. Comparison of different position

Fixed ISO 55, at the red channel, the dark current rate at (1, 1) square is higher than that at (2001, 1501) square. However, at the green channel, the dark current rate at (1, 1) square is much lower than that at (2001, 1501) square. So, position on the image does not affect the magnitude of dark noises.

c. Comparison of different ISO Speed

Looking into the values, we can observe that higher ISO Speed will lead to larger dark noise voltages per second after applying analog gain.

5. Read Noise Result

a. Comparison of different color channels

The read noises at red channel are similar to those at blue channel. The read noises at green channel are higher than those at red and blue ones.

b. Comparison of different position

The read noises at (1,1) square are similar to the read noises at (2001,1501) square. Position on the image does not affect the magnitude of read noises.

c. Comparison of different ISO Speed

We can see that higher ISO Speed will cause larger read noise voltages after applying analog gain.

Conclusions

For this project, we have tried to estimate four kinds of sensor noises, PRNU, DSNU, Dark Noise, and Read Noise. And we can see that different color channels, different ISO Speeds (gains),and different locations can all cause slight differences on the camera noises. In the real application, we usually use Gaussian Distribution to model the noises. Plotting the distribution of Photon Response and Dark Signal, we can see the distribution are similar to normal distribution with bell curve. So, Gaussian Distribution can be a reasonable estimation to model the camera noise.

Appendix

1. Code for this project is on Github : https://github.com/jerryhuang1995/PSYCH221-Camera-Noise-Model