LiYuXie

Introduction

The International Standards Organization(ISO) has a set of camera image quality metrics to quantify resolution, noise, color accuracy of cameras. However, due to different content in various images and distinct preference of different people, the image quality estimated by even the most commonly used metrics might not be coincident with the perceived image quality. As a result, We need to evaluate the metrics by comparing the metrics with the appearance of these images as they are rendered on a display.

In this project, we focus on the effectiveness of metrics for evaluating the noise level of different kinds of images by comparing results generated from metrics and observations of human. With Image Systems Engineering Toolbox(ISET), we calculate the metric for simulated camera to evaluate visible noise of a camera – vSNR. vSNR is the ratio of signal power to noise power, calculated from XYZ value of the processed image. By varying luminance, pixel size, and optical/sensor parameters, we see how the vSNR and images would change so that we evaluate whether the image quality can coincide with human observation or not for different scenes, e.g. human faces, texts, landscapes.

Background

Image Quality Metrics

Nowadays, a lot of metrics are used to perceive image quality, which can be found in the paper Development of the I3A CPIQ spatial metrics[1]. Their goals are to predict the visible differences between a pair of images. These metrics are commonly used to measure the important aspects of image quality such as sharpness, noise, contrast and so on.

visible SNR

Signal-to-noise ratio(SNR)is a measure that compares the level of a signal to the level of background noise. It is defined as the ratio of signal power to the noise power. The SNR used in imaging is a physical measure of the sensitivity of a imaging system.

The vSNR is the inverse of the standard deviation of the SCIELAB representation of a uniform field; its units are 1/ΔE[2].The vSNR metric can be used in simulations to predict how imaging system components affect noise visibility[2].

Methods

Software

Matlab 7.14

Algorithms

To evaluate the effectiveness of vSNR for measuring image quality, we investigate vSNR effectiveness with different parameters for processing different type of scenes. We use ISET to calculate metrics for different parameters (e.g. luminance, pixel size, F-number). We also use ISET to simulate how camera captures and processes different type of scenes (e.g. faces, text and landscapes). For each condition, we compare the metrics with the appearance of these images as they are rendered on a display. We use control variate method to measure and compare the influence of each variable.

Experimental Procedure

Using the script s_Psych221_winter2014_vSNR.m, change the parameters and see what happens to the image displayed and vSNR. Summarize a trend of vSNR effectiveness with different parameters. The focus is to evaluate the effectiveness of vSNR measuring the noise level and overall image quality.

(1) Luminance
(i) For human face,
Control Fov=5, f#=5, viewD=0.2, pixel=[0.5,1,2,4]. Change light level, lightLevels = [5,10,100,200]

(ii) For text,
Control Fov=5, f#=5, viewD=0.2, pixel=[0.5,1,2,4]. Change light level, lightLevels = [5,10,100,200]

(iii) For landscape,
Control Fov=5, f#=5, viewD=0.2, pixel=[0.5,1,2,4]. Change light level, lightLevels = [5,10,100,200]

(2) Pixel Size
(i) For human face,
Control light level = [5,10,100,200], Fov=5, f#=5, viewD=0.2. Change pixel = [0.5, 1, 2, 4].

(ii) For text,
Control light level = [5,10,100,200], Fov=5, f#=5, viewD=0.2. Change pixel = [0.5, 1, 2, 4].

(iii) For landscape,
Control light level = [5,10,100,200], Fov=5, f#=5, viewD=0.2. Change pixel = [0.5, 1, 2, 4].

(3) F-number
(i) For human face,
Control Fov=5, viewD=0.2, light level = [50, 100, 200]. Change fnumber = 3.
Control Fov=5, viewD=0.2, light level = [50, 100, 200]. Change fnumber = 5.
Control Fov=5, viewD=0.2, light level = [50, 100, 200]. Change fnumber = 8.

(ii) For text,
Control Fov=5, viewD=0.2, light level = [50, 100, 200]. Change fnumber = 3.
Control Fov=5, viewD=0.2, light level = [50, 100, 200]. Change fnumber = 5.
Control Fov=5, viewD=0.2, light level = [50, 100, 200]. Change fnumber = 8.

(iii) For landscape,
Control Fov=5, viewD=0.2, light level = [50, 100, 200]. Change fnumber = 3.
Control Fov=5, viewD=0.2, light level = [50, 100, 200]. Change fnumber = 5.
Control Fov=5, viewD=0.2, light level = [50, 100, 200]. Change fnumber = 8.

Results

Luminance

Human Face

Fov=5, f#=5, viewD=0.2
Horizontally (from left to right): lightLevels = [5,10,100,200]
Vertically (from top to bottom): pixel = [0.5,1,2,4]

From the image displayed, we could see that image quality in terms of noise is getting better with increasing light level in each pixel size condition. We could also see that overall image quality is getting better with increasing light level in each pixel size condition.

vSNR plot:

From the above plot we could see that vSNR increases with increasing light level in each pixel size condition. Thus, we could conclude that vSNR can effectively reflect the noise level and overall image quality of human face, in terms of light level.

Text

Fov=5, f#=5, viewD=0.2
Horizontally (from left to right): lightLevels = [5,10,100,200]
Vertically (from top to bottom): pixel = [0.5,1,2,4]

From the image displayed, we could see that image quality is getting better with increasing light level in each pixel size condition. We could also see that overall image quality is getting better with increasing light level in each pixel size condition.

vSNR plot:

From the above plot we could see that vSNR increases with increasing light level in each pixel size condition. Thus, we could conclude that vSNR can effectively reflect the noise level and overall image quality of text image, in terms of light level.

Landscape

Fov=5, f#=5, viewD=0.2
Horizontally (from left to right): lightLevels = [5,10,100,200]
Vertically (from top to bottom): pixel = [0.5,1,2,4]

From the image displayed, we could see that image quality is getting better with increasing light level in each pixel size condition. We could also see that overall image quality is getting better with increasing light level in each pixel size condition.

vSNR plot:

From the above plot we could see that vSNR increases with increasing light level in each pixel size condition. Thus, we could conclude that vSNR can effectively reflect the noise level and overall image quality of landscape image, in terms of light level.

Pixel Size

Human Face

Fov=5, f#=5, viewD=0.2
Horizontally (from left to right): lightLevels = [5,10,100,200]
Vertically (from top to bottom): pixel = [0.5,1,2,4]

From the image displayed, we could see that image quality in terms of noise is getting better with increasing pixel size in each light level condition. However, the overall image quality from the feeling of human eyes might not be the best at the largest pixel size. Human eyes may sense the best overall image quality at a proper pixel size (might be 1 in this image condition).

vSNR plot:

From the above plot we could see that vSNR increases with increasing pixel size in each light level condition. Thus, we could conclude that vSNR can effectively reflect the noise level of human face, in terms of pixel size. Considering the overall image quality, more resolution factors need to be considered.

Text

Fov=5, f#=5, viewD=0.2
Horizontally (from left to right): lightLevels = [5,10,100,200]
Vertically (from top to bottom): pixel = [0.5,1,2,4]

From the image displayed, we could see that image quality in terms of noise is getting better with increasing pixel size in each light level condition. However, the overall image quality from the feeling of human eyes might not be the best at the largest pixel size. Human eyes may sense the best overall image quality at a proper pixel size (might be 1 in this image condition).

vSNR plot:

From the above plot we could see that vSNR increases with increasing pixel size in each light level condition. Thus, we could conclude that vSNR can effectively reflect the noise level of text image, in terms of pixel size. Considering the overall image quality, more resolution factors need to be considered.

Landscape

Fov=5, f#=5, viewD=0.2
Horizontally (from left to right): lightLevels = [5,10,100,200]
Vertically (from top to bottom): pixel = [0.5,1,2,4]

From the image displayed, we could see that image quality in terms of noise is getting better with increasing pixel size in each light level condition. However, the overall image quality from the feeling of human eyes might not be the best at the largest pixel size. Human eyes may sense the best overall image quality at a proper pixel size (might be 1 in this image condition).

vSNR plot:

From the above plot we could see that vSNR increases with increasing pixel size in each light level condition. Thus, we could conclude that vSNR can effectively reflect the noise level of landscape image, in terms of pixel size. Considering the overall image quality, more resolution factors need to be considered.

F-number

Human Face

Fov=5, viewD=0.2, pixel = 1
Horizontally (from left to right): lightLevels = [50,100,200]
Vertically (from top to bottom): fnumber = [3, 5, 8]

From the image displayed, we could see that image quality in terms of noise is better when F-number is smaller. We could also see that overall image quality is better when F-number is smaller.

vSNR Plot: (from top to bottom, fnumber = 3, 5, 8)

From the above plots we could see that vSNR is higher when F-number is smaller. Thus, we could conclude that vSNR can effectively reflect the noise level and overall image quality of human face image, in terms of F-number.

Text

Fov=5, viewD=0.2, pixel = 1
Horizontally (from left to right): lightLevels = [50,100,200]
Vertically (from top to bottom): fnumber = [3, 5, 8]

From the image displayed, we could see that image quality in terms of noise is better when F-number is smaller. We could also see that overall image quality is better when F-number is smaller.

vSNR Plot: (from top to bottom, fnumber = 3, 5, 8)

From the above plots we could see that vSNR is higher when F-number is smaller. Thus, we could conclude that vSNR can effectively reflect the noise level and overall image quality of text image, in terms of F-number.

Landscape

Fov=5, viewD=0.2, pixel = 1
Horizontally (from left to right): lightLevels = [50,100,200]
Vertically (from top to bottom): fnumber = [3, 5, 8]

From the image displayed, we could see that image quality in terms of noise is better when F-number is smaller. We could also see that overall image quality is better when F-number is smaller.

vSNR Plot: (from top to bottom, fnumber = 3, 5, 8)

From the above plots we could see that vSNR is higher when F-number is smaller. Thus, we could conclude that vSNR can effectively reflect the noise level and overall image quality of landscape image, in terms of F-number.

Conclusions

Luminance--For all these three kinds of images, we can see from the displayed image that with the increase of light level, image quality in terms of noise is getting better, which is coincident with what is shown in the vSNR plot. With the increase of light level, the overall image quality is getting better. Thus, we could conclude that this camera image quality metric can effectively reflect the noise level and overall image quality in terms of light level.

Pixel Size--For all these three kinds of images, we can see from the displayed image that with the increase of pixel size, image quality in terms of noise is getting better, which is also indicated in the vSNR plots. However, the overall image quality from human observation might not be the best at the largest pixel size. Due to the influence of resolution factor, human eyes may sense the best overall image quality at a specific pixel size. Thus, we could conclude that this camera image quality metric can effectively reflect the noise level of the images in terms of pixel size, but it does not coincide with human feelings, because of the influence of resolution.

F-number--We can reach a conclusion from the displayed images that the smaller F-number is, the better image quality in terms of noise would be. This is true for all these three kinds of images. When F-number decreases, we can also have better overall image quality. This conclusion could be confirmed by the vSNR plots. In a word, this camera image quality metric could effectively reflect the noise level and overall image quality in terms of F-number, despite of the differences of image kinds.

To sum up, in order to get better image quality, we need to consider the resolution tradeoffs with sensitivity. When pixel size is larger, it will increase the amount of photons a pixel can capture, so the vSNR will be higher. That is, the image quality in terms of noise level will be better. However, larger pixels will decrease the number of sensor pixels sampling the optical irradiance image. This might lead to a low resolution image.

Which to choose, a high resolution noisy image or a low resolution clear image? This depends on many factors such as image type, the display, view distance, and so on. Some image like human faces, we might choose a relatively larger pixel size (higher vSNR) since the noise will affect the image quality remarkably. Some image like text, we might choose a relatively smaller pixel size because we need to identify the words and the resolution is more important.

References

[1] Baxter, Donald, et al. "Development of the I3A CPIQ spatial metrics." IS&T/SPIE Electronic Imaging. International Society for Optics and Photonics, 2012.

[2] J. Farrell et al., ”Using visible SNR (vSNR) to compare the image quality of pixel binning and digital resizing”, Proc SPIE 7537, 75370C (2010).

We appreciate guidance and help from Professor Farrell.

Appendix I

s_Psych221_winter2014_vSNR.m [1]
You could change the parameters (eg: light level, pixel size, fnumber) in the "Tuning the camera parameters" part.

Appendix II

Yaoxuan Li: paper study, analyze and generate scripts, do evaluation tests, create presentation PPT
Huijie Yu: paper study, analyze and generate scripts, do evaluation tests, create final wiki page
Xinyi Xie: paper study, analyze and generate scripts, do evaluation tests, create final wiki page