Objective Measures for Visually Lossless Testing of RGBG Displays
Introduction
Image compression is widely used in applications which store or transmit high resolution images with large data content. To achieve substantial reduction in bits per frame, this technique may result in lossy images reproduced after decompression. Thus, the minimization of perceivable artifacts is a key goal in implementing an effective compression algorithm, and this can be evaluated only through a correct assessment of image fidelity. In this project my goal is to evaluate the image fidelity of reconstructed images as displayed to viewers on RGBG Pentile displays.
Background
Image fidelity can be evaluated objectively or subjectively. In the subjective approach, participants with naive eyes evaluate the quality of compression techniques as presented to them in a lab-controlled environment. In contrast, the objective evaluations are independent of individual judgement. They are obtained by applying a model of display to images and measuring the artifacts in the uniform color space of CIELAB. For the quality assessment of color images, the CIELAB E metric is a commonly used tool derived from perceptual measurements of color discrimination of large uniform patches. However, CIELAB metrics are not suitable for measuring the fidelity of natural images, since many related studies found that color discrimination of human vision depends on spatial content of the image [1], [2]. The S-CIELAB metric as proposed in [3] is a spatial extension to CIELAB which incorporates sensitivities to spatial frequencies of the three opponent color channels by adding a spatial pre-processing step before the standard CIELAB E calculation. However, S-CIELAB assumes that all three subpixels are co-located which does not hold for real life displays. This project focuses on Pentile RGBG pattern which is a sophisticated subpixel layout (Fig. 2) used in AMOLED displays such as Samsung Galaxy Tab S. Hence, we need to accommodate the subpixel pattern to evaluate image fidelity in such displays compared to regular RGB stripes.
Methods
Our proposed solution is to analyze the displayed images through ISETBIO toolbox to account for the human visual system, as illustrated in the figure below. The display renders the image by low-pass filtering to minimize color aliasing, down-sampling R and B pixels, and tilting the subpixel pattern. The gamma curve of the display is also applied. The scene is modeled with the radiance emitted at each coordinate in display within the visible range over 31 wavelengths. The image is constructed onto the retina and photoreceptors by modeling the optics of the eye and chromatic aberration. After constructing the retinal image in all wavelengths, it is converted to CIE XYZ color space, and using CIELAB the compression artifacts are compared with the original image.
This technique has been recently implemented by my colleagues in Samsung Display team and the objective results were compared with the subjective test. It was concluded that using ISETBIO (on RGB and RGBG displays) eliminates the false alarms compared to the conventional S-CIELAB method (without rendering display subpixel pattern), while it increases the detection rate of flickers.In this project, the developed model was enhanced by adding the photoreceptor absorption response to retinal image before converting the image to CIE XYZ color space and using CIELAB.
To find the photoreceptor response, the eye movement has been taken into account. It has been known that eye is never still, even during fixation [1]. There are three main forms of eye movement during visual fixation: tremor, drifts, and microsaccades. Tremor is an aperiodic, wave-like motion of the eye with a frequency between 60 to 100 Hz. The tremor spatial amplitude is about the diameter of a cone. In ISETBIO, the timing and the position of tremor are modeled with gaussian and uniform random variables. Drift are superimposed on tremor and is a slow motion trajectory of the eye during the epochsof microsaccades. During drifts, the image can move across a dozen photoreceptors. In ISETBIO, drifts are assumed to be 2D Brownian motion. Microsaccades are small, jerk-like eye movements that carry the image across a range of several dozen to several hundred photoreceptors and are about 25 msec in duration. In ISETBIO, when a Microsaccades is present, tremor and drift are suppressed. In this project, all corresponding parameters to these three types of eye movement are set to default as in ISETBIO. Using these parameters, a fixed sequence comprising 20 eye positions is obtained which is illustrated below and is used for both original and reconstructed image.
Results
Here we show the obtained results for several reference original/compressed images. For each image, the error map is shown using three approaches: 1) S-CIELAB (without rendering subpixel pattern), 2) ISETBIO+CIELAB over optical image (previous work), 3) ISETBIO+CIELAB over cone absorptions (this project).
