Scielabfrontendoptics

Introduction

In this project, we are looking to see if the spatial extension of CIELAB (S-CIELAB) provides results that model front-end optics. We focus on comparing ∆E error maps between two images that have been obtained in two ways. The first error map is obtained by using S-CIELAB and finding ∆E between two original images. The second error map is taken by ∆E using standard CIELAB, but is found using cone responses that have been simulated by IsetBio.

Background

The standard form of CIELAB is useful for measuring the ∆E value between two uniform fields of color, but it has been found that the human eye responds to fields with fine differences in colors differently (1). Therefore, the S-CIELAB space was created to account for this. An image is turned into S-CIELAB space by first separating it into different color components, filtering each component on the eye’s sensitivity to each component, reconstructing each into XYZ space, then turning it into LAB. This spatial filtering performed is supposed to account for the human eye’s front-end optics.

Further experimentation has been done to compare the S-CIELAB ∆E metrics to visual differences in images that human observers can discern (2). While the S-CIELAB ∆E gives a better result than the standard CIELAB ∆E, it does not completely align with data provided by human targets. Thus, we would like to compare the S-CIELAB ∆E measurements with measurements given by human cone responses, and see to what extent S-CIELAB actually models front-end optics.

Methods

Experiment Design

We start by analyzing the noise that comes from the cone responses. This is necessary for us to interpret the results correctly, since we need to know which effects show up a as a result of noise and which ones show up as a result of S-CIELAB calculation or the front-end optics model (excluding noise). While the results below show the noise for an integration time of 50ms, we got rid of noise of the noise by using an integration time of 1000s for most experiments. In that case, we observed a maximum ∆E due to noise of about 0.7. We then compare two scenes of two uniformly colored squares of different colors to get information about the dependency of S-CIELAB, CIELAB, and the front-end optics model in complete absence of spatial frequencies. In the last experiment, we add the frequency component to the experiment by comparing spoke targets of different colors. Spoke targets contain spatial frequencies across a wide spectrum and are well suited to test to which extent S-CIELAB models front end optics. Figure 1 shows the general experiment setup for all three experiments.

Noise Analysis

In order to analyze the noise in the cone responses, we get look at a Macbeth Colorchecker. We record the and the MSE, the SNR for each of the three channels (L, M, S or R, G, B) separately on the demosaicked cone response image.We also compare two identical Macbeth Colorchecker scenes and calculate the CIELAB ∆E between the two cone responses coming from two identical scenes. We use the white from the bottom left corner of the color chart as a reference white for the ∆E calculations.

Uniform Squares

We experiment on 28 pairs of similarly colored squares, with a ∆E of 10 or lower between each color pair. We take 7 original colors (red, yellow, green, green-blue, blue, violet, and white), and modify each in 4 ways to get 4 different comparisons for each square. The 4 modifications we make are as follows:

1. Change just the red channel (turn 0 into 0.05, or 1 into 0.95)
2. Change just the green channel
3. Change just the blue channel
4. Change all three channels

We obtain two error maps for each pair of squares:

1. S-CIELAB ∆E between the two original images (white point = [1,1,1])
2. ∆E between demosaicked cone response images (white point = [0.97,0.88,0.18])

We also take the average XYZ value of each cone response and find the ∆E between these two to have a metric of an average ∆E value that is not affected by noise. This value is compared to the mean of the S-CIELAB ∆E error map (which is constant).

Spoke Targets

In the last experiment, we record the two error measurements (S-CIELAB between the scenes and CIELAB between cone responses) as we did in the uniform squares experiment. Two different modes of comparison are made: The first mode is a comparison of a spoke with two colors, and a spoke with the same two colors, but with a phase shift of 180° (see Figure 2.1). The second mode is a comparison between a bi-colored spoke and a spoke where one color is slightly changed (see Figure 2.2). We started out by comparing colors that have a very large ∆E, but we also compared colors that have an almost unnoticeable ∆E, shown in Figure 2.3.

Results

Noise Analysis

We found that the SNR (introduced by mainly photon / absorption noise) differs significantly depending on the color. Overall, blue cones show the fewest absorptions and the lowest SNR. We conducted the experiments for an integration time of 50ms, acknowledging that changing the integration time would only scales the photon noise by a constant factor and not yield significant additional information. Figure 5 shows the SNR for the cone responses. We labeled the Macbeth Colorchecker according to the scheme shown in Figure 4. From Figure 5 we can see that the SNR is generally higher for the L and M cones than it is for the S cones. Furthermore, the more absorptions we expect due to the color, the higher the SNR. This effect is especially visible for the white square, where all cones have high absorption rates.

We also calculated the CIELAB ∆E between two instances of cone responses that result from two identical scenes, to get a feeling for how much of a color (∆E) difference the noise is responsible for. Figure 7 and 8 show the difference for two identical scenes in the unbalanced case for the two white points, and Figure 9 shows the white-balanced case. The ∆E does not change significantly depending on the chosen reference white point. (It’s a bit lower for the second white point). However, the ∆E changes when comparing the white-balanced images. For some colors the ∆E becomes smaller. These colors are the colors that have a significant amount of short wavelengths (white, blue, violet, etc). For most colors, the ∆E becomes larger.