YirongMengman

Introduction

Human pattern sensitivity has been studied for a long time.[1][2] Theories of human pattern sensitivity changed from single-resolution theories to modern multiresolution theories. Contrast sensitivity functions(CSF) are used as one of experimental measures to compare the properties of human neural mechanisms and theories. Contrast sensitivity is the inverse of contrast threhold, which is the minimum amount of contrast that a target on a uniform background must have so that human can see.

The first measurement of contrast sensitivity as a function of spatial frequency was in 1956 from Schade. Observers needed to decide what contrast was necessary to just detect the patterns. Figure 1 was his experimental result. The horizontal axis was spatial frequency measured in terms of the display. The vertical axis was contrast sensitivity, that is $log(1/c)$ where c was the contrast of the pattern of detection threshold[1][3].

Fig. 1 : CFS Experiment result from Schade

He found that contrast sensitivity decreased as the spatial frequency increased and there was no improvement of contrast sensitivity at low spatial frequencies. It was also found that the contrast sensitivity function (CSF) was a product of optical and neural factors[2]. The decrease in the high spatial frequency was due to the optical blurring of the lens and the feature of retinal ganglion cells with center-surround receptive fields[3]. As for the low frequency fall off, center surround receptive field was one possible reason. Later, some researchers in neuroscience found the existence of multiple channels in vision, each of them selective to a band of spatial frequencies[4]. This finding made more and more scientists interested in measuring the CSF.

Human vision adapts quickly to new viewing conditions. Therefore, a single CFS is not enough to describe human pattern sensitivity. By using small sinusoidal grating presented in front of observers within the middle few degrees of the visual field, factors like temporal properties, the mean background illuminance background level and the wavelength composition of the stimulus all had great influence on pattern sensitivity. Also, if the sinusoidal grating was put on peripheral locations in the visual field, sensitivity decreased. Figure 2 showed the CSF measured at retina eccentricities of 0, 1.5, 4, 7.5, 14 and 30 degrees[5]. The observers’ peak contrast sensitivity was nearly 100 for gratings near 5-8 cpd and could still resolve gratings in about 50 cpd while the limit of observers was 2 cpd when measured in the visual periphery.

Fig. 2: CSFs with different retina eccentricity

Several neural factors could explain the deduction of absolute sensitivity and spatial resolution[3]. The density of retinal ganglion cells drops and therefore there is less cortical area used to represent the periphery. Also, there are fewer sensors in periphery because the density of cone mosaic falls off quickly as a function of retina eccentricity. Besides, the photoreceptors in the fovea are much smaller than those in the periphery. This change of size may have something to do with the visual sensitivity as well.

Background

In some objective contrast sensitivity experiments, observers need to make yes or no judgement about the detection of the stimulus. According to Green and Swets[7], people say “yes” if the internal magnitude of the stimulus exceeds an internal criterion. For one thing, many things, like instructions, can cause the observers to change their criterions[8]. For another thing, observers have various critierions and criterions differ over time. Some computational methods model the stages of human visual encoding, which make computations accessible and benefit the communication across various fields. For example, ISETBio [6] allows researchers to create spectral radiance scenes as input to estimate the effects of human optics, eye movements, cone absorptions etc.

For the modelling of contrast sensitivity function, various algorithms can be used to analyze the computed visual system responses and further associate them with psychophysical performance[6]. In contrast sensitivity experiments, a computer-controlled two alternative forced choice method is often used. Contrast threshold in this case is the contrast at which the observer's response is correct on a given percentage(eg. 75%). For example, in the experiments of Jyrki et.al, the subject heard two sound signals with an interval of 0.5s and he had to decide during which signal the blank stimulus field was replaced by a vertical, stationsay grating[9]. Similar to this, Cottaris, Wandell et.al [6] modelled a two-alternative forced choice version of the contrast sensitivity experiment with ISETBio. They first demonstrated that the CSF derived from ISETBio agreed with ones derived with traditional ideal-observer approaches when all conditions were matched. An SVM classifier was used as an inference engine to link visual representations to psychological performance. The efficiency of the classifier’s calculation played an important role in the final performance.

Methods

We first use ISETBio[6] to generate experiment stimulus and corresponding cone mosaic response to it. The process and corresponding functions in ISETBio toolbox is described in the following figure.

Fig. 3 : Using ISETBio to generate harmonic stimulus and corresponding cone mosaic response

In our experiment, we generate a simple individual specific contrast sensitivity function, which has no eye movement during the test, and the cone mosaic distribution is fixed during the test.

Our goal is to use computational method to compute the threshold contrast, with which, the harmonic pattern is just barely visible. We use signal detection algorithm to automatically compute the threshold contrast from the cone mosaic response to the stimulus of different frequencies. The process is described in following figure.

Fig. 4 : Using signal detection algorithm to compute threshold contrast. 1: compute likelihood parameters; 2: compute likelihood value; 3: compare likelihood ratio and update threshold

First, we generate $N$ trials of realizations of cone mosaic response to a spatially uniform pattern (null stimulus). We view the response of each single cone as independent Poisson variables. By fitting the realizations with Poisson distribution (using poissfit function in MATLAB 2020b), we can get the Poisson parameter $\Lambda$ of each cone. We can write the distribution of the response of $M$ cones to null stimulus as:

$Poiss(\Lambda _{1}^{(0)},\Lambda _{2}^{(0)},...,\Lambda _{M}^{(0)})$.

Second, for each frequency $f$, we can start with a contrast $a$, then the distribution of the response of $M$ cones to the test stimulus can be expressed as:

$Poiss(\Lambda _{1},\Lambda _{2},...,\Lambda _{M})$.

where, by definition of harmonic pattern,

$\Lambda _{k}=\Lambda _{k}^{(0)}(1+asin(2\pi fx))$.

As we express the Poisson measurements, which is the cone mosaic response, as $\lambda =(\lambda _{1},\lambda _{2},...,\lambda _{M})$, we can later generate the likelihood function of this Poisson distribution, as:

$L(\Lambda )=\prod _{i=1}^{M}{\frac {\lambda _{i}^{\Lambda _{i}}e^{-\lambda _{i}}}{\Lambda _{i}!}}$.

To make our life easier, we can further use the corresponding loglikelihood function:

$l(\Lambda )=\sum _{i=1}^{M}(\Lambda _{i}ln(\lambda _{i})-\lambda _{i})$.

With the expression of the likelihood functions, we can further conduct likelihood ratio test method to determine whether or not the test stimulus with corresponding contrast is visible. We can use Neyman-Pearson theorem[10] and restate our task as follows:

With two Hypotheses: ${\begin{cases}{\mathcal {H}}_{0},&{\mbox{null stimulus }}\\{\mathcal {H}}_{1},&{\mbox{test stimulus with nonzero contrast}}\end{cases}}$

to maximize the probability of detection for a given false alert probability $P_{FA}=\alpha$, detect ${\mathcal {H}}_{1}$ if

${\frac {p(x;{\mathcal {H}}_{1})}{p(x;{\mathcal {H}}_{0})}}>\gamma$,

where

$P_{FA}=\int \limits _{x:L(x)>\gamma }p(x;{\mathcal {H}}_{0})\,dx=\alpha$.

In our experiment, we set $\gamma$ to be proportional to the number of test realizations ($N$ trials of realizations) and inverse proportional to the number of cones ($M$ cones).

However, to make the computational procedure as realistic as in person experiments, we cannot assume a known test stimulus with known contrast ahead. In the computational procedure, the contrast $a$ is unknown, which leads to an unknown likelihood function value $p(x;{\mathcal {H}}_{1})$, which makes our computation of likelihood ratio value impossible.

To solve this problem, we use the method of generalized likelihood ratio test (GLRT)[10], where we use maximum likelihood estimation of first estimate the contrast value ${\hat {a}}$, and plug in the estimated ${\hat {a}}$ to the likelihood function $p(x;{\mathcal {H}}_{1})$. With this procedure, we can successfully conduct the signal detection algorithm.

Later we use binary search to update the test contrast until we find the threshold contrast. By repeating the above processes, we can figure out the threshold contrast of different frequency given fixed position of retina, and generate the contrast sensitivity function curve. By changing the position of our cone mosaic in retina, we can generate different CSF curves from fovea to periphery.

Results

Method Validation

We first try different integration time of cones and figure out a proper integration time for our further experiment. The parameters we use are listed in Table 1, besides the integration time. The location of the cone mosaic is at the fovea (0 degree). We have $149*149$ cones with size $2.0*2.0um^{2}$ each. The result is plot in Figure 5.

Fig. 5: 0 degree CSF with different integration time

We can observe that the CSF curves with different integration time present similar trends and are almost parallel to each other. We would like to choose $10ms$ as our standard integration time in the further experiments.

Second, we use GRABIT[11] to obtain the CSF curves generated in [6] using ISETBio. Then we can compare the results of our method with that of the literature to validate our method. The parameters we used are listed in Table 1. We have $149*149$ cones with size $2.0*2.0um^{2}$ each. The result is presented in Figure 6. We use spline interpolation to generate the smooth curve connecting each data points.

Fig. 6: 0 degree CSF, compare with CSFs in [6], the ratio between 0 degree CSF generated from Hypothesis test and Bank's 87 ideal observer

Table. 1: General parameters in Cone Mosaic creation

We can observe that the CSF curve obtained by our Hypothesis test method has the same trend as that of the literature. However, from the ratio between 0 degree CSF generated from Hypothesis test and Bank's 87 ideal observer, we can find with increasing spatial frequency, the contrast sensitivity obtained using our method increases more than that of the literature. One reason is that in the literature, they use Gabor filter to process the test stimulus such that in each view there are same numbers of harmonic pattern cycles for different spatial frequency. In other words, for higher spatial frequency, the stimulus signal occupies less field of view, which makes it more difficult for people to detect.

Effects of Cone Distribution

After validation of our method, we start generating CSF from fovea to periphery. We use two different cone mosaic distributions with the same cone types density ratio, $L:M:S=0.6:0.3:0.1$. The cone distributions are presented in Figure 7.

Fig. 7: Cone distribution generated with random seed 1 (left) and random seed 2 (right)

We compare the CSF curves of these two different cone mosaic distributions. The general parameters are listed in Table 1. The corresponding aperture size and number of cones in each mosaic view is listed in Table 2.

Fig. 8: Contrast sensitivity function from fovea to periphery of cone distribution generated with random seed 1 (left) and random seed 2 (right)

Table. 2: Aperture size and number of cones for each degree

We observe that, for a given cone types density ratio and cone mosaic distribution, from fovea to periphery, the contrast sensitivity is consistently decreasing. At the same time, when we come to the periphery (e.g. 14 degree), the contrast sensitivity values are no longer reliable, there are more errors, as the red and green spots showed in Figure 8. It might come from the noise and undersampling problem, and the simple statistical model applied in the Hypothesis test may also fail to take some important details into consideration.

For the comparison between CSFs of different cone mosaic distribution, taking the noise into account, we observe that the difference between cone mosaic distributions has negligible affects on the CSF performance, according to our results. As the signal detection task takes all the cones' responses into account, and the contrast sensitivity is a global characteristic for a single field of view, the result is reasonable.

Effects of cone types density ratio

Next, we assess the CSF performance with different cone types density. We consider two extreme cases: an L cone only retina and an S cone only retina, as shown in Figure 9.

Fig. 9: Retina with S cone only

The experiment parameters are the same as that listed in Table 1 and Table 2. The CSF results are shown in Figure 10.

Fig. 10: CSFs from fovea to periphery of L cone only retina (left) and S cone only retina (right)

We observe that for a given cone mosaic position (e.g. 0 degree) and a given spatial frequency, the contrast sensitivity of an L cone only retina is much greater than an S cone only retina. At the same time, with increasing spatial frequency, the contrast sensitivity of an S cone only retina decreases with a much higher speed than an L cone only retina, the contrast sensitivity reaches 1 (by definition, threshold contrast is smaller than 1, so 1 is the lower limit of contrast sensitivity) even at a central position (e.g. 7.5 degree).

It is reasonable considering the chromatic aberration[12] of human eyes. The lens cannot focus short wavelength lights as well as long and middle wavelength lights, as shown in Figure 11[6], and S-cones, which is sensitive to the corresponding wavelength, as shown in Figure 12[6], can only response to the blurred signal, which makes it hard to distinguish signals from noise. At the same time, as we used a uniform light spectrum, which has small portion of lights with wavelength within the region where S cones are sensitive to, also contributes some to the poor performance of the S cone only retina. Since less incident photons result in lower signal to noise ratio according to the Poisson statistics.

Fig. 11: Line spread functions of short (left, 410 nm), middle (middle, 550 nm) and long (right, 700 nm) wavelength lights

Fig. 12: Quantal efficiency of S, M and L cones

Conclusions

We have developed a method to computationally obtain human contrast sensitivity functions from fovea to periphery with the help of ISETBio [6]. We investigated the effects of integration time on the CSFs. Higher integration time leads to higher contrast sensitivity, as we tested in the range from $5ms$ to $100ms$, since higher integration times mean more photons received by cones, which leads to higher signal to noise ratio according to the Poisson statistics.

And we also investigated the effects of cone mosaic distribution and cone type density ratio on CSFs. The results show that the density ratio between each type of cones has great effects on cone mosaic responses to test stimulus and affects the contrast sensitivity. Because of chromatic aberration, a retina with higher L and M cone densities performs much better than a retina with more S cones while less L and M cones. On the other hand, the effects of cone mosaic distribution on contrast sensitivity is negligible for a given cone type densities ratio.

Our hypothesis test based observer gives a promising result and has great potential in CSF computation, as it needs less data than machine learning methods and is much easier to explain. However, our method is affected by noise and some subtle unknown patterns, which lead to unexpected upper and lower shifting in some certain spatial frequencies. And we used an empirical likelihood ratio test threshold, which, after logarithm, is proportional to the number of test trials and inversely proportional to the number of cones. A better likelihood ratio test threshold may help solving the previous mentioned problem.

Furthermore, our experiment has not included the fixational eye movements and is individual specific. As the results show that the difference between cone mosaic distributions has negligible effects on CSF, an averaged CSF among people with fixational eye movements is desired. And a more comprehensive and complicated statistical model is required to solve this problem by means of hypothesis test.

We may conduct the above experiments in future work.

Reference

[1] Otto H. Schade, Optical and Photoelectric Analog of the Eye, J. Opt. Soc. Am. 46, 721-739 (1956)

[2] Campbell, F W, and D G Green. Optical and retinal factors affecting visual resolution. The Journal of physiology vol. 181,3 (1965): 576-93. doi:10.1113/jphysiol.1965.sp007784

[3] Wandell, Brian A. Foundations of Vision Link

[4] Campbell FW, Robson JG. Application of Fourier analysis to the visibility of gratings. J Physiol. 1968 Aug;197(3):551-66. doi: 10.1113/jphysiol.1968.sp008574. PMID: 5666169; PMCID: PMC1351748.

[5] ROVAMO, J., VIRSU, V. & NÄSÄNEN, R. Cortical magnification factor predicts the photopic contrast sensitivity of peripheral vision. Nature 271, 54–56 (1978). https://doi.org/10.1038/271054a0.

[6] Nicolas P. Cottaris, Haomiao Jiang, Xiaomao Ding, Brian A. Wandell, David H. Brainard; A computational-observer model of spatial contrast sensitivity: Effects of wave-front-based optics, cone-mosaic structure, and inference engine. Journal of Vision 2019;19(4):8. doi: 10.1167/19.4.8.

[7] Green, D. M., & Swets, J. A. (1966). Signal detection theory and psychophysics. John Wiley.

[8] Pelli DG, Bex P. Measuring contrast sensitivity. Vision Res. 2013 Sep 20;90:10-4. doi: 10.1016/j.visres.2013.04.015. Epub 2013 May 3. PMID: 23643905; PMCID: PMC3744596.

[9] Rovamo J, Leinonen L, Laurinen P, Virsu V. Temporal Integration and Contrast Sensitivity in Foveal and Peripheral Vision. Perception. 1984;13(6):665-674. doi:10.1068/p130665

[10] Kay, Steven M. Fundamentals of statistical signal processing. Prentice Hall PTR, 1993.

[11] jiro (2020). GRABIT (https://www.mathworks.com/matlabcentral/fileexchange/7173-grabit), MATLAB Central File Exchange. Retrieved November 17, 2020.

[12] https://foundationsofvision.stanford.edu/chapter-2-image-formation/#ChromaticAberration

Appendix

Code and Results file data

All the code and results file data can be download here: CSFModel.zip

Work Breakdown

Mengman Nie: Literature review

Yirong Yang: GLRT observer conduction

Presentation PPT

The contents have been updated in Wiki page, the presentation PPT is the original one. Our presentation is collaborated with Mr. Okkeun Lee.

The slides can be download here: presentationSlides