MeganckSajdakWu

Predicting Human Performance Using ISETBIO. Ryan Meganck, Adam Sajdak, Stephen Wu.

Introduction

Motivating Examples

Modern displays have benefited heavily from the technological advances surrounding the manufacture and design of transistors. Display designers have been able to package an increasing amount of transistors in a given display to yield stunning images while also improving the energy efficiency of the displays. In a vacuum, the goal of these display designers would be to strive for an infinite number of pixels in a display, but in all pipelines, there is a bottleneck for performance. In this case, the pipeline consists not only of display, but the observer watching the display. The observer is limited by non-idealities in the eye as well as the image processing portion of the brain.

At very low display resolutions, it is expected that a human observer would be able to notice an increase resolution or pixel count. In this case, the display resolution is the limiting factor in the pipeline. Conversely at high resolutions, there is a point where the resolution of the display is no longer the limiting factor and the human observer can no longer resolve higher resolutions.

The purpose of this project is to determine the critical point for display performance in the visual pipeline. In other words, to find the critical resolution at various viewing distances where the observer is no longer able to discern two different images.

Vernier Acuity

The metric used in this project for predicting an observer's ability to resolve an image is Vernier Acuity. This metric describes an observer's ability to discern the alignment of two line segments. The two images below are examples of the two scenes used throughout the project. As labeled, the first scene is an aligned line segment and the second is misaligned. As the display resolution is increased, it becomes more difficult to discern if the segment is aligned. The observer was shown 1,000 images per simulation (500 aligned, 500 misaligned) and the computer algorithm would try to classify them as aligned or misaligned. A 75% classification accuracy would indicate that the human visual system was not the limiting factor in the visual pipeline.

Aligned Scene

Misaligned Scene

Image Pipeline Noise Sources

In order to simulate the effect of the human visual system on the visual pipeline, several physical non-idealities and noise sources are added. These noise sources make it much more difficult for the computer algorithm (and the human brain) to accurately classify the images.

One source of noise is the point-spread function (PSF) and blurring associated with the lens of the human eye. The lens is not ideal and has a point-spread function that dictates how light will be diffused as it passes through the lens. Furthermore, chromatic aberrations cause the diffusion to depend on the color or wavelength of the light. The image below shows a scene (from above) that has passed through the human lens. As shown, the line on the retinal image is much thicker than the original scene due to the blurring and it is much more difficult to tell that the image is misaligned.

Retinal Image of Misaligned Scene

Another source of noise in the image pipeline is fixation eye movement. Fixational eye movement the involuntary movement of the eyes when trying to focus on a single point. While these movements are typically very short, they affect the integration of photons on photoreceptors in the eye. In the context of classification, the eye movements add noise to the photon data and make it more difficult to accurately classify a scene.

The last source of noise is photon noise. This noise arises from the discrete nature of photons. Even in a fully lit scene, a given photoreceptor will receive a slightly different number of photons than a neighboring pixel during a given integration time. The end result is a noise source superimposed with the ideal photon count to produce the measured photon data.

Methods

The general procedure used for both methods in this experiment is:

1. Generate two scenes (one aligned, one misaligned).
2. Obtain cone absorption data for these scenes.
3. Perform additional processing on the data if necessary.
4. Train the system using the cone absorption data.
5. Obtain a new set of cone absorptions for the scenes.
6. Use the system to predict the classification of the test cone absorptions.
7. Compute prediction accuracy and other metrics.

Data

Training and test scenes were generated in MATLAB with the aid of a tutorial script provided by course staff. A human eye and display are modeled, and the retinal image is computed by applying functions which integrate the incident photons and approximate the noise generated by aberrations in the path from image to retina. This also includes noise induced by fixational eye movement.

The feature vector for the experiment is the vector of cone absorptions. The length of this vector varies with FOV, image size, viewing distance, and screen resolution (TODO), since not all cones in the eye are affected by the scene.

1-Nearest Neighbor

The k-nearest neighbors algorithm classifies a test point by observing the k training examples closest to the point and chooses the output label of the plurality. In this case, k = 1. The distance between two images was measured using the euclidean distance (2-norm) on vectorized retinal images. The images were vectorized so that each pixel yielded a unique dimension.

Support Vector Machine

One approach uses a support vector machine to classify test images. Given a training dataset, this is achieved by calculating the separating hyperplane which yields the largest functional margin between the two classes. Test inputs are then classified based on which side of the hyperplane they fall on.

The particular flavor of SVM chosen for this investigation is C-support vector classification (C-SVC) and the kernel trick was employed using a radial basis function (RBF) kernel to allow for interpretation of higher-dimensional features aside from the individual cone absorptions. This method required the tuning of two parameters:

C-SVC cost parameter

Controls the number of misclassified examples allowed when processing the training data to maximize the margin. This allows the system to ignore outliers if they exist.

RBF $\gamma$ parameter

Controls the scale of the kernel function.

Since these parameters are relatively independent, an exhaustive search across parameter pairs was deemed unnecessary. Instead each parameter was varied along a single dimension on a logarithmic scale and the values generating the highest prediction accuracy were chosen.

Experiments

Experiment 1: Determine the Critical PPI

Scope

The purpose of the first experiment was to determine the PPI where a human observer can start to discern misalignment in two line segments. A classification accuracy of 75% was used to signify an ability to accurately evaluate an image. The classification accuracy was plotted with respect to PPI for three different viewing distances: 0.5m, 1.0m, and 2.0m. In order to account for the discrete nature of a simulation, a simulation with 500 samples was run 20 times at a given PPI and the classification accuracy was averaged over these 20 runs.

Conclusions

Further Investigation

• Application of principal component analysis to feature vectors
• Neural network
• Modeling myopia and hyperopia
• Comparing displays
• Contrast
• anything else?

References

Paper paper paper

Appendix I

Source code Result images

Appendix II: Work Breakdown

Ryan Meganck

• 1-NN implementation
• PCA implementation

Adam Sajdak

• Testing framework

Stephen Wu

• SVM implementation and parameter tuning

Shared Responsibilities

• Wiki page
• Data analysis