Joelle Dowling
Introduction
Color Calibration is one piece of simulating the image processing pipeline. Without color calibration, or the camera sensor's spectral sensitivity, we would not know if we are accurately simulating the colors of the sensor image.
In past work, accurate simulated spectral quantum efficiencies were achieved. However, the data used to generate this model is tedious and costly to gather. Previously, the data would be collected by illuminating a surface with many different monochromatic lights. Monochromatic lights are expensive and retaking measurements for each one takes a lot of time. In this project, the data is collected by illuminating the surface with 3 different illuminants. Having this simpler data allows for quicker and cheaper data generation.
The purpose of this project is to model the spectral quantum efficiency (QE) of a camera sensor. Since we do not know the spectral transmittance of the optics and filters in the camera, we take all the wavelength dependency to lie in the sensor. Accurately modelling this can help in validation on other projects.
Background
Spectral Quantum Efficiency relates the surface reflectance measurements and the sensor response. We'll let n be the number of sampled wavelengths and m be the number of patches on the MCC. Let the surface reflectance measurements be in a n x m matrix, M. Let the sensor response, which is the RGB values, be in a m x 3 matrix, R. And let the spectral quantum efficiency be the n x 3 matrix, S. We can relate these matrices by the following linear equation:
Methods
The setup of this project has two parts. The first is to generate a model for the Google Pixel 4A's camera sensor's spectral quantum efficiency. We do this using the
This project multiple methods to improve the spectral quantum efficiency model. To initially solve for the spectral quantum efficiency matrix, the linear equation explained in the previous section was used. However, doing this alone is not sufficient to get an accurate model. This is the case, in part, because the radiance data was generated by measuring the radiance of MCC. Our data technically has 72 samples (24 per MCC x 3 illuminants), but the patches are not independent. Singular value decomposition (SVD) can be used to obtain the principal components of our samples, which results in less than 10 independent measurements. Since we are trying to get the spectral QE information for 31 wavelengths (400:10:700 nm), we are heavily under-sampled.
Furthermore, simply solving the linear equation will create an overfitted model. We can use the results of the SVD to make a new basis and represent the spectral QE as a weighted sum of the new basis. Lastly, if we believe that the camera's spectral QE is similar to Sony's characterization of the