WuYang
Psf analysis and image deblurring using a simulated camera lens http://white.stanford.edu/teach/index.php/Psych221_Project_Suggestions#Psf_analysis_and_image_deblurring_using_a_simulated_camera_lens Tony Wu, Samuel Yang
Back to Psych 221 Projects 2014
Background
What is known from the literature.
Methods
Linear Model for Digital Photography
Design of an Underwater Color Rig
Handheld
PVC Rig
PVC With Index-Compensated Geometry
Calibrating the Underwater Color Rig
Data-Collection Dives
Results
Macbeth/Xrite Color Checker Reflectances
Canon SX260HS Responsivities
Underwater Color Rig Data
85ft Dive Number 1 (with Salp capture on return!)
85ft Dive Number 2 (with plastic bottle for fun)
Extracting Color Patch RGB Information
As the rig descends the patches take on a green tint and the red patches especially lose their intensity and appear a muted gray color. A rendering of the color checker values at different depths is shown below.
Light is quickly attenuated by the water column as the rig descends. A plot of RGB color values throughout a dive is shown below to illustrate the effect. The RGB values correspond to the white color patch on the bottom left of the color target.
Pixel Saturation
During the RGB pixel extraction process, color patches that experienced saturation were flagged as unusable for illuminant estimation. Of the 24 patches on the Xrite color target, only 7 were usable. A map of the usable color patches is shown below.
Underwater Spectra
With known camera color responsivities, known color target reflectances, and a set of extracted RGB pixel data the underwater scene illuminant can be estimated. Rearranging the linear model for color digital photography, we get the following system for each camera photo:
The color patch reflectivities in the model have been merged via element-by-element multiplication to form a single combined spectral response for each color patch. There are a total of 7 usable color targets within each image and 3 pixel color values per target resulting in a total of 21 linear equations usable for estimating the illuminant. There are a total of 51 spectral bands in the responsivity matrix of the color filter array and consequently 51 unknown values in the illuminant estimate. This results in an underdetermined system. To solve the system, an additional smoothness constraint was applied by developing the following optimization objective function, inspired by the multispectral estimation techniques of Park et. al [4].
The system was solved via non-negative least squares optimization in MATLAB using the lsqnonneg function. Since natural illuminants tend to be smooth and are guaranteed to be non-negative this is a reasonable constraint. An estimated, normalized spectrum from a depth of 65 feet is shown below.
By sorting the estimated spectra by depth an intensity map of illuminants can be created, as shown below. Depth in feet is shown along the y-axis and wavelength is shown in nanometers along the x-axis. Illuminant intensity is shown in log scale. Note the dramatic attenuation of light with depth and the spectral peak near 532nm.
Conclusions and Future Work
The proof-of-concept experiment reveals future improvements and experiments for the system: The color target chosen must be analyzed explicitly to confirm the color patch spectra match those in the ISET toolbox. The acrylic technique used to shield the color card must be refined to suppress reflections. A higher resolution setting on the PR-650 colorimeter should be used to refine the color filter array characterization of the digital camera. A shorter exposure setting should be used during data-collection dives to prevent pixel saturation. Finally, more illuminant data should be collected at both the Hopkins Marine Station site and other locations during different seasons to determine the geographic and temporal diversity of the seawater absorption characteristic.
Despite design challenges, however, the underwater color rig was successfully able to characterize the changing underwater illumination with depth using relatively inexpensive equipment. By careful selection and characterization of both digital camera and color target a sufficient basis was established to generate a true multispectral estimate of the underwater illuminant spectrum. The techniques described in this report can be used to process images captured and estimate illumination at depth in a variety of applications from digital photography color balancing to monitoring marine plankton populations or estimating the health of a coral population. Hopefully the design of this inexpensive system will enable further underwater multispectral exploration.
References - Resources and related work
References
[1] Coral Watch Project http://www.coralwatch.org/
[2] Raimondo Schettini and Silvia Corchs, "Underwater Image Processing: State of the Art of Restoration and Image Enhancement Methods" EURASIP Journal on Advances in Signal Processing, Volume 2010, Article ID 746052, 14 pages
[3] Brian A. Wandell, Foundations of Vision, Chapter 9. https://www.stanford.edu/group/vista/cgi-bin/FOV/chapter-9-color/#Linear_Models
[4] Jong-Il Park, Moon-Hyun Lee, Michael D. Grossberg, and Shree K. Nayar "Multispectral Imaging Using Multiplexed Illumination" Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on
Software
Image Systems Engineering Toolbox http://imageval.com/
Canon Hack Development Kit http://chdk.wikia.com/wiki/CHDK
Appendix I - Code and Data
In the belief that the techniques used may be illustrated best by example, the MATLAB code used to perform the multispectral analysis and calibration is available below along with sample data from the project.
Code
All code was written in MATLAB R2012a for Mac OSX, Mountain Lion. External dependencies include the MATLAB Image Processing Toolkit, MATLAB Optimization Toolkit, and the ISET toolkit.
Data
A total of 2.5GB of image data were collected for this project, and are available upon request. The extracted RGB color patch data, colorimeter responses, and SX260HS responsivity data are available below.
File:CanonSX260HSResponsivity.zip
Presentation
This project was given as a 5-minute presentation to the PSYCH221 Winter 2013 class at Stanford. The presentation files used are linked below.