Ademola-IdowuKhwajaGhosh
Analysis of Real Camera Lenses - Atinuke Ademola-Idowu, Ayesha Khwaja, Pallabi Ghosh
Introduction
Image blurring can occur as a result of external or internal factors. External factors include motion blur, defocus or different image field depth and height. Internal factors include light diffraction, lens aberration, sensor resolution and anti-aliasing filter. The external factors are majorly user caused and can be corrected by necessary adjustments but the internal factors cannot be readily corrected. Therefore in order to characterize these intrinsic factors which are camera unique, the Point Spread Function has to be obtained.
Point Spread function of an imaging system can be described as the image of any point object that was captured by the camera system. Due to inbuilt factors, the point will be blurred and appear as a blob, circular or elliptical based on the point's location. So point spread function is the 2D impulse response of the system. Our algorithm aims at estimating the PSF for a given imaging system.
Background
Literature Survey
There have been several methods used by
The first seeks to
The second seeks to
Method
Overview
We use a method similar to the one followed by Delbracio et. all [1] to compute the average PSF of various parts of the camera lens. The main idea is to take into account all possible external factors that make the captured scene as close as possible to the displayed scene so that the only difference between them is that the displayed scene has been blurred by the camera's PSF. To do this, first the pattern position and its illumination conditions are estimated to take into account for the geometric and radiometric correction. Now, the PSF can be computed by inverting a linear system.
Set-up
In order to obtain the PSF of the camera lens, a test patch arranged in a 3x5 array was displayed on a monitor and captured using a Nikon D2Xs camera. We did this for different exposure values in order to determine how the PSF varies with exposure time.
We displayed the following pattern on the monitor:
The following was the scene the camera captured:
Next we try and estimate the pattern position and illumination conditions as discussed in the next section.
Radiometric Correction
A white point displayed on the scene does not appear to be white on the captured image as can be seen from Figure 2 above. Further more, the monitor makes these white points darker at the edges than at the middle of the screen. To take this into account, we use a scene of a white background captured with the same settings as were used to capture the test pattern. The idea is that each pixel is affected equally due to the above mentioned reasons in both the images and hence if we take the ratio of the intensity values of each pixel, we would get an image which is free from illumination.
The following image is obtained after we apply for radiometric correction to our captured scene:
Geometrical Registration
As can be seen above from Figure 1 and 3, the plane of the display and that of the digital pattern are not parallel to each other. To compute the PSF, we want these two planes to be parallel to each other. The plane of the display in the captured scene of figure 3 can be made parallel to the plane of the digital pattern by calculating the image homography.
The homography is an invertible transformation between two spaces, where there is one-to-one correspondence between each point of the two spaces. Let P be a point (in homogeneous coordinate space) in space 1 and P' be its corresponding point in space 2. Then, homography matrix, H, relates P and P' as P = HP'. H is a 3x3 matrix with 8 degrees of freedom and hence can be determined given the positions of four points in a plane, and the positions of these four points in another plane. To get these total of 8 points, we use manual corner detection.
Once we have the homography matrix H, we transform each point in figure 3 by pre-multiplying it with H. The result we obtain is a geometrically registered image as shown in the figure below.
PSF Estimation
We now have a illumination corrected and geometrically registered image, which defers from the test pattern in only the sense that the test pattern has been blurred by the camera PSF. This is a linear system and hence the camera PSF can be found out by inverting this linear system. We do this in the Fourier domain.
Let D be the Digital pattern, C be the illumination corrected and geometrically registered captured image and PSF be camera PSF. Then
Results
Using the algorithm as described above the PSF of the camera system was calculated as shown in the figure below:
Next we also magnify each block so that the nature of the PSF is visible more clearly, and show the results in the next figure.
We see that there is a lot of noise in the estimated PSF. This noise can be attributed to the erroneous selection of points while trying to remove perspective distortion. The process was manual, and could have been prone to human error. Although we tried to remove this error through multiple selections of points, it improved the results slightly, but didn’t remove it totally. That is the reason why, selection of different regions while doing the division of Fourier transforms, gave different erroneous regions in the final PSF estimation.
Testing for the validity of estimated PSF
To check whether our PSF estimation is correct, we generated 5 patterns, as shown below
We captured the images of these 5 patterns using 3 different camera settings with 3 different f numbers. We took the Fourier transform of each pattern and multiplied it with the Fourier Transform of the PSF. Then we took the inverse Fourier transform and compared it to the captured images. The following figure shows the results for the radial pattern. The first column is the pattern, the second column shows captured images at F# 5.6,18 and 34 and the third column shows the corresponding convolved images.
Theoretically they should be equivalent, but due to a number of factors they are not. For example the exposure duration of the captured image of the test pattern is different from the exposure duration of the captured image from which the PSF is actually computed. The exposure duration of the test pattern is larger, hence it is brighter and more blurry.
But there are some factors which are really similar in the two sets. For example, the middle column consisting of captured image, has the 1st image as the least blurry. The blurriness increases as we go down the column. The same observation can be made in the last column as well. This shows the similarity in their nature, although due to factors mentioned above they look really different.
Conclusions
The project led us to 3 main concluding points
- The results in figure 3 shows that the central block has a symmetric PSF, whereas it is assymetric in the corner blocks and bent toward the corners. That is the characteristics of any PSF. So our results seem to be correct.
- Our algorithm uses the white image to do illumination control, hence it is robust to illumination changes
- But on the other hand our PSF estimation is noisy which causes the huge difference in captured and convolved images in figure 5. We would try to reduce the noise in the future. Including the camera characteristics in the calculation might reduce the noise. Also better method of selection of points to remove perspective distortion can be useful in noise reduction.
References
[1] Mauricio Delbracio, Pablo Musé and Andrés Almansa. Non-parametric Sub-pixel Local Point Spread Function Estimation, Image Processing On Line,vol. 2012, pp. 8–21.
[2] Johannes Brauers, Claude Seiler and Til Aach. Direct PSF Estimation Using a Random Noise Target, Germany, Digital Photography, volume 7537 of SPIE Proceedings, page 75370. SPIE, (2010)
[3] Felix Heide, Mushfiqur Rouf, Matthias B. Hullin, Bjorn Labitzke, Wolfgang Heidrich, Andreas Kolb. High-Quality Computational Imaging through SImple Lenses.
Appendix A - Source codes and results
Source codes, results and images used, along with a README for this project can be downloaded from here.
Appendix B - Breakdown of Work
Atinuke: Error analysis, Reading raw camera data
Ayesha: Geometric and Radiometric correction on captured scene; Testing generated scenes with the computed PSF
Pallabi: Computing patchwise PSF, Generating different scenes for testing
Atinuke, Ayesha, Pallabi: Literature survey, Result analysis, Conclusions, Wiki page and Presentation Slides