PetykiewiczBuckley

Introduction

Haze is caused by the scattering of Rayleigh and Mie light by particles in the atmosphere, such as droplets of water or smoke. Restoring an image to non-hazy conditions is desirable because we like clear days. Since the amount of Rayleigh scattering increases proportional to $1/{\lambda }^4$, where $\lambda$ is the wavelength of light, for longer wavelengths there will be less scattering, and thus the image will appear less hazy at these wavelengths. Here, we investigate using NIR and red spectral data to add detail and thus dehaze images, using a modified version of the algorithm described in [1]. This algorithm takes detail from the long wavelength channel and adds it to a luminance channel to dehaze the image. Our investigation includes data from 400-1000 nm, and we investigate the possibilities of implementing a NIR filter for dehazing and of using the red camera sensor from three different commercial cameras to dehaze all three color channels. We perform a viewer study to assess the effectiveness of our dehazing. We also explain our results using spectral data and comparing to an online database[2] of reflectance spectra. We also dehaze RGB images taken from the dataset from [1] available here compare dehazing of these images with their NIR data and with the R pixel values, obtaining (to us) more attractive results with the R camera sensor.

Methods

We started with the hyperspectral panorama taken from the dish. Initially, we converted (scenes from) this data to XYZ by interpolating the XYZ curves given to us in ISET at the wavelengths at which the hyperspectral data was taken, and using these curves to weight the spectral data (see attached code). To view the image, we then converted the XYZ data to sRGB using ISET to display it. To dehaze the image, we used as a starting point the algorithm described in reference [1]. This algorithm uses a weighted least squares algorithm (described in [3]) to decompose the image into a multiscale representation with approximation and detail images of different degrees, and then compares pixel-wise the detail images from the NIR intensity image with detail images from the visible intensity image and synthesizes the approximation images with details from both the NIR and visible images. The approximation images are given by:
$I_{k+1}^a=W_{\lambda }(I_0)$
$\lambda ={\lambda }_0{c}^k$
where $I_k^a$ is the kth level approximation image (which can either be visible, $V_k^a$ or NIR, $N_k^a$, and where we implement levels from 0 to n. $\lambda$ is a measure of the coarseness of the approximation image, and ${\lambda }_0$ is for the first image. As in [1] we chose ${\lambda }_0=0,1$, c = 2 and n = 6, although we experimented with different values and obtained similar results. $I_0$ is either an intensity channel of the visible image $V_0$ or the NIR image $N_0$. We compared both a linear intensity channel and nonlinear (L*) channel for the dehazing, obtaining similar results. The NIR intensity image was also converted to an L* nonlinear scale before applying the filter in the case that the L* visible image channel was used. The function W performs the approximation operation using weighted least squares described in [2], the matlab code for implementing this function is freely available here. The detail images are differences of approximation images, as described in [1],[4], and are given by
$I_k^d=\frac{I_{k-1}^a-I_k^a}{I_k^a}$
An example of original, approximation and detail images is shown below.
The synthesis procedure is based on the observation that
$I_0=I_n^a\prod _{k=1}^n(I_k^d+1)=I_n^a\frac{I_{n-1}^a}{I_n^a}\frac{I_{n-2}^a}{I_{n-1}^a}\mathrm{...}\frac{I_0^a}{I_1^a}$
and that the NIR image has higher contrast when there is haze, and therefore we can take from the detail image whichever has a higher intensity to create the fused image $F_0$
$F_0=V_n^a\prod _{k=1}^n(max(V_k^d,N_k^d)+1)$
In practice, we found that simply replacing the visible detail with the NIR detail was not much different from this.

Spectral reflectances of selected materials

We modified this algorithm to use as the NIR data for dehazing (1) wavelength bands of hyperspectral data, from 574 nm to 974 nm (shorter than that made the image more hazy) (2) gaussian bands of hyperspectral data in the same range (3) Use input red camera sensor filter spectrum to both "read" (as opposed to the XYZ curves we initially used) the hyperspectral data and dehaze it. We also examined the hyperspectral data for spectral characteristics that would help us to determine the best wavelength range to use, and why specific wavelength ranges worked better than others. We performed a viewer study on five people, in which we displayed side by side images that we had dehazed and original images, and asked which they thought was better (or if they couldn't tell them apart). Images included were (i) dehazed with 700 nm (2 nm spectral band), without and (ii) with photoshop white balancing, dehazed with 775 nm (10 nm spectral band), (iii) dehazed with QImaging red sensor , without and (iv) with photoshop whiteness balancing, (vi) a panorama dehazed with the same 700 nm spectral band and (vii) a panorama dehazed with the QImaging red sensor. Once we had determined that we could use the red camera sensor to dehaze images, we downloaded [ images] (available online) used in the paper [1] and dehazed them using both the R camera sensor and the NIR data to compare these. Results are discussed in that section.

Results

Original	Dehazed with X-.194*Z	Dehazed with IR

Original	Dehazed with 684 nm	Dehazed with 694 nm
Dehazed with 704 nm	Dehazed with 714 nm	Dehazed with 734 nm
Dehazed with 775 nm	Dehazed with 854 nm	Dehazed with 914 nm

Z channel intensity	Y channel intensity	X channel intensity
700-775 nm intensity	775-850 nm intensity	850-920 nm intensity

Conclusions

- Describe what you learned. What worked? What didn't? Why? What would you do if you kept working on the project?

References

1. L. Schaul, C. Fredembach, and S. Süsstrunk, Color Image Dehazing using the Near-Infrared, IEEE International Conference on Image Processing, 2009.
2. Baldridge, A. M., S.J. Hook, C.I. Grove and G. Rivera, 2009.. The ASTER Spectral Library Version 2.0. Remote Sensing of Environment, vol 113, pp. 711-715
3. Z. Farbman, R. Fattal, D. Lischinski, and R. Szeliski, “Edgepreserving decompositions for multi-scale tone and detail manipulation,” International Conference on Computer Graphics and Interactive Techniques, pp. 1–10, 2008.
4. A. Toet, “Hierarchical image fusion,” Machine Vision and Applications, vol. 3, no. 1, pp. 1–11, 1990.
5. Y.Y. Schechner, S.G. Narasimhan, and S.K. Nayar, “Instant dehazing of images using polarization,” IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 325–332, 2001.
6. K. He, J. Sun, and X. Tang, “Single image haze removal using dark channel prior,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 1957–1963, 2009.
7. C. Fredembach and S. S¨usstrunk, “Colouring the near infrared,” IS&T 16th Color Imaging Conference, pp. 176–182, 2008.

Appendix I

• Viewer Study of Dehazing - The slideshow we showed in our viewer study

Appendix II

• Jan and Sonia split the work exactly down the center.