Simulate an Underwater Imaging System and Explore Water Absorption and Scattering Estimation Methods

Introduction

Motivation

There is a growing demand for high-speed wireless underwater data transmission. Optical communication provides one method for achieving this, given its inherently large bandwidth, relatively low attenuation, and ability to form secure point-to-point connections. However, underwater wireless optical communication (UWOC) is impacted significantly by distortions due to optical turbulence, caused by small changes in the refractive index of the water due to different medium conditions.

What has been done in the past

In the past, researchers have superimposed Laguerre-Gaussian (LG) beams to create an alphabet of spatially multiplexed symbols that can be encoded with information and decoded at the receiver by a Convolutional Neural Network (CNN). LG beams can be superimposed due to their orthogonality due to the fact they carry orbital angular momentum (OAM), and they have been shown to have inherent resilience to optical turbulence. Through signal processing and machine learning methods, researchers can reduce a certain amount of impact of oceanic optical turbulence on the received signal. However, the challenges in the oceanic environment motivate investigation into understanding the water medium better to further improve optical communication.

Goal

The goal of this project is to develop experimental designs aimed at characterizing a participating medium using camera captures. The absorption and scattering properties are fundamental characteristics of a participating medium such as water. Since measuring these two properties of real water or other media is difficult, In this project, we use camera simulations and virtual 3D environment techniques to test these parameters in order to create accurate models that faithfully model light interactions with such a medium, understand how they impact a medium, and hence simulate the appearance of an image captured by an underwater camera.

Background

Absorption

The absorption of light is the process by which water (or other medium) molecules absorb certain wavelengths of electromagnetic radiation, particularly light. This process takes place due to certain energy levels of light interacting with specific molecular structures of water molecules. The absorption of light by water is wavelength-dependent, and different light wavelengths can be absorbed differently. The absorption property of the medium is crucial to understanding underwater imaging and optics because it directly affects how light behaves through a media and the appearance of images captured by cameras.

Fig. 1: Visible light absorption spectrum of pure water (absorption coefficient vs. wavelength).

Scattering

The scattering of light is the process where light interacts with particles, molecules, or impurities in water (or other media), which causes it to deviate from its original path. When lights are scattered, the phenomenon affects the optical properties of water, light transmission underwater, and the appearance of images captured by cameras.

Fig. 2: Visible light scattering example in water.

Constant integration time

I am not sure what background knowledge should be mentioned in this section, add anything you think that's needed.

Methods

Experimental Setup

The experimental setup in our simulation of underwater images is shown below in Fig. 3. We have a camera that is facing the direction of the image. There are some mediums between them, such as water, fog, or plankton. When lights transfer through the medium and are captured by the camera, the light absorption and scattering of the medium take place, which causes the image captured by the camera to be distorted. In the figure, green lights (green arrows) indicate the water scattering scenario, and the black light (black arrow) indicates the water absorption scenario.

Fig. 3: The simulation experimental setup between camera, medium, and image.

MATLAB Simulation Tools

In this project, we have used three MATLAB simulation tools:

• iset3d toolbox We used iset3d toolbox in MATLAB to create simple 3D scenes.

• PBRT We Ran PBRT in MATLAB to produce physically accurate images by physically based and ray tracing through scenes.

• isetcam toolbox We explored isetcam to provide accurate camera simulation and modeling.

We ran these simulations through remote rendering on Stanford mux through Dr. David's computer.

Fig. 4: The output scene from MATLAB toolboxes. Example 1.

Fig. 5: The output scene from MATLAB toolboxes. Example 2.

Image Formation Model

After simulating, we can use the captured image to get the characters of the medium. First, we need to generate an image formation model. In general, the image formation in air and across short distances can be described with a simple linear model that integrate over wavelength $\lambda$: $$m=g\underset{\lambda }{\int }I(\lambda )r(\lambda )c(\lambda )d\lambda$$ where $m$ refers to the pixel value of the estimated image, $I_0$ is illuminant, $r$ is the scene reflectance, $c$ is the spectral response of camera, $g$ is the gain related to camera settings. However, we need to modify the equation as our model would be affected by the medium.

The light rays received by the camera consists two type of rays. The direct rays is the one originated from the light source, go through the media, then after getting reflected from the target image travel through the medium back and reaches the camera. The scattered rays get reflected towards the camera directly by the medium and do not interact with the target image.

For direct rays, light decays exponentially in the medium according to Beer-Lambert’s law. After the rays traveling distance $d$ in the medium with spectral absorption coefficient $\alpha$, the light intensity is: $$I_{d,i}=I_{0,i}{e}^{-d{\alpha }_i}$$ where $I_{0,i}$ is the original light intensity, subscript i refers to the ith waveband. From the equation, we extract the transition form T when light travel through the medium: $$T=e^{-d\alpha }$$ Therefore, the image formation model for direct rays is: $$m_{direct}=g\underset{\lambda }{\int }{I}_0(\lambda )T(\lambda )r(\lambda )T(\lambda )c(\lambda )d\lambda$$

For scattered rays, it's very complex because it involves complicate interaction with the medium. Therefore, we simplify it, combining all the effect and representing it as a single variable $P_{scatter}$. Then the model for scattered rays is: $$m_{scatter}=g\underset{\lambda }{\int }{I}_0(\lambda )P_{scatter}(\lambda )c(\lambda )d\lambda$$

We add the effect of two types of rays to generate the whole image formation model: $$m=g(\underset{\lambda }{\int }{I}_0(\lambda )T(\lambda )r(\lambda )T(\lambda )c(\lambda )d\lambda +\underset{\lambda }{\int }{I}_0(\lambda )P_{scatter}(\lambda )c(\lambda )d\lambda )$$ For computational convenience, we discretize the continuous integration into N wavebands with wavelength discretization step $\Delta \lambda$: $$m\approx g\Delta \lambda (\sum _{i=1}^N{I}_{0,i}{e}^{-2d{\alpha }_i}{r}_i{c}_i+\sum _{i=1}^N{I}_{0,i}{p}_i{c}_i)$$ and use the vectorization version in coding: $$m=g\Delta \lambda \cdot r^T\cdot Diag(I_0)\cdot Diag(e^{-2d\alpha })\cdot c+I_0^T\cdot Diag(p)\cdot c$$

If the value of $I_0,c,r,g$ is given, this equation becomes an affine function of $P_{scatter}$ and a convex, non-increasing function of $\alpha$. This character can help us solve the problem.

Algorithm

Our task is to estimate the characters of the medium, absorption $\alpha$ and scatter $P_{scatter}$, given known parameter $I_0,c,r,g$ and using the pixel values of the image captured by the camera in the simulation process as the ground truth. Firstly, we use the model above to estimate the pixel value with variables $\alpha$ and $P_{scatter}$. The estimated result should be as close to the captured value as possible. Therefore, we can extract the variable by solving the optimization problem: $$Minimiz{e}_{\alpha ,p}|\hat{m}-m{|}^{\gamma }$$ $$\text{subject to: }\alpha \ge 0$$ $$P_{scatter}\ge 0$$

where $m$ refers to the pixel values estimated by the model, $\hat{m}$ refers to the pixel values captured by the camera in simulation. The object is the norm of the difference, and the constraints are non-negativity of the spectral absorption coefficients and scatter.

Because the function m is a smooth, convex, non-increasing function of $\alpha$ and $P_{scatter}$ as mentioned before, the objective is a quasiconvex function when $\gamma \ge 1$ ( we set $\gamma =1$ in the experiment). Therefore, this problem is a convex optimization problem, which means it can be solved by the package CVXPY.

The variable $\alpha$ and $P_{scatter}$ can be found using iterative method, and the process is shown below:

1. Initialize ${\alpha }_0$;

2. Represent $m$ as its first order Taylor expansion with approximation center ${\alpha }_0$: $$m=m({\alpha }_0)+(\alpha -{\alpha }_0)\frac{\partial m}{\partial \alpha }{|}_{{\alpha }_0}$$ In this way, the function m is an affine function, which helps to make the computation easier.

3. Apply m into the minimization problem and solve it;

4. The minimizer becomes the new approximation center;

5. Repeat step(2) - (4) until it converges.

References

• Blasinski, Henryk, et al. “A model for estimating spectral properties of water from RGB images.” 2014 IEEE International Conference on Image Processing (ICIP), 2014, https://doi.org/10.1109/icip.2014.7025122.

• The iset3d GitHub Wiki's tutorial

• The isetcam GitHub Wiki's tutorial

Appendix I

我觉得我们可以学他的appendix I的写法

但我不确定你们想放哪些 我觉得我们可以放t_mediumExample221.m，mediumCreate.m,和algrism file的内容

Appendix II

Tianyun Zhao: Scheduled meetings with Henryk, communicated with the mentor and instructors, worked in person on the lab computer, worked on the MATLAB simulation parts, collected pixel data with simulation outputs, tested if our algorithm data was valid, and re-collected pixel data with algorithm outputs.

Cecilia Xie: Scheduled meetings with David, worked remotely on the lab computer, worked on MATLAB simulations, collected pixel data with simulation output figures, tested if our algorithm data was valid, and re-collected pixel data with algorithm output, compared the captured and modeled output figures and evaluated results.

Shiqi Xia: Communicated with the mentor and instructors, understood and generated the model, built the algorithm, coded based on the model and algorithm, ran and got the estimated character of medium using the simulation outputs, evaluated the results and gave possible solutions for future work.