Simultaneous Color Holographic Display

Introduction

A holographic display is a type of display system that produces 3D or 2D images by manipulating the wavefront of light. With the help of a spatial light modulator (SLM), a holographic display can manipulate the phase of a coherent wavefront at the pixel level. This allows it to reshape the wavefront precisely as it would originally emanate from a real object, creating an image with genuine depth cues.

Holographic displays typically use a laser as a light source, resulting in monochromatic holograms. To achieve color holograms, color holographic displays sequentially switch between RGB lasers at a high rate, leveraging the persistence of vision property of the human eye. This allows the human eye to fuse sequential monochromatic holograms into a perceived color hologram. However, this color scheme sacrifices the refresh rate of the SLM, as displaying one frame of a color target requires displaying three phase patterns, one for each RGB channel.

One potential solution to fully utilize the SLM’s refresh rate is to simultaneously activate the three primary laser lights and have the SLM modulate these three wavefronts with different wavelengths at the same time, using the same phase pattern. This approach could potentially allow for full utilization of the SLM’s refresh rate.

This project aims to investigate the effectiveness of traditional phase retrieval pipelines in this setup and explore potential improvements in reconstruction quality through the use of different loss functions.

Background

How to get light phase from intensity

Unlike conventional displays that directly control light intensity, holographic displays use a Spatial Light Modulator (SLM) to modulate the phase of light on a per-pixel basis. The modulated wavefront then propagates through free space from a starting plane $0$ to the image plane $z$. Our goal is to determine the phase pattern on the SLM such that, at the image plane $z$, the resulting intensity distribution matches a desired target intensity pattern.

Angular Spectrum Method

The Angular Spectrum Method is a computational technique used to model how wavefronts propagate through free space. It can be expressed in the following form:

$u(x,y,z,\lambda )={ℱ}^{-1}\{ℱ\{u(x,y,0,\lambda )\}\cdot \mathscr{H}(k_x,k_y,z,\lambda )\}$

$\mathscr{H}(k_x,k_y,z,\lambda )=\{\begin{array}{cc}{e}^{-\frac{i2\pi }{\lambda }\sqrt{1-(\lambda k_x{)}^2-(\lambda k_y{)}^2}z}& \text{if }\sqrt{k_x^2+k_y^2}<\frac{1}{\lambda },\\ 0& \text{otherwise}.\end{array}$

$u(x,y,0,\lambda )$ is the wavefield at the plane $z=0$. By applying the 2D Fourier transform to it, $ℱ\{u(x,y,0,\lambda )\}$, we decompose the wavefield into a superposition of plane waves traveling in various directions. This continuous distribution of plane waves is known as the angular spectrum. The spatial frequencies $k_x$ and $k_y$ determine the propagation direction of each plane wave component.

As each plane wave propagates through free space, it accumulates a distance-dependent phase shift. This phase shift is described by the transfer function $\mathscr{H}(k_x,k_y,z,\lambda )$. In the Fourier domain, by multiplying the angular spectrum $ℱ\{u(x,y,0,\lambda )\}$ by $\mathscr{H}(k_x,k_y,z,\lambda )$, we effectively propagate all plane wave components over the distance $z$.

Finally, to reconstruct the propagated wavefield at $z$, we apply the inverse Fourier transform: ${ℱ}^{-1}\{ℱ\{u(x,y,0,\lambda )\}\cdot \mathscr{H}(k_x,k_y,z,\lambda )\}$.

Image Formation Model

In our setup of the holographic display, the coherent light source that illuminates the SLM has a source field $u_{src}(x,y,\lambda )$.

The phase phase-only SLM can apply a spatially-varying delay $\varphi (x,y,\lambda )$ on the phase of the field $u_{src}(x,y,\lambda )$, so the wavefield at the SLM becomes to:

$u_{SLM}(x,y,\lambda )=e^{iq(\varphi (x,y,\lambda ))}{u}_{src}(x,y,\lambda )$

The SLM is at plane $z=0$, we can use the Angular Spectrum Method to model what the wavefield will looks like at image plane $z$.

$u_z(x,y,\lambda )=ASM(u_{SLM}(x,y,\lambda ),z)$

At the image plane, what people see is the intensity of light not the wavefield. We can get light intensity by squaring the wavefield.

$I_z(x,y,\lambda )=|u_z(x,y,\lambda ){|}^2$

In combination, the final light intensity distribution is $I_z(x,y,\lambda )=|ASM(e^{iq(\varphi (x,y,\lambda ))}{u}_{src}(x,y,\lambda ),z){|}^2$

For notational convenience, we can write the intensity pattern at the image plane $z$ as:

${𝑰}_{𝒛}\mathit{(}𝒙,𝒚,𝝀\mathit{)}\mathit{=}\mathit{|}𝑨𝑺𝑴\mathit{(}{𝒆}^{𝒊𝒒\mathit{(}𝝓\mathit{)}},𝒛\mathit{)}{\mathit{|}}^2$

This is how we can get the light intensity at the image plane $z$, by showing a phase pattern $\varphi$ at the SLM.

Iterative Method To Get Light Phase from Intensity

Now we know how to calculate light intensity from the phase pattern. However in order to show some images on the holographic display, we need to find a way to calculate phase pattern from the light intensity. People usually use gradient descent to solve this kind of inverse problem.

At iteration 0 we can generate some phase random pattern ${\varphi }^0$, using the image formation model we derive in the previous section, we can get the intensity pattern at the image plane, $|ASM(e^{iq(\varphi )},z){|}^2$ and the light wave amplitude is the square root of the intensity: $|ASM(e^{iq(\varphi )},z)|$

If the target light intensity is $I_{target}$, the target light amplitude is square root of the intensity, $a_{target}=\sqrt{I_{target}}$, we can compare $|ASM(e^{iq(\varphi )},z)|$ and $a_{target}$ with a loss function, $ℒ(|ASM(e^{iq(\varphi )},z)|,a_{target})$.

We can calculate the gradient of $\varphi$ from the loss function, $(\frac{\partial ℒ}{\partial \varphi }{)}^Tℒ(|ASM(e^{iq(\varphi )},z)|,a_{target})$ and iteratively update the value of $\varphi$, ${\varphi }^{(k)}\leftarrow {\varphi }^{(k-1)}-\alpha (\frac{\partial ℒ}{\partial \varphi }{)}^Tℒ(|ASM(e^{iq({\varphi }^{(k-1)})},z)|,a_{target})$

After enough iterations, the value of $\varphi$ should converge. In this way, we can get a phase pattern $\varphi$ that will generate the target intensity $I_{target}$ at the image plane $z$.

Simultaneous Color Scheme Related Questions

Does the laser support to turn all three color on simultaneously?

The laser we used in hardware setup is FISBA RGBeam. It has 3 diodes, and each diode emits red, green and blue light. It is possible to turn all 3 diodes on. All 3 lasers with different wavelength will pass through the same optical fiber. After passing through the collimating optics, the SLM will see a plane wave in white color.

How we can use only one phase pattern to generate three different intensity pattern?

We model how light waves propagate in free space, using the angular spectrum method.

$u(x,y,z,\lambda )={ℱ}^{-1}\{ℱ\{u(x,y,0,\lambda )\}\cdot \mathscr{H}(k_x,k_y,z,\lambda )\}$

$\mathscr{H}(k_x,k_y,z,\lambda )=\{\begin{array}{cc}{e}^{-\frac{i2\pi }{\lambda }\sqrt{1-(\lambda k_x{)}^2-(\lambda k_y{)}^2}z}& \text{if }\sqrt{k_x^2+k_y^2}<\frac{1}{\lambda },\\ 0& \text{otherwise}.\end{array}$

The transfer function in ASM is not only distance dependent but also wavelength dependent. As plane wave propagates through free space, the phase accumulation also differs depending on the wavelength of the plane wave. As a result, even the plane waves with different wavelength being applied the same phase shift at same time at the SLM. The final intensity distribution of these 3 wavelength of light are still different. This phenomenon give us some degree of freedom to use one phase pattern to match 3 different target intensity.

Methods

In the field sequential color scheme, we can generate three phase pattern to match three light intensity pattern of the RGB target. In the simultaneous color scheme we are trying to match three light intensity pattern using just one phase pattern. Even the Angular Spectrum Method indicates we can get three light intensity pattern from one phase pattern, we might not have enough degree of freedom to perfectly to match the three target intensity. There might be always some errors between the reconstruction intensity and the target intensity. A perceptual driven loss function might be useful in this case, other than just try to match wavefield amplitude, we can prioritize the matching of some visual elements that is more important to human perception.

The code base of this project is: Time-multiplexed Neural Holography: A Flexible Framework for Holographic Near-eye Displays with Fast Heavily-quantized Spatial Light Modulators. It provides a good framework to solve for phase pattern from the target intensity, and is flexible to change the loss function in the gradient descent method.

L2 Loss on light amplitude

The mean square loss, also called the L2 loss is the default loss of the code base. It prioritize minimizing the error between reconstruction amplitude and target amplitude.

$MSE=\frac{1}{n}\sum _{i=1}^n(|ASM(e^{iq(\varphi )},z)|-a_{target}{)}^2$

Using L2 loss, after 5000 iterations of gradient descent, the phase pattern, corresponding reconstruction intensity and target intensity are shown below.

CIELAB L2 loss on light intensity

From the L2 loss reconstruction, we see noticeable color shift between Reconstruction and Target images. These results indicated that using only one phase pattern, we don't have enough degree of freedom to match the target intensity.

From the class, we learnt that CIELAB is a color space that is perceptually uniform, Maybe we can convert the target intensities and reconstruction intensities in to the CIELAB color space and calculate the L2 loss in the CIELAB color space, so we can prioritize the color matching.

$CIELABLoss=MSE(RGB2LAB(|ASM(e^{iq(\varphi )},z){|}^2),RGB2LAB(I_{target}))$

In order to construct RGB2LAB function, we need to get RGB2XYZ matrix for our holographic display first. We assume the wavelength of the three primary laser are 636nm, 518nm and 441nm and power they can achieve are both 0.0035 Watts/sr/nm/m^2. The following plot is the spectral power distributions of our holographic display setup:

File:Spd holo.png.png