Non-line of Sight imaging by SPAD​

Introduction

Silicon single photon avalanche detector (Si SPAD) has become a hot topic today for its single photon sensitivity, pico-second timing resolution and CMOS compatibility and low cost. Because of its unique properties, it becomes a tool to capture transient imaging for computer vision industry. One of applications enabled by transient imaging is non-line of sight problem, usually being referred to "look around the corner", where using Si SPAD, we could locate, track and recognize the shape of objects around the corner without directly seeing it.

The object of our project is to simulate Si SPAD response in non-line of sight imaging using ray tracing and verify its algorithm to track position of "hidden" object.

Background

Difference between APD and SPAD

Traditional avalanche photo detector (APD) and SPAD both use avalanche effect to amplify signal. However, they are very different in circuit configuration and performance. APD is often reversely biased to have a gain around 10-100, while SPAD is biased above breakdown voltage with a gain above $10^6$ or even up to infinity. In timing response, APD usually have a time resolution of sub nanoseconds while SPAD have a time resolution of pico-seconds. The bandwidth of APD is limited by RC delay and timing resolution. On the other hand, SPAD is constrained by dead time, where it has to rest for a while before being sensitive to next arrival photon.

Work principle of SPAD

Because of dead time, the measurement method for using SPAD is time-correlated single photon counting (TCSPC). We would like to use bio-fluorescence as a working example. In these experiments as shown in Fig 1, the fluorescence response with regards to excitation laser is often recorded. The experiment begin by repeating excitation laser pulses with many cycles and sometimes fluorophore will emit photons. For each cycle, the emitted photon may be captured by SPAD and get registered or it may be missing. Si SPAD will also take note in photon arrival time. After repeating a certain times, a histogram of photon arrival time could be drawn and this distribution will be asymptotic to fluorescence emission response, denoted by red curve in Fig 1.

Imaging model of SPAD

With SPAD work principle in mind, SPAD imaging model is built upon Poisson distribution as it comes from discrete nature of photons and SPAD triggering response (trigger or no trigger). In equations below, h is actual measurement result, $A,\mu ,\tau ,d$ are matrix in time domain. $d$ denotes dark count rate (DCR), which is the number of counts without any photon impinging on SPAD due to detector noise. $\mu \tau$ is expected response of how many photons are injected, absorbed and successfully trigger SPAD. $\mu \tau +d$ represents timing response of photon from the environment and this has to convolve with $A$, SPAD timing response itself. The final measurement results are Poisson distribution with expected tming response from $A\times (\mu \tau +d))$.

Methods

Scene introduction

There have been multiple papers published on this topic. The one we studied is “Detection and tracking of moving objects hidden from view”. It was published last year on nature Photonics. The set-up of scene is described in Fig 3. SPAD and laser are spaced 15cm apart and they both are used to locate and track an object (10cm wide and 30cm high) behind the wall.

The working principle starts by laser focus onto the floor where photons are scattered to different directions for first reflection. Some of lucky photons hit on hidden object (as in Fig 3(c) for second reflection. Among second reflection, some photons scatter to the filed of view of SPAD camera and register its time of flight for third reflection. Photons captured in SPAD field of view also has a spacial distribution that is ellipse with focal points one at first reflection location and the other one at hidden object, as illustrated in Fig 3(d).

Reconstruction algorithm

The reconstruction of object location is solely dependent on timing response captured by each SPAD. Fig 4(a) shows measured timing response from both background and from photons reflected from hidden object. By ignoring background signal (which could be done by doing calibration without hidden object behind the wall), we could select target signal associated with time $<t{>}_i$ (Fig 4(b)) and plot spatial distribution for SPAD image sensor pixels who has response at time $<t{>}_i$ in Fig 4(c). Among those pixels at point i, we could tell how far photons travel from laser to object and to point i. Then the trace of an ellipse could be drawn to show all possible positions of an object. We could then repeat this process by drawing another ellipse using pixels who has response in time $<t{>}_i$. The object position could be determined by the most overlapped regions of different ellipses.

Results

verification of reconstruction algorithm

File:Abcd.png

Example of two iterations of the meanshift algorithm

Reference

• 1 http://www.boselec.com/products/sigtcspcwhat.html
• 2 Gariepy, Genevieve, Francesco Tonolini, Robert Henderson, Jonathan Leach, and Daniele Faccio. "Detection and tracking of moving objects hidden from view." Nature Photonics (2015).