Mobile Multi-Bounce LiDAR

Abstract

Single-photon avalanche diodes (SPADs) are emerging sensors that can measure the propagation of light in a scene, capturing higher-order reflections, shadows, and light transport that ordinary cameras are unable to. Measurement of these multi-bounce light paths is especially useful for non-line-of-sight (NLOS) imaging. The increasing availability of SPAD sensors on mobile devices (e.g. iPhone Pro LiDAR) raises the potential to enable NLOS capabilities on consumer devices in the future. Currently, these sensors are primarily employed for LiDAR-based depth estimation, with untapped potential in other applications. In light of recent advances in SPAD device development, the timing is opportune to revisit the applicability of multi-bounce LiDAR techniques on consumer-grade mobile devices.

This thesis extends the applicability of multi-bounce LiDAR techniques from research-grade SPAD hardware to consumer-grade mobile LiDARs. First, we enable single-shot capture of two-bounce signals and remove the need for laser scanning by developing a tomographic formulation for two-bounce non-line-of-sight imaging. Second, we enable real-time non-line-of-sight capture at eye-safe laser power under object and camera motion. Our approach is inspired by principles from burst photography.

We implement and evaluate the proposed algorithms in simulations and on experimental SPAD hardware. We also demonstrate real-time non-line-of-sight tracking on a consumer-grade smartphone LiDAR. Potential future applications of our results include "X-ray vision" in AR/VR, full-body tracking for AR headsets, room scanning for hard-to-reach areas, collision avoidance for autonomous vehicles, and robotic navigation.