Scene reconstruction in the presence of high-speed motion and low illumination is important in many applications such as augmented and virtual reality, drone navigation, and autonomous robotics. Traditional motion estimation techniques fail in such conditions, suffering from too much blur in the presence of high-speed motion and strong noise in low-light conditions. Single-photon cameras have recently emerged as a promising technology capable of capturing hundreds of thousands of photon frames per second thanks to their high speed and extreme sensitivity. Unfortunately, traditional computer vision techniques are not well suited for dealing with the binary-valued photon data captured by these cameras because these are corrupted by extreme Poisson noise. Here we present a method capable of estimating extreme scene motion under challenging conditions, such as low light or high dynamic range, from a sequence of high-speed image frames such as those captured by a single-photon camera. Our method relies on iteratively improving a motion estimate by grouping and aggregating frames after-the-fact, in a stratified manner. We demonstrate the creation of high-quality panoramas under fast motion and extremely low light, and super-resolution results using a custom single-photon camera prototype. For code and supplemental material see our $\href{https://wisionlab.com/project/panoramas-from-photons/}{\text{project webpage}}$.

翻译：在高速运动与低照度条件下进行场景重建，对于增强现实/虚拟现实、无人机导航及自主机器人等众多应用至关重要。传统运动估计技术在此类条件下会失效，原因在于高速运动导致图像过度模糊，而低光照环境则伴随强烈噪声。单光子相机凭借其超高速度与极端灵敏度，近期成为能够每秒捕获数十万帧光子图像的突破性技术。然而，传统计算机视觉技术难以有效处理这类相机捕获的二进制光子数据——这些数据受极端泊松噪声污染。本文提出一种方法，可在低光照或高动态范围等严峻条件下，通过单光子相机捕获的高速图像序列实现极端场景运动估计。该方法采用分层策略，通过事后分组聚合帧数据迭代优化运动估计。我们展示了在快速运动与极低光照条件下生成高质量全景图的能力，并通过定制单光子相机原型实现超分辨率重建。代码与补充材料详见项目网页$\href{https://wisionlab.com/project/panoramas-from-photons/}{\text{https://wisionlab.com/project/panoramas-from-photons/}}$。