Fluorescence lifetime imaging (FLI) has been receiving increased attention in recent years as a powerful diagnostic technique in biological and medical research. However, existing FLI systems often suffer from a tradeoff between processing speed, accuracy, and robustness. In this paper, we propose a robust approach that enables fast FLI with no degradation of accuracy. The approach is based on a SPAD TCSPC system coupled to a recurrent neural network (RNN) that accurately estimates the fluorescence lifetime directly from raw timestamps without building histograms, thereby drastically reducing transfer data volumes and hardware resource utilization, thus enabling FLI acquisition at video rate. We train two variants of the RNN on a synthetic dataset and compare the results to those obtained using center-of-mass method (CMM) and least squares fitting (LS fitting). Results demonstrate that two RNN variants, gated recurrent unit (GRU) and long short-term memory (LSTM), are comparable to CMM and LS fitting in terms of accuracy, while outperforming them in background noise by a large margin. To explore the ultimate limits of the approach, we derived the Cramer-Rao lower bound of the measurement, showing that RNN yields lifetime estimations with near-optimal precision. Moreover, our FLI model, which is purely trained on synthetic datasets, works well with never-seen-before, real-world data. To demonstrate real-time operation, we have built a FLI microscope based on Piccolo, a 32x32 SPAD sensor developed in our lab. Four quantized GRU cores, capable of processing up to 4 million photons per second, are deployed on a Xilinx Kintex-7 FPGA. Powered by the GRU, the FLI setup can retrieve real-time fluorescence lifetime images at up to 10 frames per second. The proposed FLI system is promising and ideally suited for biomedical applications.

翻译：荧光寿命成像（FLI）近年来作为一种生物医学研究中的强大诊断技术受到越来越多的关注。然而，现有FLI系统通常在处理速度、准确性和鲁棒性之间存在权衡。本文提出了一种鲁棒方法，可在不降低准确性的前提下实现快速FLI。该方法基于SPAD TCSPC系统，并与递归神经网络（RNN）耦合，通过直接从未经直方图构建的原始时间戳中精确估计荧光寿命，从而大幅减少数据传输量和硬件资源消耗，实现视频帧率的FLI采集。我们在合成数据集上训练了两种RNN变体，并将结果与质心法（CMM）和最小二乘拟合（LS fitting）的结果进行比较。结果表明，门控循环单元（GRU）和长短期记忆网络（LSTM）两种RNN变体在准确性上与CMM和LS fitting相当，而在背景噪声抑制方面显著优于两者。为探索该方法的极限，我们推导了测量的Cramér-Rao下界，表明RNN能以接近最优的精度进行寿命估计。此外，仅在合成数据集上训练的FLI模型能够良好地处理从未见过的真实数据。为验证实时操作能力，我们基于实验室自研的32×32像素SPAD传感器Piccolo搭建了FLI显微镜。我们在Xilinx Kintex-7 FPGA上部署了四个量化GRU核，其处理能力高达每秒400万个光子。在GRU驱动下，该FLI装置能以每秒10帧的速率获取实时荧光寿命图像。所提出的FLI系统具有广阔应用前景，特别适合生物医学领域。