Fluorescence lifetime imaging (FLI) is a widely used technique in the biomedical field for measuring the decay times of fluorescent molecules, providing insights into metabolic states, protein interactions, and ligand-receptor bindings. However, its broader application in fast biological processes, such as dynamic activity monitoring, and clinical use, such as in guided surgery, is limited by long data acquisition times and computationally demanding data processing. While deep learning has reduced post-processing times, time-resolved data acquisition remains a bottleneck for real-time applications. To address this, we propose a method to achieve real-time FLI using an FPGA-based hardware accelerator. Specifically, we implemented a GRU-based sequence-to-sequence (Seq2Seq) model on an FPGA board compatible with time-resolved cameras. The GRU model balances accurate processing with the resource constraints of FPGAs, which have limited DSP units and BRAM. The limited memory and computational resources on the FPGA require efficient scheduling of operations and memory allocation to deploy deep learning models for low-latency applications. We address these challenges by using STOMP, a queue-based discrete-event simulator that automates and optimizes task scheduling and memory management on hardware. By integrating a GRU-based Seq2Seq model and its compressed version, called Seq2SeqLite, generated through knowledge distillation, we were able to process multiple pixels in parallel, reducing latency compared to sequential processing. We explore various levels of parallelism to achieve an optimal balance between performance and resource utilization. Our results indicate that the proposed techniques achieved a 17.7x and 52.0x speedup over manual scheduling for the Seq2Seq model and the Seq2SeqLite model, respectively.

翻译：荧光寿命成像（FLI）是生物医学领域广泛应用的技术，用于测量荧光分子的衰减时间，从而揭示代谢状态、蛋白质相互作用以及配体-受体结合等信息。然而，该技术在快速生物过程（如动态活动监测）和临床应用（如引导手术）中的广泛应用受到数据采集时间长和计算密集型数据处理需求的限制。尽管深度学习已缩短了后处理时间，但时间分辨数据采集仍是实时应用的瓶颈。为解决这一问题，我们提出了一种基于FPGA硬件加速器实现实时FLI的方法。具体而言，我们在兼容时间分辨相机的FPGA开发板上实现了一个基于GRU的序列到序列（Seq2Seq）模型。GRU模型在FPGA有限的DSP单元和BRAM资源约束下，实现了处理精度与硬件资源的平衡。FPGA有限的内存和计算资源要求对操作调度和内存分配进行高效规划，以部署适用于低延迟应用的深度学习模型。我们通过使用STOMP（一种基于队列的离散事件模拟器）应对这些挑战，该工具可自动化并优化硬件上的任务调度与内存管理。通过集成基于GRU的Seq2Seq模型及其通过知识蒸馏生成的压缩版本（称为Seq2SeqLite），我们实现了多像素并行处理，与顺序处理相比显著降低了延迟。我们探索了不同层级的并行化方案，以在性能与资源利用率之间达到最优平衡。实验结果表明，所提技术使Seq2Seq模型和Seq2SeqLite模型相比手动调度分别实现了17.7倍和52.0倍的加速比。