We present uSplit, a dedicated approach for trained image decomposition in the context of fluorescence microscopy images. We find that best results using regular deep architectures are achieved when large image patches are used during training, making memory consumption the limiting factor to further improving performance. We therefore introduce lateral contextualization (LC), a memory efficient way to train powerful networks and show that LC leads to consistent and significant improvements on the task at hand. We integrate LC with U-Nets, Hierarchical AEs, and Hierarchical VAEs, for which we formulate a modified ELBO loss. Additionally, LC enables training deeper hierarchical models than otherwise possible and, interestingly, helps to reduce tiling artefacts that are inherently impossible to avoid when using tiled VAE predictions. We apply uSplit to five decomposition tasks, one on a synthetic dataset, four others derived from real microscopy data. LC achieves SOTA results (average improvements to the best baseline of 2.36 dB PSNR), while simultaneously requiring considerably less GPU memory.

翻译：我们提出uSplit，一种面向荧光显微图像中训练型图像分解的专用方法。研究发现，使用常规深度架构时，采用大尺寸图像块进行训练才能获得最佳结果，这使得内存消耗成为进一步提升性能的瓶颈。为此，我们引入侧向上下文化（LC）这一高效内存训练方法，证明LC能在当前任务中带来一致且显著的性能提升。我们将LC与U-Net、层次化AE及层次化VAE（为其设计了改进型ELBO损失）进行集成。此外，LC使得训练更深的层次化模型成为可能，并有助于减少使用分块VAE预测时固有的拼接伪影。我们将uSplit应用于五个分解任务（一个基于合成数据集，其余四个源自真实显微数据）。LC在显著降低GPU内存占用的同时，实现了SOTA性能（相比最佳基线平均提升2.36 dB PSNR）。