We present uSplit, a dedicated approach for trained image decomposition in the context of fluorescence microscopy images. We find that best results using regular deep architectures are achieved when large image patches are used during training, making memory consumption the limiting factor to further improving performance. We therefore introduce lateral contextualization (LC), a memory efficient way to train powerful networks and show that LC leads to consistent and significant improvements on the task at hand. We integrate LC with U-Nets, Hierarchical AEs, and Hierarchical VAEs, for which we formulate a modified ELBO loss. Additionally, LC enables training deeper hierarchical models than otherwise possible and, interestingly, helps to reduce tiling artefacts that are inherently impossible to avoid when using tiled VAE predictions. We apply uSplit to five decomposition tasks, one on a synthetic dataset, four others derived from real microscopy data. LC achieves SOTA results (average improvements to the best baseline of 2.36 dB PSNR), while simultaneously requiring considerably less GPU memory.

翻译：我们提出uSplit，一种针对荧光显微图像训练图像分解的专门方法。研究发现，使用常规深度架构时，采用大尺寸图像块进行训练可取得最优结果，这使得内存消耗成为制约性能进一步提升的关键因素。为此，我们引入侧向情境化（LC）这一内存高效训练方法，证明其在该任务上能带来持续且显著的性能提升。我们将LC与U-Net、层次化自编码器及层次化变分自编码器进行集成，并针对后者设计了改进型ELBO损失函数。值得关注的是，LC不仅支持训练比常规方法更深的层次化模型，还能有效抑制采用分块变分自编码器预测时不可避免的拼接伪影。我们使用uSplit在五个图像分解任务上开展实验（包括一个合成数据集及四个源自真实显微数据的数据集）。LC方法在显著降低GPU内存占用的同时，实现了最优结果（较最佳基线平均提升2.36 dB PSNR）。