In this work we present denoiSplit, a method to tackle a new analysis task, i.e. the challenge of joint semantic image splitting and unsupervised denoising. This dual approach has important applications in fluorescence microscopy, where semantic image splitting has important applications but noise does generally hinder the downstream analysis of image content. Image splitting involves dissecting an image into its distinguishable semantic structures. We show that the current state-of-the-art method for this task struggles in the presence of image noise, inadvertently also distributing the noise across the predicted outputs. The method we present here can deal with image noise by integrating an unsupervised denoising sub-task. This integration results in improved semantic image unmixing, even in the presence of notable and realistic levels of imaging noise. A key innovation in denoiSplit is the use of specifically formulated noise models and the suitable adjustment of KL-divergence loss for the high-dimensional hierarchical latent space we are training. We showcase the performance of denoiSplit across 4 tasks on real-world microscopy images. Additionally, we perform qualitative and quantitative evaluations and compare results to existing benchmarks, demonstrating the effectiveness of using denoiSplit: a single Variational Splitting Encoder-Decoder (VSE) Network using two suitable noise models to jointly perform semantic splitting and denoising.

翻译：本文提出denoiSplit方法，旨在解决一项新的分析任务——联合语义图像分割与无监督去噪的挑战。该双任务方法在荧光显微成像中具有重要应用：语义图像分割在该领域具有关键价值，但噪声通常会阻碍图像内容的后续分析。图像分割涉及将图像解构为其可区分的语义结构。研究表明，当前最先进的图像分割方法在噪声干扰下性能受限，会无意中将噪声扩散至各预测输出。本文提出的方法通过集成无监督去噪子任务有效处理图像噪声，即便面对显著且真实的成像噪声水平，也能提升语义图像分离效果。denoiSplit的核心创新在于：采用特定公式化噪声模型，并针对所训练的高维层级隐空间合理调整KL散度损失函数。我们在四类真实显微图像任务中验证了denoiSplit的性能，同时通过定性与定量评估，将其结果与现有基准进行对比，证明了该方法的有效性：即采用双适配噪声模型的单一变分分割编码器-解码器网络（VSE），可联合执行语义分割与去噪任务。