The locations of different mRNA molecules can be revealed by multiplexed in situ RNA detection. By assigning detected mRNA molecules to individual cells, it is possible to identify many different cell types in parallel. This in turn enables investigation of the spatial cellular architecture in tissue, which is crucial for furthering our understanding of biological processes and diseases. However, cell typing typically depends on the segmentation of cell nuclei, which is often done based on images of a DNA stain, such as DAPI. Limiting cell definition to a nuclear stain makes it fundamentally difficult to determine accurate cell borders, and thereby also difficult to assign mRNA molecules to the correct cell. As such, we have developed a computational tool that segments cells solely based on the local composition of mRNA molecules. First, a small neural network is trained to compute attractive and repulsive edges between pairs of mRNA molecules. The signed graph is then partitioned by a mutex watershed into components corresponding to different cells. We evaluated our method on two publicly available datasets and compared it against the current state-of-the-art and older baselines. We conclude that combining neural networks with combinatorial optimization is a promising approach for cell segmentation of in situ transcriptomics data.

翻译：通过多重原位RNA检测技术可揭示不同mRNA分子的空间位置。将检测到的mRNA分子分配到单个细胞后，能够同步识别多种细胞类型，进而探究组织中的空间细胞结构，这对深入理解生物过程与疾病机制至关重要。然而，细胞分型通常依赖于细胞核分割——这一过程往往基于DNA染色（如DAPI）图像进行。将细胞定义局限于核染色从根本上难以确定精确的细胞边界，从而难以将mRNA分子正确分配至对应细胞。为此，我们开发了一种仅基于mRNA分子局部组成进行细胞分割的计算工具。首先训练小型神经网络计算mRNA分子对间的引力边与斥力边，随后通过互斥分水岭算法将符号图划分为对应不同细胞的连通分量。我们在两个公开数据集上评估了该方法，并与当前最优方法及早期基线方法进行了比较。结果表明，将神经网络与组合优化相结合是处理原位转录组学数据细胞分割问题的有前景策略。