Learning compact and interpretable representations of data is a critical challenge in scientific image analysis. Here, we introduce Affinity-VAE, a generative model that enables us to impose our scientific intuition about the similarity of instances in the dataset on the learned representation during training. We demonstrate the utility of the approach in the scientific domain of cryo-electron tomography (cryo-ET) where a significant current challenge is to identify similar molecules within a noisy and low contrast tomographic image volume. This task is distinct from classification in that, at inference time, it is unknown whether an instance is part of the training set or not. We trained affinity-VAE using prior knowledge of protein structure to inform the latent space. Our model is able to create rotationally-invariant, morphologically homogeneous clusters in the latent representation, with improved cluster separation compared to other approaches. It achieves competitive performance on protein classification with the added benefit of disentangling object pose, structural similarity and an interpretable latent representation. In the context of cryo-ET data, affinity-VAE captures the orientation of identified proteins in 3D which can be used as a prior for subsequent scientific experiments. Extracting physical principles from a trained network is of significant importance in scientific imaging where a ground truth training set is not always feasible.

翻译：学习数据紧凑且可解释的表征是科学图像分析中的关键挑战。本文提出亲和性变分自编码器（Affinity-VAE），这是一种生成模型，能够在训练过程中将关于数据集中实例相似性的科学先验知识融入学习到的表征中。我们在冷冻电子断层扫描（cryo-ET）这一科学领域展示了该方法的实用性，该领域当前面临的核心挑战是在噪声高、对比度低的断层扫描图像体积中识别相似分子。该任务与分类任务的区别在于，在推理阶段无法确定实例是否属于训练集。我们利用蛋白质结构的先验知识训练亲和性变分自编码器以约束潜在空间。该模型能够在潜在表征中创建旋转不变且形态均匀的聚类，其聚类分离度优于其他方法。该模型在蛋白质分类任务中取得了具有竞争力的性能，同时兼具解耦物体姿态、结构相似性以及获得可解释潜在表征的优势。在冷冻电子断层扫描数据背景下，亲和性变分自编码器能够捕获已识别蛋白质的三维取向，这可为后续科学实验提供先验信息。在难以获得真实标注训练集的科学成像领域，从训练好的网络中提取物理原理具有至关重要的意义。