Directly predicting human epidermal growth factor receptor 2 (HER2) status from widely available hematoxylin and eosin (HE)-stained whole slide images (WSIs) can reduce technical costs and expedite treatment selection. Accurately predicting HER2 requires large collections of multi-site WSIs. Federated learning enables collaborative training of these WSIs without gigabyte-size WSIs transportation and data privacy concerns. However, federated learning encounters challenges in addressing label imbalance in multi-site WSIs from the real world. Moreover, existing WSI classification methods cannot simultaneously exploit local context information and long-range dependencies in the site-end feature representation of federated learning. To address these issues, we present a point transformer with federated learning for multi-site HER2 status prediction from HE-stained WSIs. Our approach incorporates two novel designs. We propose a dynamic label distribution strategy and an auxiliary classifier, which helps to establish a well-initialized model and mitigate label distribution variations across sites. Additionally, we propose a farthest cosine sampling based on cosine distance. It can sample the most distinctive features and capture the long-range dependencies. Extensive experiments and analysis show that our method achieves state-of-the-art performance at four sites with a total of 2687 WSIs. Furthermore, we demonstrate that our model can generalize to two unseen sites with 229 WSIs.

翻译：从广泛可用的苏木精-伊红（HE）染色全切片图像（WSI）中直接预测人表皮生长因子受体2（HER2）状态，可降低技术成本并加速治疗方案选择。准确预测HER2需要大量多中心WSI数据集。联邦学习能够在无需传输吉字节级WSI数据和避免数据隐私问题的前提下，实现对这些WSI的协同训练。然而，联邦学习在应对真实世界中多中心WSI的标签不平衡问题时面临挑战。此外，现有WSI分类方法无法在联邦学习的站点端特征表示中同时利用局部上下文信息和长程依赖关系。为解决这些问题，我们提出一种融合联邦学习的点变换器，用于从HE染色WSI中预测多中心HER2状态。我们的方法包含两项创新设计：首先提出动态标签分布策略与辅助分类器，有助于建立初始化的优质模型并缓解各站点间的标签分布差异；其次提出基于余弦距离的最远余弦采样方法，能够采样最具判别性的特征并捕获长程依赖关系。大量实验与分析表明，我们的方法在包含2687张WSI的四个站点上取得了最优性能。此外，我们证明该模型可泛化至包含229张WSI的两个未见站点。