Directly predicting human epidermal growth factor receptor 2 (HER2) status from widely available hematoxylin and eosin (HE)-stained whole slide images (WSIs) can reduce technical costs and expedite treatment selection. Accurately predicting HER2 requires large collections of multi-site WSIs. Federated learning enables collaborative training of these WSIs without gigabyte-size WSIs transportation and data privacy concerns. However, federated learning encounters challenges in addressing label imbalance in multi-site WSIs from the real world. Moreover, existing WSI classification methods cannot simultaneously exploit local context information and long-range dependencies in the site-end feature representation of federated learning. To address these issues, we present a point transformer with federated learning for multi-site HER2 status prediction from HE-stained WSIs. Our approach incorporates two novel designs. We propose a dynamic label distribution strategy and an auxiliary classifier, which helps to establish a well-initialized model and mitigate label distribution variations across sites. Additionally, we propose a farthest cosine sampling based on cosine distance. It can sample the most distinctive features and capture the long-range dependencies. Extensive experiments and analysis show that our method achieves state-of-the-art performance at four sites with a total of 2687 WSIs. Furthermore, we demonstrate that our model can generalize to two unseen sites with 229 WSIs.

翻译：直接利用广泛可用的苏木精-伊红(HE)染色全切片图像(WSI)预测人表皮生长因子受体2(HER2)状态，可降低技术成本并加快治疗方案选择。准确预测HER2需要采集大量多中心WSI。联邦学习能够在无需传输吉字节级WSI且无需担忧数据隐私的情况下，实现这些WSI的协同训练。然而，联邦学习在应对真实世界中多中心WSI的标签不平衡问题时面临挑战。此外，现有WSI分类方法无法同时在联邦学习的站点端特征表示中利用局部上下文信息与长程依赖关系。为解决上述问题，我们提出了一种基于联邦学习的点Transformer方法，用于从HE染色WSI预测多中心HER2状态。本方法包含两项创新设计：我们提出动态标签分布策略与辅助分类器，有助于建立良好初始化的模型并缓解不同站点间的标签分布差异；我们还提出基于余弦距离的最远余弦采样算法，可采样最具判别性的特征并捕获长程依赖关系。大量实验与分析表明，本方法在包含共计2687张WSI的四个中心站点上取得了最先进性能。此外，我们证明该模型可泛化至包含229张WSI的两个未参与训练的站点。