Data scarcity is a significant obstacle hindering the learning of powerful machine learning models in critical healthcare applications. Data-sharing mechanisms among multiple entities (e.g., hospitals) can accelerate model training and yield more accurate predictions. Recently, approaches such as Federated Learning (FL) and Split Learning (SL) have facilitated collaboration without the need to exchange private data. In this work, we propose a framework for medical imaging classification tasks called Federated Split learning of Vision transformer with Block Sampling (FeSViBS). The FeSViBS framework builds upon the existing federated split vision transformer and introduces a block sampling module, which leverages intermediate features extracted by the Vision Transformer (ViT) at the server. This is achieved by sampling features (patch tokens) from an intermediate transformer block and distilling their information content into a pseudo class token before passing them back to the client. These pseudo class tokens serve as an effective feature augmentation strategy and enhances the generalizability of the learned model. We demonstrate the utility of our proposed method compared to other SL and FL approaches on three publicly available medical imaging datasets: HAM1000, BloodMNIST, and Fed-ISIC2019, under both IID and non-IID settings. Code: https://github.com/faresmalik/FeSViBS

翻译：数据稀缺是阻碍关键医疗应用中强大机器学习模型学习的重大障碍。多实体（如医院）间的数据共享机制能够加速模型训练并提升预测准确性。近年来，联邦学习（FL）与分割学习（SL）等方法在不交换私有数据的前提下实现了协作。本文提出了一种面向医学影像分类任务的框架——基于块采样的视觉Transformer联邦分割学习（FeSViBS）。该框架在现有联邦分割视觉Transformer基础上引入块采样模块，利用服务器端视觉Transformer（ViT）提取的中间特征。具体而言，通过从中级Transformer块中采样特征（块令牌）并将其信息内容提炼为伪类令牌后回传至客户端。这些伪类令牌作为有效的特征增强策略，提升了学习模型的泛化能力。在HAM1000、BloodMNIST和Fed-ISIC2019三组公开医学影像数据集上，我们展示了所提方法相较于其他SL和FL方法在独立同分布与非独立同分布场景下的有效性。代码：https://github.com/faresmalik/FeSViBS