Multiple Instance Learning (MIL) addresses problems where supervision is available at the level of bags of instances and has been successfully applied in fields ranging from computational pathology to satellite imagery. Nevertheless, existing algorithms struggle in the low-label regime that characterizes many real-world applications. Flexible models overfit and rigid ones fail to adapt to the task at hand. We show that pretraining an in-context learner with a Perceiver-style architecture on synthetic data yields a model that can solve new tasks from a handful of labeled bags. At inference time, classification happens in a single forward pass and requires no gradient updates. We propose and investigate different synthetic data generators for bag-structured data and find that they capture complementary inductive biases. A model pretrained on a mixture of these generators inherits their per-task strengths and achieves the best average performance across twelve MIL benchmarks, outperforming supervised baselines that require task-specific training.

翻译：多实例学习（MIL）解决的是以实例包为单位提供监督信号的问题，已成功应用于计算病理学、卫星图像等多个领域。然而，现有算法在处理许多实际应用中常见的低标签场景时陷入困境：灵活模型容易过拟合，而刚性模型则难以适应具体任务。我们证明，采用感知机式架构的上下文学习器在合成数据上进行预训练后，能够仅凭少量标注包解决新任务。推理时，分类过程仅需一次前向传播，无需梯度更新。我们提出并研究了多种适用于包结构数据的合成数据生成器，发现它们能够捕获互补的归纳偏置。在混合生成器上预训练的模型继承了各生成器的任务适应优势，在十二个MIL基准测试中取得了最优平均性能，超越了需要特定任务训练的监督基线方法。