Vision-Language Models have demonstrated strong potential in medical image analysis and disease diagnosis. However, after deployment, their performance may deteriorate when the input data distribution shifts from that observed during development. Detecting such performance degradation is essential for clinical reliability, yet remains challenging for large pre-trained VLMs operating without labeled data. In this study, we investigate performance degradation detection under data shift in a state-of-the-art pathology VLM. We examine both input-level data shift and output-level prediction behavior to understand their respective roles in monitoring model reliability. To facilitate systematic analysis of input data shift, we develop DomainSAT, a lightweight toolbox with a graphical interface that integrates representative shift detection algorithms and enables intuitive exploration of data shift. Our analysis shows that while input data shift detection is effective at identifying distributional changes and providing early diagnostic signals, it does not always correspond to actual performance degradation. Motivated by this observation, we further study output-based monitoring and introduce a label-free, confidence-based degradation indicator that directly captures changes in model prediction confidence. We find that this indicator exhibits a close relationship with performance degradation and serves as an effective complement to input shift detection. Experiments on a large-scale pathology dataset for tumor classification demonstrate that combining input data shift detection and output confidence-based indicators enables more reliable detection and interpretation of performance degradation in VLMs under data shift. These findings provide a practical and complementary framework for monitoring the reliability of foundation models in digital pathology.

翻译：视觉语言模型在医学图像分析与疾病诊断中展现出巨大潜力。然而，部署后，当输入数据分布与开发阶段观察到的分布发生偏移时，其性能可能出现退化。检测此类性能退化对于临床可靠性至关重要，但对于无需标注数据即可运行的大型预训练视觉语言模型而言，这仍具挑战性。在本研究中，我们探究了在一种先进的病理学视觉语言模型中，数据偏移下的性能退化检测问题。我们同时考察了输入层的数据偏移与输出层的预测行为，以理解它们在监控模型可靠性中的各自作用。为促进对输入数据偏移的系统分析，我们开发了DomainSAT——一个具有图形界面的轻量级工具箱，它整合了代表性的偏移检测算法，并支持对数据偏移的直观探索。我们的分析表明，尽管输入数据偏移检测能有效识别分布变化并提供早期诊断信号，但它并不总是与实际性能退化相对应。受此观察启发，我们进一步研究了基于输出的监控方法，并提出了一种无需标注、基于置信度的退化指示器，该指示器可直接捕捉模型预测置信度的变化。我们发现该指示器与性能退化表现出紧密关联，可作为输入偏移检测的有效补充。在用于肿瘤分类的大规模病理数据集上的实验表明，结合输入数据偏移检测与基于输出置信度的指示器，能够更可靠地检测并解释数据偏移下视觉语言模型的性能退化。这些发现为监控数字病理学中基础模型的可靠性提供了一个实用且互补的框架。