Vision-language models (VLMs) have recently shown remarkable performance in navigation and localization tasks by leveraging large-scale pretraining for semantic understanding. However, applying VLMs to 6-DoF endoscopic camera localization presents several challenges: 1) the lack of large-scale, high-quality, densely annotated, and localization-oriented vision-language datasets in real-world medical settings; 2) limited capability for fine-grained pose regression; and 3) high computational latency when extracting temporal features from past frames. To address these issues, we first construct BREATH dataset, the largest in-vivo endoscopic localization dataset to date, collected in the complex human airway. Building on this dataset, we propose BREATH-VL, a hybrid framework that integrates semantic cues from VLMs with geometric information from vision-based registration methods for accurate 6-DoF pose estimation. Our motivation lies in the complementary strengths of both approaches: VLMs offer generalizable semantic understanding, while registration methods provide precise geometric alignment. To further enhance the VLM's ability to capture temporal context, we introduce a lightweight context-learning mechanism that encodes motion history as linguistic prompts, enabling efficient temporal reasoning without expensive video-level computation. Extensive experiments demonstrate that the vision-language module delivers robust semantic localization in challenging surgical scenes. Building on this, our BREATH-VL outperforms state-of-the-art vision-only localization methods in both accuracy and generalization, reducing translational error by 25.5% compared with the best-performing baseline, while achieving competitive computational latency.

翻译：视觉-语言模型（VLMs）最近通过利用大规模预训练进行语义理解，在导航与定位任务中展现出卓越性能。然而，将VLMs应用于六自由度内窥镜相机定位面临若干挑战：1）在真实医疗场景中，缺乏大规模、高质量、密集标注且面向定位的视觉-语言数据集；2）细粒度姿态回归能力有限；3）从历史帧提取时序特征时计算延迟较高。为解决这些问题，我们首先构建了BREATH数据集，这是迄今为止在复杂人体气道中采集的最大规模体内内窥镜定位数据集。基于此数据集，我们提出BREATH-VL，一个混合框架，它整合了来自VLMs的语义线索与基于视觉的配准方法提供的几何信息，以实现精确的六自由度姿态估计。我们的动机源于两种方法的互补优势：VLMs提供可泛化的语义理解，而配准方法则提供精确的几何对齐。为进一步增强VLM捕捉时序上下文的能力，我们引入了一种轻量级上下文学习机制，将运动历史编码为语言提示，从而在不进行昂贵视频级计算的情况下实现高效的时序推理。大量实验表明，该视觉-语言模块在具有挑战性的手术场景中提供了鲁棒的语义定位。在此基础上，我们的BREATH-VL在精度与泛化性上均优于最先进的纯视觉定位方法，与性能最佳的基线相比，平移误差降低了25.5%，同时实现了具有竞争力的计算延迟。