Multispectral pedestrian detection is attractive for around-the-clock applications due to the complementary information between RGB and thermal modalities. However, current models often fail to detect pedestrians in certain cases (e.g., thermal-obscured pedestrians), particularly due to the modality bias learned from statistically biased datasets. In this paper, we investigate how to mitigate modality bias in multispectral pedestrian detection using Large Language Models (LLMs). Accordingly, we design a Multispectral Chain-of-Thought (MSCoT) prompting strategy, which prompts the LLM to perform multispectral pedestrian detection. Moreover, we propose a novel Multispectral Chain-of-Thought Detection (MSCoTDet) framework that integrates MSCoT prompting into multispectral pedestrian detection. To this end, we design a Language-driven Multi-modal Fusion (LMF) strategy that enables fusing the outputs of MSCoT prompting with the detection results of vision-based multispectral pedestrian detection models. Extensive experiments validate that MSCoTDet effectively mitigates modality biases and improves multispectral pedestrian detection.

翻译：多光谱行人检测因RGB与热成像模态间的互补信息而在全天候应用中备受关注。然而，现有模型在某些情况下（如热成像被遮挡的行人）常无法有效检测行人，这主要源于从统计偏差数据集中学习到的模态偏差。本文研究如何利用大语言模型（LLMs）缓解多光谱行人检测中的模态偏差。为此，我们设计了一种多光谱思维链（MSCoT）提示策略，引导LLM执行多光谱行人检测任务。进一步，我们提出了一种新颖的多光谱思维链检测（MSCoTDet）框架，将MSCoT提示策略整合到多光谱行人检测流程中。为实现这一目标，我们设计了语言驱动的多模态融合（LMF）策略，能够将MSCoT提示的输出与基于视觉的多光谱行人检测模型的结果进行融合。大量实验验证表明，MSCoTDet能有效缓解模态偏差并显著提升多光谱行人检测性能。