Human Object Interaction (HOI) detection aims to localize and infer the relationships between a human and an object. Arguably, training supervised models for this task from scratch presents challenges due to the performance drop over rare classes and the high computational cost and time required to handle long-tailed distributions of HOIs in complex HOI scenes in realistic settings. This observation motivates us to design an HOI detector that can be trained even with long-tailed labeled data and can leverage existing knowledge from pre-trained models. Inspired by the powerful generalization ability of the large Vision-Language Models (VLM) on classification and retrieval tasks, we propose an efficient Adaptive HOI Detector with Concept-guided Memory (ADA-CM). ADA-CM has two operating modes. The first mode makes it tunable without learning new parameters in a training-free paradigm. Its second mode incorporates an instance-aware adapter mechanism that can further efficiently boost performance if updating a lightweight set of parameters can be afforded. Our proposed method achieves competitive results with state-of-the-art on the HICO-DET and V-COCO datasets with much less training time. Code can be found at https://github.com/ltttpku/ADA-CM.

翻译：人-物交互检测旨在定位并推断人与物体之间的关系。然而，从头训练监督模型面临挑战：罕见类别性能下降，且处理现实复杂场景中人-物交互的长尾分布需要高昂计算成本和时间。为此，我们设计了一种能够利用预训练模型现有知识、甚至可用长尾标注数据训练的人-物交互检测器。受大视觉语言模型在分类与检索任务中强大泛化能力启发，我们提出了一种高效的自适性人-物交互检测器——概念引导记忆法（ADA-CM）。ADA-CM具有两种工作模式：第一种模式无需学习新参数，可在无训练范式下实现调优；第二种模式集成实例感知适配器机制，在允许更新轻量级参数集时可进一步高效提升性能。在HICO-DET和V-COCO数据集上，本方法以更少训练时间实现了与主流方法相当的竞争性结果。代码见https://github.com/ltttpku/ADA-CM。