Just Recognizable Difference (JRD) boosts coding efficiency for machine vision through visibility threshold modeling, but is currently limited to a single-task scenario. To address this issue, we propose a Multi-Task JRD (MT-JRD) dataset and an Attribute-assisted MT-JRD (AMT-JRD) model for Video Coding for Machines (VCM), enhancing both prediction accuracy and coding efficiency. First, we construct a dataset comprising 27,264 JRD annotations from machines, supporting three representative tasks including object detection, instance segmentation, and keypoint detection. Secondly, we propose the AMT-JRD prediction model, which integrates Generalized Feature Extraction Module (GFEM) and Specialized Feature Extraction Module (SFEM) to facilitate joint learning across multiple tasks. Thirdly, we innovatively incorporate object attribute information into object-wise JRD prediction through the Attribute Feature Fusion Module (AFFM), which introduces prior knowledge about object size and location. This design effectively compensates for the limitations of relying solely on image features and enhances the model's capacity to represent the perceptual mechanisms of machine vision. Finally, we apply the AMT-JRD model to VCM, where the accurately predicted JRDs are applied to reduce the coding bit rate while preserving accuracy across multiple machine vision tasks. Extensive experimental results demonstrate that AMT-JRD achieves precise and robust multi-task prediction with a mean absolute error of 3.781 and error variance of 5.332 across three tasks, outperforming the state-of-the-art single-task prediction model by 6.7% and 6.3%, respectively. Coding experiments further reveal that compared to the baseline VVC and JPEG, the AMT-JRD-based VCM improves an average of 3.861% and 7.886% Bjontegaard Delta-mean Average Precision (BD-mAP), respectively.
翻译:最小可察觉差异(JRD)通过可见度阈值建模提升了机器视觉的编码效率,但目前仅局限于单任务场景。为解决这一问题,我们针对机器视频编码(VCM)提出多任务JRD(MT-JRD)数据集及属性辅助MT-JRD(AMT-JRD)模型,从而同时提升预测精度与编码效率。首先,我们构建包含27,264个来自机器的JRD标注的数据集,支持目标检测、实例分割和关键点检测三类代表性任务。其次,我们提出AMT-JRD预测模型,该模型整合了通用特征提取模块(GFEM)与专用特征提取模块(SFEM),以促进多任务联合学习。第三,我们通过属性特征融合模块(AFFM)创新性地将目标属性信息融入目标级JRD预测,该模块引入了关于目标尺寸与位置的先验知识。这一设计有效弥补了仅依赖图像特征的局限性,并增强了模型表征机器视觉感知机制的能力。最后,我们将AMT-JRD模型应用于VCM,通过精确预测的JRD降低编码比特率,同时保持多类机器视觉任务的精度。大量实验结果表明,AMT-JRD在三类任务上实现了精确且鲁棒的多任务预测,平均绝对误差为3.781,误差方差为5.332,分别比最先进的单任务预测模型提升6.7%和6.3%。编码实验进一步表明,与基准VVC和JPEG相比,基于AMT-JRD的VCM的Bjontegaard Delta-平均精度均值(BD-mAP)分别平均提升3.861%和7.886%。