Recent advances in machine learning have shown that Reinforcement Learning from Human Feedback (RLHF) can improve machine learning models and align them with human preferences. Although very successful for Large Language Models (LLMs), these advancements have not had a comparable impact in research for autonomous vehicles -- where alignment with human expectations can be imperative. In this paper, we propose to adapt similar RL-based methods to unsupervised object discovery, i.e. learning to detect objects from LiDAR points without any training labels. Instead of labels, we use simple heuristics to mimic human feedback. More explicitly, we combine multiple heuristics into a simple reward function that positively correlates its score with bounding box accuracy, \ie, boxes containing objects are scored higher than those without. We start from the detector's own predictions to explore the space and reinforce boxes with high rewards through gradient updates. Empirically, we demonstrate that our approach is not only more accurate, but also orders of magnitudes faster to train compared to prior works on object discovery.

翻译：近期机器学习领域的进展表明，从人类反馈中进行的强化学习（RLHF）可以改进机器学习模型并使其与人类偏好对齐。尽管该方法在大型语言模型（LLMs）中取得了巨大成功，但这些进展尚未对自动驾驶研究产生同等影响——而在该领域，与人类期望对齐至关重要。本文提出将类似的基于强化学习的方法适配至无监督目标发现任务，即无需任何训练标签，仅从激光雷达点云中学习检测目标。我们采用简单启发式方法替代人工标签以模拟人类反馈。具体而言，我们将多种启发式方法整合为单一奖励函数，其评分与边界框精度正相关，例如包含目标的边界框得分显著高于无目标框。我们从检测器自身预测出发探索目标空间，并通过梯度更新强化高奖励边界框。实验表明，与以往目标发现研究工作相比，本方法不仅精度更高，训练速度更是提升数个数量级。