Despite continual learning's long and well-established academic history, its application in real-world scenarios remains rather limited. This paper contends that this gap is attributable to a misalignment between the actual challenges of continual learning and the evaluation protocols in use, rendering proposed solutions ineffective for addressing the complexities of real-world setups. We validate our hypothesis and assess progress to date, using a new 3D semantic segmentation benchmark, OCL-3DSS. We investigate various continual learning schemes from the literature by utilizing more realistic protocols that necessitate online and continual learning for dynamic, real-world scenarios (eg., in robotics and 3D vision applications). The outcomes are sobering: all considered methods perform poorly, significantly deviating from the upper bound of joint offline training. This raises questions about the applicability of existing methods in realistic settings. Our paper aims to initiate a paradigm shift, advocating for the adoption of continual learning methods through new experimental protocols that better emulate real-world conditions to facilitate breakthroughs in the field.

翻译：尽管持续学习在学术界有着悠久而深厚的历史，其在实际场景中的应用仍然相当有限。本文认为，这一差距源于持续学习的实际挑战与当前评估协议之间的不匹配，导致现有解决方案无法有效应对现实环境中的复杂性。我们通过一个新的3D语义分割基准OCL-3DSS验证了这一假设，并评估了迄今的进展。我们利用更逼真的协议（例如在机器人和3D视觉应用中），研究了文献中多种持续学习方案，这些方案需要在线和持续学习以应对动态的现实世界场景。结果令人警醒：所有考虑的方法表现均不佳，显著偏离了联合离线训练的上界。这引发了关于现有方法在实际环境中适用性的质疑。本文旨在推动范式转变，倡导通过能更好模拟现实条件的新实验协议来采用持续学习方法，从而促进该领域的突破。