Evolutionary search-based techniques are commonly used for testing autonomous robotic systems. However, these approaches often rely on computationally expensive simulator-based models for test scenario evaluation. To improve the computational efficiency of the search-based testing, we propose augmenting the evolutionary search (ES) with a reinforcement learning (RL) agent trained using surrogate rewards derived from domain knowledge. In our approach, known as RIGAA (Reinforcement learning Informed Genetic Algorithm for Autonomous systems testing), we first train an RL agent to learn useful constraints of the problem and then use it to produce a certain part of the initial population of the search algorithm. By incorporating an RL agent into the search process, we aim to guide the algorithm towards promising regions of the search space from the start, enabling more efficient exploration of the solution space. We evaluate RIGAA on two case studies: maze generation for an autonomous ant robot and road topology generation for an autonomous vehicle lane keeping assist system. In both case studies, RIGAA converges faster to fitter solutions and produces a better test suite (in terms of average test scenario fitness and diversity). RIGAA also outperforms the state-of-the-art tools for vehicle lane keeping assist system testing, such as AmbieGen and Frenetic.

翻译：基于进化搜索的技术常用于测试自主机器人系统。然而，这些方法通常依赖于计算成本高昂的基于模拟器的模型进行测试场景评估。为提高基于搜索的测试的计算效率，我们提出通过强化学习（RL）智能体来增强进化搜索（ES），该智能体使用基于领域知识的替代奖励进行训练。在我们的方法中，称为RIGAA（强化学习引导的遗传算法用于自主系统测试），我们首先训练一个RL智能体学习问题的有用约束，然后利用它生成搜索算法初始种群的一部分。通过将RL智能体融入搜索过程，我们旨在从初始阶段引导算法朝向搜索空间中有前景的区域，从而实现对解空间更高效的探索。我们在两个案例研究中评估了RIGAA：自主蚂蚁机器人的迷宫生成和自主车辆车道保持辅助系统的道路拓扑生成。在两个案例研究中，RIGAA均能更快地收敛到更优解，并生成更好的测试套件（就平均测试场景适应度和多样性而言）。RIGAA在车辆车道保持辅助系统测试方面也优于最先进的工具，如AmbieGen和Frenetic。