Quality-Diversity (QD) algorithms are a new type of Evolutionary Algorithms (EAs), aiming to find a set of high-performing, yet diverse solutions. They have found many successful applications in reinforcement learning and robotics, helping improve the robustness in complex environments. Furthermore, they often empirically find a better overall solution than traditional search algorithms which explicitly search for a single highest-performing solution. However, their theoretical analysis is far behind, leaving many fundamental questions unexplored. In this paper, we try to shed some light on the optimization ability of QD algorithms via rigorous running time analysis. By comparing the popular QD algorithm MAP-Elites with $(\mu+1)$-EA (a typical EA focusing on finding better objective values only), we prove that on two NP-hard problem classes with wide applications, i.e., monotone approximately submodular maximization with a size constraint, and set cover, MAP-Elites can achieve the (asymptotically) optimal polynomial-time approximation ratio, while $(\mu+1)$-EA requires exponential expected time on some instances. This provides theoretical justification for that QD algorithms can be helpful for optimization, and discloses that the simultaneous search for high-performing solutions with diverse behaviors can provide stepping stones to good overall solutions and help avoid local optima.
翻译:质量多样性(QD)算法是一种新型进化算法(EA),旨在寻找一组高性能且多样化的解。该算法在强化学习和机器人领域取得了诸多成功应用,有助于提升复杂环境中的鲁棒性。此外,与显式搜索单个最高性能解的传统搜索算法相比,QD算法通常能经验性地找到更优的整体解。然而,其理论分析远远落后,仍有许多基本问题有待探索。本文尝试通过严格的运行时间分析,揭示QD算法的优化能力。通过比较流行的QD算法MAP-Elites与$(\mu+1)$-EA(一种仅关注寻找更优目标值的典型EA),我们证明:在两类具有广泛应用的NP难问题(即带大小约束的单调近似子模最大化问题和集合覆盖问题)上,MAP-Elites能够实现(渐近)最优的多项式时间近似比,而$(\mu+1)$-EA在某些实例上需要指数级期望时间。这为QD算法有助于优化的观点提供了理论依据,并揭示出同时搜索具有多样化行为的高性能解,可为获得整体最优解提供“垫脚石”,并帮助避免局部最优。