Quality-Diversity (QD) algorithms are a new type of Evolutionary Algorithms (EAs), aiming to find a set of high-performing, yet diverse solutions. They have found many successful applications in reinforcement learning and robotics, helping improve the robustness in complex environments. Furthermore, they often empirically find a better overall solution than traditional search algorithms which explicitly search for a single highest-performing solution. However, their theoretical analysis is far behind, leaving many fundamental questions unexplored. In this paper, we try to shed some light on the optimization ability of QD algorithms via rigorous running time analysis. By comparing the popular QD algorithm MAP-Elites with $(\mu+1)$-EA (a typical EA focusing on finding better objective values only), we prove that on two NP-hard problem classes with wide applications, i.e., monotone approximately submodular maximization with a size constraint, and set cover, MAP-Elites can achieve the (asymptotically) optimal polynomial-time approximation ratio, while $(\mu+1)$-EA requires exponential expected time on some instances. This provides theoretical justification for that QD algorithms can be helpful for optimization, and discloses that the simultaneous search for high-performing solutions with diverse behaviors can provide stepping stones to good overall solutions and help avoid local optima.
翻译:质量多样性(QD)算法是一种新型进化算法(EA),旨在寻找一组高性能且多样化的解。该算法已在强化学习和机器人领域取得诸多成功应用,有助于提升复杂环境下的鲁棒性。此外,相较于显式搜索单一最优解的传统搜索算法,QD算法通常能在经验上找到更优的整体解。然而,其理论分析严重滞后,诸多基础问题尚待探索。本文通过严格的运行时间分析,尝试阐明QD算法的优化能力。通过将主流QD算法MAP-Elites与仅关注目标值优化的典型进化算法$(\mu+1)$-EA进行对比,我们证明:在具有广泛应用的NP困难问题类别(即带大小约束的单调近似子模最大化问题和集合覆盖问题)中,MAP-Elites能够达到(渐近)最优的多项式时间近似比,而$(\mu+1)$-EA在某些实例上需要指数级期望时间。这为QD算法可辅助优化的论断提供了理论依据,并揭示了同时搜索具有多样行为的高性能解能够为整体最优解提供跳板,从而避免陷入局部最优。