Diversity optimization seeks to discover a set of solutions that elicit diverse features. Prior work has proposed Novelty Search (NS), which, given a current set of solutions, seeks to expand the set by finding points in areas of low density in the feature space. However, to estimate density, NS relies on a heuristic that considers the k-nearest neighbors of the search point in the feature space, which yields a weaker stability guarantee. We propose Density Descent Search (DDS), an algorithm that explores the feature space via CMA-ES on a continuous density estimate of the feature space that also provides a stronger stability guarantee. We experiment with DDS and two density estimation methods: kernel density estimation (KDE) and continuous normalizing flow (CNF). On several standard diversity optimization benchmarks, DDS outperforms NS, the recently proposed MAP-Annealing algorithm, and other state-of-the-art baselines. Additionally, we prove that DDS with KDE provides stronger stability guarantees than NS, making it more suitable for adaptive optimizers. Furthermore, we prove that NS is a special case of DDS that descends a KDE of the feature space.

翻译：多样性优化旨在发现一组能够引发多样化特征的解。先前研究提出了新颖性搜索算法，该算法在给定当前解集的情况下，通过寻找特征空间中低密度区域的点来扩展解集。然而，为估计密度，NS依赖于考虑特征空间中搜索点的k近邻的启发式方法，这导致其稳定性保证较弱。本文提出密度下降搜索算法，该算法通过在特征空间的连续密度估计上运行CMA-ES来探索特征空间，同时提供更强的稳定性保证。我们采用两种密度估计方法进行DDS实验：核密度估计与连续标准化流。在多个标准多样性优化基准测试中，DDS的表现优于NS、最新提出的MAP退火算法以及其他先进基线方法。此外，我们证明采用KDE的DDS比NS具有更强的稳定性保证，使其更适用于自适应优化器。更进一步，我们证明NS是DDS在特征空间KDE下降过程中的特例。