Diversity optimization seeks to discover a set of solutions that elicit diverse features. Prior work has proposed Novelty Search (NS), which, given a current set of solutions, seeks to expand the set by finding points in areas of low density in the feature space. However, to estimate density, NS relies on a heuristic that considers the k-nearest neighbors of the search point in the feature space, which yields a weaker stability guarantee. We propose Density Descent Search (DDS), an algorithm that explores the feature space via gradient descent on a continuous density estimate of the feature space that also provides stronger stability guarantee. We experiment with DDS and two density estimation methods: kernel density estimation (KDE) and continuous normalizing flow (CNF). On several standard diversity optimization benchmarks, DDS outperforms NS, the recently proposed MAP-Annealing algorithm, and other state-of-the-art baselines. Additionally, we prove that DDS with KDE provides stronger stability guarantees than NS, making it more suitable for adaptive optimizers. Furthermore, we prove that NS is a special case of DDS that descends a KDE of the feature space.

翻译：多样性优化旨在发现一组具有多样化特征的解决方案。此前工作提出了新颖性搜索（NS），该方法通过在当前解决方案集的低密度特征区域中寻找新点来扩展解集。然而，NS通过计算搜索点在特征空间中的k近邻进行密度估计，这种启发式方法稳定性较弱。我们提出密度下降搜索（DDS）算法，该算法通过特征空间连续密度估计的梯度下降进行探索，同时提供更强的稳定性保证。本文实验采用了两种密度估计方法：核密度估计（KDE）和连续归一化流（CNF）。在多个标准多样性优化基准测试中，DDS均优于NS、最新提出的MAP-退火算法及其他先进基线方法。此外，我们证明了基于KDE的DDS比NS具有更强的稳定性保证，使其更适用于自适应优化器。进一步的理论分析表明，NS实际上是DDS的一种特例，即对特征空间KDE的梯度下降过程。