Recently, an intriguing class of non-convex optimization problems has emerged in the context of learning directed acyclic graphs (DAGs). These problems involve minimizing a given loss or score function, subject to a non-convex continuous constraint that penalizes the presence of cycles in a graph. In this work, we delve into the optimization challenges associated with this class of non-convex programs. To address these challenges, we propose a bi-level algorithm that leverages the non-convex constraint in a novel way. The outer level of the algorithm optimizes over topological orders by iteratively swapping pairs of nodes within the topological order of a DAG. A key innovation of our approach is the development of an effective method for generating a set of candidate swapping pairs for each iteration. At the inner level, given a topological order, we utilize off-the-shelf solvers that can handle linear constraints. The key advantage of our proposed algorithm is that it is guaranteed to find a local minimum or a KKT point under weaker conditions compared to previous work and finds solutions with lower scores. Extensive experiments demonstrate that our method outperforms state-of-the-art approaches in terms of achieving a better score. Additionally, our method can also be used as a post-processing algorithm to significantly improve the score of other algorithms. Code implementing the proposed method is available at https://github.com/duntrain/topo.

翻译：最近，在学习有向无环图（DAG）的背景下，出现了一类引人关注的非凸优化问题。这类问题涉及在满足一个惩罚图中环存在的非凸连续约束条件下，最小化给定的损失函数或评分函数。本文深入研究了与此类非凸规划相关的优化挑战。为应对这些挑战，我们提出了一种双层算法，该算法以新颖的方式利用了非凸约束。算法的外层通过迭代交换有向无环图拓扑顺序中的节点对来优化拓扑顺序。我们方法的一个关键创新在于为每次迭代生成一组候选交换对的有效方法。在内层，给定一个拓扑顺序后，我们利用现成的求解器来处理线性约束。我们提出的算法的主要优势在于，与先前的工作相比，它能在更弱的条件下保证找到局部最小值或KKT点，并找到评分更低的解。大量实验表明，我们的方法在获得更优评分方面优于当前最先进的方法。此外，我们的方法还可作为后处理算法，显著提升其他算法的评分。实现所提方法的代码可在 https://github.com/duntrain/topo 获取。