One of the central problems studied in the theory of machine learning is the question of whether, for a given class of hypotheses, it is possible to efficiently find a {consistent} hypothesis, i.e., which has zero training error. While problems involving {\em convex} hypotheses have been extensively studied, the question of whether efficient learning is possible for non-convex hypotheses composed of possibly several disconnected regions is still less understood. Although it has been shown quite a while ago that efficient learning of weakly convex hypotheses, a parameterized relaxation of convex hypotheses, is possible for the special case of Boolean functions, the question of whether this idea can be developed into a generic paradigm has not been studied yet. In this paper, we provide a positive answer and show that the consistent hypothesis finding problem can indeed be solved in polynomial time for a broad class of weakly convex hypotheses over metric spaces. To this end, we propose a general domain-independent algorithm for finding consistent weakly convex hypotheses and prove sufficient conditions for its efficiency that characterize the corresponding hypothesis classes. To illustrate our general algorithm and its properties, we discuss several non-trivial learning examples to demonstrate how it can be used to efficiently solve the corresponding consistent hypothesis finding problem. Without the weak convexity constraint, these problems are known to be computationally intractable. We then proceed to show that the general idea of our algorithm can even be extended to the case of extensional weakly convex hypotheses, as it naturally arise, e.g., when performing vertex classification in graphs. We prove that using our extended algorithm, the problem can be solved in polynomial time provided the distances in the domain can be computed efficiently.

翻译：机器学习理论的核心问题之一在于：对于给定的假设类别，能否高效地找到**一致的**假设（即训练误差为零的假设）。尽管涉及**凸**假设的问题已被广泛研究，但对于由多个可能不连通区域组成的非凸假设能否实现高效学习，目前仍知之甚少。尽管很早之前已有研究表明，在布尔函数的特殊情况下，弱凸假设（凸假设的一种参数化松弛）的高效学习是可行的，但这一思想能否发展为通用范式尚未得到研究。本文给出了肯定答案，证明对于度量空间上一大类弱凸假设，一致假设寻找问题确实可以在多项式时间内解决。为此，我们提出了一种通用的领域无关算法，用于寻找一致的弱凸假设，并证明了其高效性的充分条件，这些条件刻画了相应的假设类别。为说明我们的通用算法及其特性，我们讨论了若干非平凡的学习实例，展示如何利用该算法高效解决相应的一致假设寻找问题。若无弱凸性约束，这些问题已知是计算上难以处理的。随后，我们证明该算法的一般思想甚至可以扩展到外延弱凸假设的情形（例如在图顶点分类中自然出现的假设）。我们证明，使用扩展算法后，只要域中的距离可以高效计算，该问题便可在多项式时间内求解。