This work continues to investigate the link between differentially private (DP) and online learning. Alon, Livni, Malliaris, and Moran (2019) showed that for binary concept classes, DP learnability of a given class implies that it has a finite Littlestone dimension (equivalently, that it is online learnable). Their proof relies on a model-theoretic result by Hodges (1997), which demonstrates that any binary concept class with a large Littlestone dimension contains a large subclass of thresholds. In a follow-up work, Jung, Kim, and Tewari (2020) extended this proof to multiclass PAC learning with a bounded number of labels. Unfortunately, Hodges's result does not apply in other natural settings such as multiclass PAC learning with an unbounded label space, and PAC learning of partial concept classes. This naturally raises the question of whether DP learnability continues to imply online learnability in more general scenarios: indeed, Alon, Hanneke, Holzman, and Moran (2021) explicitly leave it as an open question in the context of partial concept classes, and the same question is open in the general multiclass setting. In this work, we give a positive answer to these questions showing that for general classification tasks, DP learnability implies online learnability. Our proof reasons directly about Littlestone trees, without relying on thresholds. We achieve this by establishing several Ramsey-type theorems for trees, which might be of independent interest.

翻译：本研究继续探讨差分隐私（DP）学习与在线学习之间的联系。Alon、Livni、Malliaris 和 Moran（2019）证明了对于二元概念类，给定类的 DP 可学习性意味着其具有有限 Littlestone 维度（等价于该概念类是在线可学习的）。他们的证明依赖于 Hodges（1997）的一个模型论结果，该结果表明任何具有较大 Littlestone 维度的二元概念类都包含一个较大的阈值子类。在后续工作中，Jung、Kim 和 Tewari（2020）将此证明推广到标签数量有限的多类 PAC 学习。然而，Hodges 的结果并不适用于其他自然场景，例如标签空间无界的多类 PAC 学习以及部分概念类的 PAC 学习。这自然引出一个问题：在更一般的场景中，DP 可学习性是否仍然蕴含在线可学习性？事实上，Alon、Hanneke、Holzman 和 Moran（2021）在部分概念类背景下明确将此列为开放问题，而该问题在一般多类设置中同样尚未解决。在本工作中，我们对这些问题给出了肯定回答，证明了对于一般分类任务，DP 可学习性蕴含在线可学习性。我们的证明直接基于对 Littlestone 树的分析，而不依赖于阈值概念。通过建立若干关于树的拉姆齐型定理（这些定理本身可能具有独立的研究价值），我们实现了这一结论。