We study a distributed learning problem in which learning agents are embedded in a directed acyclic graph (DAG). There is a fixed and arbitrary distribution over feature/label pairs, and each agent or vertex in the graph is able to directly observe only a subset of the features -- potentially a different subset for every agent. The agents learn sequentially in some order consistent with a topological sort of the DAG, committing to a model mapping observations to predictions of the real-valued label. Each agent observes the predictions of their parents in the DAG, and trains their model using both the features of the instance that they directly observe, and the predictions of their parents as additional features. We ask when this process is sufficient to achieve \emph{information aggregation}, in the sense that some agent in the DAG is able to learn a model whose error is competitive with the best model that could have been learned (in some hypothesis class) with direct access to \emph{all} features, despite the fact that no single agent in the network has such access. We give upper and lower bounds for this problem for both linear and general hypothesis classes. Our results identify the \emph{depth} of the DAG as the key parameter: information aggregation can occur over sufficiently long paths in the DAG, assuming that all of the relevant features are well represented along the path, and there are distributions over which information aggregation cannot occur even in the linear case, and even in arbitrarily large DAGs that do not have sufficient depth (such as a hub-and-spokes topology in which the spoke vertices collectively see all the features). We complement our theoretical results with a comprehensive set of experiments.

翻译：我们研究一个分布式学习问题，其中学习智能体嵌入在一个有向无环图（DAG）中。存在一个固定且任意的特征/标签对分布，图中每个智能体或顶点只能直接观察到特征的一个子集——不同智能体可能观察到不同的子集。智能体按照与DAG拓扑排序一致的顺序依次学习，并承诺一个将观测值映射到实值标签预测的模型。每个智能体观察其父节点在DAG中的预测，并使用其直接观测到的实例特征以及父节点的预测作为附加特征来训练其模型。我们探讨此过程何时足以实现**信息聚合**，即DAG中某个智能体能够学习到一个模型，其误差可与直接访问**所有**特征时（在某个假设类中）可能学到的最佳模型相竞争，尽管网络中没有任何单个智能体具备这种访问权限。我们针对线性和一般假设类给出了该问题的上界和下界。我们的结果表明DAG的**深度**是关键参数：假设所有相关特征在路径上都能得到充分表示，信息聚合可以在DAG中足够长的路径上发生；同时存在某些分布，即使在线性情况下，甚至在深度不足的任意大DAG中（例如轮辐拓扑结构，其中辐条顶点共同看到所有特征），信息聚合也无法实现。我们通过一系列全面的实验对理论结果进行了补充。