As the network scale increases, existing fully distributed solutions start to lag behind the real-world challenges such as (1) slow information propagation, (2) network communication failures, and (3) external adversarial attacks. In this paper, we focus on hierarchical system architecture and address the problem of non-Bayesian learning over networks that are vulnerable to communication failures and adversarial attacks. On network communication, we consider packet-dropping link failures. We first propose a hierarchical robust push-sum algorithm that can achieve average consensus despite frequent packet-dropping link failures. We provide a sparse information fusion rule between the parameter server and arbitrarily selected network representatives. Then, interleaving the consensus update step with a dual averaging update with Kullback-Leibler (KL) divergence as the proximal function, we obtain a packet-dropping fault-tolerant non-Bayesian learning algorithm with provable convergence guarantees. On external adversarial attacks, we consider Byzantine attacks in which the compromised agents can send maliciously calibrated messages to others (including both the agents and the parameter server). To avoid the curse of dimensionality of Byzantine consensus, we solve the non-Bayesian learning problem via running multiple dynamics, each of which only involves Byzantine consensus with scalar inputs. To facilitate resilient information propagation across sub-networks, we use a novel Byzantine-resilient gossiping-type rule at the parameter server.

翻译：随着网络规模的扩大，现有的全分布式解决方案开始落后于实际挑战，例如：（1）信息传播缓慢，（2）网络通信故障，以及（3）外部敌手攻击。本文聚焦于分层系统架构，并解决在易受通信故障和敌手攻击的网络上的非贝叶斯学习问题。在网络通信方面，我们考虑了丢包链路故障。首先提出一种分层鲁棒推-和算法，该算法能在频繁丢包链路故障下实现平均共识。我们设计了参数服务器与任意选定的网络代表之间的稀疏信息融合规则。随后，将共识更新步骤与以Kullback-Leibler散度作为邻近函数的对偶平均更新交替进行，得到一种具有可证明收敛保证的丢包容错非贝叶斯学习算法。在外部敌手攻击方面，我们考虑拜占庭攻击，其中被攻陷的智能体可向其他节点（包括智能体和参数服务器）发送恶意校准消息。为避免拜占庭共识的维度诅咒，我们通过运行多个动力学过程来解决非贝叶斯学习问题，每个过程仅涉及标量输入的拜占庭共识。为促进子网络间弹性信息传播，我们在参数服务器处采用一种新颖的拜占庭鲁棒八卦型规则。