We consider the problem of evaluating distinct multivariate polynomials over several massive datasets in a distributed computing system with a single master node and multiple worker nodes. We focus on the general case when each multivariate polynomial is evaluated over its corresponding dataset and propose a generalization of the Lagrange Coded Computing framework (Yu et al. 2019) to perform all computations simultaneously while providing robustness against stragglers who do not respond in time, adversarial workers who respond with wrong computation and information-theoretic security of dataset against colluding workers. Our scheme introduces a small computation overhead which results in a reduction in download cost and also offers comparable resistance to stragglers over existing solutions. On top of it, we also propose two verification schemes to detect the presence of adversaries, which leads to incorrect results, without involving additional nodes.
翻译:我们考虑在具有单个主节点和多个工作节点的分布式计算系统中,对多个大规模数据集评估不同多元多项式的问题。我们聚焦于每个多元多项式在其对应数据集上评估的一般性情况,并提出对Lagrange编码计算框架(Yu等人,2019年)的通用化扩展,以同时执行所有计算,同时提供对未及时响应的掉队者、返回错误计算的对抗性工作节点以及针对共谋工作节点的数据集信息论安全性的鲁棒性。我们的方案仅引入较小的计算开销,从而降低了下载成本,并在应对掉队者方面与现有解决方案相当。在此基础上,我们还提出了两种验证方案,用于在不引入额外节点的情况下检测导致错误结果的对抗性工作节点存在。