Spreading and storing erasure-coded data in distributed systems effectively is challenging in real settings. Practical deployments must contend with unpredictable network latencies, particularly when information dispersal is integrated into consensus protocols, a prominent and latency-sensitive use case. Existing approaches address this challenge through timeout-based dissemination and adaptive communication or storage decisions driven by acknowledgments during dissemination. However, these designs focus almost exclusively on dissemination-time efficiency, complicate recovery with reconstruction procedures that require metadata that can differ per consensus value, and rely on a centralized leader to make storage decisions for all nodes. This paper introduces \textbf{Rafture}, a novel information dispersal algorithm, and its integration in a consensus protocol, that overcomes these limitations. Rafture is the first information dispersal solution to incorporate post-dissemination pruning, allowing systems to adapt storage costs after dissemination completes. It employs a simple, fixed-threshold erasure code while varying distinct fragment assignment along a second dimension. This ensures that reconstruction is always possible from $F+1$ fragments using the same interpolation method and no additional metadata. Rafture further enables nodes to adapt autonomously based on locally observed information, eliminating the need for global coordination. We evaluate Rafture in highly dynamic network settings and show that it simplifies recovery while significantly improving long-term storage consumption under variable network conditions.
翻译:在分布式系统中有效传播与存储纠删码数据在实际应用中面临挑战。实际部署必须应对不可预测的网络延迟,特别是在将信息分散机制集成到对延迟敏感的共识协议中时尤为突出。现有方法通过基于超时的传播以及利用传播过程中确认驱动的自适应通信或存储决策来应对这一挑战。然而,这些设计几乎完全聚焦于传播阶段效率,通过需要因共识值而异元数据的重建过程增加了恢复复杂性,且依赖集中式领导者为所有节点制定存储决策。本文提出\textbf{Rafture}——一种新型信息分散算法及其与共识协议的集成方案,克服了上述局限。Rafture是首个引入后传播修剪的信息分散解决方案,允许系统在传播完成后自适应调整存储成本。它采用简单固定阈值的纠删码,同时沿第二维度变化不同分片分配策略,确保始终可通过$F+1$个分片并使用相同插值方法进行重建,无需额外元数据。Rafture进一步使节点能够基于本地观测信息自主决策,消除全局协调需求。我们在高度动态网络环境下评估Rafture,结果表明该方案在简化恢复过程的同时,显著改善了动态网络条件下的长期存储消耗。