Pilotfish is the first scale-out blockchain execution engine able to harness any degree of parallelizability existing in its workload. Pilotfish allows each validator to employ multiple machines, named ExecutionWorkers, under its control to scale its execution layer. Given a sufficiently parallelizable and compute-intensive load, the number of transactions that the validator can execute increases linearly with the number of ExecutionWorkers at its disposal. In addition, Pilotfish maintains the consistency of the state, even when many validators experience simultaneous machine failures. This is possible due to the meticulous co-design of our crash-recovery protocol which leverages the existing fault tolerance in the blockchain's consensus mechanism. Finally, Pilotfish can also be seen as the first distributed deterministic execution engine that provides support for dynamic reads as transactions are not required to provide a fully accurate read and write set. This loosening of requirements would normally reduce the parallelizability available by blocking write-after-write conflicts, but our novel versioned-queues scheduling algorithm circumvents this by exploiting the lazy recovery property of Pilotfish, which only persists consistent state and re-executes any optimistic steps taken before the crash. In order to prove our claims we implemented the common path of Pilotfish with support for the MoveVM and evaluated it against the parallel execution MoveVM of Sui. Our results show that Pilotfish provides good scalability up to 8 ExecutionWorkers for a variety of workloads. In computationally-heavy workloads, Pilotfish's scalability is linear.
翻译:Pilotfish是首个能够充分利用工作负载中任意并行化程度进行横向扩展的区块链执行引擎。Pilotfish允许每个验证者在其控制下部署多台名为ExecutionWorker的机器来扩展执行层。在具有足够并行性和计算密集型负载的条件下,验证者可执行的事务数量随其可支配的ExecutionWorker数量线性增长。此外,即便多个验证者同时遭遇机器故障,Pilotfish仍能维持状态一致性。这一特性得益于我们精心协同设计的崩溃恢复协议——该协议利用了区块链共识机制中已有的容错能力。最后,Pilotfish也可视为首个支持动态读取的分布式确定性执行引擎,因其无需事务提供完全准确的读写集合。通常情况下,这种要求的放宽会因写-写冲突阻塞而降低并行性,但我们的新颖版本化队列调度算法通过利用Pilotfish的惰性恢复特性(仅持久化一致状态并重新执行崩溃前执行的乐观步骤)规避了此问题。为验证上述论断,我们实现了支持MoveVM的Pilotfish通用路径,并与Sui的并行执行MoveVM进行了对比评估。结果表明,在多种工作负载下,Pilotfish在8个ExecutionWorker内展现出良好的可扩展性。在计算密集型负载中,Pilotfish的扩展性呈线性特征。