The security of many Proof-of-Stake (PoS) payment systems relies on quorum-based State Machine Replication (SMR) protocols. While classical analyses assume purely Byzantine faults, real-world systems must tolerate both arbitrary failures and strategic, profit-driven validators. We therefore study quorum-based SMR under a hybrid model with honest, Byzantine, and rational participants. We first establish the fundamental limitations of traditional consensus mechanisms, proving two impossibility results: (1) in partially synchronous networks, no quorum-based protocol can achieve SMR when rational and Byzantine validators collectively exceed $1/3$ of the participants; and (2) even under synchronous network assumptions, SMR remains unattainable if this coalition comprises more than $2/3$ of the validator set. Assuming a synchrony bound $Δ$, we show how to extend any quorum-based SMR protocol to tolerate up to $1/3$ Byzantine and $1/3$ rational validators by modifying only its finalization rule. Our approach enforces a necessary bound on the total transaction volume finalized within any time window $Δ$ and introduces the \emph{strongest chain rule}, which enables efficient finalization of transactions when a supermajority of honest participants provably supports execution. Empirical analysis of Ethereum and Cosmos demonstrates validator participation exceeding the required $5/6$ threshold in over $99%$ of blocks, supporting the practicality of our design. Finally, we present a recovery mechanism that restores safety and liveness after consistency violations, even with up to $5/9$ Byzantine stake and $1/9$ rational stake, guaranteeing full reimbursement of provable client losses.

翻译：许多权益证明（Proof-of-Stake, PoS）支付系统的安全性依赖于基于共识机制的状态机复制（State Machine Replication, SMR）协议。尽管经典分析假设纯拜占庭容错，但实际系统必须同时容忍任意故障和策略性、逐利的验证者。因此，我们在具有诚实、拜占庭和理性参与者的混合模型下研究基于共识机制的SMR。首先，我们确立了传统共识机制的基本局限性，证明了两个不可能性结果：（1）在部分同步网络中，当理性和拜占庭验证者总数超过参与者的$1/3$时，不存在基于共识机制的协议能实现SMR；（2）即便在同步网络假设下，如果该联盟占比超过验证者集合的$2/3$，SMR仍然无法实现。假设同步界为$Δ$，我们展示了如何通过仅修改最终性规则，将任意基于共识机制的SMR协议扩展为可容忍最多$1/3$的拜占庭验证者和$1/3$的理性验证者。我们的方法强制约束了任意时间窗口$Δ$内最终确定的交易总量上限，并引入了**最强链规则**（strongest chain rule），该规则可在诚实参与者绝大多数支持执行时，高效地实现交易的最终确认。对以太坊（Ethereum）和Cosmos的实证分析表明，超过$99\%$的区块中验证者参与度超过了所需的$5/6$阈值，这支持了我们设计的实用性。最后，我们提出了一种恢复机制，能在发生一致性违反后恢复安全性和活跃性，即使存在高达$5/9$的拜占庭权益和$1/9$的理性权益，也能保证可证明的客户损失得到全额补偿。