Independently trained domain specialists can be fused post-hoc into a single model that outperforms any individual specialist, and the gain is predictable: gain = 0.82 x divergence - 2.72 (R^2 = 0.856, n=6, 3-26% divergence). This enables practitioners to estimate cooperative value before committing compute. Below ~3.3% divergence, gains approach zero.In the KALAVAI protocol, contributors fine-tune copies of a shared checkpoint independently, then submit for lightweight MoE routing (500 steps). Gains are consistent: +7.72% at 410M (+/-0.02%, 3 seeds), +7.49% at 1B (+/-0.01%, 3 seeds), +6.53% at 6.9B, each over the best specialist. The router matches domain-oracle routing within <10^{-5} nats. Cross-lingual fusion (Tamil/Yoruba/Welsh/Code) achieves +21.76%, with Yoruba perplexity falling 41.9 to 7.7. A 20-contributor federation achieves +16.71% (+/-0.07pp, 3 seeds).Three requirements bound the protocol. Shared initialisation is necessary: checkpoint mismatch degrades routing. Frozen layers are optional below ~10,000 steps and beneficial beyond. Learned routing is essential: uniform averaging degrades by -1.2% vs. best specialist, while any trained router achieves oracle-optimal assignment.

翻译：独立训练的领域专家模型可在训练后融合为单一模型，其性能优于任一独立专家，且增益可预测：增益 = 0.82 × 分歧度 - 2.72（R² = 0.856，n=6，分歧度范围3-26%）。该公式使从业者能够在投入计算资源前评估协同价值。当分歧度低于约3.3%时，增益趋近于零。在KALAVAI协议中，贡献者基于共享检查点副本独立微调模型，随后提交轻量级MoE路由（500步）进行融合。增益表现一致：在410M规模下为+7.72%（±0.02%，3个随机种子），1B规模下为+7.49%（±0.01%，3个随机种子），6.9B规模下为+6.53%（均优于最佳专家模型）。路由器的分配精度与领域专家分配基准的差异小于10⁻⁵ nats。跨语言融合（泰米尔语/约鲁巴语/威尔士语/代码）实现+21.76%的增益，其中约鲁巴语的困惑度从41.9降至7.7。包含20个贡献者的联邦融合方案实现+16.71%的增益（±0.07个百分点，3个随机种子）。该协议受三项约束限制：共享初始化是必要前提——检查点不匹配将降低路由效果；当训练步数低于约10,000步时，层冻结为可选操作，超出该阈值则有益；学习路由机制至关重要：统一平均法相比最佳专家模型降低1.2%的性能，而任何经过训练的路由器均可实现接近基准分配方案的最优分配。