We performed a billion locality sensitive hash comparisons between artificially generated data samples to answer the critical question - can we reproduce the results of generative AI models? Reproducibility is one of the pillars of scientific research for verifiability, benchmarking, trust, and transparency. Futhermore, we take this research to the next level by verifying the "correctness" of generative AI output in a non-deterministic, trustless, decentralized network. We generate millions of data samples from a variety of open source diffusion and large language models and describe the procedures and trade-offs between generating more verses less deterministic output. Additionally, we analyze the outputs to provide empirical evidence of different parameterizations of tolerance and error bounds for verification. For our results, we show that with a majority vote between three independent verifiers, we can detect image generated perceptual collisions in generated AI with over 99.89% probability and less than 0.0267% chance of intra-class collision. For large language models (LLMs), we are able to gain 100% consensus using greedy methods or n-way beam searches to generate consensus demonstrated on different LLMs. In the context of generative AI training, we pinpoint and minimize the major sources of stochasticity and present gossip and synchronization training techniques for verifiability. Thus, this work provides a practical, solid foundation for AI verification, reproducibility, and consensus for generative AI applications.

翻译：我们对人工生成的数据样本进行了数十亿次局部敏感哈希比较，以解答一个关键问题：能否复现生成式AI模型的结果？可复现性是科学研究的支柱之一，关乎可验证性、基准测试、可信度与透明度。此外，我们进一步在非确定性、无需信任的去中心化网络中验证了生成式AI输出的“正确性”。我们从多种开源扩散模型与大型语言模型中生成数百万个数据样本，描述了生成过程中确定性与非确定性输出之间的权衡流程。同时，我们分析输出结果，为不同容差与误差边界的参数化验证提供了经验证据。研究结果表明，通过三个独立验证者的多数投票机制，我们能够以超过99.89%的概率检测生成式AI中的图像感知碰撞，且同类碰撞概率低于0.0267%。对于大型语言模型，我们采用贪心方法或n路束搜索在不同模型上实现了100%的共识。在生成式AI训练方面，我们定位并最小化了随机性的主要来源，并提出了用于验证的八卦协议与同步训练技术。因此，本文为生成式AI应用的可验证性、可复现性与共识构建了实用且坚实的基础。