Log-Structured Merge-Trees (LSM-trees) dominate persistent key-value storage but suffer from high write amplification from 10x to 30x under random workloads due to repeated compaction. This overhead becomes prohibitive for large values with uniformly distributed keys, a workload common in content-addressable storage, deduplication systems, and blockchain validators. We present Tidehunter, a storage engine that eliminates value compaction by treating the Write-Ahead Log (WAL) as permanent storage rather than a temporary recovery buffer. Values are never overwritten; and small, lazily-flushed index tables map keys to WAL positions. Tidehunter introduces (a) lock-free writes that saturate NVMe drives through atomic allocation and parallel copying, (b) an optimistic index structure that exploits uniform key distributions for single-roundtrip lookups, and (c) epoch-based pruning that reclaims space without blocking writes. On a 1 TB dataset with 1 KB values, Tidehunter achieves 830K writes per second, that is 8.4x higher than RocksDB and 2.9x higher than BlobDB, while improving point queries by 1.7x and existence checks by 15.6x. We validate real-world impact by integrating Tidehunter into Sui, a high-throughput blockchain, where it maintains stable throughput and latency under loads that cause RocksDB-backed validators to collapse. Tidehunter is production-ready and is being deployed in production within Sui.

翻译：日志结构合并树（LSM-trees）在持久化键值存储领域占据主导地位，但由于重复的压缩操作，在随机工作负载下会产生高达10倍至30倍的高写入放大。对于键值均匀分布的大值数据——这种工作负载常见于内容寻址存储、重复数据删除系统和区块链验证器——此类开销变得难以承受。本文提出Tidehunter，一种通过将预写日志（WAL）作为永久存储而非临时恢复缓冲区来消除值压缩的存储引擎。值数据从不被覆写；通过小型且惰性刷新的索引表将键映射到WAL位置。Tidehunter引入了以下特性：（a）通过原子分配与并行复制实现无锁写入，从而饱和NVMe驱动器的带宽；（b）利用键值均匀分布特性实现单次往返查找的乐观索引结构；（c）基于周期的空间回收机制，可在不阻塞写入的情况下回收存储空间。在1 TB数据集（含1 KB值数据）的测试中，Tidehunter实现了每秒83万次写入，较RocksDB提升8.4倍，较BlobDB提升2.9倍，同时点查询性能提升1.7倍，存在性检查性能提升15.6倍。我们将Tidehunter集成至高吞吐量区块链Sui以验证其实际影响：在导致基于RocksDB的验证器崩溃的负载下，该系统仍能保持稳定的吞吐量与延迟。Tidehunter已达到生产就绪状态，并已在Sui的生产环境中部署。