Uniform random rotations (URRs) are a common preprocessing step in modern quantization approaches used for gradient compression, inference acceleration, KV-cache compression, model weight quantization, and approximate nearest-neighbor search in vector databases. In practice, URRs are often replaced by randomized Hadamard transforms (RHTs), which preserve orthogonality while admitting fast implementations. The remaining issue is the performance for worst-case inputs. With a URR, each coordinate is individually distributed as a shifted beta distribution, which converges to a Gaussian distribution in high dimensions. Generally, one RHT is not suitable in the worst case, as individual coordinates can be far from these distributions. We show that after composing two RHTs on any $d$-sized input vector, the marginal distribution of every fixed coordinate of the normalized rotated vector is within $O(d^{-1/2})$ of a standard Gaussian both in Kolmogorov distance and in $1$-Wasserstein distance. We then plug these bounds into the analyses of modern compression schemes, namely DRIVE and QUIC-FL, and show that two RHTs achieve performance that asymptotically matches URRs. However, we show that two RHTs may not be sufficient for Vector Quantization (VQ), which often requires weak correlation across fixed-size blocks of coordinates (as opposed to only marginal distribution convergence for single coordinates). We prove that a composition of three RHTs leads to decaying coordinate covariance. This ensures that any fixed, bounded, multi-dimensional VQ codebook optimized for URRs has the same expected error when using three RHTs, up to an additive term that vanishes with the dimension. Finally, because practical inputs are rarely adversarial, we propose a linear-time ${O}(d)$ check on the input's moments to dynamically adapt the number of RHTs used at runtime to improve performance.

翻译：均匀随机旋转（URR）是现代量化方法中的常见预处理步骤，广泛应用于梯度压缩、推理加速、KV缓存压缩、模型权重量化以及向量数据库中的近似最近邻搜索。实践中，URR常被随机化哈达玛变换（RHT）替代，后者既能保持正交性，又支持快速实现。剩余问题在于最坏情况输入下的性能。使用URR时，每个坐标独立服从偏移贝塔分布，并在高维空间中收敛至高斯分布。一般而言，单个RHT在最坏情况下并不适用，因为个别坐标可能严重偏离这些分布。我们证明，对任意d维输入向量施加两次RHT后，归一化旋转向量的每个固定坐标的边缘分布在Kolmogorov距离和1-Wasserstein距离上均与标准高斯分布相差不超过O(d^{-1/2})。随后，我们将这些边界代入现代压缩方案（即DRIVE和QUIC-FL）的分析中，并证明两次RHT可实现与URR渐近匹配的性能。然而，我们证明两次RHT可能不足以满足向量量化（VQ）的要求——后者通常需要固定大小坐标块之间的弱相关性（而非仅单个坐标的边缘分布收敛）。我们证明三次RHT的组合可导致坐标协方差衰减。这确保任何针对URR优化的固定有界多维VQ码本在使用三次RHT时具有相同的期望误差，仅相差一个随维度增加而消失的加性项。最后，鉴于实际输入极少具有对抗性，我们提出一种基于输入矩的线性时间O(d)检查方法，可在运行时动态调整RHT使用次数以提升性能。