Consider two $D$-dimensional data vectors (e.g., embeddings): $u, v$. In many embedding-based retrieval (EBR) applications where the vectors are generated from trained models, $D=256\sim 1024$ are common. In this paper, OPORP (one permutation + one random projection) uses a variant of the ``count-sketch'' type of data structures for achieving data reduction/compression. With OPORP, we first apply a permutation on the data vectors. A random vector $r$ is generated i.i.d. with moments: $E(r_i) = 0, E(r_i^2)=1, E(r_i^3) =0, E(r_i^4)=s$. We multiply (as dot product) $r$ with all permuted data vectors. Then we break the $D$ columns into $k$ equal-length bins and aggregate (i.e., sum) the values in each bin to obtain $k$ samples from each data vector. One crucial step is to normalize the $k$ samples to the unit $l_2$ norm. We show that the estimation variance is essentially: $(s-1)A + \frac{D-k}{D-1}\frac{1}{k}\left[ (1-\rho^2)^2 -2A\right]$, where $A\geq 0$ is a function of the data ($u,v$). This formula reveals several key properties: (1) We need $s=1$. (2) The factor $\frac{D-k}{D-1}$ can be highly beneficial in reducing variances. (3) The term $\frac{1}{k}(1-\rho^2)^2$ is a substantial improvement compared with $\frac{1}{k}(1+\rho^2)$, which corresponds to the un-normalized estimator. We illustrate that by letting the $k$ in OPORP to be $k=1$ and repeat the procedure $m$ times, we exactly recover the work of ``very spars random projections'' (VSRP). This immediately leads to a normalized estimator for VSRP which substantially improves the original estimator of VSRP. In summary, with OPORP, the two key steps: (i) the normalization and (ii) the fixed-length binning scheme, have considerably improved the accuracy in estimating the cosine similarity, which is a routine (and crucial) task in modern embedding-based retrieval (EBR) applications.

翻译：考虑两个 $D$ 维数据向量（如嵌入向量）：$u, v$。在许多基于嵌入的检索（EBR）应用中，向量由训练模型生成，通常 $D=256\sim 1024$。本文提出的 OPORP（一次排列加一次随机投影）采用一种变体“计数草图”型数据结构实现数据降维/压缩。通过OPORP，我们首先对数据向量施加一次排列，并生成独立同分布的随机向量 $r$，其矩满足：$E(r_i) = 0, E(r_i^2)=1, E(r_i^3) =0, E(r_i^4)=s$。我们将 $r$ 与所有排列后的数据向量进行点乘运算，然后将 $D$ 个列等分为 $k$ 个长度相同的桶，对每个桶内的值进行聚合（即求和），从而从每个数据向量中获得 $k$ 个样本。关键步骤是将这 $k$ 个样本归一化至单位 $l_2$ 范数。我们证明估计方差本质上为：$(s-1)A + \frac{D-k}{D-1}\frac{1}{k}\left[ (1-\rho^2)^2 -2A\right]$，其中 $A\geq 0$ 是数据 $(u,v)$ 的函数。该公式揭示了若干关键性质：（1）需满足 $s=1$；（2）因子 $\frac{D-k}{D-1}$ 可显著降低方差；（3）项 $\frac{1}{k}(1-\rho^2)^2$ 相比非归一化估计器的 $\frac{1}{k}(1+\rho^2)$ 具有显著改进。我们证明，令 OPORP 中 $k=1$ 并重复该过程 $m$ 次，可精确恢复“极稀疏随机投影”（VSRP）方法。这立即导出了 VSRP 的归一化估计器，显著改进了原始 VSRP 估计器。综上所述，通过 OPORP 的两个关键步骤——(i) 归一化与 (ii) 固定长度分桶方案——显著提升了余弦相似度估计的精度，而余弦相似度估计是现代基于嵌入的检索（EBR）应用中常规且至关重要的任务。