Precise estimation of cross-correlation or similarity between two random variables lies at the heart of signal detection, hyperdimensional computing, associative memories, and neural networks. Although a vast literature exists on different methods for estimating cross-correlations, the question what is the best and simplest method to estimate cross-correlations using finite samples ? is still not clear. In this paper, we first argue that the standard empirical approach might not be the optimal method even though the estimator exhibits uniform convergence to the true cross-correlation. Instead, we show that there exists a large class of simple non-linear functions that can be used to construct cross-correlators with a higher signal-to-noise ratio (SNR). To demonstrate this, we first present a general mathematical framework using Price's Theorem that allows us to analyze cross-correlators constructed using a mixture of piece-wise linear functions. Using this framework and high-dimensional embedding, we show that some of the most promising cross-correlators are based on Huber's loss functions, margin-propagation (MP) functions, and the log-sum-exp functions.

翻译：精确估计两个随机变量之间的互相关或相似性是信号检测、超维计算、联想记忆和神经网络的核心问题。尽管已有大量文献探讨了互相关估计的不同方法，但"使用有限样本估计互相关的最佳且最简方法是什么"这一问题仍不明确。本文首先论证：即使标准经验估计量能一致收敛于真实互相关值，该估计方法也可能并非最优。相反，我们证明存在一大类简单的非线性函数，可用于构建具有更高信噪比（SNR）的互相关器。为验证这一观点，我们首先提出一个基于普赖斯定理的通用数学框架，用于分析由分段线性函数混合构建的互相关器。利用该框架和髙维嵌入技术，我们发现最具潜力的互相关器包括基于Huber损失函数、边缘传播（MP）函数及对数求和指数函数的互相关器。