This paper proposes a novel multivariate definition of statistical dependence between two continuous random processes (r.p.) using a functional methodology inspired by Alfr\'ed R\'enyi. The argument of the logarithm of mutual information between pairs of samples of a r.p., named here the normalized cross density (NCD), defines a symmetric and self-adjoint positive definite function. We show that maximizing the alternating covariance estimation (ACE) recursion, applied to each of the joint probability density of input sample pairs, obeys all the properties of Renyi's maximal correlation. We propose the NCD's eigenspectrum as a novel multivariate measure of the statistical dependence between the input and output r.p. The multivariate statistical dependence can also be estimated directly from r.p. realizations. The proposed functional maximum correlation algorithm (FMCA) is applied to a machine learning architecture built from two neural networks that learn concurrently by approximating each others' outputs. We prove that the FMCA optimal solution is an equilibrium point that estimates the eigenspectrum of the cross density kernel. Preliminary results with synthetic data and medium size image datasets corroborate the theory. Four different strategies of applying the cross density kernel are proposed and thoroughly discussed to show the versatility and stability of the methodology, which transcends supervised learning. More specifically, when the two random processes are high-dimensional real-world images and a white uniform noise process, the algorithm learns a factorial code i.e., the occurrence of a code guarantees that a certain input in the training image set was present, which is quite important for feature learning.

翻译：本文提出了一种基于Alfréd Rényi启发的函数方法，用于定义两个连续随机过程之间统计依赖性的多变量新定义。随机过程样本对之间互信息对数的自变量（本文称为归一化交叉密度函数）定义了一个对称且自伴的正定函数。我们证明，应用于输入样本对联合概率密度的交替协方差估计递推最大化过程，满足Rényi最大相关的所有性质。提出将NCD的特征谱作为输入与输出随机过程之间统计依赖性的新型多变量度量，该多变量统计依赖性亦可直接从随机过程实现中估计。所提出的函数最大相关算法应用于由两个神经网络构成的机器学习架构，这两个网络通过近似彼此输出进行同步学习。我们证明FMCA的最优解是估计交叉密度核特征谱的均衡点。合成数据与中等规模图像数据集的初步实验结果验证了该理论。通过提出并深入讨论四种交叉密度核应用策略，展示了该方法超越监督学习的普适性与稳定性。具体而言，当两个随机过程分别为高维真实世界图像与白色均匀噪声过程时，该算法可学习阶乘编码——即编码的出现能保证训练图像集中特定输入的存在，这对特征学习具有重要意义。