We study the problem of unsupervised heteroscedastic covariance estimation, where the goal is to learn the multivariate target distribution $\mathcal{N}(y, \Sigma_y | x )$ given an observation $x$. This problem is particularly challenging as $\Sigma_{y}$ varies for different samples (heteroscedastic) and no annotation for the covariance is available (unsupervised). Typically, state-of-the-art methods predict the mean $f_{\theta}(x)$ and covariance $\textrm{Cov}(f_{\theta}(x))$ of the target distribution through two neural networks trained using the negative log-likelihood. This raises two questions: (1) Does the predicted covariance truly capture the randomness of the predicted mean? (2) In the absence of ground-truth annotation, how can we quantify the performance of covariance estimation? We address (1) by deriving TIC: Taylor Induced Covariance, which captures the randomness of the multivariate $f_{\theta}(x)$ by incorporating its gradient and curvature around $x$ through the second order Taylor polynomial. Furthermore, we tackle (2) by introducing TAC: Task Agnostic Correlations, a metric which leverages conditioning of the normal distribution to evaluate the covariance. We verify the effectiveness of TIC through multiple experiments spanning synthetic (univariate, multivariate) and real-world datasets (UCI Regression, LSP, and MPII Human Pose Estimation). Our experiments show that TIC outperforms state-of-the-art in accurately learning the covariance, as quantified through TAC.

翻译：我们研究无监督异方差协方差估计问题，其目标是在给定观测$x$的条件下学习多元目标分布$\mathcal{N}(y, \Sigma_y | x )$。该问题尤为具有挑战性，因为$\Sigma_{y}$随不同样本变化（异方差），且协方差无标注可用（无监督）。典型情况下，现有最优方法通过两个神经网络预测目标分布的均值$f_{\theta}(x)$与协方差$\textrm{Cov}(f_{\theta}(x))$，并采用负对数似然进行训练。这引发两个问题：(1) 预测的协方差是否真正捕获了预测均值的随机性？(2) 在缺少真实标注的情况下，如何量化协方差估计的性能？针对问题(1)，我们推导出泰勒诱导协方差（TIC），该方法通过二阶泰勒多项式融合多元$f_{\theta}(x)$在$x$附近的梯度与曲率，从而捕获其随机性。针对问题(2)，我们提出任务无关相关性（TAC）度量指标，利用正态分布的条件化特性评估协方差。通过涵盖合成数据（单变量、多变量）与真实数据集（UCI回归、LSP、MPII人体姿态估计）的多项实验，我们验证了TIC的有效性。实验表明，经TAC量化评估，TIC在准确学习协方差方面优于现有最优方法。