Deep heteroscedastic regression involves jointly optimizing the mean and covariance of the predicted distribution using the negative log-likelihood. However, recent works show that this may result in sub-optimal convergence due to the challenges associated with covariance estimation. While the literature addresses this by proposing alternate formulations to mitigate the impact of the predicted covariance, we focus on improving the predicted covariance itself. We study two questions: (1) Does the predicted covariance truly capture the randomness of the predicted mean? (2) In the absence of supervision, how can we quantify the accuracy of covariance estimation? We address (1) with a Taylor Induced Covariance (TIC), which captures the randomness of the predicted mean by incorporating its gradient and curvature through the second order Taylor polynomial. Furthermore, we tackle (2) by introducing a Task Agnostic Correlations (TAC) metric, which combines the notion of correlations and absolute error to evaluate the covariance. We evaluate TIC-TAC across multiple experiments spanning synthetic and real-world datasets. Our results show that not only does TIC accurately learn the covariance, it additionally facilitates an improved convergence of the negative log-likelihood. Our code is available at https://github.com/vita-epfl/TIC-TAC

翻译：深度异方差回归涉及使用负对数似然联合优化预测分布的均值和协方差。然而，近期研究表明，由于协方差估计相关的挑战，这可能导致次优收敛。尽管现有文献通过提出替代公式来减轻预测协方差的影响以解决此问题，我们则专注于改进预测协方差本身。我们研究两个问题：(1) 预测协方差是否真正捕捉了预测均值的随机性？(2) 在缺乏监督的情况下，我们如何量化协方差估计的准确性？我们通过泰勒诱导协方差（TIC）解决(1)，该方法通过二阶泰勒多项式结合预测均值的梯度和曲率来捕捉其随机性。此外，我们通过引入任务无关相关性（TAC）指标来解决(2)，该指标结合了相关性和绝对误差的概念来评估协方差。我们在涵盖合成和真实数据集的多个实验中评估TIC-TAC。结果表明，TIC不仅能够准确学习协方差，还促进了负对数似然的改进收敛。我们的代码可在 https://github.com/vita-epfl/TIC-TAC 获取。