Modelling statistical relationships beyond the conditional mean is crucial in many settings. Conditional density estimation (CDE) aims to learn the full conditional probability density from data. Though highly expressive, neural network based CDE models can suffer from severe over-fitting when trained with the maximum likelihood objective. Due to the inherent structure of such models, classical regularization approaches in the parameter space are rendered ineffective. To address this issue, we develop a model-agnostic noise regularization method for CDE that adds random perturbations to the data during training. We demonstrate that the proposed approach corresponds to a smoothness regularization and prove its asymptotic consistency. In our experiments, noise regularization significantly and consistently outperforms other regularization methods across seven data sets and three CDE models. The effectiveness of noise regularization makes neural network based CDE the preferable method over previous non- and semi-parametric approaches, even when training data is scarce.

翻译：在许多环境下,模拟超出有条件平均值的统计关系至关重要。 有条件密度估计(CDE)旨在从数据中学习完全的有条件概率密度。 虽然高度直观,但以神经网络为基础的CDE模型在经过培训时可能严重超标。由于这些模型的内在结构,参数空间的典型正规化方法变得无效。为了解决这一问题,我们为CDE开发了一种模型 -- -- 不可知性噪音规范化方法,在培训期间为数据添加随机扰动。我们证明,拟议方法与平稳性规范相对应,并证明它无症状的一致性。在我们的实验中,噪声规范化明显和一贯地超越了七个数据集和三个CDE模型的其他正规化方法。噪音规范化的效力使得以神经网络为基础的CDE比以前的非参数和半参数方法更可取,即使培训数据稀缺。