Kernel conditional mean embeddings (CMEs) offer a powerful framework for representing conditional distribution, but they often face scalability and expressiveness challenges. In this work, we propose a new method that effectively combines the strengths of deep learning with CMEs in order to address these challenges. Specifically, our approach leverages the end-to-end neural network (NN) optimization framework using a kernel-based objective. This design circumvents the computationally expensive Gram matrix inversion required by current CME methods. To further enhance performance, we provide efficient strategies to optimize the remaining kernel hyperparameters. In conditional density estimation tasks, our NN-CME hybrid achieves competitive performance and often surpasses existing deep learning-based methods. Lastly, we showcase its remarkable versatility by seamlessly integrating it into reinforcement learning (RL) contexts. Building on Q-learning, our approach naturally leads to a new variant of distributional RL methods, which demonstrates consistent effectiveness across different environments.

翻译：核条件均值嵌入（CMEs）为表示条件分布提供了强大的框架，但常面临可扩展性和表达能力方面的挑战。本文提出一种新方法，通过有效结合深度学习与CME的优势来解决这些问题。具体而言，我们的方法利用基于核目标函数的端到端神经网络（NN）优化框架，既避免了当前CME方法所需的计算昂贵的Gram矩阵求逆，又提供了高效策略来优化剩余核超参数以进一步提升性能。在条件密度估计任务中，我们的NN-CME混合方法取得了具有竞争力的性能，且通常超越现有深度学习方法。最后，我们通过将其无缝集成到强化学习（RL）场景中展示了其卓越的通用性——基于Q-learning，该方法自然衍生出分布强化学习的一种新变体，并在不同环境中均表现出稳定的有效性。