The conditional mean embedding (CME) encodes Markovian stochastic kernels through their actions on probability distributions embedded within the reproducing kernel Hilbert spaces (RKHS). The CME plays a key role in several well-known machine learning tasks such as reinforcement learning, analysis of dynamical systems, etc. We present an algorithm to learn the CME incrementally from data via an operator-valued stochastic gradient descent. As is well-known, function learning in RKHS suffers from scalability challenges from large data. We utilize a compression mechanism to counter the scalability challenge. The core contribution of this paper is a finite-sample performance guarantee on the last iterate of the online compressed operator learning algorithm with fast-mixing Markovian samples, when the target CME may not be contained in the hypothesis space. We illustrate the efficacy of our algorithm by applying it to the analysis of an example dynamical system.

翻译：条件均值嵌入（CME）通过其作用于再生核希尔伯特空间（RKHS）中嵌入的概率分布，对马尔可夫随机核进行编码。CME在强化学习、动力系统分析等多项知名机器学习任务中发挥着关键作用。我们提出了一种算法，通过算子值随机梯度下降从数据中增量学习CME。众所周知，RKHS中的函数学习面临大数据带来的可扩展性挑战。我们利用压缩机制来应对这一可扩展性挑战。本文的核心贡献在于：当目标CME可能不包含在假设空间中时，针对具有快速混合马尔可夫样本的在线压缩算子学习算法的最后一次迭代，给出了有限样本性能保证。通过将其应用于一个示例动力系统的分析，我们验证了所提算法的有效性。