For models consisting of a classifier in some representation space, learning online from a non-stationary data stream often necessitates changes in the representation. So, the question arises of what is the best way to adapt the classifier to shifts in representation. Current methods only slowly change the classifier to representation shift, introducing noise into learning as the classifier is misaligned to the representation. We propose DeepCCG, an empirical Bayesian approach to solve this problem. DeepCCG works by updating the posterior of a class conditional Gaussian classifier such that the classifier adapts in one step to representation shift. The use of a class conditional Gaussian classifier also enables DeepCCG to use a log conditional marginal likelihood loss to update the representation. To perform the update to the classifier and representation, DeepCCG maintains a fixed number of examples in memory and so a key part of DeepCCG is selecting what examples to store, choosing the subset that minimises the KL divergence between the true posterior and the posterior induced by the subset. We explore the behaviour of DeepCCG in online continual learning (CL), demonstrating that it performs well against a spectrum of online CL methods and that it reduces the change in performance due to representation shift.

翻译：对于在某种表征空间中由分类器构成的模型而言，从非平稳数据流中在线学习通常需要改变表征方式。因此，一个关键问题是如何最优地调整分类器以适应表征偏移。现有方法仅能缓慢地调整分类器以适应表征偏移，导致分类器与表征失配而在学习过程中引入噪声。我们提出DeepCCG——一种用于解决该问题的经验贝叶斯方法。DeepCCG通过更新类别条件高斯分类器的后验分布，使分类器能够一步适应表征偏移。类别条件高斯分类器的应用还使得DeepCCG能够利用对数条件边际似然损失来更新表征。为执行分类器与表征的更新，DeepCCG在内存中维护固定数量的样本，因此其核心在于选择存储哪些样本——选择能最小化真实后验与子集诱导后验之间KL散度的子集。我们探究了DeepCCG在在线持续学习中的表现，结果表明其性能优于多种在线持续学习方法，并能有效降低表征偏移带来的性能变化。