We introduce a novel continual learning problem: how to sequentially update the weights of a personalized 2D and 3D generative face model as new batches of photos in different appearances, styles, poses, and lighting are captured regularly. We observe that naive sequential fine-tuning of the model leads to catastrophic forgetting of past representations of the individual's face. We then demonstrate that a simple random sampling-based experience replay method is effective at mitigating catastrophic forgetting when a relatively large number of images can be stored and replayed. However, for long-term deployment of these models with relatively smaller storage, this simple random sampling-based replay technique also forgets past representations. Thus, we introduce a novel experience replay algorithm that combines random sampling with StyleGAN's latent space to represent the buffer as an optimal convex hull. We observe that our proposed convex hull-based experience replay is more effective in preventing forgetting than a random sampling baseline and the lower bound.

翻译：我们提出了一种新颖的持续学习问题：当定期捕获具有不同外观、风格、姿态和光照条件的新批次照片时，如何顺序更新个性化2D和3D生成式人脸模型的权重。我们观察到，对模型进行简单的顺序微调会导致对个体面部历史表征的灾难性遗忘。随后我们证明，在能够存储和回放相对大量图像的情况下，基于随机采样的简单体验回放方法能有效缓解灾难性遗忘。然而，对于存储容量相对较小的长期模型部署场景，这种基于随机采样的简单回放技术同样会遗忘历史表征。为此，我们提出了一种创新的体验回放算法，该方法将随机采样与StyleGAN的潜在空间相结合，通过最优凸包来表示回放缓冲区。实验表明，我们提出的基于凸包的体验回放方法在防止遗忘方面优于随机采样基线和下界基准。