In the last few years, Neural Painting (NP) techniques became capable of producing extremely realistic artworks. This paper advances the state of the art in this emerging research domain by proposing the first approach for Interactive NP. Considering a setting where a user looks at a scene and tries to reproduce it on a painting, our objective is to develop a computational framework to assist the users creativity by suggesting the next strokes to paint, that can be possibly used to complete the artwork. To accomplish such a task, we propose I-Paint, a novel method based on a conditional transformer Variational AutoEncoder (VAE) architecture with a two-stage decoder. To evaluate the proposed approach and stimulate research in this area, we also introduce two novel datasets. Our experiments show that our approach provides good stroke suggestions and compares favorably to the state of the art. Additional details, code and examples are available at https://helia95.github.io/inp-website.

翻译：近年来，神经绘画技术已能生成极为逼真的艺术作品。本文通过首次提出交互式神经绘画方法，推动了这一新兴研究领域的发展。针对用户观察场景并尝试在画布上再现的场景，我们旨在开发一种计算框架，通过推荐可能用于完成作品的下一笔触来辅助用户创作。为实现此目标，我们提出I-Paint方法，这是一种基于条件变压器变分自编码器架构的新型方法，其解码器采用两阶段设计。为评估所提方法并推动该领域研究，我们同时引入两个新型数据集。实验表明，我们的方法能够提供优质的笔触推荐，且性能优于现有技术。更多细节、代码及示例请访问https://helia95.github.io/inp-website。