We present a method for generating alpha mattes using a limited data source. We pretrain a novel transformerbased model (StyleMatte) on portrait datasets. We utilize this model to provide image-mask pairs for the StyleGAN3-based network (StyleMatteGAN). This network is trained unsupervisedly and generates previously unseen imagemask training pairs that are fed back to StyleMatte. We demonstrate that the performance of the matte pulling network improves during this cycle and obtains top results on the human portraits and state-of-the-art metrics on animals dataset. Furthermore, StyleMatteGAN provides high-resolution, privacy-preserving portraits with alpha mattes, making it suitable for various image composition tasks. Our code is available at https://github.com/chroneus/stylematte
翻译:我们提出了一种利用有限数据源生成alpha遮罩的方法。我们首先在肖像数据集上预训练一个基于Transformer的新型模型(StyleMatte),并利用该模型为基于StyleGAN3的网络(StyleMatteGAN)提供图像-遮罩对。该网络以无监督方式训练,生成此前未见过的图像-遮罩训练对,并反馈给StyleMatte。我们证明,在此循环过程中,抠图网络的性能得到提升,在人物肖像数据集上取得了顶尖结果,并在动物数据集上达到了最先进的指标。此外,StyleMatteGAN能够生成高分辨率、保护隐私的带alpha遮罩肖像,适用于各种图像合成任务。我们的代码开源于 https://github.com/chroneus/stylematte。