Intrinsic images, in the original sense, are image-like maps of scene properties like depth, normal, albedo or shading. This paper demonstrates that StyleGAN can easily be induced to produce intrinsic images. The procedure is straightforward. We show that, if StyleGAN produces $G({w})$ from latents ${w}$, then for each type of intrinsic image, there is a fixed offset ${d}_c$ so that $G({w}+{d}_c)$ is that type of intrinsic image for $G({w})$. Here ${d}_c$ is {\em independent of ${w}$}. The StyleGAN we used was pretrained by others, so this property is not some accident of our training regime. We show that there are image transformations StyleGAN will {\em not} produce in this fashion, so StyleGAN is not a generic image regression engine. It is conceptually exciting that an image generator should ``know'' and represent intrinsic images. There may also be practical advantages to using a generative model to produce intrinsic images. The intrinsic images obtained from StyleGAN compare well both qualitatively and quantitatively with those obtained by using SOTA image regression techniques; but StyleGAN's intrinsic images are robust to relighting effects, unlike SOTA methods.
翻译:固有图像(Intrinsic images),按其原初定义,是指场景属性(如深度、法向、反照率或明暗)的类图像映射。本文证明,StyleGAN 可以轻易地被引导生成固有图像。其操作步骤直接了当:我们证明,若 StyleGAN 从潜变量 ${w}$ 生成 $G({w})$,则对于每种固有图像类型,存在一个固定偏移量 ${d}_c$,使得 $G({w}+{d}_c)$ 即为 $G({w})$ 所对应的该类固有图像。此处 ${d}_c$ {\em 与 ${w}$ 无关}。我们所用的 StyleGAN 系由他人预训练,因此该性质并非源于我们的训练方案。此外,我们指出存在某些图像变换是 StyleGAN {\em 无法}以这种方式生成的,故 StyleGAN 并非通用的图像回归引擎。一个图像生成器竟能“知晓”并表示固有图像,这在概念上令人振奋。利用生成模型生成固有图像还可能具有实际优势。由 StyleGAN 获得的固有图像,在定性和定量层面均与最先进图像回归技术所得结果相当;然而,不同于现有最优方法,StyleGAN 生成的固有图像对重光照效果具有鲁棒性。