We introduce LightIt, a method for explicit illumination control for image generation. Recent generative methods lack lighting control, which is crucial to numerous artistic aspects of image generation such as setting the overall mood or cinematic appearance. To overcome these limitations, we propose to condition the generation on shading and normal maps. We model the lighting with single bounce shading, which includes cast shadows. We first train a shading estimation module to generate a dataset of real-world images and shading pairs. Then, we train a control network using the estimated shading and normals as input. Our method demonstrates high-quality image generation and lighting control in numerous scenes. Additionally, we use our generated dataset to train an identity-preserving relighting model, conditioned on an image and a target shading. Our method is the first that enables the generation of images with controllable, consistent lighting and performs on par with specialized relighting state-of-the-art methods.

翻译：我们提出LightIt，一种用于图像生成中显式照明控制的方法。现有的生成方法缺乏照明控制，而这对图像生成的诸多艺术层面（如设定整体氛围或电影感外观）至关重要。为突破这些限制，我们提出以着色图和法线图作为生成条件。我们采用包含投射阴影的单次反射着色进行照明建模。首先训练一个着色估计模块，用于生成真实世界图像与着色图配对的数据集；随后以估计的着色图和法线图为输入训练控制网络。我们的方法在多类场景中实现了高质量图像生成与照明控制。此外，我们利用生成的数据集训练了一个身份保持的重光照模型，该模型以图像和目标着色为条件。本方法是首个实现了可控、一致照明图像生成的方法，且性能与专用重光照领域的最新方法持平。