In this paper, we propose a technique for making humans in photographs protrude like reliefs. Unlike previous methods which mostly focus on the face and head, our method aims to generate art works that describe the whole body activity of the character. One challenge is that there is no ground-truth for supervised deep learning. We introduce a sigmoid variant function to manipulate gradients tactfully and train our neural networks by equipping with a loss function defined in gradient domain. The second challenge is that actual photographs often across different light conditions. We used image-based rendering technique to address this challenge and acquire rendering images and depth data under different lighting conditions. To make a clear division of labor in network modules, a two-scale architecture is proposed to create high-quality relief from a single photograph. Extensive experimental results on a variety of scenes show that our method is a highly effective solution for generating digital 2.5D artwork from photographs.
翻译:摘要:本文提出了一种技术,旨在使照片中的人物如浮雕般立体凸显。与以往主要关注人脸及头部的方法不同,本方法旨在生成描述人物全身活动的艺术作品。面临的挑战之一是缺乏用于监督深度学习的真实数据。我们引入了一种Sigmoid变体函数,巧妙地对梯度进行操作,并通过在梯度域中定义损失函数来训练神经网络。第二个挑战在于真实照片往往在不同光照条件下拍摄。我们采用基于图像的渲染技术解决该问题,获取不同光照条件下的渲染图像与深度数据。为实现网络模块的清晰分工,我们提出了一种双尺度架构,从而从单张照片生成高质量浮雕。在多种场景下的大量实验结果表明,本方法是生成数字2.5D艺术作品的极为有效方案。