A foveated image can be entirely reconstructed from a sparse set of samples distributed according to the retinal sensitivity of the human visual system, which rapidly decreases with increasing eccentricity. The use of Generative Adversarial Networks has recently been shown to be a promising solution for such a task, as they can successfully hallucinate missing image information. As in the case of other supervised learning approaches, the definition of the loss function and the training strategy heavily influence the quality of the output. In this work,we consider the problem of efficiently guiding the training of foveated reconstruction techniques such that they are more aware of the capabilities and limitations of the human visual system, and thus can reconstruct visually important image features. Our primary goal is to make the training procedure less sensitive to distortions that humans cannot detect and focus on penalizing perceptually important artifacts. Given the nature of GAN-based solutions, we focus on the sensitivity of human vision to hallucination in case of input samples with different densities. We propose psychophysical experiments, a dataset, and a procedure for training foveated image reconstruction. The proposed strategy renders the generator network flexible by penalizing only perceptually important deviations in the output. As a result, the method emphasized the recovery of perceptually important image features. We evaluated our strategy and compared it with alternative solutions by using a newly trained objective metric, a recent foveated video quality metric, and user experiments. Our evaluations revealed significant improvements in the perceived image reconstruction quality compared with the standard GAN-based training approach.

翻译：具有中央凹特性的图像可以从根据人类视觉系统视网膜敏感度（随离心率增加而迅速下降）分布的稀疏采样点中完全重建。近期研究显示，生成对抗网络在此类任务中展现出前景，因其能有效补全缺失的图像信息。与其他监督学习方法类似，损失函数定义和训练策略对输出质量有显著影响。本研究聚焦于如何高效引导中央凹重建技术的训练过程，使其更契合人类视觉系统的能力与局限，从而重建视觉上重要的图像特征。我们的核心目标是降低训练过程对不可察觉失真的敏感度，并着重惩罚感知上重要的人为伪影。鉴于基于GAN的解决方案的特性，我们重点研究了在不同采样密度输入情况下人类视觉对幻觉的敏感性。本文提出了一套包含心理物理实验、数据集和训练流程的中央凹图像重建方法。所提策略通过仅惩罚输出中感知显著偏差来增强生成器网络的灵活性，从而优先恢复感知重要的图像特征。我们采用新训练的目标度量指标、新型中央凹视频质量度量及用户实验，将本策略与替代方案进行对比评估。结果表明，与传统GAN训练方法相比，本方法在感知图像重建质量方面取得了显著提升。