Table look-up realization of image restoration CNNs has the potential of achieving competitive image quality while being much faster and resource frugal than the straightforward CNN implementation. The main technical challenge facing the LUT-based CNN algorithm designers is to manage the table size without overly restricting the receptive field. The prevailing strategy is to reuse the table for small pixel patches of different orientations (apparently assuming a degree of isotropy) and then fuse the look-up results. The fusion is currently done by average pooling, which we find being ill suited to anisotropic signal structures. To alleviate the problem, we investigate and discuss anisotropic pooling methods to replace naive averaging for improving the performance of the current LUT-realizable CNN restoration methods. First, we introduce the method of generalized median pooling which leads to measurable gains over average pooling. We then extend this idea by learning data-dependent pooling coefficients for each orientation, so that they can adaptively weigh the contributions of differently oriented pixel patches. Experimental results on various restoration benchmarks show that our anisotropic pooling strategy yields both perceptually and numerically superior results compared to existing LUT-realizable CNN methods.

翻译：基于查找表（LUT）实现的图像修复卷积神经网络（CNN）在保持竞争力的图像质量的同时，相比直接的CNN实现具有更快的速度和更低的资源消耗。基于LUT的CNN算法设计面临的主要技术挑战是在不过度限制感受野的前提下管理查找表的大小。当前的主流策略是对不同方向的小像素块（显然假设了一定程度的各向同性）复用同一查找表，然后融合查找结果。目前的融合方法采用平均池化，但我们发现该方法不适用于各向异性的信号结构。为缓解此问题，我们研究并讨论了各向异性池化方法，以替代简单的平均池化，从而提升当前基于LUT实现的CNN修复方法的性能。首先，我们引入了广义中值池化方法，该方法相比平均池化带来了可量化的性能提升。随后，我们通过为每个方向学习数据依赖的池化系数来扩展这一思路，使其能够自适应地权衡不同方向像素块的贡献。在各种修复基准测试上的实验结果表明，与现有的基于LUT实现的CNN方法相比，我们的各向异性池化策略在感知质量和数值指标上均取得了更优的结果。