3D lookup tables (3D LUTs) are a key component for image enhancement. Modern image signal processors (ISPs) have dedicated support for these as part of the camera rendering pipeline. Cameras typically provide multiple options for picture styles, where each style is usually obtained by applying a unique handcrafted 3D LUT. Current approaches for learning and applying 3D LUTs are notably fast, yet not so memory-efficient, as storing multiple 3D LUTs is required. For this reason and other implementation limitations, their use on mobile devices is less popular. In this work, we propose a Neural Implicit LUT (NILUT), an implicitly defined continuous 3D color transformation parameterized by a neural network. We show that NILUTs are capable of accurately emulating real 3D LUTs. Moreover, a NILUT can be extended to incorporate multiple styles into a single network with the ability to blend styles implicitly. Our novel approach is memory-efficient, controllable and can complement previous methods, including learned ISPs. Code, models and dataset available at: https://github.com/mv-lab/nilut
翻译:三维查找表(3D LUT)是图像增强的关键组件。现代图像信号处理器(ISP)在相机渲染管线中专门支持这一技术。相机通常提供多种图像风格选项,每种风格通常通过应用独特的手工制作3D LUT获得。当前学习和应用3D LUT的方法速度显著且高效,但在存储多个3D LUT时需要耗费大量内存,因此存在内存效率不足的问题。受此限制及其他实现因素影响,这类方法在移动设备上的应用并不普及。本文提出神经隐式查找表(NILUT)——一种由神经网络参数化定义的连续三维颜色变换。我们证明NILUT能够精确模拟真实3D LUT。此外,NILUT可扩展为在单一网络中融合多种风格,并具备隐式混合风格的能力。这种新方法兼具内存高效性与可控性,能对包括已学习的ISP在内的现有方法形成补充。代码、模型及数据集见:https://github.com/mv-lab/nilut