We introduce the problem of knot-based inverse perceptual art. Given multiple target images and their corresponding viewing configurations, the objective is to find a 3D knot-based tubular structure whose appearance resembles the target images when viewed from the specified viewing configurations. To solve this problem, we first design a differentiable rendering algorithm for rendering tubular knots embedded in 3D for arbitrary perspective camera configurations. Utilizing this differentiable rendering algorithm, we search over the space of knot configurations to find the ideal knot embedding. We represent the knot embeddings via homeomorphisms of the desired template knot, where the homeomorphisms are parametrized by the weights of an invertible neural network. Our approach is fully differentiable, making it possible to find the ideal 3D tubular structure for the desired perceptual art using gradient-based optimization. We propose several loss functions that impose additional physical constraints, ensuring that the tube is free of self-intersection, lies within a predefined region in space, satisfies the physical bending limits of the tube material and the material cost is within a specified budget. We demonstrate through results that our knot representation is highly expressive and gives impressive results even for challenging target images in both single view as well as multiple view constraints. Through extensive ablation study we show that each of the proposed loss function is effective in ensuring physical realizability. To the best of our knowledge, we are the first to propose a fully differentiable optimization framework for knot-based inverse perceptual art. Both the code and data will be made publicly available.

翻译：[translated abstract in Chinese] 我们提出了基于绳结的逆向感知艺术问题。给定多个目标图像及其对应的视角配置，目标是寻找一种三维绳结状管状结构，使其在指定视角配置下呈现的外观与目标图像相似。为解决此问题，我们首先设计了一种可微分渲染算法，用于渲染嵌入三维空间中的管状绳结，该算法适用于任意透视摄像机配置。借助此可微分渲染算法，我们在绳结配置空间中进行搜索，以找到理想的绳结嵌入。我们通过期望模板绳结的同胚映射来表示绳结嵌入，其中同胚映射由可逆神经网络的权重参数化。我们的方法完全可微，因此能够利用基于梯度的优化为期望的感知艺术找到理想的三维管状结构。我们提出了多个损失函数，用于施加额外的物理约束，确保管状结构无自交、位于预设空间区域内、满足管材的物理弯曲极限，并且材料成本在指定预算内。通过实验结果，我们展示了绳结表示具有高表达力，即使在具有挑战性的目标图像（包括单视角与多视角约束）下也能获得令人印象深刻的结果。广泛的消融研究表明，我们提出的每个损失函数在确保物理可实现性方面均有效。据我们所知，我们是首个提出基于绳结的逆向感知艺术的完全可微优化框架的工作。代码与数据将公开发布。