We present ASSIST, an object-wise neural radiance field as a panoptic representation for compositional and realistic simulation. Central to our approach is a novel scene node data structure that stores the information of each object in a unified fashion, allowing online interaction in both intra- and cross-scene settings. By incorporating a differentiable neural network along with the associated bounding box and semantic features, the proposed structure guarantees user-friendly interaction on independent objects to scale up novel view simulation. Objects in the scene can be queried, added, duplicated, deleted, transformed, or swapped simply through mouse/keyboard controls or language instructions. Experiments demonstrate the efficacy of the proposed method, where scaled realistic simulation can be achieved through interactive editing and compositional rendering, with color images, depth images, and panoptic segmentation masks generated in a 3D consistent manner.

翻译：摘要：本文提出ASSIST，一种基于对象级神经辐射场的全景表示方法，用于组合式且真实感的仿真。其核心是一种新型场景节点数据结构，该结构以统一方式存储各对象信息，支持场景内及跨场景的在线交互。通过结合可微神经网络、关联边界框与语义特征，所提结构保障了对独立对象的用户友好型交互操作，从而扩展新视角仿真规模。场景中的对象可通过鼠标/键盘控制或语言指令实现查询、添加、复制、删除、变换及替换。实验表明，该方法通过交互式编辑与组合渲染，能够以三维一致的方式生成彩色图像、深度图像及全景分割掩码，从而实现可扩展的真实感仿真。