Augmented Reality (AR) can simulate various visual perceptions, such as how individuals with colorblindness see the world. However, these simulations require developers to predefine each visual effect, limiting flexibility. We present ShadAR, an AR application enabling real-time transformation of visual perception through shader generation using large language models (LLMs). ShadAR allows users to express their visual intent via natural language, which is interpreted by an LLM to generate corresponding shader code. This shader is then compiled real-time to modify the AR headset viewport. We present our LLM-driven shader generation pipeline and demonstrate its ability to transform visual perception for inclusiveness and creativity.

翻译：增强现实（AR）能够模拟多种视觉感知，例如色盲人群观察世界的方式。然而，这些模拟需要开发者预先定义每种视觉效果，限制了系统的灵活性。本文提出ShadAR——一款通过大语言模型（LLM）驱动着色器生成实现实时视觉感知转换的AR应用。ShadAR允许用户通过自然语言描述视觉意图，由LLM解析并生成相应的着色器代码。该着色器经实时编译后可直接修改AR头戴设备的视口显示。我们详细阐述了基于LLM的着色器生成技术流程，并展示了该系统在促进视觉包容性与创造性表达方面的应用潜力。