Recent developments in Large Language Models (LLMs) have demonstrated their remarkable capabilities across a range of tasks. Questions, however, persist about the nature of LLMs and their potential to integrate common-sense human knowledge when performing tasks involving information about the real physical world. This paper delves into these questions by exploring how LLMs can be extended to interact with and reason about the physical world through IoT sensors and actuators, a concept that we term "Penetrative AI". The paper explores such an extension at two levels of LLMs' ability to penetrate into the physical world via the processing of sensory signals. Our preliminary findings indicate that LLMs, with ChatGPT being the representative example in our exploration, have considerable and unique proficiency in employing the embedded world knowledge for interpreting IoT sensor data and reasoning over them about tasks in the physical realm. Not only this opens up new applications for LLMs beyond traditional text-based tasks, but also enables new ways of incorporating human knowledge in cyber-physical systems.

翻译：近期大语言模型（LLM）的发展已展现出其在各类任务中的卓越能力。然而，关于LLM的本质及其在执行涉及真实物理世界信息的任务时整合常识性人类知识的潜力，仍存在诸多疑问。本文通过探索如何借助物联网传感器与执行器扩展LLM与物理世界交互及推理的能力（我们称之为"穿透式人工智能"），深入探讨了这些问题。本文从LLM通过处理传感信号"穿透"物理世界的两个能力层级展开探索。初步研究结果表明，以ChatGPT为代表的LLM在运用内嵌世界知识解读物联网传感器数据并基于此进行物理领域任务推理方面，展现出显著且独特的优势。这不仅为LLM开辟了超越传统文本任务的新应用场景，也为在网络-物理系统中融入人类知识提供了新途径。