This study presents a conceptual framework and a prototype assessment for Large Language Model (LLM)-based Building Energy Management System (BEMS) AI agents to facilitate context-aware energy management in smart buildings through natural language interaction. The proposed framework comprises three modules: perception (sensing), central control (brain), and action (actuation and user interaction), forming a closed feedback loop that captures, analyzes, and interprets energy data to respond intelligently to user queries and manage connected appliances. By leveraging the autonomous data analytics capabilities of LLMs, the BEMS AI agent seeks to offer context-aware insights into energy consumption, cost prediction, and device scheduling, thereby addressing limitations in existing energy management systems. The prototype's performance was evaluated using 120 user queries across four distinct real-world residential energy datasets and different evaluation metrics, including latency, functionality, capability, accuracy, and cost-effectiveness. The generalizability of the framework was demonstrated using ANOVA tests. The results revealed promising performance, measured by response accuracy in device control (86%), memory-related tasks (97%), scheduling and automation (74%), and energy analysis (77%), while more complex cost estimation tasks highlighted areas for improvement with an accuracy of 49%. This benchmarking study moves toward formalizing the assessment of LLM-based BEMS AI agents and identifying future research directions, emphasizing the trade-off between response accuracy and computational efficiency.

翻译：本研究提出了一个概念框架和原型评估，用于基于大型语言模型（LLM）的建筑能源管理系统（BEMS）人工智能代理，旨在通过自然语言交互促进智能建筑中的上下文感知能源管理。所提出的框架包含三个模块：感知（传感）、中央控制（大脑）以及执行（驱动与用户交互），形成一个闭环反馈系统，能够捕获、分析和解读能源数据，从而智能响应用户查询并管理连接的电器设备。通过利用LLM的自主数据分析能力，BEMS人工智能代理致力于提供关于能源消耗、成本预测和设备调度的上下文感知洞察，从而解决现有能源管理系统的局限性。该原型的性能通过使用四个不同的真实世界住宅能源数据集中的120条用户查询以及多种评估指标（包括延迟、功能性、能力、准确性和成本效益）进行了评估。该框架的泛化能力通过方差分析（ANOVA）测试得到了验证。结果显示其性能表现良好，具体体现在设备控制（86%）、记忆相关任务（97%）、调度与自动化（74%）以及能源分析（77%）的响应准确率上，而更复杂的成本估算任务则凸显了需要改进的领域，其准确率为49%。这项基准研究旨在推动基于LLM的BEMS人工智能代理评估的规范化，并指明未来的研究方向，同时强调了响应准确性与计算效率之间的权衡关系。