Large language models (LLMs) have opened new opportunities for transforming natural language user intents into executable actions. This capability enables embodied AI agents to perform complex tasks, without involvement of an expert, making human-robot interaction (HRI) more convenient. However these developments raise significant security and privacy challenges such as self-preferencing, where a single LLM service provider dominates the market and uses this power to promote their own preferences. LLM oracles have been recently proposed as a mechanism to decentralize LLMs by executing multiple LLMs from different vendors and aggregating their outputs to obtain a more reliable and trustworthy final result. However, the accuracy of these approaches highly depends on the aggregation method. The current aggregation methods mostly use semantic similarity between various LLM outputs, not suitable for robotic task planning, where the temporal order of tasks is important. To fill the gap, we propose an LLM oracle with a new aggregation method for robotic task planning. In addition, we propose a decentralized multi-robot infrastructure based on Hyperledger Fabric that can host the proposed oracle. The proposed infrastructure enables users to express their natural language intent to the system, which then can be decomposed into subtasks. These subtasks require coordinating different robots from different vendors, while enforcing fine-grained access control management on the data. To evaluate our methodology, we created the SkillChain-RTD benchmark made it publicly available. Our experimental results demonstrate the feasibility of the proposed architecture, and the proposed aggregation method outperforms other aggregation methods currently in use.

翻译：大型语言模型（LLM）为将自然语言用户意图转化为可执行动作开辟了新途径。该能力使得具身智能体能够执行复杂任务而无需专家介入，从而使人机交互（HRI）更为便捷。然而，这些进展也引发了严重的安全与隐私挑战，例如自我偏好问题——即单一LLM服务提供商垄断市场并利用其主导地位推行自身偏好。近期提出的LLM预言机机制通过执行来自不同供应商的多个LLM并聚合其输出，以获得更可靠、可信的最终结果，从而实现LLM的去中心化。但此类方法的准确性高度依赖于聚合策略。现有聚合方法多采用不同LLM输出间的语义相似度计算，并不适用于注重任务时序性的机器人任务规划场景。为填补这一空白，我们提出一种采用新型聚合方法的LLM预言机用于机器人任务规划。此外，我们构建了基于Hyperledger Fabric的分布式多机器人基础设施以承载该预言机。该基础设施允许用户向系统表达自然语言意图，系统可将其分解为子任务。这些子任务需要协调来自不同供应商的机器人，同时对数据实施细粒度访问控制管理。为评估方法性能，我们创建了SkillChain-RTD基准测试集并已公开。实验结果表明：所提架构具备可行性，且新型聚合方法性能优于当前主流聚合方法。