Businesses and software platforms are increasingly turning to Large Language Models (LLMs) such as GPT-3.5, GPT-4, GLM-3, and LLaMa-2 for chat assistance with file access or as reasoning agents for customer service. However, current LLM-based customer service models have limited integration with customer profiles and lack the operational capabilities necessary for effective service. Moreover, existing API integrations emphasize diversity over the precision and error avoidance essential in real-world customer service scenarios. To address these issues, we propose an LLM agent named CHOPS (CHat with custOmer Profile in existing System), designed to: (1) efficiently utilize existing databases or systems for accessing user information or interacting with these systems following existing guidelines; (2) provide accurate and reasonable responses or carry out required operations in the system while avoiding harmful operations; and (3) leverage a combination of small and large LLMs to achieve satisfying performance at a reasonable inference cost. We introduce a practical dataset, the CPHOS-dataset, which includes a database, guiding files, and QA pairs collected from CPHOS, an online platform that facilitates the organization of simulated Physics Olympiads for high school teachers and students. We have conducted extensive experiments to validate the performance of our proposed CHOPS architecture using the CPHOS-dataset, with the aim of demonstrating how LLMs can enhance or serve as alternatives to human customer service. Code for our proposed architecture and dataset can be found at {https://github.com/JingzheShi/CHOPS}.

翻译：企业和软件平台正日益采用大型语言模型（LLM），如GPT-3.5、GPT-4、GLM-3和LLaMa-2，用于具备文件访问功能的聊天辅助或作为客户服务的推理代理。然而，当前基于LLM的客户服务模型与客户档案的集成有限，且缺乏有效服务所需的操作能力。此外，现有的API集成过于强调多样性，而忽视了实际客户服务场景中至关重要的精确性与错误规避。为解决这些问题，我们提出了一种名为CHOPS（在现有系统中结合客户档案的对话）的LLM代理，其设计目标为：（1）高效利用现有数据库或系统，依据既定规范访问用户信息或与这些系统交互；（2）提供准确合理的响应或在系统中执行所需操作，同时避免有害操作；（3）结合使用小型与大型LLM，以合理的推理成本实现令人满意的性能。我们引入了一个实用数据集CPHOS-dataset，其中包含从CPHOS平台收集的数据库、指导文件及问答对；CPHOS是一个为高中师生组织模拟物理奥林匹克竞赛的在线平台。我们通过大量实验，使用CPHOS-dataset验证了所提出的CHOPS架构的性能，旨在展示LLM如何增强或替代人工客户服务。所提架构及数据集的代码可在{https://github.com/JingzheShi/CHOPS}获取。