Businesses and software platforms are increasingly turning to Large Language Models (LLMs) such as GPT-3.5, GPT-4, GLM-3, and LLaMa-2 for chat assistance with file access or as reasoning agents for customer service. However, current LLM-based customer service models have limited integration with customer profiles and lack the operational capabilities necessary for effective service. Moreover, existing API integrations emphasize diversity over the precision and error avoidance essential in real-world customer service scenarios. To address these issues, we propose an LLM agent named CHOPS (CHat with custOmer Profile in existing System), designed to: (1) efficiently utilize existing databases or systems for accessing user information or interacting with these systems following existing guidelines; (2) provide accurate and reasonable responses or carry out required operations in the system while avoiding harmful operations; and (3) leverage a combination of small and large LLMs to achieve satisfying performance at a reasonable inference cost. We introduce a practical dataset, the CPHOS-dataset, which includes a database, guiding files, and QA pairs collected from CPHOS, an online platform that facilitates the organization of simulated Physics Olympiads for high school teachers and students. We have conducted extensive experiments to validate the performance of our proposed CHOPS architecture using the CPHOS-dataset, with the aim of demonstrating how LLMs can enhance or serve as alternatives to human customer service. Code for our proposed architecture and dataset can be found at {https://github.com/JingzheShi/CHOPS}.

翻译：企业和软件平台正日益转向使用大型语言模型（如GPT-3.5、GPT-4、GLM-3和LLaMa-2）进行文件访问的聊天辅助或作为客户服务的推理代理。然而，当前基于LLM的客户服务模型与客户档案的整合有限，且缺乏有效服务所需的操作能力。此外，现有的API集成侧重于多样性，而非现实客户服务场景中至关重要的精确性与错误规避。为解决这些问题，我们提出了一种名为CHOPS（基于现有系统的客户档案对话代理）的LLM代理，其设计目标包括：（1）高效利用现有数据库或系统以访问用户信息，或遵循现有规范与这些系统进行交互；（2）提供准确合理的响应或在系统中执行必要操作，同时避免有害操作；（3）结合使用小型与大型LLM，以合理的推理成本实现令人满意的性能。我们引入了一个实用数据集CPHOS-dataset，其中包含从CPHOS平台收集的数据库、指导文件和问答对；CPHOS是一个为高中师生组织模拟物理奥林匹克竞赛的在线平台。我们通过大量实验，使用CPHOS-dataset验证了所提出的CHOPS架构的性能，旨在展示LLM如何增强或替代人工客户服务。所提架构及数据集的代码可见于{https://github.com/JingzheShi/CHOPS}。