The development of Large Language Models (LLMs) has created transformative opportunities for the financial industry, especially in the area of financial trading. However, how to integrate LLMs with trading systems has become a challenge. To address this problem, we propose an intelligent trade order recognition pipeline that enables the conversion of trade orders into a standard format in trade execution. The system improves the ability of human traders to interact with trading platforms while addressing the problem of misinformation acquisition in trade execution. In addition, we have created a trade order dataset of 500 pieces of data to simulate real-world trading scenarios. Moreover, we designed several metrics to provide a comprehensive assessment of dataset reliability and the generative power of big models in finance by experimenting with five state-of-the-art LLMs on our dataset. The results indicate that while LLMs demonstrate high generation rates (87.50% to 98.33%) and perfect follow-up rates, they face significant challenges in accuracy (5% to 10%) and completeness, with high missing rates (14.29% to 67.29%). In addition, LLMs tend to over-interrogate, suggesting that large models tend to collect more information, carrying certain challenges for information security.

翻译：大型语言模型（LLMs）的发展为金融行业，特别是在金融交易领域，带来了变革性的机遇。然而，如何将LLMs与交易系统有效整合已成为一项挑战。为解决此问题，我们提出了一种智能交易指令识别流程，能够将交易指令转换为交易执行中的标准格式。该系统提升了人类交易员与交易平台的交互能力，同时解决了交易执行中信息误获取的问题。此外，我们创建了一个包含500条数据的交易指令数据集，以模拟真实世界的交易场景。进一步地，我们设计了一系列评估指标，通过在我们的数据集上对五种最先进的LLMs进行实验，全面评估了数据集的可靠性以及大模型在金融领域的生成能力。结果表明，尽管LLMs展现出较高的生成率（87.50%至98.33%）和完美的后续指令遵循率，但在准确性（5%至10%）和完整性方面面临显著挑战，且缺失率较高（14.29%至67.29%）。此外，LLMs倾向于过度询问，这表明大模型倾向于收集更多信息，这对信息安全带来了一定的挑战。