Users increasingly query LLM-enabled web chatbots for help with scam defense. The Consumer Financial Protection Bureau's complaints database is a rich data source for evaluating LLM performance on user scam queries, but currently the corpus does not distinguish between scam and non-scam fraud. We developed an LLM ensemble approach to distinguishing scam and fraud CFPB complaints and describe initial findings regarding the strengths and weaknesses of LLMs in the scam defense context.

翻译：用户越来越多地通过支持大语言模型的网络聊天机器人寻求防诈骗帮助。消费者金融保护局的投诉数据库是评估大语言模型处理用户诈骗查询性能的丰富数据源，但当前该语料库未区分诈骗与非诈骗性质的欺诈行为。我们开发了一种大语言模型集成方法，用于区分消费者金融保护局投诉中的诈骗与欺诈类别，并阐述了关于大语言模型在防诈骗场景中优势与局限性的初步发现。