Stance detection aims to identify the attitude expressed in a document towards a given target. Techniques such as Chain-of-Thought (CoT) prompting have advanced this task, enhancing a model's reasoning capabilities through the derivation of intermediate rationales. However, CoT relies primarily on a model's pre-trained internal knowledge during reasoning, thereby neglecting the valuable external information that is previously unknown to the model. This omission, especially within the unsupervised reasoning process, can affect the model's overall performance. Moreover, while CoT enhances Large Language Models (LLMs), smaller LMs, though efficient operationally, face challenges in delivering nuanced reasoning. In response to these identified gaps, we introduce the Ladder-of-Thought (LoT) for the stance detection task. Constructed through a dual-phase Progressive Optimization Framework, LoT directs the small LMs to assimilate high-quality external knowledge, refining the intermediate rationales produced. These bolstered rationales subsequently serve as the foundation for more precise predictions - akin to how a ladder facilitates reaching elevated goals. LoT achieves a balance between efficiency and performance. Our empirical evaluations underscore LoT's efficacy, marking a 16% improvement over GPT-3.5 and a 10% enhancement compared to GPT-3.5 with CoT on stance detection task.
翻译:立场检测旨在识别文档中对特定目标所表达的态度。链式思维(CoT)提示等技术通过推导中间推理过程增强了模型的推理能力,推动了该任务的发展。然而,CoT在推理过程中主要依赖模型预训练的内部知识,从而忽视了模型先前未知的有价值外部信息。这种忽略,尤其是在无监督推理过程中,可能影响模型的整体性能。此外,虽然CoT增强了大型语言模型(LLMs),但较小规模的语言模型虽在运行上高效,却在提供细致推理方面面临挑战。针对这些已知不足,我们提出面向立场检测任务的“思维阶梯”(LoT)。通过双阶段渐进优化框架构建的LoT,引导小型语言模型吸收高质量外部知识,精炼其生成的中间推理过程。这些强化的推理过程随后成为更精确预测的基础——正如阶梯助力达成更高目标。LoT实现了效率与性能的平衡。我们的实证评估证实了LoT的有效性,在立场检测任务上相比GPT-3.5提升16%,相比采用CoT的GPT-3.5提升10%。