We present a novel dataset for the controlled composition of counterarguments designed for further applications in argument refining, mining, and evaluation. Our dataset constitutes enriched counter-arguments to posts in the Reddit ChangeMyView dataset that are integrated with evidence retrieved from high-quality sources and generated based on user preferences, adjusting the critical attributes of evidence and argument style. The resultant Counterfire corpus comprises arguments generated from GPT-3.5 turbo, Koala, and PaLM 2 models and two of their finetuned variants (N = 32,000). Model evaluation indicates strong paraphrasing abilities with evidence, albeit limited word overlap, while demonstrating high style integration (0.9682 for 'reciprocity'), showing the ability of LLM to assimilate diverse styles. Of all models, GPT-3.5 turbo showed the highest scores in argument quality evaluation, showing consistent accuracy (score >0.8). In further analyses, reciprocity-style counterarguments display higher counts in most categories, possibly indicating a more creatively persuasive use of evidence. In contrast, human-written counterarguments exhibited greater argumentative richness and diversity across categories. Despite human-written arguments being favored as the most persuasive in human evaluation, the 'No Style' generated text surprisingly exhibited the highest score, prompting further exploration and investigation on the trade-offs in generation for facts and style.

翻译：我们提出一个用于反论点受控组合生成的新型数据集，旨在推动论点精炼、挖掘与评估的进一步应用。该数据集包含针对Reddit平台ChangeMyView数据集中帖子的增强型反论点，这些论点整合了从高质量来源检索的证据，并根据用户偏好调整关键属性（证据与论证风格）。由此生成的《回击》语料库包含来自GPT-3.5 turbo、Koala、PaLM 2模型及其两种微调变体的论点（N=32,000）。模型评估表明，虽词汇重叠有限，但模型具备强大的证据释意能力，同时展现出高风格融合度（"互惠"风格达0.9682），显示大语言模型（LLM）对多样化风格的吸收能力。在所有模型中，GPT-3.5 turbo在论点质量评估中得分最高，准确率持续超过0.8。进一步分析发现，采用互惠风格的反论点在多数类别中数量更多，可能暗示证据使用的创造性说服力。相比之下，人工撰写的反论点在各类别中展现出更丰富的论证多样性与深度。尽管在人类评估中人工论点被认为最具说服力，但"无风格"生成的文本意外获得最高分数，这促使我们进一步探索事实生成与风格表现之间的权衡机制。