Large Language Models (LLMs), despite extensive pretraining on broad internet corpora, often struggle to adapt effectively to specialized domains. There is growing interest in fine-tuning these models for such domains; however, progress is constrained by the scarcity and limited coverage of high-quality, task-relevant data. To address this, synthetic data generation methods such as paraphrasing or knowledge extraction are commonly applied. Although these approaches excel at factual recall and conceptual knowledge, they suffer from two critical shortcomings: (i) they provide minimal support for interpretive reasoning capabilities in these specialized domains, and (ii) they often produce synthetic corpora that are excessively large and redundant, resulting in poor sample efficiency. To overcome these gaps, we propose an adversarial question-generation framework that produces a compact set of semantically challenging questions. These questions are constructed by comparing the outputs of the model to be adapted and a robust expert model grounded in reference documents, using an iterative, feedback-driven process designed to reveal and address comprehension gaps. Evaluation on specialized subsets of the LegalBench corpus demonstrates that our method achieves greater accuracy with substantially fewer synthetic samples.

翻译：尽管大型语言模型（LLMs）已在广泛的互联网语料上进行过大量预训练，但其在适应专业领域时仍常面临困难。针对这些领域对模型进行微调的需求日益增长，然而高质量、任务相关数据的稀缺性与有限覆盖范围制约了进展。为此，通常采用诸如复述或知识抽取等合成数据生成方法。尽管这些方法在事实回忆与概念知识方面表现优异，却存在两个关键缺陷：（i）它们对专业领域中的解释性推理能力支持甚微；（ii）生成的合成语料库往往体量过大且冗余度高，导致样本效率低下。为弥补这些不足，我们提出一种对抗式问题生成框架，该框架能产生一组紧凑的语义挑战性问题。这些问题通过将待适配模型与基于参考文档的鲁棒专家模型的输出进行对比来构建，采用迭代的、反馈驱动的流程，旨在揭示并弥补理解差距。在LegalBench语料库专业子集上的评估表明，我们的方法能够以显著更少的合成样本实现更高的准确率。