While large language models (LLMs) are increasingly used to generate writing feedback, there remains no systematic comparison of LLM and expert feedback on the dimensions that writing research identifies as central to revision: goal-orientation, anchoring to specific sentences, and prioritization. We introduce FOXGLOVE, a dataset of 696 feedback comments written by trained writing instructors on 69 twelfth-grade argumentative essays, paired with 1,644 comments generated from four frontier LLMs under a shared protocol, totaling 2,340 comments. We provide expert quality ratings on a subset of both instructor and LLM comments. We find that instructors and LLMs distribute feedback similarly across goals and essay positions, yet instructors and models diverge on the specific sentences on which to provide feedback. Additionally, we find that models tend to write more complex feedback and use fewer questions than instructors. LLM feedback also receives higher ratings on most dimensions of quality, as rated by instructors, but much of this advantage appears to be attributable to lengthier comments. FOXGLOVE enables systematic comparison of where human and LLM feedback align, diverge, and differ.

翻译：尽管大型语言模型（LLM）被越来越多地用于生成写作反馈，但在写作研究确定为修改核心的维度——目标导向性、对具体句子的锚定性以及优先排序——上，仍缺乏对LLM与专家反馈的系统性比较。我们提出FOXGLOVE数据集，包含由训练有素的写作指导教师针对69篇十二年级议论文撰写的696条反馈评论，以及基于统一协议由四个前沿LLM生成的1,644条评论，共计2,340条评论。我们提供了针对部分教师评论与LLM评论的专家质量评分。研究发现，教师与LLM在反馈的目标分布和文章段落位置上具有相似性，但在提供反馈的具体句子上存在分歧。此外，模型倾向于撰写更复杂的反馈，且使用问句的频率低于教师。尽管LLM反馈在大多数质量维度上获得了教师评定的更高分数，但这种优势很大程度上可归因于其更长的评论篇幅。FOXGLOVE数据集能够系统性比较人类与LLM反馈在哪些方面相互契合、存在分歧或表现出差异。