Motivation: Patient-generated text contains critical information on patients' lived experiences, social context, and care engagement, but remains largely unstructured, limiting its use in patient-centered outcomes research. Prior work introduced the PV-Miner benchmark and PVMinerLLM models for structured extraction. However, supervised fine-tuning (SFT) alone struggles with rare, fine-grained, and unevenly distributed errors, particularly in token-critical structured outputs. Results: We present PVminerLLM2, an improved set of LLMs for structured patient voice extraction that applies preference optimization to address token-critical errors beyond the reach of supervised fine-tuning. Our method introduces (i) a preference objective with token-level gated stabilization term that prevents degradation of absolute token likelihood under preference optimization, and (ii) confusion-aware preference pair construction to better capture low-separation distinctions. We further incorporate token-importance weighting and inverse-frequency reweighing to address token imbalance and class skew. Across multiple model sizes, PVMinerLLM2 consistently outperforms strong baselines, achieving gains of up to 4.43% (Code), 3.50% (Sub-code), and 1.55% (Span), and outperforms baseline LLM trained with existing preference optimization methods. Availability and Implementation: The supplementary material, code, evaluation scripts, and trained models for PVminerLLM2 are publicly available at: https://github.com/Data-Mining-Lab-Yale/PVminerLLM2

翻译：动机：患者生成的文本包含患者生活体验、社会环境与护理参与的关键信息，但大多为非结构化形式，限制了其在以患者为中心的结果研究中的应用。前期工作提出了PV-Miner基准测试及PVMinerLLM模型用于结构化提取。然而，仅靠监督微调（SFT）难以处理罕见、细粒度且分布不均的错误，尤其是在对token关键的结构化输出中。结果：我们提出PVminerLLM2——一组经过改进的大语言模型，通过应用偏好优化来解决监督微调无法触及的token关键错误。本方法引入了：(i) 带有token级门控稳定项的偏好目标，可防止偏好优化下绝对token似然值的退化；(ii) 基于混淆感知的偏好对构建策略，以更好捕捉低区分度的差异。我们进一步引入token重要性加权与逆频率重加权，以应对token不平衡与类别偏斜问题。在多个模型规模下，PVMinerLLM2始终优于强基线模型，在代码、子代码和跨度任务上分别取得了最高4.43%、3.50%和1.55%的性能提升，并超越了采用现有偏好优化方法训练的基线大语言模型。可用性与实现：PVminerLLM2的补充材料、代码、评估脚本及训练模型已公开于：https://github.com/Data-Mining-Lab-Yale/PVminerLLM2