Constraint Acquisition (CA) and related research on the validation and enhancement of Mathematical Programming (MP) models from domain knowledge artifacts are currently limited by inadequate benchmarks. This deficiency impedes reproducibility and cross-study comparability, slowing the maturation of CA methods. Existing benchmarks were designed for solver evaluation rather than for assessing CA algorithms. They are loosely organized, treat individual problems inconsistently, and omit the domain knowledge artifacts required by CA methods. This work presents MPMMine, a benchmark suite designed to assess algorithms that discover, validate, and enhance MP models using diverse domain knowledge artifacts. MPMMine is guided by consistency, standardization, completeness, extensibility, openness, and version control. It adopts a uniform structure and relies on open formats: MiniZinc, CommonMark, and JSON. It provides multiple models per problem, tens of instances per model, and thousands of solutions and non-solutions in both integer and continuous domains, alongside natural-language descriptions to support text-to-model methods.

翻译：约束获取(CA)及相关基于领域知识制品对数学规划(MP)模型进行验证与增强的研究，目前受限于不充分的基准测试。这种缺陷阻碍了可重复性和跨研究可比性，延缓了CA方法的成熟。现有基准测试专为求解器评估而非CA算法评估设计，其组织松散，对单问题实例处理标准不一，且缺少CA方法所需的领域知识制品。本研究提出基准测试套件MPMMine，旨在评估利用多样化领域知识制品发现、验证及增强MP模型的各类算法。MPMMine遵循一致性、标准化、完备性、可扩展性、开放性和版本控制原则，采用统一结构并依托开放式格式：MiniZinc、CommonMark和JSON。该套件为每个问题提供多个模型、每个模型配备数十个实例，并在整数与连续域中提供数千个解与非解实例，另附自然语言描述以支持文本到模型方法。