Commit messages (CMs) are an essential part of version control. By providing important context in regard to what has changed and why, they strongly support software maintenance and evolution. But writing good CMs is difficult and often neglected by developers. So far, there is no tool suitable for practice that automatically assesses how well a CM is written, including its meaning and context. Since this task is challenging, we ask the research question: how well can the CM quality, including semantics and context, be measured with machine learning methods? By considering all rules from the most popular CM quality guideline, creating datasets for those rules, and training and evaluating state-of-the-art machine learning models to check those rules, we can answer the research question with: sufficiently well for practice, with the lowest F$_1$ score of 82.9\%, for the most challenging task. We develop a full-fledged open-source framework that checks all these CM quality rules. It is useful for research, e.g., automatic CM generation, but most importantly for software practitioners to raise the quality of CMs and thus the maintainability and evolution speed of their software.

翻译：提交信息是版本控制的重要组成部分。通过提供关于变更内容及原因的上下文信息，提交信息对软件维护与演进具有重要支撑作用。然而，编写高质量的提交信息较为困难，且常被开发者忽视。目前尚缺乏适用于实践、能够自动评估提交信息质量（包括其语义与上下文）的工具。鉴于该任务的挑战性，我们提出研究问题：如何利用机器学习方法有效衡量包含语义和上下文的提交信息质量？通过整合最流行提交信息质量指南中的所有规则，为这些规则构建数据集，并训练与评估最先进的机器学习模型以检验这些规则，我们得出研究结论：该方法能够充分满足实践需求，在最困难任务上的最低F₁分数达82.9%。我们开发了一套完整的开源框架用于检查所有提交信息质量规则。该框架对自动提交信息生成等研究领域具有实用价值，更重要的是能够帮助软件从业者提升提交信息质量，进而提高软件的可维护性与演进速度。