Models such as finite state automata are widely used to abstract the behavior of software systems by capturing the sequences of events observable during their execution. Nevertheless, models rarely exist in practice and, when they do, get easily outdated; moreover, manually building and maintaining models is costly and error-prone. As a result, a variety of model inference methods that automatically construct models from execution traces have been proposed to address these issues. However, performing a systematic and reliable accuracy assessment of inferred models remains an open problem. Even when a reference model is given, most existing model accuracy assessment methods may return misleading and biased results. This is mainly due to their reliance on statistical estimators over a finite number of randomly generated traces, introducing avoidable uncertainty about the estimation and being sensitive to the parameters of the random trace generative process. This paper addresses this problem by developing a systematic approach based on analytic combinatorics that minimizes bias and uncertainty in model accuracy assessment by replacing statistical estimation with deterministic accuracy measures. We experimentally demonstrate the consistency and applicability of our approach by assessing the accuracy of models inferred by state-of-the-art inference tools against reference models from established specification mining benchmarks.

翻译：诸如有限状态自动机等模型被广泛用于通过捕获软件系统执行过程中可观察的事件序列来抽象其行为。然而，在实践中模型很少存在，即便存在也容易过时；此外，手动构建和维护模型既昂贵又容易出错。因此，研究者提出了多种模型推理方法，能从执行轨迹中自动构建模型以解决这些问题。然而，对推理模型进行系统且可靠的精度评估仍是一个开放性问题。即使给定参考模型，大多数现有的模型精度评估方法也可能返回误导性的有偏结果。这主要是因为这些方法依赖基于有限数量随机生成轨迹的统计估计量，引入了关于估计结果的可避免的不确定性，并且对随机轨迹生成过程的参数敏感。本文通过开发一种基于解析组合学的系统方法来解决该问题，该方法通过用确定性精度指标替换统计估计，最大程度减少了模型精度评估中的偏差和不确定性。我们通过将当前最先进推理工具推断出的模型精度与既定规范挖掘基准中的参考模型进行比较，通过实验证明了该方法的一致性和适用性。