Models such as finite state automata are widely used to abstract the behavior of software systems by capturing the sequences of events observable during their execution. Nevertheless, models rarely exist in practice and, when they do, get easily outdated; moreover, manually building and maintaining models is costly and error-prone. As a result, a variety of model inference methods that automatically construct models from execution traces have been proposed to address these issues. However, performing a systematic and reliable accuracy assessment of inferred models remains an open problem. Even when a reference model is given, most existing model accuracy assessment methods may return misleading and biased results. This is mainly due to their reliance on statistical estimators over a finite number of randomly generated traces, introducing avoidable uncertainty about the estimation and being sensitive to the parameters of the random trace generative process. This paper addresses this problem by developing a systematic approach based on analytic combinatorics that minimizes bias and uncertainty in model accuracy assessment by replacing statistical estimation with deterministic accuracy measures. We experimentally demonstrate the consistency and applicability of our approach by assessing the accuracy of models inferred by state-of-the-art inference tools against reference models from established specification mining benchmarks.

翻译：有限状态自动机等模型广泛用于通过捕捉软件系统执行过程中可观测事件的序列来抽象其行为。然而，实践中模型很少存在，即便存在也容易过时；此外，手动构建和维护模型成本高昂且易出错。为此，研究者提出了多种从执行轨迹自动构建模型的模型推理方法以解决这些问题。然而，对推理模型进行系统且可靠的准确性评估仍是一个开放性问题。即便给定参考模型，现有大多数模型准确性评估方法仍可能返回有误导性和偏倚的结果。这主要源于它们依赖基于有限数量随机生成轨迹的统计估计量，引入了关于估计的不确定性且对随机轨迹生成过程的参数敏感。本文通过开发基于解析组合数学的系统方法解决此问题，该方法用确定性准确性度量替代统计估计，最小化模型准确性评估中的偏倚和不确定性。我们通过评估利用最先进的推理工具从既定规范挖掘基准中的参考模型推理出的模型准确性，实验证明了该方法的一致性和适用性。