Vehicle models have a long history of research and as of today are able to model the involved physics in a reasonable manner. However, each new vehicle has its new characteristics or parameters. The identification of these is the main task of an engineer. To validate whether the correct parameter set has been chosen is a tedious task and often can only be performed by experts. Metrics known commonly used in literature are able to compare different results under certain aspects. However, they fail to answer the question: Are the models accurate enough? In this article, we propose the usage of a custom metric trained on the knowledge of experts to tackle this problem. Our approach involves three main steps: first, the formalized collection of subject matter experts' opinion on the question: Having seen the measurement and simulation time series in comparison, is the model quality sufficient? From this step, we obtain a data set that is able to quantify the sufficiency of a simulation result based on a comparison to corresponding experimental data. In a second step, we compute common model metrics on the measurement and simulation time series and use these model metrics as features to a regression model. Third, we fit a regression model to the experts' opinions. This regression model, i.e., our custom metric, can than predict the sufficiency of a new simulation result and gives a confidence on this prediction.

翻译：车辆模型的研究历史悠久，至今已能够以合理方式模拟相关物理过程。然而，每种新型车辆都具有其独特的特性或参数。识别这些参数是工程师的主要任务。验证所选参数集是否正确是一项繁琐的工作，通常只能由专家完成。文献中常用的度量标准能够在特定方面比较不同结果，但无法回答核心问题：模型是否足够精确？本文提出利用基于专家知识训练的自定义度量来解决这一问题。我们的方法包含三个主要步骤：首先，系统收集领域专家对以下问题的判断意见：在对比观测测量数据与仿真时序数据后，模型质量是否满足要求？通过此步骤，我们获得能够基于仿真结果与实验数据对比来量化仿真充分性的数据集。第二步，我们计算测量数据与仿真时序数据的常用模型度量指标，并将这些指标作为回归模型的特征。第三步，我们通过回归模型拟合专家判断意见。该回归模型（即我们的自定义度量）能够预测新仿真结果的充分性，并提供相应的置信度评估。