Data quality is an important consideration in many engineering applications and projects. Data collection procedures do not always involve careful utilization of the most precise instruments and strictest protocols. As a consequence, data are invariably affected by imprecision and sometimes sharply varying levels of quality of the data. Different mathematical representations of imprecision have been suggested, including a classical approach to censored data which is considered optimal when the proposed error model is correct, and a weaker approach called interval statistics based on partial identification that makes fewer assumptions. Maximizing the quality of statistical results is often crucial to the success of many engineering projects, and a natural question that arises is whether data of differing qualities should be pooled together or we should include only precise measurements and disregard imprecise data. Some worry that combining precise and imprecise measurements can depreciate the overall quality of the pooled data. Some fear that excluding data of lesser precision can increase their overall uncertainty about results because lower sample size implies more sampling uncertainty. This paper explores these concerns and describes simulation results that show when it is advisable to combine fairly precise data with rather imprecise data by comparing analyses using different mathematical representations of imprecision. Pooling data sets is preferred when the low-quality data set does not exceed a certain level of uncertainty. However, so long as the data are random, it may be legitimate to reject the low-quality data if its reduction of sampling uncertainty does not counterbalance the effect of its imprecision on the overall uncertainty.
翻译:数据质量是许多工程应用和项目中的重要考量因素。数据收集过程并不总能确保使用最精密的仪器和最严格的规程,因此数据不可避免地会受到不精确性的影响,且数据质量往往呈现剧烈变化。针对不精确性已提出多种数学表示方法,包括一种经典删失数据处理方法(在误差模型正确时被认为是最优的),以及基于部分识别假设较少的区间统计方法。提升统计结果质量对许多工程项目的成功至关重要,由此自然产生一个问题:不同质量的数据应当合并使用,还是仅保留精确测量值而舍弃不精确数据?有人担忧合并精确与不精确测量值会降低整体数据质量,也有人担心剔除精度较低的数据会增加结果的不确定性——因为样本量减少会导致抽样不确定性增大。本文通过比较采用不同不精确性数学表示方法的分析结果,探讨了这些关切,并展示了何时适合将较高精度数据与相当不精确数据合并的仿真结论。当低质量数据集的不确定性未超过特定阈值时,优先采用数据合并策略;但在数据满足随机性的条件下,若低质量数据减少抽样不确定性的程度无法抵消其不精确性对整体不确定性的影响,则舍弃该低质量数据可能是合理的。