Machine learning (ML)-based monitoring systems have been extensively developed to enhance the print quality of additive manufacturing (AM). In-situ and in-process data acquired using sensors can be used to train ML models that detect process anomalies, predict part quality, and adjust process parameters. However, the reproducibility of the proposed AM monitoring systems has not been investigated. There has not been a method to evaluate and improve reproducibility in the joint domain of AM and ML. Consequently, some crucial information for reproducing the research is usually missing from the publications; thus, systems reproduced based on the publications often cannot achieve the claimed performance. This paper establishes the definition of reproducibility in this domain, proposes a reproducibility investigation pipeline, and composes a reproducibility checklist. A research is reproducible if a performance comparable to the original research can be obtained when reproduced by a different team using a different experiment setup. The reproducibility investigation pipeline sequentially guides the readers through all the necessary reproduction steps, during which the reproducibility checklist will help extract the reproducibility information from the publication. A case study that reproduced a vision-based warping detection system demonstrated the usage and validated the efficacy of the proposed pipeline and checklist. It has been observed that the reproducibility checklist can help the authors verify that all the information critical to reproducibility is provided in the publications. The investigation pipeline can help identify the missing reproducibility information, which should be acquired from the original authors to achieve the claimed performance.
翻译:基于机器学习(ML)的监控系统已被广泛开发,以提升增材制造(AM)的打印质量。利用传感器获取的现场与过程数据可用于训练ML模型,以检测过程异常、预测零件质量并调整工艺参数。然而,现有AM监控系统的可复现性尚未得到充分研究。目前尚缺乏在AM与ML交叉领域评估和改进可复现性的方法。因此,现有出版物通常缺失复现研究所需的关键信息;基于这些出版物复现的系统往往无法达到宣称的性能。本文界定了该领域的可复现性定义,提出了可复现性研究流程,并制定了可复现性核查清单。若不同团队使用不同实验设置进行复现时,能获得与原始研究相当的性能,则该研究被视为可复现。可复现性研究流程逐步引导读者完成所有必要的复现步骤,在此过程中,可复现性核查清单将协助从出版物中提取关键复现信息。通过复现基于视觉的翘曲检测系统的案例研究,展示了所提流程与清单的使用方法,并验证了其有效性。研究发现,可复现性核查清单能帮助作者确认出版物是否提供了所有影响可复现性的关键信息。研究流程则有助于识别缺失的可复现性信息,这些信息需向原作者获取,以实现宣称的性能。