Modern multi-centre randomized controlled trials (MCRCTs) collect massive amounts of tabular data, and are monitored intensively for irregularities by humans. We began by empirically evaluating 6 modern machine learning-based outlier detection algorithms on the task of identifying irregular data in 838 datasets from 7 real-world MCRCTs with a total of 77,001 patients from over 44 countries. Our results reinforce key findings from prior work in the outlier detection literature on data from other domains. Existing algorithms often succeed at identifying irregularities without any supervision, with at least one algorithm exhibiting positive performance 70.6% of the time. However, performance across datasets varies substantially with no single algorithm performing consistently well, motivating new techniques for unsupervised model selection or other means of aggregating potentially discordant predictions from multiple candidate models. We propose the Meta-learned Probabilistic Ensemble (MePE), a simple algorithm for aggregating the predictions of multiple unsupervised models, and show that it performs favourably compared to recent meta-learning approaches for outlier detection model selection. While meta-learning shows promise, small ensembles outperform all forms of meta-learning on average, a negative result that may guide the application of current outlier detection approaches in healthcare and other real-world domains.

翻译：现代多中心随机对照试验（MCRCTs）收集了大量表格数据，并需由人力密集监测异常情况。我们首先对6种基于现代机器学习的异常值检测算法进行了实证评估，任务是从7个真实世界MCRCT的838个数据集中识别异常数据，这些数据集涵盖来自44个国家共77,001名患者。实验结果强化了针对其他领域数据的异常值检测文献中的关键结论：现有算法通常能在无监督条件下成功识别异常，其中至少一种算法在70.6%的情况下表现出正向性能。然而，不同数据集的性能差异显著，且无单一算法始终保持稳定表现，这促使我们探索无监督模型选择或聚合多个候选模型潜在矛盾预测的新方法。我们提出了元学习概率集成（MePE）——一种用于聚合多个无监督模型预测的简单算法，并证明其性能优于近期基于元学习的异常值检测模型选择方法。尽管元学习展现出潜力，但小型集成在平均性能上超越了所有形式的元学习，这一负面结论或可指导当前异常值检测方法在医疗及其他现实领域的应用。