Combining data from various sources empowers researchers to explore innovative questions, for example those raised by conducting healthcare monitoring studies. However, the lack of a unique identifier often poses challenges. Record linkage procedures determine whether pairs of observations collected on different occasions belong to the same individual using partially identifying variables (e.g. birth year, postal code). Existing methodologies typically involve a compromise between computational efficiency and accuracy. Traditional approaches simplify this task by condensing information, yet they neglect dependencies among linkage decisions and disregard the one-to-one relationship required to establish coherent links. Modern approaches offer a comprehensive representation of the data generation process, at the expense of computational overhead and reduced flexibility. We propose a flexible method, that adapts to varying data complexities, addressing registration errors and accommodating changes of the identifying information over time. Our approach balances accuracy and scalability, estimating the linkage using a Stochastic Expectation Maximisation algorithm on a latent variable model. We illustrate the ability of our methodology to connect observations using large real data applications and demonstrate the robustness of our model to the linking variables quality in a simulation study. The proposed algorithm FlexRL is implemented and available in an open source R package.

翻译：整合来自不同来源的数据使研究人员能够探索创新性问题，例如开展医疗健康监测研究所提出的问题。然而，缺乏唯一标识符常常带来挑战。记录链接程序利用部分识别变量（如出生年份、邮政编码）判断在不同场合收集的观测值对是否属于同一个体。现有方法通常需要在计算效率与准确性之间进行权衡。传统方法通过压缩信息来简化此任务，但它们忽略了链接决策间的依赖关系，且未考虑建立一致链接所需的一对一对应关系。现代方法虽能全面表征数据生成过程，却以计算开销增加和灵活性降低为代价。我们提出一种灵活的方法，能够适应不同的数据复杂性，处理登记错误并容纳识别信息随时间的变化。我们的方法在准确性与可扩展性之间取得平衡，通过基于隐变量模型的随机期望最大化算法来估计链接。我们通过大规模实际数据应用展示了该方法连接观测值的能力，并在模拟研究中证明了模型对链接变量质量的鲁棒性。所提出的算法 FlexRL 已实现并发布于开源 R 软件包中。