Data sharing is a necessity for innovative progress in many domains, especially in healthcare. However, the ability to share data is hindered by regulations protecting the privacy of natural persons. Synthetic tabular data provide a promising solution to address data sharing difficulties but does not inherently guarantee privacy. Still, there is a lack of agreement on appropriate methods for assessing the privacy-preserving capabilities of synthetic data, making it difficult to compare results across studies. To the best of our knowledge, this is the first work to identify properties that constitute good universal privacy evaluation metrics for synthetic tabular data. The goal of such metrics is to enable comparability across studies and to allow non-technical stakeholders to understand how privacy is protected. We identify four principles for the assessment of metrics: Comparability, Applicability, Interpretability, and Representativeness (CAIR). To quantify and rank the degree to which evaluation metrics conform to the CAIR principles, we design a rubric using a scale of 1-4. Each of the four properties is scored on four parameters, yielding 16 total dimensions. We study the applicability and usefulness of the CAIR principles and rubric by assessing a selection of metrics popular in other studies. The results provide granular insights into the strengths and weaknesses of existing metrics that not only rank the metrics but highlight areas of potential improvements. We expect that the CAIR principles will foster agreement among researchers and organizations on which universal privacy evaluation metrics are appropriate for synthetic tabular data.

翻译：数据共享是许多领域（尤其是医疗领域）创新进步的必要条件。然而，保护自然人隐私的法规阻碍了数据共享能力。合成表格数据为解决数据共享难题提供了有前景的方案，但本身并不能天然保证隐私性。目前，学界对评估合成数据隐私保护能力的合适方法缺乏共识，导致跨研究结果难以比较。据我们所知，这是首个识别构成合成表格数据优秀通用隐私评估指标特征的研究。此类指标的目标是实现跨研究可比性，并使非技术利益相关者理解隐私保护机制。我们提出评估指标的四大原则：可比性（Comparability）、适用性（Applicability）、可解释性（Interpretability）和代表性（Representativeness，简称CAIR）。为量化评估指标符合CAIR原则的程度并进行排序，我们设计了采用1-4分制的评分量表。每项属性通过四个参数评分，共产生16个评估维度。我们通过选取其他研究中常用的指标进行评估，验证了CAIR原则及评分量表的适用性与有效性。结果不仅对现有指标进行了排序，还深入揭示了其优劣势及潜在改进方向。我们预期CAIR原则将促进研究人员与机构就合成表格数据适用的通用隐私评估指标达成共识。