LLM-powered coding and development assistants have become prevalent to programmers' workflows. However, concerns about the trustworthiness of LLMs for code persist despite their widespread use. Much of the existing research focused on either training or evaluation, raising questions about whether stakeholders in training and evaluation align in their understanding of model trustworthiness and whether they can move toward a unified direction. In this paper, we propose a vision for a unified trustworthiness auditing framework, DataTrust, which adopts a data-centric approach that synergistically emphasizes both training and evaluation data and their correlations. DataTrust aims to connect model trustworthiness indicators in evaluation with data quality indicators in training. It autonomously inspects training data and evaluates model trustworthiness using synthesized data, attributing potential causes from specific evaluation data to corresponding training data and refining indicator connections. Additionally, a trustworthiness arena powered by DataTrust will engage crowdsourced input and deliver quantitative outcomes. We outline the benefits that various stakeholders can gain from DataTrust and discuss the challenges and opportunities it presents.

翻译：基于大模型的编程与开发助手已广泛融入程序员的工作流程。然而，尽管应用普遍，代码大模型的可信度问题依然备受关注。现有研究多集中于训练或评估的单一环节，这引发了一个疑问：训练与评估的各方利益相关者是否对模型可信度持有统一的理解，以及他们能否朝着一致的方向推进。本文提出了一种统一的可信度审计框架愿景——DataTrust，该框架采用以数据为中心的方法，协同强调训练数据、评估数据及其关联性。DataTrust旨在将评估中的模型可信度指标与训练中的数据质量指标相连接。它能够自主检查训练数据，利用合成数据评估模型可信度，并将特定评估数据中发现的潜在问题归因至相应的训练数据，从而优化指标间的关联。此外，由DataTrust驱动的可信度竞技场将吸纳众包输入并产出量化结果。我们概述了不同利益相关者可从DataTrust中获得的益处，并讨论了该框架面临的挑战与机遇。