Large Language models (LLMs) can generate complicated source code from natural language prompts. However, LLMs can generate output that deviates from what the user wants, requiring supervision and editing. To support this process, we offer techniques to localize where generations might be misaligned from user intent. We first create a dataset of "Minimal Intent Aligning Patches" of repaired LLM generated programs. Each program uses test cases to verify correctness. After creating a dataset of programs, we measure how well various techniques can assign a well-calibrated probability to indicate which parts of code will be edited in a minimal patch (i.e., give a probability that corresponds with empirical odds it is edited). We compare white-box probing (where we propose a technique for efficient arbitrary-span querying), against black-box reflective and self-consistency based approaches. We find probes with a small supervisor model can achieve low calibration error and Brier Skill Score of approx 0.2 estimating edited lines on code generated by models many orders of magnitude larger. We discuss the generalizability of the techniques, and the connections to AI oversight and control, finding a probe trained only on code shows some signs of generalizing to natural language errors if new probability scaling is allowed.

翻译：大型语言模型（LLM）能够根据自然语言提示生成复杂的源代码。然而，LLM生成的输出可能偏离用户意图，因此需要监督与编辑。为支持这一过程，我们提出了定位生成内容可能偏离用户意图位置的技术。我们首先创建了一个包含"最小意图对齐补丁"的修复后LLM生成程序数据集，其中每个程序均通过测试用例验证其正确性。在构建程序数据集后，我们评估了多种技术为代码中哪些部分将在最小补丁中被编辑（即给出与经验编辑概率对应的预测概率）分配良好校准概率的能力。我们比较了白盒探测（提出了一种高效任意跨度查询技术）与基于黑盒反思及自一致性的方法。研究发现：采用小型监督模型的探测方法在校准误差方面表现优异，其Brier技巧评分约为0.2，能够有效预测由规模大数个数量级的模型所生成代码的编辑行。我们探讨了这些技术的泛化能力及其与AI监督控制的关联，发现仅通过代码训练的探测模型在允许新概率缩放的情况下，对自然语言错误也展现出一定的泛化迹象。