We study code-to-metric regression: predicting numeric outcomes of code executions, a challenging task due to the open-ended nature of programming languages. While prior methods have resorted to heavy and domain-specific feature engineering, we show that a single unified Regression Language Model (RLM) using a frozen LLM encoder can simultaneously predict directly from text, (i) the memory footprint of code across multiple high-level languages such as Python and C++, (ii) the latency of Triton GPU kernels, and (iii) the accuracy and speed of trained neural networks represented in ONNX. In particular, a relatively small 300M parameter RLM based on T5Gemma, obtains $>$0.9 Spearman-rank on competitive programming submissions from APPS, and a single unified model achieves $>$0.5 average Spearman-rank across 17 separate languages from CodeNet. Furthermore, the RLM can obtain the highest average Kendall-Tau of 0.46 on five classic NAS design spaces previously dominated by graph neural networks, and simultaneously predict architecture latencies on numerous hardware platforms.

翻译：我们研究代码到度量的回归：预测代码执行的数值结果，这是一项因编程语言的开放性而具有挑战性的任务。尽管先前的方法依赖于繁重且特定领域的特征工程，但我们证明，一个使用冻结的大语言模型编码器的统一回归语言模型，能够直接从文本同时预测以下指标：(i) 跨多种高级语言（如Python和C++）的代码内存占用，(ii) Triton GPU内核的延迟，以及(iii) 以ONNX表示的已训练神经网络的准确性和速度。特别地，一个基于T5Gemma、参数规模相对较小的300M回归语言模型，在来自APPS的竞赛编程提交上获得了超过0.9的斯皮尔曼秩相关系数；而单个统一模型在来自CodeNet的17种不同语言上取得了平均超过0.5的斯皮尔曼秩相关系数。此外，该回归语言模型在五个经典的NAS设计空间（此前由图神经网络主导）上获得了最高的平均肯德尔tau系数0.46，并能同时预测多种硬件平台上的架构延迟。