Fine-tuning large language models (LLMs) is compute-intensive and error-prone: model performance depends sensitively on data quality and hyperparameter choices, and naïve runs can even degrade model performance. This raises a practical question:can we predict fine-tuning performance before committing to a full training run? We present TUNEAHEAD, a lightweight framework for pre-hoc prediction of fine-tuning performance. TUNEAHEAD encodes each candidate run as a meta-feature vector that combines static dataset descriptors with dynamic probe features from a short standardized probe. A predictor maps these features to performance estimates, while SHAP-based attributions provide interpretable diagnostics that reveal which specific features drive the prediction. Across 1,300+ fine-tuning runs on Qwen2.5-7B-Instruct, TUNEAHEAD consistently outperforms strong baselines such as Early-Stop Extrapolation and ProxyLM. On a held-out test set of 370 runs, TUNEAHEAD achieves an RMSE of 1.47 percentage points and places 95.1% of predictions within +3/-3 percentage points of the true score. These accurate continuous predictions support practical go/no-go screening policies that can reduce unnecessary full fine-tuning while retaining most promising runs.

翻译：微调大型语言模型（LLMs）计算密集且容易出错：模型性能对数据质量和超参数选择高度敏感，而简单运行甚至可能降低模型性能。这引发了一个实际问题：我们能否在投入完整训练运行之前预测微调性能？我们提出TUNEAHEAD，一个用于预先预测微调性能的轻量级框架。TUNEAHEAD将每个候选运行编码为一个元特征向量，该向量结合了静态数据集描述符和来自标准化短时探测的动态探测特征。一个预测器将这些特征映射到性能估计值，而基于SHAP的归因则提供可解释的诊断，揭示哪些特定特征驱动了预测结果。在Qwen2.5-7B-Instruct上的1300多次微调运行中，TUNEAHEAD持续优于Early-Stop Extrapolation和ProxyLM等强基线。在包含370次运行的保留测试集上，TUNEAHEAD实现了1.47个百分点的均方根误差，并将95.1%的预测值控制在真实分数的正负3个百分点以内。这些精确的连续预测支持实用的“通过/不通过”筛选策略，可以在保留最有潜力的运行的同时减少不必要的完整微调。