Conventional reward modeling relies on gradient descent over neural weights, creating opaque, data-hungry "black boxes." We propose a paradigm shift from implicit to explicit reward parameterization, recasting optimization from continuous weight spaces to the discrete space of natural language rubrics. We introduce a training-free framework based on iterative rubric learning: it locally induces discriminative criteria via verification-driven refinement, and globally compresses the candidate criteria pool into a compact core set by maximizing an information-theoretic coding rate objective. We organize the compressed core set into a hierarchical rubric structure -- high-level evaluation dimensions supported by concrete verification checks -- serving as an interpretable, portable reward function. Empirically, our approach challenges prevailing data scaling assumptions: using only 70 preference pairs, our rubric-guided judges outperform fully trained reward models on diverse benchmarks. For instance, Qwen3-8B equipped with our learned rubrics achieves 80.91% on RewardBench2, surpassing the specialized Skywork-Reward-V2-Qwen3-8B (78.20%). These results demonstrate that alignment signals are highly compressible and can be effectively captured through explicit symbolic search.

翻译：传统的奖励建模依赖于对神经权重的梯度下降，这会产生不透明且数据依赖的“黑箱”。我们提出了一种从隐式到显式奖励参数化的范式转变，将优化问题从连续的权重空间重新构建到自然语言评估准则的离散空间。我们引入了一种基于迭代准则学习的免训练框架：该框架通过验证驱动的精炼在局部归纳出判别性准则，并通过最大化信息论编码率目标在全局将候选准则池压缩为一个紧凑的核心集。我们将压缩后的核心集组织成一种层次化的准则结构——由具体的验证检查支撑的高层评估维度——作为一种可解释、可移植的奖励函数。实证结果表明，我们的方法挑战了主流的数据规模假设：仅使用70个偏好对，我们基于准则指导的评估器就在多个基准测试中超越了完全训练的奖励模型。例如，配备我们学习到的准则的Qwen3-8B在RewardBench2上达到了80.91%，超过了专用的Skywork-Reward-V2-Qwen3-8B（78.20%）。这些结果表明，对齐信号具有高度可压缩性，并且可以通过显式的符号搜索被有效捕获。