Efficient numerical optimization methods can improve performance and reduce the environmental impact of computing in many applications. This work presents a proof-of-concept study combining primitive state representations and agent-environment interactions as first-order optimizers in the setting of budget-limited optimization. Through reinforcement learning (RL) over a set of training instances of an optimization problem class, optimal policies for sequential update selection of algorithmic iteration steps are approximated in generally formulated low-dimensional partial state representations that consider aspects of progress and resource use. For the investigated case studies, deployment of the trained agents to unseen instances of the quadratic optimization problem classes outperformed conventional optimal algorithms with optimized hyperparameters. The results show that elementary RL methods combined with succinct partial state representations can be used as heuristics to manage complexity in RL-based optimization, paving the way for agentic optimization approaches.

翻译：高效数值优化方法能够在众多应用中提升计算性能并降低环境影响。本研究提出一项概念验证，将原始状态表示与智能体-环境交互结合，作为预算有限优化场景下的一阶优化器。通过对某类优化问题的训练实例集进行强化学习，在综合考虑进度与资源使用的一般化低维部分状态表示中，近似逼近算法迭代步序贯更新的最优策略。在所研究的案例中，将训练后的智能体部署至二次优化问题类的未见过实例时，其表现优于采用优化超参数的常规最优算法。结果表明，将基础强化学习方法与简洁的部分状态表示相结合，可作为管理基于强化学习优化中复杂性的启发式策略，为智能体驱动的优化方法奠定基础。