Optimising deep neural networks is a challenging task due to complex training dynamics, high computational requirements, and long training times. To address this difficulty, we propose the framework of Generalisable Agents for Neural Network Optimisation (GANNO) -- a multi-agent reinforcement learning (MARL) approach that learns to improve neural network optimisation by dynamically and responsively scheduling hyperparameters during training. GANNO utilises an agent per layer that observes localised network dynamics and accordingly takes actions to adjust these dynamics at a layerwise level to collectively improve global performance. In this paper, we use GANNO to control the layerwise learning rate and show that the framework can yield useful and responsive schedules that are competitive with handcrafted heuristics. Furthermore, GANNO is shown to perform robustly across a wide variety of unseen initial conditions, and can successfully generalise to harder problems than it was trained on. Our work presents an overview of the opportunities that this paradigm offers for training neural networks, along with key challenges that remain to be overcome.

翻译：深度神经网络的优化因复杂的训练动态、高昂的计算需求及漫长的训练时间而极具挑战。为解决这一难题，我们提出了面向神经网络优化的可泛化智能体（GANNO）框架——一种采用多智能体强化学习（MARL）的方法，通过动态响应式地调度训练过程中的超参数，学习提升神经网络优化效果。GANNO为每个网络层分配一个智能体，该智能体观测局部网络动态，并在层级别采取相应行动调整这些动态，从而协同提升全局性能。本文中，我们利用GANNO控制逐层学习率，实验表明该框架能够生成有效且自适应的调度方案，其性能可与手工设计的启发式方法相媲美。此外，GANNO在多种未见过的初始条件下展现出稳健性能，并能成功泛化到训练时未曾遇到的更复杂问题上。我们的工作不仅揭示了这一范式在神经网络训练中的潜在机遇，也指出了尚待克服的关键挑战。