Optimising deep neural networks is a challenging task due to complex training dynamics, high computational requirements, and long training times. To address this difficulty, we propose the framework of Generalisable Agents for Neural Network Optimisation (GANNO) -- a multi-agent reinforcement learning (MARL) approach that learns to improve neural network optimisation by dynamically and responsively scheduling hyperparameters during training. GANNO utilises an agent per layer that observes localised network dynamics and accordingly takes actions to adjust these dynamics at a layerwise level to collectively improve global performance. In this paper, we use GANNO to control the layerwise learning rate and show that the framework can yield useful and responsive schedules that are competitive with handcrafted heuristics. Furthermore, GANNO is shown to perform robustly across a wide variety of unseen initial conditions, and can successfully generalise to harder problems than it was trained on. Our work presents an overview of the opportunities that this paradigm offers for training neural networks, along with key challenges that remain to be overcome.

翻译：深度神经网络的优化因复杂的训练动态、高计算需求及长训练时间而具有挑战性。为解决这一难题，我们提出面向神经网络优化的通用智能体框架（GANNO）——一种通过多智能体强化学习（MARL）方法，在训练过程中动态自适应调度超参数以改进神经网络优化的方案。GANNO为每个网络层分配一个智能体，该智能体观测局部网络动态，并在层级别执行相应动作以调整这些动态，从而协同提升全局性能。本文使用GANNO控制层级学习率，实验表明该框架可生成有效且自适应的调度策略，其性能可媲美人工设计的启发式方法。此外，GANNO在多种未见过的初始条件下展现出稳健性能，并能成功泛化至比训练环境更复杂的问题。本研究阐述了该范式为神经网络训练带来的机遇，同时指出了仍需克服的关键挑战。