Feedback-driven optimization, such as traditional machine learning training, is a static process that lacks real-time adaptability of hyperparameters. Tuning solutions for optimization require trial and error paired with checkpointing and schedulers, in many cases feedback from the algorithm is overlooked. Adjusting hyperparameters during optimization usually requires the program to be restarted, wasting utilization and time, while placing unnecessary strain on memory and processors. We present LiveTune, a novel framework allowing real-time parameter adjustment of optimization loops through LiveVariables. Live Variables allow for continuous feedback-driven optimization by storing parameters on designated ports on the system, allowing them to be dynamically adjusted. Extensive evaluations of our framework on standard machine learning training pipelines show saving up to 60 seconds and 5.4 Kilojoules of energy per hyperparameter change. We also show the feasibility and value of LiveTune in a reinforcement learning application where the users change the dynamics of the reward structure while the agent is learning showing 5x improvement over the baseline. Finally, we outline a fully automated workflow to provide end-to-end, unsupervised feedback-driven optimization.

翻译：反馈驱动优化（如传统机器学习训练）本质上是静态过程，缺乏超参数的实时自适应能力。现有的优化调优方案需要试错，并依赖检查点与调度器，在许多情况下忽略了算法本身的反馈。在优化过程中调整超参数通常需要重启程序，这不仅浪费利用率和时间，还给内存和处理器带来不必要的负担。本文提出LiveTune——一种通过LiveVariables实现优化循环实时参数调整的新型框架。LiveVariables通过在系统指定端口存储参数实现持续反馈驱动优化，支持参数的动态调整。在标准机器学习训练管道上的广泛评估表明，每次超参数变更可节省高达60秒时间和5.4千焦耳能耗。我们还在强化学习应用中展示了LiveTune的可行性与价值：当用户动态调整奖励结构时（智能体持续学习），该方法相较基线实现了5倍性能提升。最后，我们概述了实现端到端无监督反馈驱动优化的全自动化工作流。