Reinforcement learning (RL) is a subset of artificial intelligence (AI) where agents learn the best action by interacting with the environment, making it suitable for tasks that do not require labeled data or direct supervision. Hyperparameters (HP) tuning refers to choosing the best parameter that leads to optimal solutions in RL algorithms. Manual or random tuning of the HP may be a crucial process because variations in this parameter lead to changes in the overall learning aspects and different rewards. In this paper, a novel and automatic HP-tuning method called Q-FOX is proposed. This uses both the FOX optimizer, a new optimization method inspired by nature that mimics red foxes' hunting behavior, and the commonly used, easy-to-implement RL Q-learning algorithm to solve the problem of HP tuning. Moreover, a new objective function is proposed which prioritizes the reward over the mean squared error (MSE) and learning time (steps). Q-FOX has been evaluated on two OpenAI Gym environment control tasks: Cart Pole and Frozen Lake. It exposed greater cumulative rewards than HP tuning with other optimizers, such as PSO, GA, Bee, or randomly selected HP. The cumulative reward for the Cart Pole task was 32.08, and for the Frozen Lake task was 0.95. Despite the robustness of Q-FOX, it has limitations. It cannot be used directly in real-word problems before choosing the HP in a simulation environment because its processes work iteratively, making it time-consuming. The results indicate that Q-FOX has played an essential role in HP tuning for RL algorithms to effectively solve different control tasks.

翻译：强化学习（RL）是人工智能（AI）的一个子领域，其中智能体通过与环境的交互学习最优动作，因此适用于无需标注数据或直接监督的任务。超参数（HP）调优是指在RL算法中选择能产生最优解的参数。手动或随机调优超参数可能是一个关键过程，因为该参数的变化会导致整体学习层面的变化并产生不同奖励。本文提出一种名为Q-FOX的新型自动超参数调优方法，该方法结合了FOX优化器（一种受自然界红狐捕猎行为启发的新优化方法）与广泛使用且易于实现的RL Q-learning算法，以解决超参数调优问题。此外，本文还提出一种新的目标函数，该函数优先考虑奖励而非均方误差（MSE）和学习时间（步数）。Q-FOX在两个OpenAI Gym环境控制任务（Cart Pole和Frozen Lake）上进行了评估。与采用PSO、GA、Bee等其他优化器或随机选择超参数的方法相比，Q-FOX获得了更高的累积奖励。Cart Pole任务的累积奖励为32.08，Frozen Lake任务的累积奖励为0.95。尽管Q-FOX具有鲁棒性，但仍存在局限性。由于其过程以迭代方式运行且耗时，在仿真环境中选定超参数之前，它无法直接应用于现实问题。结果表明，Q-FOX在RL算法的超参数调优中发挥了关键作用，可有效解决不同的控制任务。