Proportional-Integrator-Derivative (PID) controller is used in a wide range of industrial and experimental processes. There are a couple of offline methods for tuning PID gains. However, due to the uncertainty of model parameters and external disturbances, real systems such as Quadrotors need more robust and reliable PID controllers. In this research, a self-tuning PID controller using a Reinforcement-Learning-based Neural Network for attitude and altitude control of a Quadrotor has been investigated. An Incremental PID, which contains static and dynamic gains, has been considered and only the variable gains have been tuned. To tune dynamic gains, a model-free actor-critic-based hybrid neural structure was used that was able to properly tune PID gains, and also has done the best as an identifier. In both tunning and identification tasks, a Neural Network with two hidden layers and sigmoid activation functions has been learned using Adaptive Momentum (ADAM) optimizer and Back-Propagation (BP) algorithm. This method is online, able to tackle disturbance, and fast in training. In addition to robustness to mass uncertainty and wind gust disturbance, results showed that the proposed method had a better performance when compared to a PID controller with constant gains.

翻译：比例-积分-微分（PID）控制器广泛应用于工业和实验过程。目前存在若干离线方法用于整定PID增益。然而，由于模型参数的不确定性和外部扰动，四旋翼飞行器等实际系统需要更鲁棒可靠的PID控制器。本研究提出了一种基于强化学习神经网络的自整定PID控制器，用于四旋翼飞行器的姿态和高度控制。考虑采用包含静态和动态增益的增量式PID，并仅对可变增益进行整定。为整定动态增益，采用了一种基于无模型演员-评论家混合神经结构，该结构既能有效整定PID增益，又能作为辨识器发挥最佳性能。在整定和辨识任务中，均采用具有两个隐藏层和Sigmoid激活函数的神经网络，通过自适应动量（ADAM）优化器和反向传播（BP）算法进行训练。该方法具有在线特性、抗扰动能力及快速训练优势。结果表明，与恒定增益PID控制器相比，所提方法不仅对质量不确定性和阵风扰动具有鲁棒性，且表现出更优的控制性能。