TY - JOUR
T1 - Double Q-PID algorithm for mobile robot control
AU - Carlucho, Ignacio
AU - De Paula, Mariano
AU - Acosta, Gerardo G.
N1 - Funding Information:
We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan X Pascal GPU used for this research. Particularly, we would like to thanks the National Research and Technology Council of Argentina ( CONICET ) for the economic support of Eng. Ignacio Carlucho with a Ph.D. fellowship.
Publisher Copyright:
© 2019 Elsevier Ltd
PY - 2019/12/15
Y1 - 2019/12/15
N2 - Many expert systems have been developed for self-adaptive PID controllers of mobile robots. However, the high computational requirements of the expert systems layers, developed for the tuning of the PID controllers, still require previous expert knowledge and high efficiency in algorithmic and software execution for real-time applications. To address these problems, in this paper we propose an expert agent-based system, based on a reinforcement learning agent, for self-adapting multiple low-level PID controllers in mobile robots. For the formulation of the artificial expert agent, we develop an incremental model-free algorithm version of the double Q-Learning algorithm for fast on-line adaptation of multiple low-level PID controllers. Fast learning and high on-line adaptability of the artificial expert agent is achieved by means of a proposed incremental active-learning exploration-exploitation procedure, for a non-uniform state space exploration, along with an experience replay mechanism for multiple value functions updates in the double Q-learning algorithm. A comprehensive comparative simulation study and experiments in a real mobile robot demonstrate the high performance of the proposed algorithm for a real-time simultaneous tuning of multiple adaptive low-level PID controllers of mobile robots in real world conditions.
AB - Many expert systems have been developed for self-adaptive PID controllers of mobile robots. However, the high computational requirements of the expert systems layers, developed for the tuning of the PID controllers, still require previous expert knowledge and high efficiency in algorithmic and software execution for real-time applications. To address these problems, in this paper we propose an expert agent-based system, based on a reinforcement learning agent, for self-adapting multiple low-level PID controllers in mobile robots. For the formulation of the artificial expert agent, we develop an incremental model-free algorithm version of the double Q-Learning algorithm for fast on-line adaptation of multiple low-level PID controllers. Fast learning and high on-line adaptability of the artificial expert agent is achieved by means of a proposed incremental active-learning exploration-exploitation procedure, for a non-uniform state space exploration, along with an experience replay mechanism for multiple value functions updates in the double Q-learning algorithm. A comprehensive comparative simulation study and experiments in a real mobile robot demonstrate the high performance of the proposed algorithm for a real-time simultaneous tuning of multiple adaptive low-level PID controllers of mobile robots in real world conditions.
KW - Double Q-learning
KW - Double Q-PID
KW - Incremental learning
KW - Mobile robots
KW - Multi-platforms
KW - Reinforcement learning
UR - http://www.scopus.com/inward/record.url?scp=85068505390&partnerID=8YFLogxK
U2 - 10.1016/j.eswa.2019.06.066
DO - 10.1016/j.eswa.2019.06.066
M3 - Article
AN - SCOPUS:85068505390
SN - 0957-4174
VL - 137
SP - 292
EP - 307
JO - Expert Systems with Applications
JF - Expert Systems with Applications
ER -