Adaptive low-level control of autonomous underwater vehicles using deep reinforcement learning

Ignacio Carlucho, Mariano De Paula*, Sen Wang, Yvan Petillot, Gerardo G. Acosta

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

118 Citations (SciVal)
499 Downloads (Pure)


Low-level control of autonomous underwater vehicles (AUVs) has been extensively addressed by classical control techniques. However, the variable operating conditions and hostile environments faced by AUVs have driven researchers towards the formulation of adaptive control approaches. The reinforcement learning (RL) paradigm is a powerful framework which has been applied in different formulations of adaptive control strategies for AUVs. However, the limitations of RL approaches have lead towards the emergence of deep reinforcement learning which has become an attractive and promising framework for developing real adaptive control strategies to solve complex control problems for autonomous systems. However, most of the existing applications of deep RL use video images to train the decision making artificial agent but obtaining camera images only for an AUV control purpose could be costly in terms of energy consumption. Moreover, the rewards are not easily obtained directly from the video frames. In this work we develop a deep RL framework for adaptive control applications of AUVs based on an actor-critic goal-oriented deep RL architecture, which takes the available raw sensory information as input and as output the continuous control actions which are the low-level commands for the AUV’s thrusters. Experiments on a real AUV demonstrate the applicability of the stated deep RL approach for an autonomous robot control problem.
Original languageEnglish
Pages (from-to)71-86
Number of pages16
JournalRobotics and Autonomous Systems
Early online date15 Jun 2018
Publication statusPublished - Sept 2018


  • Adaptive low-level control
  • Autonomous robot
  • AUV
  • Deep reinforcement learning

ASJC Scopus subject areas

  • Software
  • Control and Systems Engineering
  • Mathematics(all)
  • Computer Science Applications


Dive into the research topics of 'Adaptive low-level control of autonomous underwater vehicles using deep reinforcement learning'. Together they form a unique fingerprint.

Cite this