AUV Position Tracking Control Using End-to-End Deep Reinforcement Learning

Ignacio Carlucho, Mariano De Paula, Sen Wang, Bruno V. Menna, Yvan R. Petillot, Gerardo G. Acosta

Research output: Chapter in Book/Report/Conference proceedingConference contribution

28 Citations (Scopus)
452 Downloads (Pure)


In this article we consider the navigation problem for an autonomous underwater vehicle (AUV) for reaching a desired way-point. The navigation problem in underwater vehicles presents major problems, the highly coupled dynamics of the vehicles and the unknown parameters of the dynamic model, make the need for complex control architectures. However, current developments in reinforcement learning show promising results for robotics applications. In particular underwater autonomous vehicles could benefit from this new techniques, achieving adaptive behavior for real-time problem solving. Based on this developments the navigation problem is solved using deep reinforcement learning, in particular the deep deterministic policy gradient. In this proposal a model free approach is used, where the raw sensor information is used as inputs to a policy network, and the outputs of this network are directly mapped to the thrusters. In addition an adaptive goal driven architecture is used to allow the agent to reach variable way points consistently. The obtained simulated results show its capacity for successfully solving AUV navigation problems.
Original languageEnglish
Title of host publicationOCEANS 2018 MTS/IEEE Charleston
ISBN (Electronic)9781538648148
Publication statusPublished - 10 Jan 2019
EventOCEANS 2018 MTS/IEEE Charleston - Charleston, United States
Duration: 22 Oct 201825 Oct 2018

Publication series

ISSN (Print)0197-7385


ConferenceOCEANS 2018 MTS/IEEE Charleston
Abbreviated titleOCEANS 2018
Country/TerritoryUnited States


  • AUV
  • Deep reinforcement learning
  • Navigation
  • Reinforcement learning

ASJC Scopus subject areas

  • Renewable Energy, Sustainability and the Environment
  • Oceanography


Dive into the research topics of 'AUV Position Tracking Control Using End-to-End Deep Reinforcement Learning'. Together they form a unique fingerprint.

Cite this