AUV Position Tracking Control Using End-to-End Deep Reinforcement Learning

Ignacio Carlucho; Mariano De Paula; Sen Wang; Bruno V. Menna; Yvan Petillot; Gerardo G. Acosta

doi:10.1109/OCEANS.2018.8604791

AUV Position Tracking Control Using End-to-End Deep Reinforcement Learning

Ignacio Carlucho, Mariano De Paula, Sen Wang, Bruno V. Menna, Yvan Petillot, Gerardo G. Acosta

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Citations (Scopus)

175 Downloads (Pure)

Abstract

In this article we consider the navigation problem for an autonomous underwater vehicle (AUV) for reaching a desired way-point. The navigation problem in underwater vehicles presents major problems, the highly coupled dynamics of the vehicles and the unknown parameters of the dynamic model, make the need for complex control architectures. However, current developments in reinforcement learning show promising results for robotics applications. In particular underwater autonomous vehicles could benefit from this new techniques, achieving adaptive behavior for real-time problem solving. Based on this developments the navigation problem is solved using deep reinforcement learning, in particular the deep deterministic policy gradient. In this proposal a model free approach is used, where the raw sensor information is used as inputs to a policy network, and the outputs of this network are directly mapped to the thrusters. In addition an adaptive goal driven architecture is used to allow the agent to reach variable way points consistently. The obtained simulated results show its capacity for successfully solving AUV navigation problems.

Original language	English
Title of host publication	OCEANS 2018 MTS/IEEE Charleston
Publisher	IEEE
ISBN (Electronic)	9781538648148
DOIs	https://doi.org/10.1109/OCEANS.2018.8604791
Publication status	Published - 10 Jan 2019
Event	OCEANS 2018 MTS/IEEE Charleston - Charleston, United States Duration: 22 Oct 2018 → 25 Oct 2018

Publication series

Name	OCEANS MTS/IEEE
Publisher	IEEE
ISSN (Print)	0197-7385

Conference

Conference	OCEANS 2018 MTS/IEEE Charleston
Country	United States
City	Charleston
Period	22/10/18 → 25/10/18

Keywords

AUV
Deep reinforcement learning
Navigation
Reinforcement learning

ASJC Scopus subject areas

Renewable Energy, Sustainability and the Environment
Oceanography

Access to Document

10.1109/OCEANS.2018.8604791

Download pdf
© 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Accepted author manuscript, 1.26 MBLicence: Copyright Publisher

Link to publication in Scopus

Fingerprint Dive into the research topics of 'AUV Position Tracking Control Using End-to-End Deep Reinforcement Learning'. Together they form a unique fingerprint.

Cite this

Carlucho, I., De Paula, M., Wang, S., Menna, B. V., Petillot, Y., & Acosta, G. G. (2019). AUV Position Tracking Control Using End-to-End Deep Reinforcement Learning. In OCEANS 2018 MTS/IEEE Charleston [8604791] (OCEANS MTS/IEEE). IEEE. https://doi.org/10.1109/OCEANS.2018.8604791

@inproceedings{67a626f2c33e4d4d84ce708316bd7eaf,

title = "AUV Position Tracking Control Using End-to-End Deep Reinforcement Learning",

abstract = "In this article we consider the navigation problem for an autonomous underwater vehicle (AUV) for reaching a desired way-point. The navigation problem in underwater vehicles presents major problems, the highly coupled dynamics of the vehicles and the unknown parameters of the dynamic model, make the need for complex control architectures. However, current developments in reinforcement learning show promising results for robotics applications. In particular underwater autonomous vehicles could benefit from this new techniques, achieving adaptive behavior for real-time problem solving. Based on this developments the navigation problem is solved using deep reinforcement learning, in particular the deep deterministic policy gradient. In this proposal a model free approach is used, where the raw sensor information is used as inputs to a policy network, and the outputs of this network are directly mapped to the thrusters. In addition an adaptive goal driven architecture is used to allow the agent to reach variable way points consistently. The obtained simulated results show its capacity for successfully solving AUV navigation problems.",

keywords = "AUV, Deep reinforcement learning, Navigation, Reinforcement learning",

author = "Ignacio Carlucho and {De Paula}, Mariano and Sen Wang and Menna, {Bruno V.} and Yvan Petillot and Acosta, {Gerardo G.}",

year = "2019",

month = jan,

day = "10",

doi = "10.1109/OCEANS.2018.8604791",

language = "English",

series = "OCEANS MTS/IEEE",

publisher = "IEEE",

booktitle = "OCEANS 2018 MTS/IEEE Charleston",

address = "United States",

note = "OCEANS 2018 MTS/IEEE Charleston ; Conference date: 22-10-2018 Through 25-10-2018",

}

TY - GEN

T1 - AUV Position Tracking Control Using End-to-End Deep Reinforcement Learning

AU - Carlucho, Ignacio

AU - De Paula, Mariano

AU - Wang, Sen

AU - Menna, Bruno V.

AU - Petillot, Yvan

AU - Acosta, Gerardo G.

PY - 2019/1/10

Y1 - 2019/1/10

N2 - In this article we consider the navigation problem for an autonomous underwater vehicle (AUV) for reaching a desired way-point. The navigation problem in underwater vehicles presents major problems, the highly coupled dynamics of the vehicles and the unknown parameters of the dynamic model, make the need for complex control architectures. However, current developments in reinforcement learning show promising results for robotics applications. In particular underwater autonomous vehicles could benefit from this new techniques, achieving adaptive behavior for real-time problem solving. Based on this developments the navigation problem is solved using deep reinforcement learning, in particular the deep deterministic policy gradient. In this proposal a model free approach is used, where the raw sensor information is used as inputs to a policy network, and the outputs of this network are directly mapped to the thrusters. In addition an adaptive goal driven architecture is used to allow the agent to reach variable way points consistently. The obtained simulated results show its capacity for successfully solving AUV navigation problems.

AB - In this article we consider the navigation problem for an autonomous underwater vehicle (AUV) for reaching a desired way-point. The navigation problem in underwater vehicles presents major problems, the highly coupled dynamics of the vehicles and the unknown parameters of the dynamic model, make the need for complex control architectures. However, current developments in reinforcement learning show promising results for robotics applications. In particular underwater autonomous vehicles could benefit from this new techniques, achieving adaptive behavior for real-time problem solving. Based on this developments the navigation problem is solved using deep reinforcement learning, in particular the deep deterministic policy gradient. In this proposal a model free approach is used, where the raw sensor information is used as inputs to a policy network, and the outputs of this network are directly mapped to the thrusters. In addition an adaptive goal driven architecture is used to allow the agent to reach variable way points consistently. The obtained simulated results show its capacity for successfully solving AUV navigation problems.

KW - AUV

KW - Deep reinforcement learning

KW - Navigation

KW - Reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=85061839317&partnerID=8YFLogxK

U2 - 10.1109/OCEANS.2018.8604791

DO - 10.1109/OCEANS.2018.8604791

M3 - Conference contribution

T3 - OCEANS MTS/IEEE

BT - OCEANS 2018 MTS/IEEE Charleston

PB - IEEE

T2 - OCEANS 2018 MTS/IEEE Charleston

Y2 - 22 October 2018 through 25 October 2018

ER -