Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

Linhai Xie, Sen Wang, Stefano Rosa, Andrew Markham, Niki Trigoni

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)
2 Downloads (Pure)

Abstract

Deep Reinforcement Learning (DRL) has been applied successfully to many robotic applications. However, the large number of trials needed for training is a key issue. Most of existing techniques developed to improve training efficiency (e.g. imitation) target on general tasks rather than being tailored for robot applications, which have their specific context to benefit from. We propose a novel framework, Assisted Reinforcement Learning, where a classical controller (e.g. a PID controller) is used as an alternative, switchable policy to speed up training of DRL for local planning and navigation problems. The core idea is that the simple control law allows the robot to rapidly learn sensible primitives, like driving in a straight line, instead of random exploration. As the actor network becomes more advanced, it can then take over to perform more complex actions, like obstacle avoidance. Eventually, the simple controller can be discarded entirely. We show that not only does this technique train faster, it also is less sensitive to the structure of the DRL network and consistently outperforms a standard Deep Deterministic Policy Gradient network. We demonstrate the results in both simulation and real-world experiments.
Original languageEnglish
Title of host publication2018 IEEE International Conference on Robotics and Automation (ICRA)
PublisherIEEE
Pages6276-6283
Number of pages8
ISBN (Electronic)9781538630815
DOIs
Publication statusPublished - 13 Sep 2018
EventIEEE International Conference on Robotics and Automation 2018 - Brisbane, Australia
Duration: 21 May 201825 May 2018

Conference

ConferenceIEEE International Conference on Robotics and Automation 2018
Abbreviated titleICRA 2018
CountryAustralia
CityBrisbane
Period21/05/1825/05/18

Fingerprint Dive into the research topics of 'Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning'. Together they form a unique fingerprint.

  • Cite this

    Xie, L., Wang, S., Rosa, S., Markham, A., & Trigoni, N. (2018). Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning. In 2018 IEEE International Conference on Robotics and Automation (ICRA) (pp. 6276-6283). IEEE. https://doi.org/10.1109/ICRA.2018.8461203