Learning Complex Teamwork Tasks using a Sub-task Curriculum

Elliot Fosong, Arrasy Rahman, Ignacio Carlucho, Stefano V. Albrecht

Research output: Working paperPreprint

75 Downloads (Pure)

Abstract

Training a team to complete a complex task via multi-agent reinforcement learning can be difficult due to challenges such as policy search in a large policy space, and non-stationarity caused by mutually adapting agents. To facilitate efficient learning of complex multi-agent tasks, we propose an approach which uses an expert-provided curriculum of simpler multi-agent sub-tasks. In each sub-task of the curriculum, a subset of the entire team is trained to acquire sub-task-specific policies. The sub-teams are then merged and transferred to the target task, where their policies are collectively fined tuned to solve the more complex target task. We present MEDoE, a flexible method which identifies situations in the target task where each agent can use its sub-task-specific skills, and uses this information to modulate hyperparameters for learning and exploration during the fine-tuning process. We compare MEDoE to multi-agent reinforcement learning baselines that train from scratch in the full task, and with na\"ive applications of standard multi-agent reinforcement learning techniques for fine-tuning. We show that MEDoE outperforms baselines which train from scratch or use na\"ive fine-tuning approaches, requiring significantly fewer total training timesteps to solve a range of complex teamwork tasks.
Original languageEnglish
DOIs
Publication statusPublished - 9 Feb 2023

Keywords

  • cs.MA
  • cs.AI
  • cs.LG

Fingerprint

Dive into the research topics of 'Learning Complex Teamwork Tasks using a Sub-task Curriculum'. Together they form a unique fingerprint.

Cite this