Incrementally learning semantic attributes through dialogue interaction: Robotics track

Andrea Vanzo, Jose L. Part, Yanchao Yu, Daniele Nardi, Oliver Lemon

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Citations (Scopus)


Enabling a robot to properly interact with users plays a key role in the effective deployment of robotic platforms in domestic environments. Robots must be able to rely on interaction to improve their behaviour and adaptively understand their operational world. Semantic mapping is the task of building a representation of the environment, that can be enhanced through interaction with the user. In this task, a proper and effective acquisition of semantic attributes of targeted entities is essential for the task accomplishment itself. In this paper, we focus on the problem of learning dialogue policies to support semantic attribute acquisition, so that the effort required by humans in providing knowledge to the robot through dialogue is minimized. To this end, we design our Dialogue Manager as a multi-objective Markov Decision Process, solving the optimisation problem through Reinforcement Learning. The Dialogue Manager interfaces with an online incremental visual classifier, based on a Load-Balancing Self-Organizing Incremental Neural Network (LB- SOINN). Experiments in a simulated scenario show the effectiveness of the proposed solution, suggesting that perceptual information can be properly exploited to reduce human tutoring cost. Moreover, a dialogue policy trained on a small amount of data generalises well to larger datasets, and so the proposed online scheme, as well as the real-time nature of the processing, are suited for an extensive deployment in real scenarios. To this end, this paper provides a demonstration of the complete system on a real robot.

Original languageEnglish
Title of host publicationAAMAS '18 Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems
PublisherInternational Foundation for Autonomous Agents and Multiagent Systems
Number of pages9
ISBN (Print)9781510868083
Publication statusPublished - 9 Jul 2018
Event17th International Conference on Autonomous Agents and Multiagent Systems 2018 - Stockholm, Sweden
Duration: 10 Jul 201815 Jul 2018


Conference17th International Conference on Autonomous Agents and Multiagent Systems 2018
Abbreviated titleAAMAS 2018
Internet address


  • Dialogue management
  • Incremental learning
  • Interactive learning
  • Reinforcement learning

ASJC Scopus subject areas

  • Artificial Intelligence
  • Software
  • Control and Systems Engineering


Dive into the research topics of 'Incrementally learning semantic attributes through dialogue interaction: Robotics track'. Together they form a unique fingerprint.

Cite this