Combining chat and task-based multimodal dialogue for more engaging HRI: A scalable method using reinforcement learning

We develop the first system to combine task-based and chatbot-style dialogue in a multimodal system for Human-Robot Interaction. We show that Reinforcement Learning is beneficial for training dialogue management (DM) in such systems - providing a scalable method for training from data and/or simulated users. We first train in simulation, and evaluate the benefits of a combined chat/task policy over systems which can only perform chat or task-based conversation. In a real user evaluation, we then show that a trained combined chat/task multimodal dialogue policy results in longer dialogue interactions than a rule-based approach, suggesting that the learned dialogue policy provides a more engaging mixture of chat and task interaction than a rule-based DM method.