Machine Learning for Social Multi-Party Human-Robot Interaction

Simon Keizer, Mary Ellen Foster, Zhuoran Wang, Oliver Lemon

Research output: Contribution to journalArticlepeer-review

33 Citations (Scopus)


We describe a variety of machine-learning techniques that are being applied to social multiuser human--robot interaction using a robot bartender in our scenario. We first present a data-driven approach to social state recognition based on supervised learning. We then describe an approach to social skills execution—that is, action selection for generating socially appropriate robot behavior—which is based on reinforcement learning, using a data-driven simulation of multiple users to train execution policies for social skills. Next, we describe how these components for social state recognition and skills execution have been integrated into an end-to-end robot bartender system, and we discuss the results of a user evaluation. Finally, we present an alternative unsupervised learning framework that combines social state recognition and social skills execution based on hierarchical Dirichlet processes and an infinite POMDP interaction manager. The models make use of data from both human--human interactions collected in a number of German bars and human--robot interactions recorded in the evaluation of an initial version of the system.
Original languageEnglish
Article number14
Number of pages32
JournalACM Transactions on Interactive Intelligent Systems
Issue number3
Publication statusPublished - Oct 2014


Dive into the research topics of 'Machine Learning for Social Multi-Party Human-Robot Interaction'. Together they form a unique fingerprint.

Cite this