Spoken conversational AI in video games – Emotional dialogue management increases user engagement

Jamie Fraser, Ioannis Papaioannou, Oliver Lemon

Research output: Chapter in Book/Report/Conference proceedingConference contribution

26 Citations (Scopus)


In a traditional role-playing game (RPG) conversing with a Non-Playable Character (NPC) typically appears somewhat unrealistic and can break immersion and user engagement. In commercial games, the player usually selects one of several possible predefined conversation options which are displayed as text or labels on the screen, to progress the conversation. In contrast, we first present a spoken conversational interface, built using a state-of-the-art open-domain social conversational AI developed for the Amazon Alexa Challenge, which was modified for use in a video game. This system is designed to keep users engaged in the conversation – which we measure by time taken speaking with the character. In particular, we use emotion detection and emotional dialogue management to enhance the conversational experience. We then evaluate the contribution of emotion detection and conversational responses in a spoken dialogue system for a role-playing video game. In order to do this, two prototypes of the same game were created: one system using sentiment analysis and emotional modelling and the other system that does not detect or react to emotions. Both systems use a spoken conversational AI system where the user can freely talk to a Non-Playable-Character using unconstrained speech input.

Original languageEnglish
Title of host publicationProceedings of the 18th International Conference on Intelligent Virtual Agents
PublisherAssociation for Computing Machinery
Number of pages6
ISBN (Electronic)9781450360135
Publication statusPublished - 5 Nov 2018
Event18th ACM International Conference on Intelligent Virtual Agents 2018 - Sydney, Australia
Duration: 5 Nov 20188 Nov 2018


Conference18th ACM International Conference on Intelligent Virtual Agents 2018
Abbreviated titleIVR 2018
Internet address


  • Dialogue systems
  • Emotion
  • Sentiment analysis
  • Speech
  • Video games

ASJC Scopus subject areas

  • Artificial Intelligence
  • Human-Computer Interaction


Dive into the research topics of 'Spoken conversational AI in video games – Emotional dialogue management increases user engagement'. Together they form a unique fingerprint.

Cite this