Spoken conversational AI in video games – Emotional dialogue management increases user engagement

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In a traditional role-playing game (RPG) conversing with a Non-Playable Character (NPC) typically appears somewhat unrealistic and can break immersion and user engagement. In commercial games, the player usually selects one of several possible predefined conversation options which are displayed as text or labels on the screen, to progress the conversation. In contrast, we first present a spoken conversational interface, built using a state-of-the-art open-domain social conversational AI developed for the Amazon Alexa Challenge, which was modified for use in a video game. This system is designed to keep users engaged in the conversation – which we measure by time taken speaking with the character. In particular, we use emotion detection and emotional dialogue management to enhance the conversational experience. We then evaluate the contribution of emotion detection and conversational responses in a spoken dialogue system for a role-playing video game. In order to do this, two prototypes of the same game were created: one system using sentiment analysis and emotional modelling and the other system that does not detect or react to emotions. Both systems use a spoken conversational AI system where the user can freely talk to a Non-Playable-Character using unconstrained speech input.

Original languageEnglish
Title of host publicationProceedings of the 18th International Conference on Intelligent Virtual Agents
PublisherACM
Pages179-184
Number of pages6
ISBN (Electronic)9781450360135
DOIs
Publication statusPublished - 5 Nov 2018
Event18th ACM International Conference on Intelligent Virtual Agents 2018 - Sydney, Australia
Duration: 5 Nov 20188 Nov 2018
https://iva2018.westernsydney.edu.au/

Conference

Conference18th ACM International Conference on Intelligent Virtual Agents 2018
Abbreviated titleIVR 2018
CountryAustralia
CitySydney
Period5/11/188/11/18
Internet address

Fingerprint

Labels

Keywords

  • Dialogue systems
  • Emotion
  • Sentiment analysis
  • Speech
  • Video games

ASJC Scopus subject areas

  • Artificial Intelligence
  • Human-Computer Interaction

Cite this

Fraser, J., Papaioannou, I., & Lemon, O. (2018). Spoken conversational AI in video games – Emotional dialogue management increases user engagement. In Proceedings of the 18th International Conference on Intelligent Virtual Agents (pp. 179-184). ACM. https://doi.org/10.1145/3267851.3267896
Fraser, Jamie ; Papaioannou, Ioannis ; Lemon, Oliver. / Spoken conversational AI in video games – Emotional dialogue management increases user engagement. Proceedings of the 18th International Conference on Intelligent Virtual Agents. ACM, 2018. pp. 179-184
@inproceedings{8b6d69f57a7a48d5b63d40a8d6e32a58,
title = "Spoken conversational AI in video games – Emotional dialogue management increases user engagement",
abstract = "In a traditional role-playing game (RPG) conversing with a Non-Playable Character (NPC) typically appears somewhat unrealistic and can break immersion and user engagement. In commercial games, the player usually selects one of several possible predefined conversation options which are displayed as text or labels on the screen, to progress the conversation. In contrast, we first present a spoken conversational interface, built using a state-of-the-art open-domain social conversational AI developed for the Amazon Alexa Challenge, which was modified for use in a video game. This system is designed to keep users engaged in the conversation – which we measure by time taken speaking with the character. In particular, we use emotion detection and emotional dialogue management to enhance the conversational experience. We then evaluate the contribution of emotion detection and conversational responses in a spoken dialogue system for a role-playing video game. In order to do this, two prototypes of the same game were created: one system using sentiment analysis and emotional modelling and the other system that does not detect or react to emotions. Both systems use a spoken conversational AI system where the user can freely talk to a Non-Playable-Character using unconstrained speech input.",
keywords = "Dialogue systems, Emotion, Sentiment analysis, Speech, Video games",
author = "Jamie Fraser and Ioannis Papaioannou and Oliver Lemon",
year = "2018",
month = "11",
day = "5",
doi = "10.1145/3267851.3267896",
language = "English",
pages = "179--184",
booktitle = "Proceedings of the 18th International Conference on Intelligent Virtual Agents",
publisher = "ACM",

}

Fraser, J, Papaioannou, I & Lemon, O 2018, Spoken conversational AI in video games – Emotional dialogue management increases user engagement. in Proceedings of the 18th International Conference on Intelligent Virtual Agents. ACM, pp. 179-184, 18th ACM International Conference on Intelligent Virtual Agents 2018, Sydney, Australia, 5/11/18. https://doi.org/10.1145/3267851.3267896

Spoken conversational AI in video games – Emotional dialogue management increases user engagement. / Fraser, Jamie; Papaioannou, Ioannis; Lemon, Oliver.

Proceedings of the 18th International Conference on Intelligent Virtual Agents. ACM, 2018. p. 179-184.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Spoken conversational AI in video games – Emotional dialogue management increases user engagement

AU - Fraser, Jamie

AU - Papaioannou, Ioannis

AU - Lemon, Oliver

PY - 2018/11/5

Y1 - 2018/11/5

N2 - In a traditional role-playing game (RPG) conversing with a Non-Playable Character (NPC) typically appears somewhat unrealistic and can break immersion and user engagement. In commercial games, the player usually selects one of several possible predefined conversation options which are displayed as text or labels on the screen, to progress the conversation. In contrast, we first present a spoken conversational interface, built using a state-of-the-art open-domain social conversational AI developed for the Amazon Alexa Challenge, which was modified for use in a video game. This system is designed to keep users engaged in the conversation – which we measure by time taken speaking with the character. In particular, we use emotion detection and emotional dialogue management to enhance the conversational experience. We then evaluate the contribution of emotion detection and conversational responses in a spoken dialogue system for a role-playing video game. In order to do this, two prototypes of the same game were created: one system using sentiment analysis and emotional modelling and the other system that does not detect or react to emotions. Both systems use a spoken conversational AI system where the user can freely talk to a Non-Playable-Character using unconstrained speech input.

AB - In a traditional role-playing game (RPG) conversing with a Non-Playable Character (NPC) typically appears somewhat unrealistic and can break immersion and user engagement. In commercial games, the player usually selects one of several possible predefined conversation options which are displayed as text or labels on the screen, to progress the conversation. In contrast, we first present a spoken conversational interface, built using a state-of-the-art open-domain social conversational AI developed for the Amazon Alexa Challenge, which was modified for use in a video game. This system is designed to keep users engaged in the conversation – which we measure by time taken speaking with the character. In particular, we use emotion detection and emotional dialogue management to enhance the conversational experience. We then evaluate the contribution of emotion detection and conversational responses in a spoken dialogue system for a role-playing video game. In order to do this, two prototypes of the same game were created: one system using sentiment analysis and emotional modelling and the other system that does not detect or react to emotions. Both systems use a spoken conversational AI system where the user can freely talk to a Non-Playable-Character using unconstrained speech input.

KW - Dialogue systems

KW - Emotion

KW - Sentiment analysis

KW - Speech

KW - Video games

UR - http://www.scopus.com/inward/record.url?scp=85058487547&partnerID=8YFLogxK

U2 - 10.1145/3267851.3267896

DO - 10.1145/3267851.3267896

M3 - Conference contribution

AN - SCOPUS:85058487547

SP - 179

EP - 184

BT - Proceedings of the 18th International Conference on Intelligent Virtual Agents

PB - ACM

ER -

Fraser J, Papaioannou I, Lemon O. Spoken conversational AI in video games – Emotional dialogue management increases user engagement. In Proceedings of the 18th International Conference on Intelligent Virtual Agents. ACM. 2018. p. 179-184 https://doi.org/10.1145/3267851.3267896