Detection of Listener Uncertainty in Robot-Led Second Language Conversation Practice

Ronald Cumbal, José Lopes, Olov Engwall

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Citations (Scopus)


Uncertainty is a frequently occurring affective state that learners experience during the acquisition of a second language. This state can constitute both a learning opportunity and a source of learner frustration. An appropriate detection could therefore benefit the learning process by reducing cognitive instability. In this study, we use a dyadic practice conversation between an adult second-language learner and a social robot to elicit events of uncertainty through the manipulation of the robot's spoken utterances (increased lexical complexity or prosody modifications). The characteristics of these events are then used to analyze multi-party practice conversations between a robot and two learners. Classification models are trained with multimodal features from annotated events of listener (un)certainty. We report the performance of our models on different settings, (sub)turn segments and multimodal inputs.

Original languageEnglish
Title of host publicationICMI '20: Proceedings of the 2020 International Conference on Multimodal Interaction
PublisherAssociation for Computing Machinery
Number of pages5
ISBN (Electronic)9781450375818
Publication statusPublished - 21 Oct 2020
Event22nd ACM International Conference on Multimodal Interaction 2020 - Virtual, Online, Netherlands
Duration: 25 Oct 202029 Oct 2020


Conference22nd ACM International Conference on Multimodal Interaction 2020
Abbreviated titleICMI 2020
CityVirtual, Online


  • conversation
  • robot assisted language learning
  • social robotics

ASJC Scopus subject areas

  • Hardware and Architecture
  • Human-Computer Interaction
  • Computer Science Applications
  • Computer Vision and Pattern Recognition


Dive into the research topics of 'Detection of Listener Uncertainty in Robot-Led Second Language Conversation Practice'. Together they form a unique fingerprint.

Cite this