Uncertainty in robot assisted second language conversation practice

Ronald Cumbal, José Lopes, Olov Engwall

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)


Moments of uncertainty are common for learners when practicing a second language. The appropriate management of these events could help avoid the development of frustration and benefit the learner's experience. Therefore, its detection is crucial in language practice conversations. In this study, an experimental conversation between an adult second language learner and a social robot is employed to visually characterize the learners' uncertainty. The robot's output is manipulated in prosody and lexical levels to provoke uncertainty during the conversation. These reactions are then processed to obtain Facial Action Units (AUs) and Gaze features. Preliminary results show distinctive behavioral patterns of uncertainty among the participants. Based on these results, a new annotation scheme is proposed, which will expand the data used to train sequential models to detect uncertainty. As future steps, the robotic conversational partner will use this information to adapt its behavior in dialogue generation and language complexity.
Original languageEnglish
Title of host publicationHRI '20: Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction
PublisherAssociation for Computing Machinery
Number of pages3
ISBN (Electronic)9781450370578
Publication statusPublished - 23 Mar 2020
Event15th Annual ACM/IEEE International Conference on Human Robot Interaction 2020 - Corn Exchange, Cambridge, United Kingdom
Duration: 23 Mar 202026 Mar 2020


Conference15th Annual ACM/IEEE International Conference on Human Robot Interaction 2020
Abbreviated titleHRI 2020
Country/TerritoryUnited Kingdom
Internet address


  • Affective states
  • Human-robot interaction
  • Robot assisted language learning
  • Uncertainty

ASJC Scopus subject areas

  • Artificial Intelligence
  • Human-Computer Interaction
  • Electrical and Electronic Engineering


Dive into the research topics of 'Uncertainty in robot assisted second language conversation practice'. Together they form a unique fingerprint.

Cite this