Identification of Low-engaged Learners in Robot-led Second Language Conversations with Adults

Olov Engwall, Ronald Cumbal, José Lopes, Mikael Ljung, Linnea Månsson

Research output: Contribution to journalArticlepeer-review

5 Citations (Scopus)
34 Downloads (Pure)


The main aim of this study is to investigate if verbal, vocal, and facial information can be used to identify low-engaged second language learners in robot-led conversation practice. The experiments were performed on voice recordings and video data from 50 conversations, in which a robotic head talks with pairs of adult language learners using four different interaction strategies with varying robot-learner focus and initiative. It was found that these robot interaction strategies influenced learner activity and engagement. The verbal analysis indicated that learners with low activity rated the robot significantly lower on two out of four scales related to social competence. The acoustic vocal and video-based facial analysis, based on manual annotations or machine learning classification, both showed that learners with low engagement rated the robot's social competencies consistently, and in several cases significantly, lower, and in addition rated the learning effectiveness lower. The agreement between manual and automatic identification of low-engaged learners based on voice recordings or face videos was further found to be adequate for future use. These experiments constitute a first step towards enabling adaption to learners' activity and engagement through within-and between-strategy changes of the robot's interaction with learners.

Original languageEnglish
Article number18
JournalACM Transactions on Human-Robot Interaction
Issue number2
Early online date4 Mar 2022
Publication statusPublished - Jun 2022


  • facial emotion expressions
  • Robot-assisted language learning
  • speech emotion recognition
  • user engagement

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Artificial Intelligence


Dive into the research topics of 'Identification of Low-engaged Learners in Robot-led Second Language Conversations with Adults'. Together they form a unique fingerprint.

Cite this