A Dialogue-Based Interface for Active Learning of Activities of Daily Living

Ronnie Smith, Mauro Dragone

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)


While Human Activity Recognition (HAR) systems may benefit from Active Learning (AL) by allowing users to self-annotate their Activities of Daily Living (ADLs), many proposed methods for collecting such annotations are for short-term data collection campaigns for specific datasets. We present a reusable dialogue-based approach to user interaction for active learning in HAR systems, which utilises a dataset of natural language descriptions of common activities (which we make publicly available) and semantic similarity measures. Our approach involves system-initiated dialogue, including follow-up questions to reduce ambiguity in user responses where appropriate. We apply our work to an existing CASAS dataset in an active learning scenario, to demonstrate our work in context, in which a natural language interface provides knowledge that can help interpret other multi-modal sensor data. We provide results highlighting the potential of our dialogue- and semantic similarity-based approach. We evaluate our work: (i) technically, as an effective way to seek users' input for active learning of ADLs; and (ii) qualitatively, through a user study in which users were asked to use our approach and an established method, and to subsequently compare the two. Results show the potential of our approach as a user-friendly mechanism for annotation of sensor data as part of an active learning system.

Original languageEnglish
Title of host publicationIUI '22: 27th International Conference on Intelligent User Interfaces
PublisherAssociation for Computing Machinery
Number of pages12
ISBN (Electronic)9781450391443
Publication statusPublished - 22 Mar 2022
Event27th International Conference on Intelligent User Interfaces 2022 - Virtual, Online, Finland
Duration: 22 Mar 202225 Mar 2022


Conference27th International Conference on Intelligent User Interfaces 2022
Abbreviated titleIUI 2022
CityVirtual, Online


  • Active Learning (AL)
  • Human Activity Recognition (HAR) labelling
  • Human-in-the-Loop (HITL) annotation
  • natural language
  • semantic similarity

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction


Dive into the research topics of 'A Dialogue-Based Interface for Active Learning of Activities of Daily Living'. Together they form a unique fingerprint.

Cite this