Keep Me in the Loop: Increasing Operator Situation Awareness through a Conversational Multimodal Interface

David A. Robb, Francisco Javier Chiyah Garcia, Atanas Laskov, Xingkun Liu, Pedro Patron, Helen Hastie

Research output: Chapter in Book/Report/Conference proceedingConference contribution

42 Downloads (Pure)

Abstract

Autonomous systems are designed to carry out activities in remote, hazardous environments without the need for operators to micro-manage them. It is, however, essential that operators maintain situation awareness in order to monitor vehicle status and handle unforeseen circumstances that may affect their intended behaviour, such as a change in the environment. We present MIRIAM, a multimodal interface that combines visual indicators of status with a conversational agent component.
This multimodal interface offers a fluid and natural way for operators to gain information on vehicle status and faults, mission progress and to set reminders. We describe the system and an evaluation study providing evidence that such an interactive multimodal interface can assist in maintaining situation awareness for operators of autonomous systems, irrespective of cognitive styles.
Original languageEnglish
Title of host publicationICMI '18 Proceedings of the 20th ACM International Conference on Multimodal Interaction
PublisherACM
Pages384-392
Number of pages9
ISBN (Print)9781450356923
DOIs
Publication statusPublished - 2 Oct 2018

    Fingerprint

Keywords

  • Multimodal output
  • natural language generation
  • autonomous systems
  • situation awareness
  • cognitive styles

Cite this

Robb, D. A., Garcia, F. J. C., Laskov, A., Liu, X., Patron, P., & Hastie, H. (2018). Keep Me in the Loop: Increasing Operator Situation Awareness through a Conversational Multimodal Interface. In ICMI '18 Proceedings of the 20th ACM International Conference on Multimodal Interaction (pp. 384-392). ACM. https://doi.org/10.1145/3242969.3242974