Comparing dialogue strategies for learning grounded language from human tutors

Research output: Chapter in Book/Report/Conference proceedingConference contribution

39 Downloads (Pure)

Abstract

We address the problem of interactively learning perceptually grounded word meanings in a multimodal dialogue system. Human tutors can correct, question, and confirm the statements of a dialogue agent which is trying to interactively learn the meanings of perceptual words, e.g.\ colours and shapes.
We show that different learner and tutor dialogue strategies lead to different learning rates, accuracy of learned meanings, and effort/costs for human tutors. For example, we show that a learner which can handle corrections in dialogue, and its own uncertainty about what it sees, can learn meanings that are as accurate as a fully-supervised learner, but with less cost/effort to the human tutor.
Original languageEnglish
Title of host publicationJerSem
Subtitle of host publicationProceedings of the 20th Workshop on the Semantics and Pragmatics of Dialogue
EditorsJulie Hunter, Mandy Simons, Matthew Stone
PublisherRutgers University
Pages44-54
Number of pages11
Publication statusPublished - 16 Jul 2016
Event20th Workshop Series on the Semantics and Pragmatics of Dialogue 2016 - New Brunswick, United States
Duration: 16 Jul 201618 Jul 2016

Publication series

NameSemDial Proceedings
ISSN (Print)2308-2275

Conference

Conference20th Workshop Series on the Semantics and Pragmatics of Dialogue 2016
CountryUnited States
CityNew Brunswick
Period16/07/1618/07/16

Keywords

  • Robotics, Development, Language action, Social interaction, Learning
  • Semantics
  • Natural language processing

Fingerprint Dive into the research topics of 'Comparing dialogue strategies for learning grounded language from human tutors'. Together they form a unique fingerprint.

  • Datasets

    HandCraft Visual Object Dataset

    Yu, Y. (Creator), Heriot-Watt University, 1 Feb 2016

    Dataset

    Cite this

    Yu, Y., Lemon, O., & Eshghi, A. (2016). Comparing dialogue strategies for learning grounded language from human tutors. In J. Hunter, M. Simons, & M. Stone (Eds.), JerSem: Proceedings of the 20th Workshop on the Semantics and Pragmatics of Dialogue (pp. 44-54). (SemDial Proceedings). Rutgers University. http://semantics.rutgers.edu/jersem/proceedings/Semdial_2016_JerSem_proceedings.pdf