Comparing dialogue strategies for learning grounded language from human tutors

Yanchao Yu; Oliver Lemon; Arash Eshghi

Comparing dialogue strategies for learning grounded language from human tutors

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

138 Downloads (Pure)

Abstract

We address the problem of interactively learning perceptually grounded word meanings in a multimodal dialogue system. Human tutors can correct, question, and confirm the statements of a dialogue agent which is trying to interactively learn the meanings of perceptual words, e.g.\ colours and shapes.
We show that different learner and tutor dialogue strategies lead to different learning rates, accuracy of learned meanings, and effort/costs for human tutors. For example, we show that a learner which can handle corrections in dialogue, and its own uncertainty about what it sees, can learn meanings that are as accurate as a fully-supervised learner, but with less cost/effort to the human tutor.

Original language	English
Title of host publication	JerSem
Subtitle of host publication	Proceedings of the 20th Workshop on the Semantics and Pragmatics of Dialogue
Editors	Julie Hunter, Mandy Simons, Matthew Stone
Publisher	Rutgers University
Pages	44-54
Number of pages	11
Publication status	Published - 16 Jul 2016
Event	20th Workshop Series on the Semantics and Pragmatics of Dialogue 2016 - New Brunswick, United States Duration: 16 Jul 2016 → 18 Jul 2016

Publication series

Name	SemDial Proceedings
ISSN (Print)	2308-2275

Conference

Conference	20th Workshop Series on the Semantics and Pragmatics of Dialogue 2016
Country/Territory	United States
City	New Brunswick
Period	16/07/16 → 18/07/16

Keywords

Robotics, Development, Language action, Social interaction, Learning
Semantics
Natural language processing

Access to Document

Download pdfAccepted author manuscript, 266 KB

http://semantics.rutgers.edu/jersem/proceedings/Semdial_2016_JerSem_proceedings.pdf

HandCraft Visual Object Dataset
Yu, Y. (Creator), Heriot-Watt University, 1 Feb 2016
Dataset

Cite this

Yu, Y., Lemon, O., & Eshghi, A. (2016). Comparing dialogue strategies for learning grounded language from human tutors. In J. Hunter, M. Simons, & M. Stone (Eds.), JerSem: Proceedings of the 20th Workshop on the Semantics and Pragmatics of Dialogue (pp. 44-54). (SemDial Proceedings). Rutgers University. http://semantics.rutgers.edu/jersem/proceedings/Semdial_2016_JerSem_proceedings.pdf

@inproceedings{7abe6045583e4083acb938a466cc28df,

title = "Comparing dialogue strategies for learning grounded language from human tutors",

abstract = "We address the problem of interactively learning perceptually grounded word meanings in a multimodal dialogue system. Human tutors can correct, question, and confirm the statements of a dialogue agent which is trying to interactively learn the meanings of perceptual words, e.g.\ colours and shapes.We show that different learner and tutor dialogue strategies lead to different learning rates, accuracy of learned meanings, and effort/costs for human tutors. For example, we show that a learner which can handle corrections in dialogue, and its own uncertainty about what it sees, can learn meanings that are as accurate as a fully-supervised learner, but with less cost/effort to the human tutor.",

keywords = "Robotics, Development, Language action, Social interaction, Learning, Semantics, Natural language processing",

author = "Yanchao Yu and Oliver Lemon and Arash Eshghi",

year = "2016",

month = jul,

day = "16",

language = "English",

series = "SemDial Proceedings",

publisher = "Rutgers University",

pages = "44--54",

editor = "Julie Hunter and Mandy Simons and Stone, { Matthew}",

booktitle = "JerSem",

address = "United States",

note = "20th Workshop Series on the Semantics and Pragmatics of Dialogue 2016 ; Conference date: 16-07-2016 Through 18-07-2016",

}

Yu, Y, Lemon, O & Eshghi, A 2016, Comparing dialogue strategies for learning grounded language from human tutors. in J Hunter, M Simons & M Stone (eds), JerSem: Proceedings of the 20th Workshop on the Semantics and Pragmatics of Dialogue. SemDial Proceedings, Rutgers University, pp. 44-54, 20th Workshop Series on the Semantics and Pragmatics of Dialogue 2016, New Brunswick, United States, 16/07/16. <http://semantics.rutgers.edu/jersem/proceedings/Semdial_2016_JerSem_proceedings.pdf>

Comparing dialogue strategies for learning grounded language from human tutors. / Yu, Yanchao; Lemon, Oliver ; Eshghi, Arash.
JerSem: Proceedings of the 20th Workshop on the Semantics and Pragmatics of Dialogue. ed. / Julie Hunter; Mandy Simons; Matthew Stone. Rutgers University, 2016. p. 44-54 (SemDial Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Comparing dialogue strategies for learning grounded language from human tutors

AU - Yu, Yanchao

AU - Lemon, Oliver

AU - Eshghi, Arash

PY - 2016/7/16

Y1 - 2016/7/16

N2 - We address the problem of interactively learning perceptually grounded word meanings in a multimodal dialogue system. Human tutors can correct, question, and confirm the statements of a dialogue agent which is trying to interactively learn the meanings of perceptual words, e.g.\ colours and shapes.We show that different learner and tutor dialogue strategies lead to different learning rates, accuracy of learned meanings, and effort/costs for human tutors. For example, we show that a learner which can handle corrections in dialogue, and its own uncertainty about what it sees, can learn meanings that are as accurate as a fully-supervised learner, but with less cost/effort to the human tutor.

AB - We address the problem of interactively learning perceptually grounded word meanings in a multimodal dialogue system. Human tutors can correct, question, and confirm the statements of a dialogue agent which is trying to interactively learn the meanings of perceptual words, e.g.\ colours and shapes.We show that different learner and tutor dialogue strategies lead to different learning rates, accuracy of learned meanings, and effort/costs for human tutors. For example, we show that a learner which can handle corrections in dialogue, and its own uncertainty about what it sees, can learn meanings that are as accurate as a fully-supervised learner, but with less cost/effort to the human tutor.

KW - Robotics, Development, Language action, Social interaction, Learning

KW - Semantics

KW - Natural language processing

M3 - Conference contribution

T3 - SemDial Proceedings

SP - 44

EP - 54

BT - JerSem

A2 - Hunter, Julie

A2 - Simons, Mandy

A2 - Stone, Matthew

PB - Rutgers University

T2 - 20th Workshop Series on the Semantics and Pragmatics of Dialogue 2016

Y2 - 16 July 2016 through 18 July 2016

ER -

Comparing dialogue strategies for learning grounded language from human tutors

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Datasets

HandCraft Visual Object Dataset

Cite this