History for visual dialog: Do we really need it?

Shubham Agarwal; Trung Bui; Joon Young Lee; Ioannis Konstas; Verena Rieser

doi:10.18653/v1/2020.acl-main.728

History for visual dialog: Do we really need it?

Shubham Agarwal^*
, Trung Bui
, Joon Young Lee
, Ioannis Konstas
, Verena Rieser

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

57 Citations (Scopus)

Abstract

Visual Dialog involves “understanding” the dialog history (what has been discussed previously) and the current question (what is asked), in addition to grounding information in the image, to generate the correct response. In this paper, we show that co-attention models which explicitly encode dialog history outperform models that don't, achieving state-of-the-art performance (72 % NDCG on val set). However, we also expose shortcomings of the crowd-sourcing dataset collection procedure by showing that history is indeed only required for a small amount of the data and that the current evaluation metric encourages generic replies. To that end, we propose a challenging subset (VisDialConv) of the VisDial val set and provide a benchmark of 63% NDCG.

Original language	English
Title of host publication	Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Editors	Dan Jurafsky, Joyce Chai, Natalie Schluter, Joel Tetreault
Publisher	Association for Computational Linguistics
Pages	8182-8197
Number of pages	16
ISBN (Electronic)	9781952148255
DOIs	https://doi.org/10.18653/v1/2020.acl-main.728
Publication status	Published - Jul 2020
Event	58th Annual Meeting of the Association for Computational Linguistics 2020 - Virtual, Online, United States Duration: 5 Jul 2020 → 10 Jul 2020

Conference

Conference	58th Annual Meeting of the Association for Computational Linguistics 2020
Abbreviated title	ACL 2020
Country/Territory	United States
City	Virtual, Online
Period	5/07/20 → 10/07/20

ASJC Scopus subject areas

Computer Science Applications
Linguistics and Language
Language and Linguistics

Access to Document

10.18653/v1/2020.acl-main.728

Cite this

Agarwal, S., Bui, T., Lee, J. Y., Konstas, I., & Rieser, V. (2020). History for visual dialog: Do we really need it? In D. Jurafsky, J. Chai, N. Schluter, & J. Tetreault (Eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (pp. 8182-8197). Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.728

@inproceedings{d8d200aa0a3245d1bbaf5b5e09fea524,

title = "History for visual dialog: Do we really need it?",

abstract = "Visual Dialog involves “understanding” the dialog history (what has been discussed previously) and the current question (what is asked), in addition to grounding information in the image, to generate the correct response. In this paper, we show that co-attention models which explicitly encode dialog history outperform models that don't, achieving state-of-the-art performance (72 \% NDCG on val set). However, we also expose shortcomings of the crowd-sourcing dataset collection procedure by showing that history is indeed only required for a small amount of the data and that the current evaluation metric encourages generic replies. To that end, we propose a challenging subset (VisDialConv) of the VisDial val set and provide a benchmark of 63\% NDCG.",

author = "Shubham Agarwal and Trung Bui and Lee, \{Joon Young\} and Ioannis Konstas and Verena Rieser",

note = "Publisher Copyright: {\textcopyright} 2020 Association for Computational Linguistics; 58th Annual Meeting of the Association for Computational Linguistics 2020, ACL 2020 ; Conference date: 05-07-2020 Through 10-07-2020",

year = "2020",

month = jul,

doi = "10.18653/v1/2020.acl-main.728",

language = "English",

pages = "8182--8197",

editor = "Jurafsky, \{Dan \} and Chai, \{Joyce \} and Natalie Schluter and Joel Tetreault",

booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics",

}

Agarwal, S, Bui, T, Lee, JY, Konstas, I & Rieser, V 2020, History for visual dialog: Do we really need it? in D Jurafsky, J Chai, N Schluter & J Tetreault (eds), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, pp. 8182-8197, 58th Annual Meeting of the Association for Computational Linguistics 2020, Virtual, Online, United States, 5/07/20. https://doi.org/10.18653/v1/2020.acl-main.728

History for visual dialog: Do we really need it? / Agarwal, Shubham; Bui, Trung; Lee, Joon Young et al.
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. ed. / Dan Jurafsky; Joyce Chai; Natalie Schluter; Joel Tetreault. Association for Computational Linguistics, 2020. p. 8182-8197.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - History for visual dialog

T2 - 58th Annual Meeting of the Association for Computational Linguistics 2020

AU - Agarwal, Shubham

AU - Bui, Trung

AU - Lee, Joon Young

AU - Konstas, Ioannis

AU - Rieser, Verena

PY - 2020/7

Y1 - 2020/7

N2 - Visual Dialog involves “understanding” the dialog history (what has been discussed previously) and the current question (what is asked), in addition to grounding information in the image, to generate the correct response. In this paper, we show that co-attention models which explicitly encode dialog history outperform models that don't, achieving state-of-the-art performance (72 % NDCG on val set). However, we also expose shortcomings of the crowd-sourcing dataset collection procedure by showing that history is indeed only required for a small amount of the data and that the current evaluation metric encourages generic replies. To that end, we propose a challenging subset (VisDialConv) of the VisDial val set and provide a benchmark of 63% NDCG.

AB - Visual Dialog involves “understanding” the dialog history (what has been discussed previously) and the current question (what is asked), in addition to grounding information in the image, to generate the correct response. In this paper, we show that co-attention models which explicitly encode dialog history outperform models that don't, achieving state-of-the-art performance (72 % NDCG on val set). However, we also expose shortcomings of the crowd-sourcing dataset collection procedure by showing that history is indeed only required for a small amount of the data and that the current evaluation metric encourages generic replies. To that end, we propose a challenging subset (VisDialConv) of the VisDial val set and provide a benchmark of 63% NDCG.

UR - https://www.scopus.com/pages/publications/85097972033

U2 - 10.18653/v1/2020.acl-main.728

DO - 10.18653/v1/2020.acl-main.728

M3 - Conference contribution

AN - SCOPUS:85097972033

SP - 8182

EP - 8197

BT - Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics

A2 - Jurafsky, Dan

A2 - Chai, Joyce

A2 - Schluter, Natalie

A2 - Tetreault, Joel

PB - Association for Computational Linguistics

Y2 - 5 July 2020 through 10 July 2020

ER -

History for visual dialog: Do we really need it?

Abstract

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this