Improving Context Modelling in Multimodal Dialogue Generation

Shubham Agarwal, Ondrej Dusek, Ioannis Konstas, Verena Rieser

Research output: Chapter in Book/Report/Conference proceedingConference contribution

12 Citations (Scopus)

Abstract

In this work, we investigate the task of textual response generation in a multimodal task-oriented dialogue system. Our work is based on the recently released Multimodal Dialogue (MMD) dataset (Saha et al., 2017) in the fashion domain. We introduce a multimodal extension to the Hierarchical Recurrent Encoder-Decoder (HRED) model and show that this extension outperforms strong baselines in terms of text-based similarity metrics. We also showcase the shortcomings of current vision and language models by performing an error analysis on our system’s output.
Original languageEnglish
Title of host publicationProceedings of the 11th International Conference on Natural Language Generation
PublisherAssociation for Computational Linguistics
Pages129-134
Number of pages6
ISBN (Electronic)9781948087865
Publication statusPublished - 5 Nov 2018
Event11th International Conference of Natural Language Generation 2018 - Tilburg University, Tilburg, Netherlands
Duration: 5 Nov 20168 Nov 2018
https://inlg2018.uvt.nl/

Conference

Conference11th International Conference of Natural Language Generation 2018
Abbreviated titleINLG'18
Country/TerritoryNetherlands
CityTilburg
Period5/11/168/11/18
Internet address

Fingerprint

Dive into the research topics of 'Improving Context Modelling in Multimodal Dialogue Generation'. Together they form a unique fingerprint.

Cite this