Semantic Noise Matters for Neural Natural Language Generation

Ondrej Dusek; David Howcroft; Verena Rieser

doi:10.18653/v1/W19-8652

Semantic Noise Matters for Neural Natural Language Generation

Ondrej Dusek, David Howcroft, Verena Rieser

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

40 Downloads (Pure)

Abstract

Neural natural language generation (NNLG) systems are known for their pathological outputs, i.e. generating text which is unrelated to the input specification. In this paper, we show the impact of semantic noise on state-of-the-art NNLG models which implement different semantic control mechanisms. We find that cleaned data can improve semantic correctness by up to 97%, while maintaining fluency. We also find that the most common error is omitting information, rather than hallucination.

Original language	English
Title of host publication	Proceedings of the 12th International Conference on Natural Language Generation
Publisher	Association for Computational Linguistics
Pages	421–426
Number of pages	6
ISBN (Electronic)	9781950737949
DOIs	https://doi.org/10.18653/v1/W19-8652
Publication status	Published - 2019
Event	12th International Conference on Natural Language Generation 2019 - Tokyo, Japan Duration: 28 Oct 2019 → 1 Nov 2019

Conference

Conference	12th International Conference on Natural Language Generation 2019
Country/Territory	Japan
City	Tokyo
Period	28/10/19 → 1/11/19

Access to Document

10.18653/v1/W19-8652Licence: CC BY

Download pdfFinal published version, 257 KBLicence: CC BY

Cite this

@inproceedings{1d723d118e4a4ea0b54b1db966bd90d3,

title = "Semantic Noise Matters for Neural Natural Language Generation",

abstract = "Neural natural language generation (NNLG) systems are known for their pathological outputs, i.e. generating text which is unrelated to the input specification. In this paper, we show the impact of semantic noise on state-of-the-art NNLG models which implement different semantic control mechanisms. We find that cleaned data can improve semantic correctness by up to 97%, while maintaining fluency. We also find that the most common error is omitting information, rather than hallucination.",

author = "Ondrej Dusek and David Howcroft and Verena Rieser",

year = "2019",

doi = "10.18653/v1/W19-8652",

language = "English",

pages = "421–426 ",

booktitle = "Proceedings of the 12th International Conference on Natural Language Generation",

publisher = "Association for Computational Linguistics",

note = "12th International Conference on Natural Language Generation 2019 ; Conference date: 28-10-2019 Through 01-11-2019",

}

Dusek, O, Howcroft, D & Rieser, V 2019, Semantic Noise Matters for Neural Natural Language Generation. in Proceedings of the 12th International Conference on Natural Language Generation. Association for Computational Linguistics, pp. 421–426 , 12th International Conference on Natural Language Generation 2019, Tokyo, Japan, 28/10/19. https://doi.org/10.18653/v1/W19-8652

TY - GEN

T1 - Semantic Noise Matters for Neural Natural Language Generation

AU - Dusek, Ondrej

AU - Howcroft, David

AU - Rieser, Verena

PY - 2019

Y1 - 2019

N2 - Neural natural language generation (NNLG) systems are known for their pathological outputs, i.e. generating text which is unrelated to the input specification. In this paper, we show the impact of semantic noise on state-of-the-art NNLG models which implement different semantic control mechanisms. We find that cleaned data can improve semantic correctness by up to 97%, while maintaining fluency. We also find that the most common error is omitting information, rather than hallucination.

AB - Neural natural language generation (NNLG) systems are known for their pathological outputs, i.e. generating text which is unrelated to the input specification. In this paper, we show the impact of semantic noise on state-of-the-art NNLG models which implement different semantic control mechanisms. We find that cleaned data can improve semantic correctness by up to 97%, while maintaining fluency. We also find that the most common error is omitting information, rather than hallucination.

U2 - 10.18653/v1/W19-8652

DO - 10.18653/v1/W19-8652

M3 - Conference contribution

SP - 421

EP - 426

BT - Proceedings of the 12th International Conference on Natural Language Generation

PB - Association for Computational Linguistics

T2 - 12th International Conference on Natural Language Generation 2019

Y2 - 28 October 2019 through 1 November 2019

ER -

Semantic Noise Matters for Neural Natural Language Generation

Abstract

Conference

Access to Document

Fingerprint

Cite this