The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering

Sabrina Chiesurin; Dimitris Dimakopoulos; Marco Antonio Sobrevilla Cabezudo; Arash Eshghi; Ioannis Papaioannou; Verena Rieser; Ioannis Konstas

doi:10.18653/v1/2023.findings-acl.60

The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering

Sabrina Chiesurin
, Dimitris Dimakopoulos
, Marco Antonio Sobrevilla Cabezudo
, Arash Eshghi
, Ioannis Papaioannou
, Verena Rieser
, Ioannis Konstas

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

20 Citations (Scopus)

1 Downloads (Pure)

Abstract

Large language models are known to produce output which sounds fluent and convincing, but is also often wrong, e.g. “unfaithful" with respect to a rationale as retrieved from a knowledge base. In this paper, we show that task-based systems which exhibit certain advanced linguistic dialog behaviors, such as lexical alignment (repeating what the user said), are in fact preferred and trusted more, whereas other phenomena, such as pronouns and ellipsis are dis-preferred. We use open-domain question answering systems as our test-bed for task based dialog generation and compare several open- and closed-book models. Our results highlight the danger of systems that appear to be trustworthy by parroting user input while providing an unfaithful response.

Original language	English
Title of host publication	Findings of the Association for Computational Linguistics: ACL 2023
Publisher	Association for Computational Linguistics
Pages	947-959
Number of pages	13
ISBN (Electronic)	9781959429623
DOIs	https://doi.org/10.18653/v1/2023.findings-acl.60
Publication status	Published - 9 Jul 2023
Event	61st Annual Meeting of the Association for Computational Linguistics 2023 - Toronto, Canada Duration: 9 Jul 2023 → 14 Jul 2023

Conference

Conference	61st Annual Meeting of the Association for Computational Linguistics 2023
Abbreviated title	ACL 2023
Country/Territory	Canada
City	Toronto
Period	9/07/23 → 14/07/23

ASJC Scopus subject areas

Computer Science Applications
Linguistics and Language
Language and Linguistics

Access to Document

10.18653/v1/2023.findings-acl.60Licence: CC BY

Download pdf
©2023 Association for Computational Linguistics © 1963–2025 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License.
Final published version, 539 KBLicence: CC BY

Cite this

Chiesurin, S., Dimakopoulos, D., Cabezudo, M. A. S., Eshghi, A., Papaioannou, I., Rieser, V., & Konstas, I. (2023). The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering. In Findings of the Association for Computational Linguistics: ACL 2023 (pp. 947-959). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.findings-acl.60

@inproceedings{10e1abc36ae147019ec61f7ee39da782,

title = "The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering",

abstract = "Large language models are known to produce output which sounds fluent and convincing, but is also often wrong, e.g. “unfaithful{"} with respect to a rationale as retrieved from a knowledge base. In this paper, we show that task-based systems which exhibit certain advanced linguistic dialog behaviors, such as lexical alignment (repeating what the user said), are in fact preferred and trusted more, whereas other phenomena, such as pronouns and ellipsis are dis-preferred. We use open-domain question answering systems as our test-bed for task based dialog generation and compare several open- and closed-book models. Our results highlight the danger of systems that appear to be trustworthy by parroting user input while providing an unfaithful response.",

author = "Sabrina Chiesurin and Dimitris Dimakopoulos and Cabezudo, \{Marco Antonio Sobrevilla\} and Arash Eshghi and Ioannis Papaioannou and Verena Rieser and Ioannis Konstas",

note = "Publisher Copyright: {\textcopyright} 2023 Association for Computational Linguistics.; 61st Annual Meeting of the Association for Computational Linguistics 2023, ACL 2023 ; Conference date: 09-07-2023 Through 14-07-2023",

year = "2023",

month = jul,

day = "9",

doi = "10.18653/v1/2023.findings-acl.60",

language = "English",

pages = "947--959",

booktitle = "Findings of the Association for Computational Linguistics: ACL 2023",

publisher = "Association for Computational Linguistics",

}

Chiesurin, S, Dimakopoulos, D, Cabezudo, MAS, Eshghi, A, Papaioannou, I, Rieser, V & Konstas, I 2023, The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering. in Findings of the Association for Computational Linguistics: ACL 2023. Association for Computational Linguistics, pp. 947-959, 61st Annual Meeting of the Association for Computational Linguistics 2023, Toronto, Ontario, Canada, 9/07/23. https://doi.org/10.18653/v1/2023.findings-acl.60

The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering. / Chiesurin, Sabrina; Dimakopoulos, Dimitris; Cabezudo, Marco Antonio Sobrevilla et al.
Findings of the Association for Computational Linguistics: ACL 2023. Association for Computational Linguistics, 2023. p. 947-959.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering

AU - Chiesurin, Sabrina

AU - Dimakopoulos, Dimitris

AU - Cabezudo, Marco Antonio Sobrevilla

AU - Eshghi, Arash

AU - Papaioannou, Ioannis

AU - Rieser, Verena

AU - Konstas, Ioannis

PY - 2023/7/9

Y1 - 2023/7/9

N2 - Large language models are known to produce output which sounds fluent and convincing, but is also often wrong, e.g. “unfaithful" with respect to a rationale as retrieved from a knowledge base. In this paper, we show that task-based systems which exhibit certain advanced linguistic dialog behaviors, such as lexical alignment (repeating what the user said), are in fact preferred and trusted more, whereas other phenomena, such as pronouns and ellipsis are dis-preferred. We use open-domain question answering systems as our test-bed for task based dialog generation and compare several open- and closed-book models. Our results highlight the danger of systems that appear to be trustworthy by parroting user input while providing an unfaithful response.

AB - Large language models are known to produce output which sounds fluent and convincing, but is also often wrong, e.g. “unfaithful" with respect to a rationale as retrieved from a knowledge base. In this paper, we show that task-based systems which exhibit certain advanced linguistic dialog behaviors, such as lexical alignment (repeating what the user said), are in fact preferred and trusted more, whereas other phenomena, such as pronouns and ellipsis are dis-preferred. We use open-domain question answering systems as our test-bed for task based dialog generation and compare several open- and closed-book models. Our results highlight the danger of systems that appear to be trustworthy by parroting user input while providing an unfaithful response.

UR - https://www.scopus.com/pages/publications/85168615098

U2 - 10.18653/v1/2023.findings-acl.60

DO - 10.18653/v1/2023.findings-acl.60

M3 - Conference contribution

AN - SCOPUS:85168615098

SP - 947

EP - 959

BT - Findings of the Association for Computational Linguistics: ACL 2023

PB - Association for Computational Linguistics

T2 - 61st Annual Meeting of the Association for Computational Linguistics 2023

Y2 - 9 July 2023 through 14 July 2023

ER -

Chiesurin S, Dimakopoulos D, Cabezudo MAS, Eshghi A, Papaioannou I, Rieser V et al. The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering. In Findings of the Association for Computational Linguistics: ACL 2023. Association for Computational Linguistics. 2023. p. 947-959 doi: 10.18653/v1/2023.findings-acl.60

The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering

Abstract

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this