SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems

Emily Dinan; Gavin Abercrombie; A. Stevie Bergman; Shannon Spruit; Dirk Hovy; Y-Lan Boureau; Verena Rieser

doi:10.18653/v1/2022.acl-long.284

SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems

Emily Dinan, Gavin Abercrombie, A. Stevie Bergman, Shannon Spruit, Dirk Hovy, Y-Lan Boureau, Verena Rieser

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

The social impact of natural language processing and its applications has received increasing attention. In this position paper, we focus on the problem of safety for end-to-end conversational AI. We survey the problem landscape therein, introducing a taxonomy of three observed phenomena: the Instigator, Yea-Sayer, and Impostor effects. We then empirically assess the extent to which current tools can measure these effects and current systems display them. We release these tools as part of a “first aid kit” (SafetyKit) to quickly assess apparent safety concerns. Our results show that, while current tools are able to provide an estimate of the relative safety of systems in various settings, they still have several shortcomings. We suggest several future directions and discuss ethical considerations.

Original language	English
Title of host publication	Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Publisher	Association for Computational Linguistics
Pages	4113–4133
Number of pages	21
DOIs	https://doi.org/10.18653/v1/2022.acl-long.284
Publication status	Published - May 2022

Access to Document

10.18653/v1/2022.acl-long.284Licence: CC BY

Cite this

Dinan, E., Abercrombie, G., Bergman, A. S., Spruit, S., Hovy, D., Boureau, Y.-L., & Rieser, V. (2022). SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 4113–4133). Association for Computational Linguistics. https://doi.org/10.18653/v1/2022.acl-long.284

@inproceedings{b4c3e03ad55745f3937919c2327a4ec0,

title = "SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems",

abstract = "The social impact of natural language processing and its applications has received increasing attention. In this position paper, we focus on the problem of safety for end-to-end conversational AI. We survey the problem landscape therein, introducing a taxonomy of three observed phenomena: the Instigator, Yea-Sayer, and Impostor effects. We then empirically assess the extent to which current tools can measure these effects and current systems display them. We release these tools as part of a “first aid kit” (SafetyKit) to quickly assess apparent safety concerns. Our results show that, while current tools are able to provide an estimate of the relative safety of systems in various settings, they still have several shortcomings. We suggest several future directions and discuss ethical considerations.",

author = "Emily Dinan and Gavin Abercrombie and Bergman, {A. Stevie} and Shannon Spruit and Dirk Hovy and Y-Lan Boureau and Verena Rieser",

year = "2022",

month = may,

doi = "10.18653/v1/2022.acl-long.284",

language = "English",

pages = "4113–4133",

booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",

publisher = "Association for Computational Linguistics",

}

Dinan, E, Abercrombie, G, Bergman, AS, Spruit, S, Hovy, D, Boureau, Y-L & Rieser, V 2022, SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems. in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, pp. 4113–4133. https://doi.org/10.18653/v1/2022.acl-long.284

SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems. / Dinan, Emily; Abercrombie, Gavin; Bergman, A. Stevie et al.
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2022. p. 4113–4133.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems

AU - Dinan, Emily

AU - Abercrombie, Gavin

AU - Bergman, A. Stevie

AU - Spruit, Shannon

AU - Hovy, Dirk

AU - Boureau, Y-Lan

AU - Rieser, Verena

PY - 2022/5

Y1 - 2022/5

N2 - The social impact of natural language processing and its applications has received increasing attention. In this position paper, we focus on the problem of safety for end-to-end conversational AI. We survey the problem landscape therein, introducing a taxonomy of three observed phenomena: the Instigator, Yea-Sayer, and Impostor effects. We then empirically assess the extent to which current tools can measure these effects and current systems display them. We release these tools as part of a “first aid kit” (SafetyKit) to quickly assess apparent safety concerns. Our results show that, while current tools are able to provide an estimate of the relative safety of systems in various settings, they still have several shortcomings. We suggest several future directions and discuss ethical considerations.

AB - The social impact of natural language processing and its applications has received increasing attention. In this position paper, we focus on the problem of safety for end-to-end conversational AI. We survey the problem landscape therein, introducing a taxonomy of three observed phenomena: the Instigator, Yea-Sayer, and Impostor effects. We then empirically assess the extent to which current tools can measure these effects and current systems display them. We release these tools as part of a “first aid kit” (SafetyKit) to quickly assess apparent safety concerns. Our results show that, while current tools are able to provide an estimate of the relative safety of systems in various settings, they still have several shortcomings. We suggest several future directions and discuss ethical considerations.

U2 - 10.18653/v1/2022.acl-long.284

DO - 10.18653/v1/2022.acl-long.284

M3 - Conference contribution

SP - 4113

EP - 4133

BT - Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

PB - Association for Computational Linguistics

ER -

SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems

Abstract

Access to Document

Fingerprint

Cite this