Risk-graded Safety for Handling Medical Queries in Conversational AI

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Downloads (Pure)

Abstract

Conversational AI systems can engage in unsafe behaviour when handling users’ medical queries that may have severe consequences and could even lead to deaths. Systems therefore need to be capable of both recognising the seriousness of medical inputs and producing responses with appropriate levels of risk. We create a corpus of human written English language medical queries and the responses of different types of systems. We label these with both crowdsourced and expert annotations. While individual crowdworkers may be unreliable at grading the seriousness of the prompts, their aggregated labels tend to agree with professional opinion to a greater extent on identifying the medical queries and recognising the risk types posed by the responses. Results of classification experiments suggest that, while these tasks can be automated, caution should be exercised, as errors can potentially be very serious.
Original languageEnglish
Title of host publicationProceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)
EditorsYulan He, Heng Ji, Sujian Li, Yang Liu, Chua-Hui Chang
PublisherAssociation for Computational Linguistics
Pages234–243
Number of pages10
Volume2
ISBN (Print)9781955917643
Publication statusPublished - Nov 2022
Event2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing: Fairness in Natural Language Processing - Taipei, Taiwan, Province of China
Duration: 21 Nov 202223 Nov 2022
Conference number: 2/12
http://www.wikicfp.com/cfp/servlet/event.showcfp?eventid=158597

Conference

Conference2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing
Abbreviated titleAACL-IJCNLP 2022
Country/TerritoryTaiwan, Province of China
CityTaipei
Period21/11/2223/11/22
Internet address

Fingerprint

Dive into the research topics of 'Risk-graded Safety for Handling Medical Queries in Conversational AI'. Together they form a unique fingerprint.

Cite this