Abstract
Although the data-driven approaches of some recent bot building platforms make it possible for a wide range of users to easily create dialogue systems, those platforms don't offer tools for quickly identifying which log dialogues contain problems. Thus, in this paper, we (1) introduce a new task, log dialogue ranking, where the ranker places problematic dialogues higher (2) provide a collection of human-bot conversations in the restaurant inquiry task labelled with dialogue quality for ranker training and evaluation (3) present a detailed description of the data collection pipeline, which is entirely based on crowd-sourcing (4) finally report a benchmark result of dialogue ranking, which shows the usability of the data and sets a baseline for future studies.
| Original language | English |
|---|---|
| Pages | 3920-3924 |
| Number of pages | 5 |
| DOIs | |
| Publication status | Published - 2020 |
| Event | 21st Annual Conference of the International Speech Communication Association 2020 - Shanghai, China Duration: 25 Oct 2020 → 29 Oct 2020 |
Conference
| Conference | 21st Annual Conference of the International Speech Communication Association 2020 |
|---|---|
| Abbreviated title | INTERSPEECH 2020 |
| Country/Territory | China |
| City | Shanghai |
| Period | 25/10/20 → 29/10/20 |
Keywords
- Dialogue quality
- Dialogue ranking
- Dialogue system
- Language resource
ASJC Scopus subject areas
- Language and Linguistics
- Human-Computer Interaction
- Signal Processing
- Software
- Modelling and Simulation