Adaptive information presentation for spoken dialogue systems: Evaluation with real users

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Citations (Scopus)

Abstract

We present evaluation results with human subjects
for a novel data-driven approach to Natural
Language Generation in spoken dialogue
systems. We evaluate a trained Information
Presentation (IP) strategy in a deployed
tourist-information spoken dialogue system.
The IP problem is formulated as statistical decision
making under uncertainty using Reinforcement
Learning, where both content planning
and attribute selection are jointly optimised
based on data collected in a Wizard-of-
Oz study. After earlier work testing and training
this model in simulation, we now present
results from an extensive online user study,
involving 131 users and more than 800 test
dialogues, which explores its contribution to
overall ‘global’ task success. We find that
the trained Information Presentation strategy
significantly improves dialogue task completion,
with up to a 9.7% increase (30% relative)
compared to the deployed dialogue system
which uses conventional, hand-coded presentation
prompts. We also present subjective
evaluation results and discuss the implications
of these results for future work in dialogue
management and NLG.
Original languageEnglish
Title of host publication13th European Workshop on Natural Language Generation
PublisherAssociation for Computational Linguistics
Pages102-109
Number of pages8
Publication statusPublished - 2011

Fingerprint Dive into the research topics of 'Adaptive information presentation for spoken dialogue systems: Evaluation with real users'. Together they form a unique fingerprint.

Cite this