Reinforcement learning approaches to natural language generation in interactive systems

In this chapter we will describe a new approach to generating natural language in interactive systems -- one which shares many features with more traditional planning approaches, but which uses statistical machine learning models to develop {\em adaptive} natural language generation (NLG) components for interactive applications. We employ statistical models of users, of generation contexts, and of natural language itself. This approach has several potential advantages: the ability to train models on real data, the availability of precise mathematical methods for optimisation, and the capacity to adapt robustly to previously unseen situations. Rather than emulating human behaviour in generation (which can be sub-optimal) these methods can find strategies for NLG which improve on human performance. Recently, some very encouraging test results have been obtained with real users of systems developed using these methods.

In this chapter we will explain the motivations behind this approach, and will present several case studies, with reference to recent empirical results in the areas of information presentation and referring expression generation, including new work on the generation of temporal referring expressions. Finally, we provide a critical outlook for future work on statistical approaches to adaptive NLG.