Natural multimodal dialogue systems

Abstract

In the context of natural multimodal dialogue systems, we address the challenging issue of the definition of cooperative answers in an appropriate multimodal form. Highlighting the intertwined relation of multimodal outputs with the content, we focus on the Dialogic strategy component, a component that defines from the set of possible contents to answer a user's request, the content to be presented to the user and its multimodal presentation. The content selection and the presentation allocation managed by the Dialogic strategy component are based on various constraints such as the availability of a modality and the user's preferences. Considering three generic types of dialogue strategies and their corresponding handled types of information as well as three generic types of presentation tasks, we present a first implementation of the Dialogic strategy component based on rules. By providing a graphical interface to configure the component by editing the rules, we show how the component can be easily modified by ergonomists at design time for exploring different solutions. In further work we envision letting the user modify the component at runtime.