Toward Affective Dialogue Management using Partially Observable Markov Decision Processes

Designing and developing affective dialogue systems have recently received much interest from the dialogue research community. A distinctive feature of these systems is affect modeling. Previous work was mainly focused on showing system's emotions to the user in order to achieve the designer's goal such as helping the student to practice nursing tasks or persuading the user to change their dietary behavior. A challenging problem is to infer the user's affective state and to adapt the system's behavior accordingly. This thesis addresses this problem from an engineering perspective using Partially Observable Markov Decision Process (POMDP) techniques and a Rapid Dialogue Prototyping Methodology (RDPM). We argue that the POMDPs are suitable for use in designing affective dialogue management models for three main reasons. First, the POMDP model allows for realistic modeling of the user's affective state, the user's intention, and other (user's) hidden state components by incorporating them into the state space. Second, recent dialogue management research has shown that the POMDP-based dialogue manager is able to cope well with uncertainty that can occur at many levels inside a dialogue system from speech recognition, natural language understanding to dialogue management. Third, the POMDP environment can be used to create a simulated user which is useful for learning and evaluation of competing dialogue strategies. In the first part of this thesis, we first present the RDPM for a quick production of frame-based dialogue models for traditional (i.e., non-affect sensitive) singleapplication dialogue systems. The usability of the RDPM has been validated through the implementation of several prototype dialogue systems. We then present a novel approach to developing interfaces for multi-application systems which are dialogue systems that allow the user to navigate between a large set of applications smoothly and transparently. The work in this part provides an essential infrastructure for implementing our prototype POMDP-based dialogue manager. In the second part, we first describe a factored POMDP approach to affective dialogue management. This approach illustrates that POMDPs are an elegant model for building affective dialogue systems. Further, the POMDP-based dialogue strategy outperforms all other known strategies from the literature when tested with smallscale dialogue problems. However, a well-known drawback of POMDP-based dialogue managers is that computing a near-optimal dialogue policy is extremely computationally expensive. We then propose a tractable hybrid DDN-POMDP method to tackle many of these scalability problems. The central contribution of our method (com- pared with other POMDP-based dialogue management methods from the literature) is the ability to handle frame-based dialogue problems with hundreds of slots and hundreds of slot values. Keywords: dialogue modeling, dialogue management, dialogue systems, rapid prototyping, partially observable Markov decision processes, multimodal, multi-application, multi-domain, affective computing.