Toward Affective Dialogue Management using Partially Observable Markov Decision Processes

B.H.T. Bui Huu Trung

Abstract

Designing and developing affective dialogue systems have recently received much interest from the dialogue research community. A distinctive feature of these systems is affect modeling. Previous work was mainly focused on showing system's emotions to the user in order to achieve the designer's goal such as helping the student to practice nursing tasks or persuading the user to change their dietary behavior. A challenging problem is to infer the user's affective state and to adapt the system's behavior accordingly. This thesis addresses this problem from an engineering perspective using Partially Observable Markov Decision Process (POMDP) techniques and a Rapid Dialogue Prototyping Methodology (RDPM). We argue that the POMDPs are suitable for use in designing affective dialogue management models for three main reasons. First, the POMDP model allows for realistic modeling of the user's affective state, the user's intention, and other (user's) hidden state components by incorporating them into the state space. Second, recent dialogue management research has shown that the POMDP-based dialogue manager is able to cope well with uncertainty that can occur at many levels inside a dialogue system from speech recognition, natural language understanding to dialogue management. Third, the POMDP environment can be used to create a simulated user which is useful for learning and evaluation of competing dialogue strategies. In the first part of this thesis, we first present the RDPM for a quick production of frame-based dialogue models for traditional (i.e., non-affect sensitive) singleapplication dialogue systems. The usability of the RDPM has been validated through the implementation of several prototype dialogue systems. We then present a novel approach to developing interfaces for multi-application systems which are dialogue systems that allow the user to navigate between a large set of applications smoothly and transparently. The work in this part provides an essential infrastructure for implementing our prototype POMDP-based dialogue manager. In the second part, we first describe a factored POMDP approach to affective dialogue management. This approach illustrates that POMDPs are an elegant model for building affective dialogue systems. Further, the POMDP-based dialogue strategy outperforms all other known strategies from the literature when tested with smallscale dialogue problems. However, a well-known drawback of POMDP-based dialogue managers is that computing a near-optimal dialogue policy is extremely computationally expensive. We then propose a tractable hybrid DDN-POMDP method to tackle many of these scalability problems. The central contribution of our method (com- pared with other POMDP-based dialogue management methods from the literature) is the ability to handle frame-based dialogue problems with hundreds of slots and hundreds of slot values. Keywords: dialogue modeling, dialogue management, dialogue systems, rapid prototyping, partially observable Markov decision processes, multimodal, multi-application, multi-domain, affective computing.
Original languageUndefined
Awarding Institution
  • University of Twente
Supervisors/Advisors
  • Nijholt, Antinus , Supervisor
  • Zwiers, Jakob , Advisor
Sponsors
Date of Award9 Oct 2008
Place of PublicationEnschede
Publisher
Print ISBNs978-90-365-2714-9
DOIs
StatePublished - 9 Oct 2008

Fingerprint

Decision process
Affective
Dialogue systems
Modeling
Methodology
Prototype
Language understanding
Speech recognition
Distinctive features
Evaluation
Dietary change
Natural language
Usability
Management research
Rapid prototyping
Process model
Key words
Managers
Uncertainty

Keywords

  • multi-domain
  • multi-application
  • dialogue management
  • partially observable Markov decision processes
  • Rapid prototyping
  • Dialogue Systems
  • Multimodal
  • Affective Computing
  • IR-59997
  • METIS-252057
  • Dialogue modeling
  • HMI-MI: MULTIMODAL INTERACTIONS
  • EWI-13613

Cite this

Bui Huu Trung, B. H. T. (2008). Toward Affective Dialogue Management using Partially Observable Markov Decision Processes Enschede: Twente University Press (TUP) DOI: 10.3990/1.9789036527149
Bui Huu Trung, B.H.T.. / Toward Affective Dialogue Management using Partially Observable Markov Decision Processes. Enschede : Twente University Press (TUP), 2008. 172 p.
@misc{f0cde62967514967b5fad44a1aed8efe,
title = "Toward Affective Dialogue Management using Partially Observable Markov Decision Processes",
abstract = "Designing and developing affective dialogue systems have recently received much interest from the dialogue research community. A distinctive feature of these systems is affect modeling. Previous work was mainly focused on showing system's emotions to the user in order to achieve the designer's goal such as helping the student to practice nursing tasks or persuading the user to change their dietary behavior. A challenging problem is to infer the user's affective state and to adapt the system's behavior accordingly. This thesis addresses this problem from an engineering perspective using Partially Observable Markov Decision Process (POMDP) techniques and a Rapid Dialogue Prototyping Methodology (RDPM). We argue that the POMDPs are suitable for use in designing affective dialogue management models for three main reasons. First, the POMDP model allows for realistic modeling of the user's affective state, the user's intention, and other (user's) hidden state components by incorporating them into the state space. Second, recent dialogue management research has shown that the POMDP-based dialogue manager is able to cope well with uncertainty that can occur at many levels inside a dialogue system from speech recognition, natural language understanding to dialogue management. Third, the POMDP environment can be used to create a simulated user which is useful for learning and evaluation of competing dialogue strategies. In the first part of this thesis, we first present the RDPM for a quick production of frame-based dialogue models for traditional (i.e., non-affect sensitive) singleapplication dialogue systems. The usability of the RDPM has been validated through the implementation of several prototype dialogue systems. We then present a novel approach to developing interfaces for multi-application systems which are dialogue systems that allow the user to navigate between a large set of applications smoothly and transparently. The work in this part provides an essential infrastructure for implementing our prototype POMDP-based dialogue manager. In the second part, we first describe a factored POMDP approach to affective dialogue management. This approach illustrates that POMDPs are an elegant model for building affective dialogue systems. Further, the POMDP-based dialogue strategy outperforms all other known strategies from the literature when tested with smallscale dialogue problems. However, a well-known drawback of POMDP-based dialogue managers is that computing a near-optimal dialogue policy is extremely computationally expensive. We then propose a tractable hybrid DDN-POMDP method to tackle many of these scalability problems. The central contribution of our method (com- pared with other POMDP-based dialogue management methods from the literature) is the ability to handle frame-based dialogue problems with hundreds of slots and hundreds of slot values. Keywords: dialogue modeling, dialogue management, dialogue systems, rapid prototyping, partially observable Markov decision processes, multimodal, multi-application, multi-domain, affective computing.",
keywords = "multi-domain, multi-application, dialogue management, partially observable Markov decision processes, Rapid prototyping, Dialogue Systems, Multimodal, Affective Computing, IR-59997, METIS-252057, Dialogue modeling, HMI-MI: MULTIMODAL INTERACTIONS, EWI-13613",
author = "{Bui Huu Trung}, B.H.T.",
note = "10.3990/1.9789036527149",
year = "2008",
month = "10",
doi = "10.3990/1.9789036527149",
isbn = "978-90-365-2714-9",
publisher = "Twente University Press (TUP)",
address = "Netherlands",
school = "University of Twente",

}

Toward Affective Dialogue Management using Partially Observable Markov Decision Processes. / Bui Huu Trung, B.H.T.

Enschede : Twente University Press (TUP), 2008. 172 p.

Research output: ScientificPhD Thesis - Research UT, graduation UT

TY - THES

T1 - Toward Affective Dialogue Management using Partially Observable Markov Decision Processes

AU - Bui Huu Trung,B.H.T.

N1 - 10.3990/1.9789036527149

PY - 2008/10/9

Y1 - 2008/10/9

N2 - Designing and developing affective dialogue systems have recently received much interest from the dialogue research community. A distinctive feature of these systems is affect modeling. Previous work was mainly focused on showing system's emotions to the user in order to achieve the designer's goal such as helping the student to practice nursing tasks or persuading the user to change their dietary behavior. A challenging problem is to infer the user's affective state and to adapt the system's behavior accordingly. This thesis addresses this problem from an engineering perspective using Partially Observable Markov Decision Process (POMDP) techniques and a Rapid Dialogue Prototyping Methodology (RDPM). We argue that the POMDPs are suitable for use in designing affective dialogue management models for three main reasons. First, the POMDP model allows for realistic modeling of the user's affective state, the user's intention, and other (user's) hidden state components by incorporating them into the state space. Second, recent dialogue management research has shown that the POMDP-based dialogue manager is able to cope well with uncertainty that can occur at many levels inside a dialogue system from speech recognition, natural language understanding to dialogue management. Third, the POMDP environment can be used to create a simulated user which is useful for learning and evaluation of competing dialogue strategies. In the first part of this thesis, we first present the RDPM for a quick production of frame-based dialogue models for traditional (i.e., non-affect sensitive) singleapplication dialogue systems. The usability of the RDPM has been validated through the implementation of several prototype dialogue systems. We then present a novel approach to developing interfaces for multi-application systems which are dialogue systems that allow the user to navigate between a large set of applications smoothly and transparently. The work in this part provides an essential infrastructure for implementing our prototype POMDP-based dialogue manager. In the second part, we first describe a factored POMDP approach to affective dialogue management. This approach illustrates that POMDPs are an elegant model for building affective dialogue systems. Further, the POMDP-based dialogue strategy outperforms all other known strategies from the literature when tested with smallscale dialogue problems. However, a well-known drawback of POMDP-based dialogue managers is that computing a near-optimal dialogue policy is extremely computationally expensive. We then propose a tractable hybrid DDN-POMDP method to tackle many of these scalability problems. The central contribution of our method (com- pared with other POMDP-based dialogue management methods from the literature) is the ability to handle frame-based dialogue problems with hundreds of slots and hundreds of slot values. Keywords: dialogue modeling, dialogue management, dialogue systems, rapid prototyping, partially observable Markov decision processes, multimodal, multi-application, multi-domain, affective computing.

AB - Designing and developing affective dialogue systems have recently received much interest from the dialogue research community. A distinctive feature of these systems is affect modeling. Previous work was mainly focused on showing system's emotions to the user in order to achieve the designer's goal such as helping the student to practice nursing tasks or persuading the user to change their dietary behavior. A challenging problem is to infer the user's affective state and to adapt the system's behavior accordingly. This thesis addresses this problem from an engineering perspective using Partially Observable Markov Decision Process (POMDP) techniques and a Rapid Dialogue Prototyping Methodology (RDPM). We argue that the POMDPs are suitable for use in designing affective dialogue management models for three main reasons. First, the POMDP model allows for realistic modeling of the user's affective state, the user's intention, and other (user's) hidden state components by incorporating them into the state space. Second, recent dialogue management research has shown that the POMDP-based dialogue manager is able to cope well with uncertainty that can occur at many levels inside a dialogue system from speech recognition, natural language understanding to dialogue management. Third, the POMDP environment can be used to create a simulated user which is useful for learning and evaluation of competing dialogue strategies. In the first part of this thesis, we first present the RDPM for a quick production of frame-based dialogue models for traditional (i.e., non-affect sensitive) singleapplication dialogue systems. The usability of the RDPM has been validated through the implementation of several prototype dialogue systems. We then present a novel approach to developing interfaces for multi-application systems which are dialogue systems that allow the user to navigate between a large set of applications smoothly and transparently. The work in this part provides an essential infrastructure for implementing our prototype POMDP-based dialogue manager. In the second part, we first describe a factored POMDP approach to affective dialogue management. This approach illustrates that POMDPs are an elegant model for building affective dialogue systems. Further, the POMDP-based dialogue strategy outperforms all other known strategies from the literature when tested with smallscale dialogue problems. However, a well-known drawback of POMDP-based dialogue managers is that computing a near-optimal dialogue policy is extremely computationally expensive. We then propose a tractable hybrid DDN-POMDP method to tackle many of these scalability problems. The central contribution of our method (com- pared with other POMDP-based dialogue management methods from the literature) is the ability to handle frame-based dialogue problems with hundreds of slots and hundreds of slot values. Keywords: dialogue modeling, dialogue management, dialogue systems, rapid prototyping, partially observable Markov decision processes, multimodal, multi-application, multi-domain, affective computing.

KW - multi-domain

KW - multi-application

KW - dialogue management

KW - partially observable Markov decision processes

KW - Rapid prototyping

KW - Dialogue Systems

KW - Multimodal

KW - Affective Computing

KW - IR-59997

KW - METIS-252057

KW - Dialogue modeling

KW - HMI-MI: MULTIMODAL INTERACTIONS

KW - EWI-13613

U2 - 10.3990/1.9789036527149

DO - 10.3990/1.9789036527149

M3 - PhD Thesis - Research UT, graduation UT

SN - 978-90-365-2714-9

PB - Twente University Press (TUP)

ER -

Bui Huu Trung BHT. Toward Affective Dialogue Management using Partially Observable Markov Decision Processes. Enschede: Twente University Press (TUP), 2008. 172 p. Available from, DOI: 10.3990/1.9789036527149