Follow-up question handling in the IMIX and Ritel systems: A comparative study

B.W. van Schooten, N. Webb (Editor), B. Webber (Editor), Hendrikus J.A. op den Akker, S. Rosset, O. Galibert, A. Max, G. Illouz

  • 7 Citations

Abstract

One of the basic topics of question answering (QA) dialogue systems is how follow-up questions should be interpreted by a QA system. In this paper, we shall discuss our experience with the IMIX and Ritel systems, for both of which a follow-up question handling scheme has been developed, and corpora have been collected. These two systems are each other's opposites in many respects: IMIX is multimodal, non-factoid, black-box QA, while Ritel is speech, factoid, keyword-based QA. Nevertheless, we will show that they are quite comparable, and that it is fruitful to examine the similarities and differences. We shall look at how the systems are composed, and how real, non-expert, users interact with the systems. We shall also provide comparisons with systems from the literature where possible, and indicate where open issues lie and in what areas existing systems may be improved. We conclude that most systems have a common architecture with a set of common subtasks, in particular detecting follow-up questions and finding referents for them. We characterise these tasks using the typical techniques used for performing them, and data from our corpora. We also identify a special type of follow-up question, the discourse question, which is asked when the user is trying to understand an answer, and propose some basic methods for handling it.
Original languageUndefined
Article number10.1017/S1351324908004920
Pages (from-to)97-118
Number of pages22
JournalNatural language engineering
Volume15
Issue number1
DOIs
StatePublished - Jan 2009

Fingerprint

system
Question answering
user
communication technology
architecture
speech
discourse
comparison
area
method
data
Dialogue systems
Referent
Key words

Keywords

  • EWI-15183
  • EC Grant Agreement nr.: FP6/506909
  • IR-62769
  • Question Answering
  • METIS-263765
  • Dialogue Systems

Cite this

van Schooten, B. W., Webb, N. (Ed.), Webber, B. (Ed.), op den Akker, H. J. A., Rosset, S., Galibert, O., ... Illouz, G. (2009). Follow-up question handling in the IMIX and Ritel systems: A comparative study. 15(1), 97-118. [10.1017/S1351324908004920]. DOI: 10.1017/S1351324908004920

van Schooten, B.W.; Webb, N. (Editor); Webber, B. (Editor); op den Akker, Hendrikus J.A.; Rosset, S.; Galibert, O.; Max, A.; Illouz, G. / Follow-up question handling in the IMIX and Ritel systems: A comparative study.

Vol. 15, No. 1, 10.1017/S1351324908004920, 01.2009, p. 97-118.

Research output: Scientific - peer-reviewArticle

@article{ab6adfa4274b4d79b91151d76b19c799,
title = "Follow-up question handling in the IMIX and Ritel systems: A comparative study",
abstract = "One of the basic topics of question answering (QA) dialogue systems is how follow-up questions should be interpreted by a QA system. In this paper, we shall discuss our experience with the IMIX and Ritel systems, for both of which a follow-up question handling scheme has been developed, and corpora have been collected. These two systems are each other's opposites in many respects: IMIX is multimodal, non-factoid, black-box QA, while Ritel is speech, factoid, keyword-based QA. Nevertheless, we will show that they are quite comparable, and that it is fruitful to examine the similarities and differences. We shall look at how the systems are composed, and how real, non-expert, users interact with the systems. We shall also provide comparisons with systems from the literature where possible, and indicate where open issues lie and in what areas existing systems may be improved. We conclude that most systems have a common architecture with a set of common subtasks, in particular detecting follow-up questions and finding referents for them. We characterise these tasks using the typical techniques used for performing them, and data from our corpora. We also identify a special type of follow-up question, the discourse question, which is asked when the user is trying to understand an answer, and propose some basic methods for handling it.",
keywords = "EWI-15183, EC Grant Agreement nr.: FP6/506909, IR-62769, Question Answering, METIS-263765, Dialogue Systems",
author = "{van Schooten}, B.W. and N. Webb and B. Webber and {op den Akker}, {Hendrikus J.A.} and S. Rosset and O. Galibert and A. Max and G. Illouz",
note = "10.1017/S1351324908004920",
year = "2009",
month = "1",
doi = "10.1017/S1351324908004920",
volume = "15",
pages = "97--118",
number = "1",

}

van Schooten, BW, Webb, N (ed.), Webber, B (ed.), op den Akker, HJA, Rosset, S, Galibert, O, Max, A & Illouz, G 2009, 'Follow-up question handling in the IMIX and Ritel systems: A comparative study' vol 15, no. 1, 10.1017/S1351324908004920, pp. 97-118. DOI: 10.1017/S1351324908004920

Follow-up question handling in the IMIX and Ritel systems: A comparative study. / van Schooten, B.W.; Webb, N. (Editor); Webber, B. (Editor); op den Akker, Hendrikus J.A.; Rosset, S.; Galibert, O.; Max, A.; Illouz, G.

Vol. 15, No. 1, 10.1017/S1351324908004920, 01.2009, p. 97-118.

Research output: Scientific - peer-reviewArticle

TY - JOUR

T1 - Follow-up question handling in the IMIX and Ritel systems: A comparative study

AU - van Schooten,B.W.

AU - op den Akker,Hendrikus J.A.

AU - Rosset,S.

AU - Galibert,O.

AU - Max,A.

AU - Illouz,G.

A2 - Webb,N.

A2 - Webber,B.

N1 - 10.1017/S1351324908004920

PY - 2009/1

Y1 - 2009/1

N2 - One of the basic topics of question answering (QA) dialogue systems is how follow-up questions should be interpreted by a QA system. In this paper, we shall discuss our experience with the IMIX and Ritel systems, for both of which a follow-up question handling scheme has been developed, and corpora have been collected. These two systems are each other's opposites in many respects: IMIX is multimodal, non-factoid, black-box QA, while Ritel is speech, factoid, keyword-based QA. Nevertheless, we will show that they are quite comparable, and that it is fruitful to examine the similarities and differences. We shall look at how the systems are composed, and how real, non-expert, users interact with the systems. We shall also provide comparisons with systems from the literature where possible, and indicate where open issues lie and in what areas existing systems may be improved. We conclude that most systems have a common architecture with a set of common subtasks, in particular detecting follow-up questions and finding referents for them. We characterise these tasks using the typical techniques used for performing them, and data from our corpora. We also identify a special type of follow-up question, the discourse question, which is asked when the user is trying to understand an answer, and propose some basic methods for handling it.

AB - One of the basic topics of question answering (QA) dialogue systems is how follow-up questions should be interpreted by a QA system. In this paper, we shall discuss our experience with the IMIX and Ritel systems, for both of which a follow-up question handling scheme has been developed, and corpora have been collected. These two systems are each other's opposites in many respects: IMIX is multimodal, non-factoid, black-box QA, while Ritel is speech, factoid, keyword-based QA. Nevertheless, we will show that they are quite comparable, and that it is fruitful to examine the similarities and differences. We shall look at how the systems are composed, and how real, non-expert, users interact with the systems. We shall also provide comparisons with systems from the literature where possible, and indicate where open issues lie and in what areas existing systems may be improved. We conclude that most systems have a common architecture with a set of common subtasks, in particular detecting follow-up questions and finding referents for them. We characterise these tasks using the typical techniques used for performing them, and data from our corpora. We also identify a special type of follow-up question, the discourse question, which is asked when the user is trying to understand an answer, and propose some basic methods for handling it.

KW - EWI-15183

KW - EC Grant Agreement nr.: FP6/506909

KW - IR-62769

KW - Question Answering

KW - METIS-263765

KW - Dialogue Systems

U2 - 10.1017/S1351324908004920

DO - 10.1017/S1351324908004920

M3 - Article

VL - 15

SP - 97

EP - 118

IS - 1

M1 - 10.1017/S1351324908004920

ER -

van Schooten BW, Webb N, (ed.), Webber B, (ed.), op den Akker HJA, Rosset S, Galibert O et al. Follow-up question handling in the IMIX and Ritel systems: A comparative study. 2009 Jan;15(1):97-118. 10.1017/S1351324908004920. Available from, DOI: 10.1017/S1351324908004920