On the Evaluation of Snippet Selection for WebCLEF

A. Overwijk; D. Nguyen; Dong-Phuong Nguyen; C. Hauff; Rudolf Berend Trieschnigg; Djoerd Hiemstra; Franciska M.G. de Jong

doi:10.1007/978-3-642-04447-2_103

On the Evaluation of Snippet Selection for WebCLEF

A. Overwijk, D. Nguyen, Dong-Phuong Nguyen, C. Hauff, Rudolf Berend Trieschnigg, Djoerd Hiemstra, Franciska M.G. de Jong

Databases (Former)

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

1 Citation (Scopus)

24 Downloads (Pure)

Abstract

WebCLEF is about supporting a user who is an expert in writing a survey article on a specific topic with a clear goal and audience by generating a ranked list with relevant snippets. This paper focuses on the evaluation methodology of WebCLEF. We show that the evaluation method and test set used for WebCLEF 2007 cannot be used to evaluate new systems and give recommendations how to improve the evaluation.

Original language	English
Title of host publication	Evaluating Systems for Multilingual and Multimodal Information Access
Subtitle of host publication	9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008, Aarhus, Denmark, September 17-19, 2008, Revised Selected Papers
Editors	Carol Peters, Thomas Deselaers, Nicola Ferro, Julio Gonzalo
Place of Publication	Berlin, Heidelberg
Publisher	Springer
Pages	794-797
Number of pages	4
ISBN (Electronic)	978-3-642-04447-2
ISBN (Print)	978-3-642-04446-5
DOIs	https://doi.org/10.1007/978-3-642-04447-2_103
Publication status	Published - 2009
Event	9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008 - Aarhus, Denmark Duration: 17 Sep 2008 → 19 Sep 2008 Conference number: 9

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer
Volume	5706
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Workshop

Workshop	9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008
Abbreviated title	CLEF
Country/Territory	Denmark
City	Aarhus
Period	17/09/08 → 19/09/08

Keywords

Measurement
Performance
Experimentation

Access to Document

10.1007/978-3-642-04447-2_103

Overwijk2009onAccepted author version, 26.4 KB

Cite this

Overwijk, A., Nguyen, D., Nguyen, D-P., Hauff, C., Trieschnigg, R. B., Hiemstra, D., & de Jong, F. M. G. (2009). On the Evaluation of Snippet Selection for WebCLEF. In C. Peters, T. Deselaers, N. Ferro, & J. Gonzalo (Eds.), Evaluating Systems for Multilingual and Multimodal Information Access: 9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008, Aarhus, Denmark, September 17-19, 2008, Revised Selected Papers (pp. 794-797). [10.1007/978-3-642-04447-2_103] (Lecture Notes in Computer Science; Vol. 5706). Springer. https://doi.org/10.1007/978-3-642-04447-2_103

Overwijk, A. ; Nguyen, D. ; Nguyen, Dong-Phuong et al. / On the Evaluation of Snippet Selection for WebCLEF. Evaluating Systems for Multilingual and Multimodal Information Access: 9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008, Aarhus, Denmark, September 17-19, 2008, Revised Selected Papers. editor / Carol Peters ; Thomas Deselaers ; Nicola Ferro ; Julio Gonzalo. Berlin, Heidelberg : Springer, 2009. pp. 794-797 (Lecture Notes in Computer Science).

@inproceedings{79ee735dd07b4356b501237dec4163eb,

title = "On the Evaluation of Snippet Selection for WebCLEF",

abstract = "WebCLEF is about supporting a user who is an expert in writing a survey article on a specific topic with a clear goal and audience by generating a ranked list with relevant snippets. This paper focuses on the evaluation methodology of WebCLEF. We show that the evaluation method and test set used for WebCLEF 2007 cannot be used to evaluate new systems and give recommendations how to improve the evaluation.",

keywords = "Measurement, Performance, Experimentation",

author = "A. Overwijk and D. Nguyen and Dong-Phuong Nguyen and C. Hauff and Trieschnigg, {Rudolf Berend} and Djoerd Hiemstra and {de Jong}, {Franciska M.G.}",

year = "2009",

doi = "10.1007/978-3-642-04447-2_103",

language = "English",

isbn = "978-3-642-04446-5",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "794--797",

editor = "Carol Peters and Thomas Deselaers and Nicola Ferro and Julio Gonzalo",

booktitle = "Evaluating Systems for Multilingual and Multimodal Information Access",

note = "9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008, CLEF ; Conference date: 17-09-2008 Through 19-09-2008",

}

Overwijk, A, Nguyen, D, Nguyen, D-P, Hauff, C, Trieschnigg, RB, Hiemstra, D & de Jong, FMG 2009, On the Evaluation of Snippet Selection for WebCLEF. in C Peters, T Deselaers, N Ferro & J Gonzalo (eds), Evaluating Systems for Multilingual and Multimodal Information Access: 9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008, Aarhus, Denmark, September 17-19, 2008, Revised Selected Papers., 10.1007/978-3-642-04447-2_103, Lecture Notes in Computer Science, vol. 5706, Springer, Berlin, Heidelberg, pp. 794-797, 9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008, Aarhus, Denmark, 17/09/08. https://doi.org/10.1007/978-3-642-04447-2_103

On the Evaluation of Snippet Selection for WebCLEF. / Overwijk, A.; Nguyen, D.; Nguyen, Dong-Phuong et al.

Evaluating Systems for Multilingual and Multimodal Information Access: 9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008, Aarhus, Denmark, September 17-19, 2008, Revised Selected Papers. ed. / Carol Peters; Thomas Deselaers; Nicola Ferro; Julio Gonzalo. Berlin, Heidelberg : Springer, 2009. p. 794-797 10.1007/978-3-642-04447-2_103 (Lecture Notes in Computer Science; Vol. 5706).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - On the Evaluation of Snippet Selection for WebCLEF

AU - Overwijk, A.

AU - Nguyen, D.

AU - Nguyen, Dong-Phuong

AU - Hauff, C.

AU - Trieschnigg, Rudolf Berend

AU - Hiemstra, Djoerd

AU - de Jong, Franciska M.G.

N1 - Conference code: 9

PY - 2009

Y1 - 2009

N2 - WebCLEF is about supporting a user who is an expert in writing a survey article on a specific topic with a clear goal and audience by generating a ranked list with relevant snippets. This paper focuses on the evaluation methodology of WebCLEF. We show that the evaluation method and test set used for WebCLEF 2007 cannot be used to evaluate new systems and give recommendations how to improve the evaluation.

AB - WebCLEF is about supporting a user who is an expert in writing a survey article on a specific topic with a clear goal and audience by generating a ranked list with relevant snippets. This paper focuses on the evaluation methodology of WebCLEF. We show that the evaluation method and test set used for WebCLEF 2007 cannot be used to evaluate new systems and give recommendations how to improve the evaluation.

KW - Measurement

KW - Performance

KW - Experimentation

U2 - 10.1007/978-3-642-04447-2_103

DO - 10.1007/978-3-642-04447-2_103

M3 - Conference contribution

SN - 978-3-642-04446-5

T3 - Lecture Notes in Computer Science

SP - 794

EP - 797

BT - Evaluating Systems for Multilingual and Multimodal Information Access

A2 - Peters, Carol

A2 - Deselaers, Thomas

A2 - Ferro, Nicola

A2 - Gonzalo, Julio

PB - Springer

CY - Berlin, Heidelberg

T2 - 9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008

Y2 - 17 September 2008 through 19 September 2008

ER -

Overwijk A, Nguyen D, Nguyen D-P, Hauff C, Trieschnigg RB, Hiemstra D et al. On the Evaluation of Snippet Selection for WebCLEF. In Peters C, Deselaers T, Ferro N, Gonzalo J, editors, Evaluating Systems for Multilingual and Multimodal Information Access: 9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008, Aarhus, Denmark, September 17-19, 2008, Revised Selected Papers. Berlin, Heidelberg: Springer. 2009. p. 794-797. 10.1007/978-3-642-04447-2_103. (Lecture Notes in Computer Science). doi: 10.1007/978-3-642-04447-2_103