Requirements engineering paper classification and evaluation criteria: A proposal and a discussion

Roelf J. Wieringa, Neil Maiden, Nancy Mead, Colette Rolland

Research output: Contribution to journalArticleAcademicpeer-review

379 Citations (Scopus)

Abstract

In recent years, members of the steering committee of the IEEE Requirements Engineering (RE) Conference have discussed paper classification and evaluation criteria for RE papers. The immediate trigger for this discussion was our concern about differences in opinion that sometimes arise in program committees about the criteria to be used in evaluating papers. If program committee members do not all use the same criteria, or if they use criteria different from those used by authors, then papers might be rejected or accepted for the wrong reasons. Surely not all papers should be evaluated according to the same criteria. Some papers describe new techniques but do not report on empirical research; others describe new conceptual frameworks for investigating certain RE problems; others report on industrial experience with existing RE techniques. Other kinds of papers can also be easily recognized. All of these types of papers should be evaluated according to different criteria. But we are far from a consensus about what classes of paper we should distinguish, and what the criteria are for each of these classes.
Original languageUndefined
Pages (from-to)102-107
Number of pages6
JournalRequirements engineering
Volume11
Issue number1
DOIs
Publication statusPublished - Mar 2006

Keywords

  • IR-57657
  • METIS-237985
  • EWI-1314
  • SCS-Services
  • Requirements engineering research Research methods Paper classification Paper evaluation criteria
  • IS-Design science methodology

Cite this

Wieringa, Roelf J. ; Maiden, Neil ; Mead, Nancy ; Rolland, Colette. / Requirements engineering paper classification and evaluation criteria: A proposal and a discussion. In: Requirements engineering. 2006 ; Vol. 11, No. 1. pp. 102-107.
@article{90addd2676094e83b3bcdbe1f191a13b,
title = "Requirements engineering paper classification and evaluation criteria: A proposal and a discussion",
abstract = "In recent years, members of the steering committee of the IEEE Requirements Engineering (RE) Conference have discussed paper classification and evaluation criteria for RE papers. The immediate trigger for this discussion was our concern about differences in opinion that sometimes arise in program committees about the criteria to be used in evaluating papers. If program committee members do not all use the same criteria, or if they use criteria different from those used by authors, then papers might be rejected or accepted for the wrong reasons. Surely not all papers should be evaluated according to the same criteria. Some papers describe new techniques but do not report on empirical research; others describe new conceptual frameworks for investigating certain RE problems; others report on industrial experience with existing RE techniques. Other kinds of papers can also be easily recognized. All of these types of papers should be evaluated according to different criteria. But we are far from a consensus about what classes of paper we should distinguish, and what the criteria are for each of these classes.",
keywords = "IR-57657, METIS-237985, EWI-1314, SCS-Services, Requirements engineering research Research methods Paper classification Paper evaluation criteria, IS-Design science methodology",
author = "Wieringa, {Roelf J.} and Neil Maiden and Nancy Mead and Colette Rolland",
year = "2006",
month = "3",
doi = "10.1007/s00766-005-0021-6",
language = "Undefined",
volume = "11",
pages = "102--107",
journal = "Requirements engineering",
issn = "0947-3602",
publisher = "Springer",
number = "1",

}

Requirements engineering paper classification and evaluation criteria: A proposal and a discussion. / Wieringa, Roelf J.; Maiden, Neil; Mead, Nancy; Rolland, Colette.

In: Requirements engineering, Vol. 11, No. 1, 03.2006, p. 102-107.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Requirements engineering paper classification and evaluation criteria: A proposal and a discussion

AU - Wieringa, Roelf J.

AU - Maiden, Neil

AU - Mead, Nancy

AU - Rolland, Colette

PY - 2006/3

Y1 - 2006/3

N2 - In recent years, members of the steering committee of the IEEE Requirements Engineering (RE) Conference have discussed paper classification and evaluation criteria for RE papers. The immediate trigger for this discussion was our concern about differences in opinion that sometimes arise in program committees about the criteria to be used in evaluating papers. If program committee members do not all use the same criteria, or if they use criteria different from those used by authors, then papers might be rejected or accepted for the wrong reasons. Surely not all papers should be evaluated according to the same criteria. Some papers describe new techniques but do not report on empirical research; others describe new conceptual frameworks for investigating certain RE problems; others report on industrial experience with existing RE techniques. Other kinds of papers can also be easily recognized. All of these types of papers should be evaluated according to different criteria. But we are far from a consensus about what classes of paper we should distinguish, and what the criteria are for each of these classes.

AB - In recent years, members of the steering committee of the IEEE Requirements Engineering (RE) Conference have discussed paper classification and evaluation criteria for RE papers. The immediate trigger for this discussion was our concern about differences in opinion that sometimes arise in program committees about the criteria to be used in evaluating papers. If program committee members do not all use the same criteria, or if they use criteria different from those used by authors, then papers might be rejected or accepted for the wrong reasons. Surely not all papers should be evaluated according to the same criteria. Some papers describe new techniques but do not report on empirical research; others describe new conceptual frameworks for investigating certain RE problems; others report on industrial experience with existing RE techniques. Other kinds of papers can also be easily recognized. All of these types of papers should be evaluated according to different criteria. But we are far from a consensus about what classes of paper we should distinguish, and what the criteria are for each of these classes.

KW - IR-57657

KW - METIS-237985

KW - EWI-1314

KW - SCS-Services

KW - Requirements engineering research Research methods Paper classification Paper evaluation criteria

KW - IS-Design science methodology

U2 - 10.1007/s00766-005-0021-6

DO - 10.1007/s00766-005-0021-6

M3 - Article

VL - 11

SP - 102

EP - 107

JO - Requirements engineering

JF - Requirements engineering

SN - 0947-3602

IS - 1

ER -