Review: Design parameters of rating scales for Web sites

With the increasing popularity of the Internet, more and more online questionnaires are being conducted. However, little research is being done on their construction, in particular on their design. The authors of this paper have conducted such a study, within the scope of rating scales for Web sites; however, their conclusions are of interest for a broader range of applications. The authors studied the influence of three design parameters (response format, questionnaire layout, and interaction mechanism) on users' behavior on rating scales, determined through perceived ease of use, perceived usefulness, disorientation, and flow. In parallel, the concepts of factor structure, reliability, validity, and sensitivity are introduced. However, their relation with perceived ease of use, perceived usefulness, disorientation, and flow remains fuzzy throughout the paper. Moreover, the relation between these concepts and the design parameters is not convincing. Hence, the psychometric properties of the research are questionable. Through two experiments, the authors found that, first, visual analogue and discrete Likert scales are equally good, and, second, a design that supports direct interaction (with radio buttons) and one item per page is preferred by users over indirect interaction (with drop-down boxes) and all items on one page. This finding was despite the fact that responses were faster with all items on one page. The research report is preceded by a thorough literature overview, and accompanied by a vast amount of analysis. Regrettably, the result is a 35-page paper, which makes the authors' important message less accessible.