The quality and qualities of classroom observation systems

Marjoleine Jolie Dobbelaer

Research output: ThesisPhD Thesis - Research UT, graduation UT

200 Downloads (Pure)

Abstract

Teaching quality is often measured by means of classroom observation using a classroom observation system (COS). Generating valid and reliable scores when using a COS is not self-explanatory. Many authors point to issues that need to be taken into account when developing, selecting, and using a COS in order to generate valid and reliable scores. A framework that brings together these issues was developed in the first study of this dissertation. The first part of the framework was designed to evaluate the extent to which the design of a COS will allow it to meet the various criteria set by users. The second and third parts of the framework were designed to be used for assessing the reliability and validity of a COS in a specific context.

In the second study of this dissertation, the framework was used to review COSs that were available to measure teaching quality in primary education. An extensive worldwide literature search for COSs, resulting in 27 COSs that met the inclusion criteria. All COSs were reviewed by two reviewers. Reviewers were, on average, positive about the scoring tools, however, insufficient empirical evidence was found for the reliable and valid use of the scores for most COSs.
In the third study, the teaching quality scores awarded by external raters, students, and teachers were compared. Little is known about how these three relate, whereas such knowledge may have important implications for measuring teaching quality. Data were collected in 25 grade nine mathematics classrooms. Students’ and teachers’ perceptions were measured for three lessons per participating teacher, and these lessons were also rated by external raters. The results showed only a significant but low correlation between external raters and students. Thus, teaching quality scores will differ depending on which of the three methods is used. Those who aim to measure teaching quality should therefore carefully select one, or combine multiple methods.

The dissertation ends with implications for COS developers, COS users, and governments and other institutions, as well as recommendations for future research.






Original languageEnglish
QualificationDoctor of Philosophy
Awarding Institution
  • University of Twente
Supervisors/Advisors
  • Visscher, A.J., Supervisor
Thesis sponsors
Award date22 Feb 2019
Place of PublicationEnschede
Publisher
Print ISBNs978-90-365-4716-1
DOIs
Publication statusPublished - 31 Jan 2019

Fingerprint

classroom
earning a doctorate
Teaching
teacher
student
primary education
school grade
inclusion
mathematics
evidence

Cite this

Dobbelaer, Marjoleine Jolie. / The quality and qualities of classroom observation systems. Enschede : Ipskamp Printing, 2019. 199 p.
@phdthesis{96626c73b49541c8a0303b31c2a345bb,
title = "The quality and qualities of classroom observation systems",
abstract = "Teaching quality is often measured by means of classroom observation using a classroom observation system (COS). Generating valid and reliable scores when using a COS is not self-explanatory. Many authors point to issues that need to be taken into account when developing, selecting, and using a COS in order to generate valid and reliable scores. A framework that brings together these issues was developed in the first study of this dissertation. The first part of the framework was designed to evaluate the extent to which the design of a COS will allow it to meet the various criteria set by users. The second and third parts of the framework were designed to be used for assessing the reliability and validity of a COS in a specific context. In the second study of this dissertation, the framework was used to review COSs that were available to measure teaching quality in primary education. An extensive worldwide literature search for COSs, resulting in 27 COSs that met the inclusion criteria. All COSs were reviewed by two reviewers. Reviewers were, on average, positive about the scoring tools, however, insufficient empirical evidence was found for the reliable and valid use of the scores for most COSs. In the third study, the teaching quality scores awarded by external raters, students, and teachers were compared. Little is known about how these three relate, whereas such knowledge may have important implications for measuring teaching quality. Data were collected in 25 grade nine mathematics classrooms. Students’ and teachers’ perceptions were measured for three lessons per participating teacher, and these lessons were also rated by external raters. The results showed only a significant but low correlation between external raters and students. Thus, teaching quality scores will differ depending on which of the three methods is used. Those who aim to measure teaching quality should therefore carefully select one, or combine multiple methods. The dissertation ends with implications for COS developers, COS users, and governments and other institutions, as well as recommendations for future research.",
author = "Dobbelaer, {Marjoleine Jolie}",
year = "2019",
month = "1",
day = "31",
doi = "10.3990/1.9789036547161",
language = "English",
isbn = "978-90-365-4716-1",
publisher = "Ipskamp Printing",
address = "Netherlands",
school = "University of Twente",

}

Dobbelaer, MJ 2019, 'The quality and qualities of classroom observation systems', Doctor of Philosophy, University of Twente, Enschede. https://doi.org/10.3990/1.9789036547161

The quality and qualities of classroom observation systems. / Dobbelaer, Marjoleine Jolie.

Enschede : Ipskamp Printing, 2019. 199 p.

Research output: ThesisPhD Thesis - Research UT, graduation UT

TY - THES

T1 - The quality and qualities of classroom observation systems

AU - Dobbelaer, Marjoleine Jolie

PY - 2019/1/31

Y1 - 2019/1/31

N2 - Teaching quality is often measured by means of classroom observation using a classroom observation system (COS). Generating valid and reliable scores when using a COS is not self-explanatory. Many authors point to issues that need to be taken into account when developing, selecting, and using a COS in order to generate valid and reliable scores. A framework that brings together these issues was developed in the first study of this dissertation. The first part of the framework was designed to evaluate the extent to which the design of a COS will allow it to meet the various criteria set by users. The second and third parts of the framework were designed to be used for assessing the reliability and validity of a COS in a specific context. In the second study of this dissertation, the framework was used to review COSs that were available to measure teaching quality in primary education. An extensive worldwide literature search for COSs, resulting in 27 COSs that met the inclusion criteria. All COSs were reviewed by two reviewers. Reviewers were, on average, positive about the scoring tools, however, insufficient empirical evidence was found for the reliable and valid use of the scores for most COSs. In the third study, the teaching quality scores awarded by external raters, students, and teachers were compared. Little is known about how these three relate, whereas such knowledge may have important implications for measuring teaching quality. Data were collected in 25 grade nine mathematics classrooms. Students’ and teachers’ perceptions were measured for three lessons per participating teacher, and these lessons were also rated by external raters. The results showed only a significant but low correlation between external raters and students. Thus, teaching quality scores will differ depending on which of the three methods is used. Those who aim to measure teaching quality should therefore carefully select one, or combine multiple methods. The dissertation ends with implications for COS developers, COS users, and governments and other institutions, as well as recommendations for future research.

AB - Teaching quality is often measured by means of classroom observation using a classroom observation system (COS). Generating valid and reliable scores when using a COS is not self-explanatory. Many authors point to issues that need to be taken into account when developing, selecting, and using a COS in order to generate valid and reliable scores. A framework that brings together these issues was developed in the first study of this dissertation. The first part of the framework was designed to evaluate the extent to which the design of a COS will allow it to meet the various criteria set by users. The second and third parts of the framework were designed to be used for assessing the reliability and validity of a COS in a specific context. In the second study of this dissertation, the framework was used to review COSs that were available to measure teaching quality in primary education. An extensive worldwide literature search for COSs, resulting in 27 COSs that met the inclusion criteria. All COSs were reviewed by two reviewers. Reviewers were, on average, positive about the scoring tools, however, insufficient empirical evidence was found for the reliable and valid use of the scores for most COSs. In the third study, the teaching quality scores awarded by external raters, students, and teachers were compared. Little is known about how these three relate, whereas such knowledge may have important implications for measuring teaching quality. Data were collected in 25 grade nine mathematics classrooms. Students’ and teachers’ perceptions were measured for three lessons per participating teacher, and these lessons were also rated by external raters. The results showed only a significant but low correlation between external raters and students. Thus, teaching quality scores will differ depending on which of the three methods is used. Those who aim to measure teaching quality should therefore carefully select one, or combine multiple methods. The dissertation ends with implications for COS developers, COS users, and governments and other institutions, as well as recommendations for future research.

U2 - 10.3990/1.9789036547161

DO - 10.3990/1.9789036547161

M3 - PhD Thesis - Research UT, graduation UT

SN - 978-90-365-4716-1

PB - Ipskamp Printing

CY - Enschede

ER -