Essay Reliability [electronic resource] : Form and Meaning / Doug Shale.
This study is an attempt at a cohesive characterization of the concept of essay reliability. As such, it takes as a basic premise that previous and current practices in reporting reliability estimates for essay tests have certain shortcomings. The study provides an analysis of these shortcomings--pa...
Saved in:
| Online Access: |
Full Text (via ERIC) |
|---|---|
| Main Author: | |
| Format: | Electronic eBook |
| Language: | English |
| Published: |
[S.l.] :
Distributed by ERIC Clearinghouse,
1986.
|
| Subjects: |
| Summary: | This study is an attempt at a cohesive characterization of the concept of essay reliability. As such, it takes as a basic premise that previous and current practices in reporting reliability estimates for essay tests have certain shortcomings. The study provides an analysis of these shortcomings--partly to encourage a fuller understanding of the concept of reliability as applied to essay testing, and partly to build the case that the framework of generalizability theory offers a much more satisfactory way of characterizing the concept. The study applies generalizability theory to existing research to illustrate that this approach is an improvement over the usual methods of estimating essay reliability. The paper also argues that the classical approach to reliability has lead to a preoccupation with inter-marker agreement which in turn has led to a formulation of the "reliability problem" that makes it not susceptible of solution. Conceptual grounds for tolerating inter-marker disagreement are advanced and the paper discusses conditions under which this may be so. However, the paper explains how generalizability theory remains an appropriate framework for estimating the reliability of essay scores whatever assumptions one chooses to make regarding inter-marker consistency. (Author) |
|---|---|
| Item Description: | ERIC Document Number: ED270484. ERIC Note: Paper presented at the Annual Meeting of the American Educational Research Association (70th, San Francisco, CA, April 16-20, 1986). |
| Physical Description: | 42 p. |
| Audience: | Researchers. |