The Generalizability of Scoring TIMSS Open-Ended Items [electronic resource] / Teresa A. Smith.
The Third International Mathematics and Science Study (TIMSS) measured mathematics and science achievement of middle school students in more than 40 countries. About one quarter of the tests' nearly 300 items were free response items requiring students to generate their own answers. Scoring the...
Saved in:
| Online Access: |
Full Text (via ERIC) |
|---|---|
| Main Author: | |
| Format: | Electronic eBook |
| Language: | English |
| Published: |
[S.l.] :
Distributed by ERIC Clearinghouse,
1997.
|
| Subjects: |
| Summary: | The Third International Mathematics and Science Study (TIMSS) measured mathematics and science achievement of middle school students in more than 40 countries. About one quarter of the tests' nearly 300 items were free response items requiring students to generate their own answers. Scoring these responses used a two-digit diagnostic code rubric with the first digit determining the correctness of the response and the second used to identify certain types of responses showing common approaches or misconceptions. This paper discusses relative contributions of the sources of error variance in student and country-level scores on these open-ended items due to rater effects as a function of item type. Fifty student booklets from 7 English speaking countries were analyzed for 350 student responses for each item. Generalizability studies determined the variability of scores associated with effects due to raters. Generalizability coefficients indicated a high degree of reliability in the relative rankings of a country's average score on TIMSS free-response ratings based on data from the cross-country coding study. The generalizability for an individual's score on a particular item was found to be somewhat less stable for some items, but this was not a concern, since the purpose of the TIMSS was to report country-level averages. (Contains six tables and five figures.) (SLD) |
|---|---|
| Item Description: | ERIC Document Number: ED414281. ERIC Note: Paper presented at the Annual Meeting of the American Educational Research Association (Chicago, IL, March 24-28, 1997). |
| Physical Description: | 22 p. |