A Generalizability Study of the Angoff Method Applied to Setting Cutoff Scores of Professional Certification Tests [microform] / Ronald T. Cope.
This study used generalizability theory and other statistical concepts to assess the application of the Angoff method to setting cutoff scores on two professional certification tests. A panel of ten judges gave pre- and post-feedback Angoff probability ratings of items of two forms of a professional...
Saved in:
| Online Access: |
Request ERIC Document |
|---|---|
| Main Author: | |
| Format: | Microfilm Book |
| Language: | English |
| Published: |
[S.l.] :
Distributed by ERIC Clearinghouse,
1987.
|
| Subjects: |
| Summary: | This study used generalizability theory and other statistical concepts to assess the application of the Angoff method to setting cutoff scores on two professional certification tests. A panel of ten judges gave pre- and post-feedback Angoff probability ratings of items of two forms of a professional certification test, and another panel of nine judges similarly rated and received feedback for the items of two forms of a similar test for more advanced certification. Feedback for a new item consisted of mean judge rating of the item. Feedback for an old item consisted of mean rating plus the item's p-value for a group of examinees who had scored close to the previous cutoff score. Results show that: (1) mean post-feedback ratings were significantly greater than initial ratings for all four test forms; (2) hypothesized greater internal consistency of second ratings was observed only for the first judge group and test; (3) correlations of mean (over judges) first and second ratings were lower for old items than for new items; (4) judges varied widely in the correlation of their first and second ratings; and (5) most judges were strongly influenced by the marginal-group p-values given for old items. (Author/JAZ) |
|---|---|
| Item Description: | ERIC Note: Paper presented at the Annual Meeting of the American Educational Research Association (Washington, DC, April 20-24, 1987). ERIC Document Number: ED282921. |
| Physical Description: | 16 p. |
| Audience: | Researchers. |
| Reproduction Note: | Microfiche. |
| Action Note: | committed to retain 20240101 20490101 Alliance Shared Trust https://www.coalliance.org/shared-print-archiving-policies |