An educational testing company hired five appraisers to evaluate the written portion of a standardized essay test. To complete the training, each appraiser must be able to rate essays consistently with standard ratings. Each appraiser rates 52 essays that have a known standard rating.
You can use this data to demonstrate Attribute Agreement Analysis.Because the data include a known standard for each sample, the test engineer can assess the consistency and correctness of ratings compared to the standard as well as compared with other appraisers.
|Appraiser||The name of the appraiser evaluating the essay: Simpson, Montgomery, Holmes, Duncan, or Hayes|
|Sample||The essay (sample) identifier: 1 to 52|
|Rating||The appraiser's rating for the sample essay: -2, -1, 0, 1, or 2|
|Standard||The known standard rating for the sample essay: -2, -1, 0, 1, or 2|