Provides an evaluation of how well appraisers can match each other and the standard (expert). You can use attribute agreement analysis for binary data (good or bad), nominal data (yellow, blue, brown), or ordinal data (1, 2, 3, 4, where categories are value-ordered). Evaluation includes various % agreement analyses as well as Kappa and Kendall’s metrics.
|When to Use||Purpose|
|Start of project||Verify you can consistently measure categorical process outputs before attempting to perform a baseline analysis.|
|Mid-project||Verify you can consistently measure appropriate categorical process inputs.|
|End of project||Verify you can consistently measure categorical process outputs after improvements have been made.|
|End of project||Verify you can consistently measure key categorical inputs that need to be controlled to maintain the improvements.|
Discrete Y that can be binary (for example, good or bad), nominal, or ordinal.
For the ordinal data case, the output includes Kendall’s statistics, as shown below: