Assessment agreement for Attribute Agreement Analysis

Find definitions and interpretation guidance for every statistic in the assessment agreement table.

In This Topic

Appraiser
# Inspected
# Matched
Percent
95% CI

Appraiser

The name or number of each appraiser.

# Inspected

The total number of items that are evaluated.

# Matched

# Matched is the number of times that the appraiser ratings agree. The number of matches has a different meaning for each of the agreement tables.

# Matched for the Within Appraiser table: The number of times that the appraiser rates each sample the same.; For example, an appraiser rates all 50 samples the same for each trial.
# Matched for the Each Appraiser vs Standard table: The number of times that the appraiser rates each sample correctly.; For example, an appraiser rates 46 of 50 samples correctly for each trial.
# Matched for the Between Appraisers table: The number of times that the appraisers agree with each other's ratings.; For example, all appraisers rate 37 of 50 samples the same for each trial.
# Matched for the All Appraisers vs Standard table: The number of times that all the appraisers rate each sample correctly for all trials.; For example, all appraisers rate 37 of 50 samples consistently and correctly for every trial.

Percent

Percent is the percentage of times that the appraiser ratings agree. The percentage matched has a different meaning for each of the agreement tables.

Percent for the Within Appraiser table: The percentage of times that the appraiser rates each sample the same.; For example, an appraiser rates all 50 samples the same across each trial, for a 100% match.
Percent for the Each Appraiser vs Standard table: The percentage of times that the appraiser rates each sample correctly.; For example, an appraiser rates 46 of 50 samples correctly, for a 92% match.
Percent for the Between Appraisers table: The percentage of times that the appraisers agree with each other's ratings.; For example, all appraisers rate 37 of 50 samples the same across all trials, for a 74% match.
Percent for the All Appraisers vs Standard table: The percentage of times that all the appraisers rate each sample correctly for all trials.; For example, all appraisers rate 37 of 50 samples consistently and correctly across all trials, for a 74% match.

95% CI

The confidence intervals provide a range of likely values for the percentage of appraiser ratings that match each other or the standard value.

Because confidence intervals are calculated from data from representative samples, the range of likely values varies slightly each time you perform the study. A 95% confidence level indicates that, if you perform this analysis 100 times, approximately 95 confidence intervals will contain the true percentage matched. The true percentage matched represents the actual percentage of matches that would occur in the process, not just in the sample for an analysis.

95% CI for the Within Appraiser table: This confidence interval provides a range of likely values for the percentage of samples that an appraiser rates consistently.; For example, an appraiser may rate all 50 samples the same across both trials, for a 100% match, and a confidence interval between 94.18% and 100%.
95% CI for the Each Appraiser vs Standard table: The confidence interval provides a range of likely values for the percentage of samples that an appraiser rates correctly across all trials.; For example, an appraiser may rate 46 of 50 samples correctly, for a 92% match, and a confidence interval between 80.77% and 97.78%.
95% CI for the Between Appraisers table: This confidence interval provides a range of likely values for the percentage of samples that all appraisers rate consistently.; For example, all appraisers may rate 37 of 50 samples the same across all trials, for a 74% match, and a confidence interval between 59.66% and 85.37%.
95% CI for the All Appraisers vs Standard table: This confidence interval provides a range of likely values for the percentage of samples that all appraisers rate consistently and correctly.; For example, all appraisers may rate 37 of 50 samples consistently and correctly across all trials, for a 74% match, and a confidence interval between 59.66% and 85.37%.