Worksheet randomization for an attribute agreement analysis

Minitab provides different ways you can randomize the runs for your Attribute Agreement Analysis. The AIAG recommends an analysis with randomization of the runs within appraiser.

Randomize runs within appraisers

Suppose you have 50 samples, 3 appraisers, and 2 replicates (trials), for a total of 300 runs, and you want to randomize runs within appraiser.

Appraiser 1 evaluates the first 50 samples, before Appraiser 2 begins to evaluate.

After each appraiser completes the first replicate, Appraiser 1 begins the second replicate and so on.

C1	C2-T	C3-T
RunOrder	Samples	Appraisers
1	24	1
2	6	1
3	8	1
4	22	1
5	33	1
6	41	1
7	32	1
8	36	1
9	19	1
10	3	1
…	…	…
151	1	1
152	50	1
153	44	1
…	…	…
300	9	3

Note

Replicate 2 is in a different order from replicate 1 because of randomization.

When you choose to also randomize appraiser sequence, any of the appraisers can be assigned to their evaluations first, any of the remaining appraisers can be assigned to evaluate second, and so on. In this case, Appraiser 3 may be assigned to the first set, Appraiser 1 to the second set, and Appraiser 2 to the last set.

Randomize all runs

Suppose you have 50 samples, 3 appraisers, and 2 replicates (trials), for a total of 300 runs, and you want to randomize all runs. Each time you create another worksheet and randomize all runs again, you will have a different run order. Because randomization of all runs encompasses both sets of replicates, an appraiser may rate the second replicate of a particular sample before all samples and appraisers have evaluated the first time.

Run 4 is the first replicate of Appraiser 1 measuring sample 24; run 24 is the second replicate.

C1	C2-T	C3-T
RunOrder	Samples	Appraisers
1	48	2
2	28	3
3	10	2
4	24	1
…	…	…
23	24	3
24	24	1
25	28	2
26	24	3
…	…	…
300	37	1

Do not randomize

Suppose you have 50 samples, 3 appraisers, and 2 replicates (trials), for a total of 300 runs, and you do not want to randomize. The run order is listed sequentially, 1 through 300. The samples are listed in order for each appraiser, then the entire sequence is repeated for the second replicate.

Rows 1-150 are the first replicate. Rows 151-300 are the second replicate.

Each of the three appraisers evaluate each of the 50 samples, sequentially for the first replicate, then for the second replicate.

C1	C2-T	C3-T
RunOrder	Samples	Appraisers
1	1	1
2	1	2
3	1	3
4	2	1
5	2	2
6	2	3
…	…	…
150	50	1
151	50	2
152	50	3
153	1	1
154	1	2
155	1	3
…	…	…
298	50	1
299	50	2
300	50	3

Standard order

Standard order is the non-randomized order of the runs. If you choose to display the standard order of the runs, it will be stored in C1 and the other columns will be shifted.