Methods and formulas for Comparisons for general linear models

Select the method or formula of your choice.

Grouping information table for multiple pairwise comparisons

Minitab uses the confidence interval results for the difference between two level means to obtain the grouping information. The grouping information is in a matrix. Suppose a term has k levels, then the maximum dimension of the matrix is k x k. If all levels are in one group, then the dimension is k x 1, with letter "A" for all factor levels. If all levels are in different groups, the dimension is k x k with letters on the diagonal only.

Minitab uses this algorithm to determine the content of the matrix:
  1. Sort all the least square means at different levels of a term in descending order, denoted as 1, 2, ... , k.
  2. Define a k x k matrix with value 0 in every cell where k = the number of factor levels.
  3. For column j, where j = 1, ... , Minitab does the following:
    1. Checks the confidence intervals of mean j – mean r, where r = j + 1, .. , k. If the interval for r contains 0, set the rth row and the jth column cell, j) as 1.
    2. Sets (j, j) cell as 1 if at least one cell in column j has value 1.
    3. Calculates the row sums from column 1 to column j for row I = j + 1, ... , k. If min (all row sums) >= 1, terminate the loop; otherwise, increase j by 1 and go to step a.
  4. For every row i, Minitab checks the sum of all column values for the row ≥ 1. If the sum is zero, set the cell of row i and column j = 1, where column j is the first column in the matrix with 0 values. This procedure produces a matrix with values 1 and 0. The total number of groups is the number of nonzero columns.
  5. Minitab matches letters to columns (e.g. A to Column 1, B to Column 2, etc.) and assigns cells with value 1 the correct letter.

Grouping information table for multiple comparisons with a control

Minitab uses the confidence interval results for the difference between each level mean and the control level to obtain the grouping information. The grouping information is in a matrix with one column.

Minitab assigns the letter "A" to the control level.

If an interval contains 0, then the level mean is in the same group as the control level. Minitab assigns the letter "A" to the level mean.

If an interval does not contain zero, then no letter is assigned.

Tukey method

Formula

Tukey's method is available for all pairwise comparisons. The formula for the confidence intervals is:

The formula for the test statistic is:

The adjusted p-value has the following expression:

For details on the calculation of the adjusted p-value, see the references below.

To find the individual error rate from the simultaneous error rate, use the following formula:

Notation

TermDescription
the least squares mean for the ith factor level or factor level combination
the least squares mean for the jth factor level or factor level combination
rthe number of means
Q1 − α, r, uthe 1 − α percentile of the Studentized range distribution for r means to compare with u degrees of freedom
uthe degrees of freedom for error from the model
SEEthe standard error of the estimated difference between the least squares means
αthe simultaneous probability of making a Type I error, based on the Studentized range distribution
α*the individual probability of making a Type I error, based on the t-distribution for one comparison

References

1 Braun, H. I., editor (1994). The collected works of John W. Tukey: Volume VIII Multiple comparisons 1948-1983. New York: Chapman and Hall.

2 J.C. Hsu (1996). Multiple Comparisons: Theory and methods. Chapman & Hall.

Fisher method

Formula

Minitab offers different confidence interval methods for comparing treatment means. For the Fisher method, the confidence interval endpoints and p-values are the same whether the comparisons are pairwise or with a control. The Fisher method uses the individual confidence level. The formula for the confidence intervals is:

The formula for the test statistic is:

The p-value = 2*P{ T u > tu}

To find the simultaneous confidence level from the individual error rate, use the following formula:

Notation

TermDescription
the least squares mean for the ith factor level or factor level combination
the least squares mean for the jth factor level or factor level combination
t1-α*/2, uthe upper α*/2 percentile of the Student's t-distribution with u degrees of freedom
TuRandom variable with a T distribution that has u degrees of freedom for error
Qrandom variable with a Studentized range distribution
αthe simultaneous probability of making a Type I error
α*the individual probability of making a Type I error
uthe degrees of freedom for error from the model
SEEthe standard error of the estimated difference between the least squares means

Bonferroni method

Formula

Minitab offers different confidence interval methods for comparing treatment means. Because the Bonferroni method makes no assumptions about the dependence among the comparisons, it is the most conservative method. "Conservative" in this context indicates that the true confidence level is likely to be greater than the confidence level that is displayed. The formula for the confidence intervals is:

The formula for the test statistic is:

Notation

TermDescription
the least squares mean for the ith factor level or factor level combination
the least squares mean for the jth factor level or factor level combination
t1-α*/2, uthe upper α' /2 percentile of the Student's t-distribution with u degrees of freedom
αthe simultaneous probability of making a Type I error
α'α / c
cthe number of comparisons
uthe degrees of freedom for error from the model
SEEthe standard error of the estimated difference between the least squares means

The number of comparisons depends on whether the comparisons are pairwise or with a control. Let k be the number of means that are being compared. The number of comparisons are in the table below.

Pairwise k (k – 1) / 2
With a control k – 1

Sidak method

Formula

Minitab offers different confidence interval methods for comparing treatment means. The Sidak method treats the comparisons as if they were independent, which produces a conservative approximation of the true error rate. The Sidak method is slightly more powerful than the Bonferroni method.

The formula for the confidence intervals is:

The formula for the test statistic is:

The adjusted p-value = 1 − (1 − p)c.

Notation

TermDescription
the least squares mean for the ith factor level or factor level combination
the least squares mean for the jth factor level or factor level combination
t1-α*/2, uthe upper α' /2 percentile of the Student's t-distribution with u degrees of freedom
αthe simultaneous probability of making a Type I error
α'1 – (1 – α ) 1/ k
uthe degrees of freedom for error from the model
SEEthe standard error of the estimated difference between the least squares means
pThe unadjusted p-value from the t-distribution for a comparison
cthe number of comparisons

The number of comparisons depends on whether the comparisons are pairwise or with a control. Let k be the number of factor combinations. The number of comparisons are in the table below.

Pairwise k (k – 1) / 2
With a control k – 1

Dunnett method

Formula

Minitab offers different confidence interval methods for comparing treatment means. Dunnet's method is available for comparisons with a control. The formula for the confidence intervals is:

The formula for the test statistic is:

The adjusted p-value is the result of integrating the distribution that Dunnett proposes for the test statistic. For details, see the references below.

Notation

TermDescription
the least squares mean for the ith factor level or factor level combination
the least squares mean for the jth factor level or factor level combination
D1-α, k-1, uthe upper α percentile of the distribution that Dunnett proposes with k − 1 comparisons and u degrees of freedom
αthe simultaneous probability of making a Type I error
kthe number of means to compare
uthe degrees of freedom for error from the model
SEEthe standard error of the estimated difference between the least squares means

References

1 Dunnett, C. W. (January 01, 1955). A multiple comparison procedure for comparing several treatments with a control. Journal of the American Statistical Association, 50, 1096-1121.

2 J.C. Hsu (1996). Multiple Comparisons: Theory and methods. Chapman & Hall.

By using this site you agree to the use of cookies for analytics and personalized content.  Read our policy