# What is the adjusted p-value in multiple comparisons?

Use for multiple comparisons in ANOVA, the adjusted p-value indicates which factor level comparisons within a family of comparisons (hypothesis tests) are significantly different. If the adjusted p-value is less than alpha, then you reject the null hypothesis. The adjustment limits the family error rate to the alpha level you choose. If you use a regular p-value for multiple comparisons, then the family error rate grows with each additional comparison. The adjusted p-value also represents the smallest family error rate at which a particular null hypothesis will be rejected.

It is important to consider the family error rate when making multiple comparisons because your chances of committing a type I error for a series of comparisons is greater than the error rate for any one comparison alone.

Suppose you compare the hardness of 4 different blends of paint. You analyze the data and get the following output:

 Tukey Simultaneous Tests for Differences of Means
 Difference of Levels Difference of Means SE of Difference 95% CI T-Value Adjusted P-Value Blend 2-Blend 1 -6.167 2.281 (-12.553, 0.219) -2.70 0.0606 Blend 3-Blend 1 -1.750 2.281 (-8.136, 4.636) -0.77 0.8682 Blend 4-Blend 1 3.333 2.281 (-3.053, 9.719) 1.46 0.4779 Blend 3-Blend 2 4.417 2.281 (-1.969, 10.803) 1.94 0.2450 Blend 4-Blend 2 9.500 2.281 (3.114, 15.886) 4.17 0.0025 Blend 4-Blend 3 5.083 2.281 (-1.303, 11.469) 2.23 0.1495
 Individual confidence level = 98.89%

You choose an alpha of 0.05 which, in conjunction with the adjusted p-value, limits the family error rate to 0.05. At this level, the differences between blends 4 and 2 are significant. If you lower the family error rate to 0.01, the differences between blends 4 and 2 are still significant.

By using this site you agree to the use of cookies for analytics and personalized content.  Read our policy