Why are F- and p-values estimates shown as asterisks in the output?

The asterisks represent missing values that cannot be calculated because the model is saturated and there are not enough degrees of freedom for error.

Consider this example of a saturated full factorial DOE model: a 3-factor, 2-level design with factors A, B, and C, no replicates, no center points, and no blocks. This design has 8 experimental runs.

When analyzing the design, you choose to fit the saturated model by including all main effects (A, B, C) and all interaction terms (AB, AC, BC, ABC). The resulting ANOVA table gives asterisks for the SS values for Residual Error, the MS value for Residual Error, all F statistics, and all p-values:

Analysis of Variance Source DF Adj SS Adj MS F-Value P-Value Model 7 71.9880 10.2840 * * Linear 3 15.7405 5.2468 * * C5 1 2.3336 2.3336 * * C6 1 2.2047 2.2047 * * C7 1 11.2023 11.2023 * * 2-Way Interactions 3 51.1369 17.0456 * * C5*C6 1 34.4645 34.4645 * * C5*C7 1 16.6473 16.6473 * * C6*C7 1 0.0251 0.0251 * * 3-Way Interactions 1 5.1106 5.1106 * * C5*C6*C7 1 5.1106 5.1106 * * Error 0 * * Total 7 71.9880
The missing values are in the table because it is impossible for Minitab to calculate these statistics. It's impossible to calculate them because there are 0 degrees of freedom (DF) for residual error, as the following calculations demonstrate:
  • Total DF = number of runs - 1
  • Main effect DF = number of factor levels - 1
  • Interaction effect DF = DF for component factors, multiplied together
  • Residual error DF = Total DF - sum of DF for all terms included in the model
So, using the previous example:
  • Total DF = 8 - 1 = 7 (8 rows of data)
  • DF for Factor A = 2 - 1 = 1 (Factor A has 2 levels)
  • DF for Factor B = 2 - 1 = 1
  • DF for Factor C = 2 - 1 = 1
  • DF for Interaction AB = (1)*(1) = 1 (Factor A has 1 DF, Factor B has 1 DF)
  • DF for Interaction AC = (1)*(1) = 1
  • DF for Interaction BC = (1)*(1) = 1
  • DF for Interaction ABC = (1)*(1)*(1) = 1
  • DF for Residual Error = 7 - (1 + 1 + 1 + 1 + 1 + 1 + 1) = 0

Zero degrees of freedom for error cause the calculations to fail as follows. Each value in the Adj MS column is calculated by dividing the values in the Adj SS column by the corresponding values in the DF column (Adj MS for Factor A = Adj SS / DF = 0.0621 / 1 = 0.0621). But, the Adj MS for Residual Error, commonly referred to as the mean square of the error (MSE), cannot be calculated because it is impossible to divide anything by 0 degrees of freedom.

Furthermore, Minitab calculates each value in the table's F column by dividing each Adj MS value by the MSE. For example, the F-value for Factor A would equal 0.0621 / MSE. But because MSE cannot be calculated, F cannot be calculated either.

Finally, the p-value is calculated from the F-statistic. Therefore, if F is missing the p-value must also be missing.

Missing p-values and F-statistics will occur in the ANOVA table whenever you have a 2-level design with one replicate, and you include all the terms in your model. To remedy the situation, re-fit your model without one or more of your interaction terms. To determine which highest-order interaction to remove from a saturated model, use the effects plots to estimate the statistical significance of the interactions.

For example, Minitab can calculate all values in the ANOVA table for the main effects and 2-way interactions if you choose Stat > DOE > Factorial > Analyze Factorial Design, click the Model button, and remove the ABC interaction term from the model:

Analysis of Variance Source DF Adj SS Adj MS F-Value P-Value Model 6 66.8774 11.1462 2.18 0.476 Linear 3 15.7405 5.2468 1.03 0.604 C5 1 2.3336 2.3336 0.46 0.622 C6 1 2.2047 2.2047 0.43 0.630 C7 1 11.2023 11.2023 2.19 0.378 2-Way Interactions 3 51.1369 17.0456 3.34 0.378 C5*C6 1 34.4645 34.4645 6.74 0.234 C5*C7 1 16.6473 16.6473 3.26 0.322 C6*C7 1 0.0251 0.0251 0.00 0.955 Error 1 5.1106 5.1106 Total 7 71.9880

Now Minitab calculates all the values because there is 1 DF remaining for error, which means Minitab can calculate the MSE, F, and p-values.

By using this site you agree to the use of cookies for analytics and personalized content.  Read our policy