Why are F- and p-values estimates shown as asterisks in the output?

The asterisks represent missing values that cannot be calculated because the model is saturated and there are not enough degrees of freedom for error.

Consider this example of a saturated full factorial DOE model: a 3-factor, 2-level design with factors A, B, and C, no replicates, no center points, and no blocks. This design has 8 experimental runs.

When analyzing the design, you choose to fit the saturated model by including all main effects (A, B, C) and all interaction terms (AB, AC, BC, ABC). The resulting ANOVA table gives asterisks for the SS values for Residual Error, the MS value for Residual Error, all F statistics, and all p-values:

General Factorial Regression: C8 versus C5, C6, C7

Analysis of Variance Source DF Adj SS Adj MS F-Value P-Value Model 7 71.9880 10.2840 * * Linear 3 63.9164 21.3055 * * C5 1 7.9818 7.9818 * * C6 1 1.3035 1.3035 * * C7 1 54.6310 54.6310 * * 2-Way Interactions 3 7.8648 2.6216 * * C5*C6 1 3.7888 3.7888 * * C5*C7 1 1.5921 1.5921 * * C6*C7 1 2.4839 2.4839 * * 3-Way Interactions 1 0.2068 0.2068 * * C5*C6*C7 1 0.2068 0.2068 * * Error 0 * * Total 7 71.9880

The missing values are in the table because it is impossible for Minitab to calculate these statistics. It's impossible to calculate them because there are 0 degrees of freedom (DF) for residual error, as the following calculations demonstrate:

Total DF = number of runs - 1
Main effect DF = number of factor levels - 1
Interaction effect DF = DF for component factors, multiplied together
Residual error DF = Total DF - sum of DF for all terms included in the model

So, using the previous example:

Total DF = 8 - 1 = 7 (8 rows of data)
DF for Factor A = 2 - 1 = 1 (Factor A has 2 levels)
DF for Factor B = 2 - 1 = 1
DF for Factor C = 2 - 1 = 1
DF for Interaction AB = (1)*(1) = 1 (Factor A has 1 DF, Factor B has 1 DF)
DF for Interaction AC = (1)*(1) = 1
DF for Interaction BC = (1)*(1) = 1
DF for Interaction ABC = (1)*(1)*(1) = 1
DF for Residual Error = 7 - (1 + 1 + 1 + 1 + 1 + 1 + 1) = 0

Zero degrees of freedom for error cause the calculations to fail as follows. Each value in the Adj MS column is calculated by dividing the values in the Adj SS column by the corresponding values in the DF column (Adj MS for Factor A = Adj SS / DF = 0.0621 / 1 = 0.0621). But, the Adj MS for Residual Error, commonly referred to as the mean square of the error (MSE), cannot be calculated because it is impossible to divide anything by 0 degrees of freedom.

Furthermore, Minitab calculates each value in the table's F column by dividing each Adj MS value by the MSE. For example, the F-value for Factor A would equal 0.0621 / MSE. But because MSE cannot be calculated, F cannot be calculated either.

Finally, the p-value is calculated from the F-statistic. Therefore, if F is missing the p-value must also be missing.

Missing p-values and F-statistics will occur in the ANOVA table whenever you have a 2-level design with one replicate, and you include all the terms in your model. To remedy the situation, re-fit your model without one or more of your interaction terms. To determine which highest-order interaction to remove from a saturated model, use the effects plots to estimate the statistical significance of the interactions.

For example, Minitab can calculate all values in the ANOVA table for the main effects and 2-way interactions if you choose Stat > DOE > Factorial > Analyze Factorial Design, click the Model button, and remove the ABC interaction term from the model:

General Factorial Regression: C8 versus C5, C6, C7

Analysis of Variance Source DF Adj SS Adj MS F-Value P-Value Model 6 60.7857 10.1309 0.90 0.666 Linear 3 53.3165 17.7722 1.59 0.515 C5 1 2.2047 2.2047 0.20 0.734 C6 1 34.4645 34.4645 3.08 0.330 C7 1 16.6473 16.6473 1.49 0.437 2-Way Interactions 3 7.4692 2.4897 0.22 0.876 C5*C6 1 2.3336 2.3336 0.21 0.727 C5*C7 1 5.1106 5.1106 0.46 0.622 C6*C7 1 0.0251 0.0251 0.00 0.970 Error 1 11.2023 11.2023 Total 7 71.9880

Now Minitab calculates all the values because there is 1 DF remaining for error, which means Minitab can calculate the MSE, F, and p-values.