Interpret the key results for Two-way ANOVA

Complete the following steps to interpret a two-way ANOVA. Key output includes the p-value, the group means, R2, and the residual plots.

Step 1: Determine whether the main effects and interaction effect are statistically significant

To determine whether each main effect and the interaction effect is statistically significant, compare the p-value for each term to your significance level to assess the null hypothesis. Usually, a significance level (denoted as α or alpha) of 0.05 works well. A significance level of 0.05 indicates a 5% risk of concluding that an effect exists when there is no actual effect.
  • The null hypothesis for a main effect is that the response mean for all factor levels are equal.
  • The null hypothesis for an interaction effect is that the response mean for the level of one factor does not depend on the value of the other factor level.
The statistical significance of the effect depends on the p-value, as follows:
  • If the p-value is greater than the significance level you selected, the effect is not statistically significant.
  • If the p-value is less than or equal to the significance level you selected, then the effect for the term is statistically significant.

    The following shows how to interpret significant main effects and interaction effects.

    • If the main effect of a factor is significant, the difference between some of the factor level means are statistically significant.
    • If an interaction term is statistically significant, the relationship between a factor and the response differs by the level of the other factor. In this case, you should not interpret the main effects without considering the interaction effect.
Analysis of Variance
Source
DF
Adj SS
Adj MS
F-Value
P-Value
SinterTime
MetalType
SinterTime*MetalType
Key Result: P-Value

In these results, you can conclude the following, based on the p-values and a significance level of 0.05:
  • The p-value for MetalType is 0.0318, which indicates that the levels of MetalType are associated with different strengths.
  • The p-value for SinterTime is 0.2094, which indicates that the levels of SinterTime are not associated with different strengths.
  • The p-value for the interaction between MetalType*SinterTime is 0.0082, which indicates that the relationship between MetalType and Strength depends on the value of SinterTime.
  • Because the interaction effect between MetalType and SinterTime is statistically significant, you cannot interpret the main effects without considering the interaction effect.

Step 2: Assess the means

If the p-value in the ANOVA table indicates a statistically significant main effect or interaction effect, use the means table to understand the group differences.

For main effects, the table displays the groups within each factor and their fitted means. For interaction effects, the table displays all possible combinations of groups across both factors.

Means
Term
Fitted Mean
SE Mean
100
150
200
1
2
3
100 1
100 2
100 3
150 1
150 2
150 3
200 1
200 2
200 3
Key Result: Fitted Mean

Examine the means table to understand the differences between the groups in your data. Look for differences in group means. If the interaction term is statistically significant, do not interpret the main effects without considering the interaction effects.

In these results, the interaction effect is statistically significant. The interaction effect indicates that the relationship between MetalType and Strength depends on the value of SinterTime. For example, if you use MetalType 2, SinterTime150 is associated with the highest mean strength. However, if you use MetalType 1, SinterTime 100 is associated with the highest mean strength.

Step 3: Determine how well the model fits your data

To determine how well the model fits your data, examine the goodness-of-fit statistics in the model summary table.
S
Use S to assess how well the model describes the response.

S is measured in the units of the response variable and represents the standard deviation of how far the data values fall from the fitted values. The lower the value of S, the better the model describes the response. However, a low S value by itself does not indicate that the model meets the model assumptions. You should check the residual plots to verify the assumptions.

R-sq

R2 is the percentage of variation in the response that is explained by the model. The higher the R2 value, the better the model fits your data. R2 is always between 0% and 100%.

R2 always increases when you add additional predictors to a model. For example, the best five-predictor model will always have an R2 that is at least as high the best four-predictor model. Therefore, R2 is most useful when you compare models of the same size.

A high R2 value does not indicate that the model meets the model assumptions. You should check the residual plots to verify the assumptions.

R-sq (adj)

Use adjusted R2 when you want to compare models that have different numbers of predictors. R2 always increases when you add a predictor to the model, even when there is no real improvement to the model. The adjusted R2 value incorporates the number of predictors in the model to help you choose the correct model.

R-sq (pred)

Use predicted R2 to determine how well your model predicts the response for new observations. Models that have larger predicted R2 values have better predictive ability.

A predicted R2 that is substantially less than R2 may indicate that the model is over-fit. An over-fit model occurs when you add terms for effects that are not important in the population, although they may appear important in the sample data. The model becomes tailored to the sample data and therefore, may not be useful for making predictions about the population.

Predicted R2 can also be more useful than adjusted R2 for comparing models because it is calculated with observations that are not included in the model calculation.

Model Summary
S
R-sq
R-sq(adj)
R-sq(pred)
Key Results: S, R-sq, R-sq (adj), R-sq (pred)

In these results, the predictors explain 63.28% of the variation in the response. The adjusted R2 is 46.96%, which is a decrease of 17%. The low predicted R2 value (17.37%) indicates that the model does not predict new observations as well as it fits the sample data. Thus, you should not use the model to make generalizations beyond the sample data.

Step 4: Determine whether your model meets the assumptions of the analysis

Use the residual plots to help you determine whether the model is adequate and meets the assumptions of the analysis. If the assumptions are not met, the model may not fit the data well and you should use caution when you interpret the results.

Residuals versus fits plot

Use the residuals versus fits plot to verify the assumption that the residuals are randomly distributed and have constant variance. Ideally, the points should fall randomly on both sides of 0, with no recognizable patterns in the points.

The patterns in the following table may indicate that the model does not meet the model assumptions.
Pattern What the pattern may indicate
Fanning or uneven spreading of residuals across fitted values Nonconstant variance
A point that is far away from zero An outlier

In this residual versus fits plot, the points appear randomly scattered on the plot. None of the groups appear to have substantially different variability.

Residuals versus order plot

Use the residuals versus order plot to verify the assumption that the residuals are independent from one another. Independent residuals show no trends or patterns when displayed in time order. Patterns in the points may indicate that residuals near each other may be correlated, and thus, not independent. Ideally, the residuals on the plot should fall randomly around the center line:
If you see a pattern, investigate the cause. The following types of patterns may indicate that the residuals are dependent.
Trend
Shift
Cycle

In this residual versus order plot, the residuals fall randomly around the centerline.

Normality plot of the residuals

Use the normal probability plot of residuals to verify the assumption that the residuals are normally distributed. The normal probability plot of the residuals should approximately follow a straight line.

The patterns in the following table may indicate that the model does not meet the model assumptions.
Pattern What the pattern may indicate
Not a straight line Nonnormality
A point that is far away from the line An outlier
Changing slope An unidentified variable

In this normal probability plot, the residuals appear to deviate from the straight line. Even though the residuals are nonnormally distributed, ANOVA test results are often robust to violations of this assumption.

By using this site you agree to the use of cookies for analytics and personalized content.  Read our policy