Interpret the key results for Fit Mixed Effects Model

Complete the following steps to interpret a mixed effects model.

Step 1: Determine whether the random terms significantly affect the response

To determine whether a random term significantly affects the response, compare the p-value for the term in the Variance Components table to your significance level. Usually, a significance level (denoted as α or alpha) of 0.05 works well. A significance level of 0.05 indicates a 5% risk of concluding that an effect exists when there is no actual effect.
P-value ≤ α: The random term significantly affects the response
If the p-value is less than or equal to the significance level, you can conclude that the random term does significantly affect the response. This means that the variance of the random term is significantly different from zero.
P-value > α: The random term does not significantly affect the response
If the p-value is greater than the significance level, you cannot conclude that the random term significantly affects the response. You may want to refit the model without the non-significant term to assess the effect of the term on other results..

Variance Components

SourceVar% of TotalSE VarZ-ValueP-Value
Field0.07791972.93%0.0675801.1529960.124
Error0.02892427.07%0.0105622.7386130.003
Total0.106843       
-2 Log likelihood = 7.736012
Key Results: P-Value

In these results, field is the random term and the p-value for field is 0.124. Because this value is greater than 0.05, you do not have enough evidence to conclude that different fields contribute to the amount of variation in the yield.

Step 2: Determine whether the fixed effect terms significantly affect the response

To determine whether a term significantly affects the response, compare the p-value to your significance level. Usually, a significance level (denoted as α or alpha) of 0.05 works well. A significance level of 0.05 indicates a 5% risk of concluding that an affect exists when there is no actual affect.

The interpretation of each p-value depends on whether it is for the coefficient of a fixed factor term or for a covariate term.

Fixed factor term

For a fixed factor term, the null hypothesis is that the fixed factor term does not significantly affect the response.
P-value ≤ α: The fixed factor term significantly affects the response

If the p-value is less than or equal to the significance level, you can conclude that the fixed factor term does significantly affect the response. The rejection of the null hypothesis indicates that one level effect is significantly different from the other level effects of the term.

P-value > α: The fixed factor term does not significantly affect the response
If the p-value is greater than the significance level, you cannot conclude that the fixed factor term significantly affects the response. You may want to refit the model without the term.

Covariate term

For a covariate term, the null hypothesis is that no association exists between the term and the response.
P-value ≤ α: The association is statistically significant
If the p-value is less than or equal to the significance level, you can conclude that there is a statistically significant association between the response and the covariate term.
P-value > α: The association is not statistically significant
If the p-value is greater than the significance level, you cannot conclude that there is a statistically significant association between the response and the covariate term. You may want to refit the model without the covariate term.

Tests of Fixed Effects

TermDF NumDF DenF-ValueP-Value
Variety5.0015.0026.290.000
Key Results: P-Values

Variety is the fixed factor term, and the p-value for the variety term is less than 0.000. Because this value is less than 0.05, you can conclude that the level means are not all equal, meaning the variety of alfalfa has an effect on the yield.

To obtain a better understanding of the main effects, go to Factorial Plots.

Step 3: Determine how well the model fits your data

To determine how well the model fits your data, examine the goodness-of-fit statistics in the Model Summary table.

S

S is the estimated standard deviation of the error term. The lower the value of S, the better the conditional fitted equation describes the response at the selected factor settings. However, an S value by itself doesn't completely describe model adequacy. Also examine the key results from other tables and the residual plots.

R-sq

R2 is the percentage of variation in the response that is explained by the model. It is calculated as 1 minus the ratio of the error sum of squares (which is the variation that is not explained by model) to the total sum of squares (which is the total variation in the model).

R-sq (adj)

Use adjusted R2 when you want to compare models with the same covariance structure but have a different number of fixed factors and covariates. Assuming the models have the same covariance structure, R2 increases when you add additional fixed factors or covariates. The adjusted R2 value incorporates the number of fixed factors and covariates in the model to help you choose the correct model.

AICc and BIC
When you show the expanded results of the analysis, Minitab shows two more statistics. These statistics are the corrected Akaike’s Information Criterion (AICc) and the Bayesian Information Criterion (BIC). Use these statistics to compare different models. For each statistic, smaller values are desirable.
Consider the following points when you interpret the goodness-of-fit statistics:
  • To get more precise and less biased estimates for the parameters in a model, usually, the number of rows in a data set should be much larger than the number of parameters in the model. To get reasonably good estimates for the variance components of the random terms, you should have enough representative levels for each random factor.

  • Goodness-of-fit statistics are just one measure of how well the model fits the data. Even when a model has a desirable value, you should check the residual plots to verify that the model meets the model assumptions.

Model Summary

SR-sqR-sq(adj)AICcBIC
0.17007192.33%90.20%12.5413.52
Key Results: S, R-sq, R-sq (adj)

In these results, the estimated standard deviation (S) of the random error term is 0.17. The model explains 92.33% of the variation in the yield of alfalfa plants. After adjusting for the number of fixed factor parameters in the model, the percentage reduces to 90.2%.

Step 4: Evaluate how each level of a fixed effect term affects the response

If the p-value indicates that a term is significant, you can examine the coefficients for the term to understand how the term relates to the response. The interpretation of each coefficient depends on whether it is for a fixed factor term or for a covariate term.

The coefficients for a fixed factor term display how the level means for the term differ. You can also perform a multiple comparisons analysis for the term to further classify the level effects into groups that are statistically the same or statistically different.

The coefficient for a covariate term represents the change in the mean response associated with a 1-unit change in that term, while everything else in the model is the same. The sign of the coefficient indicates the direction of the relationship between the term and the response. The size of the coefficient usually provides a good way to assess the practical significance of the term on the response variable.

Coefficients

TermCoefSE CoefDFT-ValueP-Value
Constant3.0945830.1438223.0021.5166920.000
Variety         
  10.3854170.07762615.004.9650160.000
  20.1454170.07762615.001.8732870.081
  30.1079170.07762615.001.3902050.185
  4-0.3195830.07762615.00-4.1169380.001
  50.3954170.07762615.005.0938380.000
Key Results: Coefficients

Of the six varieties of alfalfa in the experiment, the output displays the coefficients for five types. By default, Minitab removes one factor level to avoid perfect multicollinearity. The coefficients for the main effects represent the difference between each level mean and the overall mean. For example, Variety 1 is associated with an alfalfa yield that is approximately 0.385 units greater than the overall mean.

Step 5: Determine whether your model meets the assumptions of the analysis

Use the residual plots to help you determine whether the model is adequate and meets the assumptions of the analysis. If the assumptions are not met, the model may not fit the data well and you should use caution when you interpret the results.

Note

You can plot marginal and conditional residuals. A marginal residual equals the difference between an observed response value and the corresponding estimated mean response without conditioning on the levels of the random factors. In contrast, given the specific levels of the random factors, a conditional residual equals the difference between an observed response value and the corresponding conditional mean response. Use the conditional residuals to check the normality of the error term in the model.

Residuals versus fits plot

The residuals versus fits graph plots the residuals on the y-axis and the fitted values on the x-axis. Use this graph to identify rows of data with much larger residuals than other rows. Further investigate those rows to see whether they are collected correctly. In addition, you can also use this plot to look for specific patterns in the residuals that may indicate additional variables to consider.

Residuals versus order plot

The residuals versus order plot displays the residuals in the order that the data were collected. Use this graph to identify rows of data with much larger residuals than other rows. Further investigate those rows to see whether they are collected correctly. If the plot shows a pattern in time order, you can try to include a time-dependent term in the model to remove the pattern.