Rank deficiency and full rank in ANOVA models

Linear models are full rank when there are an adequate number of observations per factor level combination to be able to estimate all terms included in the model. When not enough observations are in the data to fit the model, Minitab removes terms until the model is small enough to fit. It is possible that other models may fit the data better.

Suppose you have a two-factor GLM model. You try to fit the model with terms A B and A*B and receive an error about "rank deficiency." This indicates that there are not enough observations per factor level combination. Try removing the interaction term (A*B).

Rank deficiency is a condition that can prevent Minitab from performing matrix calculations. For example, consider the following data set with two predictor variables and one response variable:

X1 and X2 are the predictor variables and Y is the response variable. The regression analysis in Minitab uses least squares to calculate the estimated coefficients b₀, b₁, b₂, in the following linear equation:

Y = b₀ + b₁X₁ + b₂X₂

The least squares procedure is equivalent to solving the set of matrix equations

b = (X^TX)^-1X^TY

where b is a column vector containing the estimated model coefficients, X is a matrix whose first column is a column of ones (used for estimating the intercept/constant) and whose remaining columns are the columns of predictor data (X1, X2,…), and Y is the column vector of response data. For the previous data set, the matrices are:

Minitab uses the QR decomposition to calculate the estimates of the parameters (b₀, b₁, and b₂) and the standard deviations of the parameters. The calculation depends on the eigenvalues of the (X^TX) matrix. If some eigenvalues of the (X^TX) are essentially zero, the square matrix (X^TX) is either singular, or close to being singular, and Minitab will not be able to do the calculations.

Rank deficiency occurs if any X variable columns can be written as a linear combination of the other X columns. Two examples are shown, using C1, C2, and C3 as predictor (X) variables:

Example 1

Example 2

In the first example, notice that C1 + C2 = C3.

In the second example, notice that 2*C1 = C2.

If you try to perform regression (or ANOVA) using these predictors, Minitab will remove terms from the model in order to perform the analysis.

Rank deficiency can also occur with categorical data:

Example 3

In this example, notice that the machine column has the exact same pattern as the operator column. If you perform ANOVA with this data set, Minitab will remove terms from the model in order to perform the analysis.