Mixed effects models contain both fixed and random effects. The general form of the mixed effects model is:
y = Xβ + Z1μ1+ Z2μ2 + ... + Zcμc + ε
Notation
Term
Description
y
the n x 1 vector of response values
X
the n x p design matrix for the fixed effect terms, p ≤ n
β
a p x 1 vector of unknown parameters
the n x mi design matrix for the random term in the model
μi
an mi x 1 vector of independent variables from N(0, )
ε
an n x 1 vector of independent variables from N(0, )
n
the number of observations
p
the number of parameters in
c
the number of random terms in the model
Variance-covariance matrix
Based on the model assumption for the general form of the mixed effects model, the response vector, y, has a multivariate normal distribution with mean vector Xβ and the following variance-covariance matrix:
σ2, σ21, ... , σ2c are called variance components.
By factoring from the variance, you can find a representation of H(θ), which is in the computation of the log-likelihood of mixed effects models.
V(σ2) = σ2H(θ) = σ2[In + θ1Z1Z'1 + ... + θcZcZ'c]
Notation
Term
Description
θi
, the ratio of the variance of the random term over the error variance
Log-likelihood
When the model contains a random factor, by default the unknown parameter estimates come from minimizing twice the negative of the restricted log-likelihood function. The minimization is equivalent to maximizing the restricted log-likelihood function. Minitab uses an iterative algorithm to minimize the restricted log-likelihood function. The function to minimize is:
Notation
Term
Description
H
In + θ1Z1Z'1 + ... + θcZcZ'c
|H|
the determinant of H
H-1
the inverse of H
mi
the number of levels for the random term
the error variance component
In
the identity matrix with n rows and columns
Restricted Maximum Likelihood (REML) estimation
By default, Minitab calculates parameter estimates that maximize the restricted likelihood function, which is equivalent to minimizing the following function:
To minimize the function, Minitab differentiates the function with respect to β, σ2, and θi and sets the differentials equal to 0:
where
Algebraic rearrangement of the first two equations to solve for the estimated parameters with respect to the differentiation give the following equations:
The derivative with respect to cannot be explicitly solved for the . Minitab uses Newton's method to estimate with the following steps:
Use the Minimum Norm Quadratic Unbiased Estimates (MINQUE)12 of the variance components to construct the initial values of σ2 and θi.
Estimate β and σ2 with the equations for and .
Find θi with Newton's method to minimize L(β, σ2, θ).
Repeat steps 2 and 3 until convergence.
The converged solutions for are the variance ratio estimates. The variance component for the random term is as follows:
Notation
Term
Description
tr(·)
the trace of the matrix
X'
the transpose of X
1 Rao, C.R. (1971 a). Estimation of variance covariance components - MINQUE theory. Journal of Multivariate Analysis 1, 257–275.
2 Rao, C.R. (1971 b). Minimum variance quadratic unbiased estimation of variance components. Journal of Multivariate Analysis 1, 445–456.
random term in the model
)
)
, the ratio of the variance of the 
random term over the error variance
random term
cannot be explicitly solved for the 
. Minitab uses Newton's method to estimate 
with the following steps:
are the variance ratio estimates. The variance component for the 
random term is as follows:
