Mixed effects models contain both fixed and random effects. The general form of the mixed effects model is:
y = Xβ + Z1μ1+ Z2μ2 + ... + Zcμc + ε
the n x 1 vector of response values
the n x p design matrix for the fixed effect terms, p ≤ n
a p x 1 vector of unknown parameters
the n x mi design matrix for the random term in the model
an mi x 1 vector of independent variables from N(0, )
an n x 1 vector of independent variables from N(0, )
the number of observations
the number of parameters in
the number of random terms in the model
Based on the model assumption for the general form of the mixed effects model, the response vector, y, has a multivariate normal distribution with mean vector Xβ and the following variance-covariance matrix:
σ2, σ21, ... , σ2c are called variance components.
By factoring from the variance, you can find a representation of H(θ), which is in the computation of the log-likelihood of mixed effects models.
V(σ2) = σ2H(θ) = σ2[In + θ1Z1Z'1 + ... + θcZcZ'c]
, the ratio of the variance of the random term over the error variance
When the model contains a random factor, by default the unknown parameter estimates come from minimizing twice the negative of the restricted log-likelihood function. The minimization is equivalent to maximizing the restricted log-likelihood function. Minitab uses an iterative algorithm to minimize the restricted log-likelihood function. The function to minimize is:
In + θ1Z1Z'1 + ... + θcZcZ'c
the determinant of H
the inverse of H
the number of levels for the random term
the error variance component
the identity matrix with n rows and columns
Restricted Maximum Likelihood (REML) estimation
By default, Minitab calculates parameter estimates that maximize the restricted likelihood function, which is equivalent to minimizing the following function:
To minimize the function, Minitab differentiates the function with respect to β, σ2, and θi and sets the differentials equal to 0:
Algebraic rearrangement of the first two equations to solve for the estimated parameters with respect to the differentiation give the following equations:
The derivative with respect to cannot be explicitly solved for the . Minitab uses Newton's method to estimate with the following steps:
Use the Minimum Norm Quadratic Unbiased Estimates (MINQUE)12 of the variance components to construct the initial values of σ2 and θi.
Estimate β and σ2 with the equations for and .
Find θi with Newton's method to minimize L(β, σ2, θ).
Repeat steps 2 and 3 until convergence.
The converged solutions for are the variance ratio estimates. The variance component for the random term is as follows:
the trace of the matrix
the transpose of X
1 Rao, C.R. (1971 a). Estimation of variance covariance components - MINQUE theory. Journal of Multivariate Analysis 1, 257–275.
2 Rao, C.R. (1971 b). Minimum variance quadratic unbiased estimation of variance components. Journal of Multivariate Analysis 1, 445–456.