Mixed effects models contain both fixed and random effects. The general form of the mixed effects model is:
y = Xβ + Z1μ1+ Z2μ2 + ... + Zcμc + ε
Notation
Term
Description
y
the n x 1 vector of response values
X
the n x p design matrix for the fixed effect terms, p ≤ n
β
a p x 1 vector of unknown parameters
the n x mi design matrix for the random term in the model
μi
an mi x 1 vector of independent variables from N(0, )
ε
an n x 1 vector of independent variables from N(0, )
n
the number of observations
p
the number of parameters in
c
the number of random terms in the model
Variance-covariance matrix
Based on the model assumption for the general form of the mixed effects model, the response vector, y, has a multivariate normal distribution with mean vector Xβ and the following variance-covariance matrix:
σ2, σ21, ... , σ2c are called variance components.
By factoring from the variance, you can find a representation of H(θ), which is in the computation of the log-likelihood of mixed effects models.
V(σ2) = σ2H(θ) = σ2[In + θ1Z1Z'1 + ... + θcZcZ'c]
Notation
Term
Description
θi
, the ratio of the variance of the random term over the error variance
Log-likelihood
When the model contains a random factor, by default the unknown parameter estimates come from minimizing twice the negative of the restricted log-likelihood function. The minimization is equivalent to maximizing the restricted log-likelihood function. Minitab uses an iterative algorithm to minimize the restricted log-likelihood function. The function to minimize is:
Notation
Term
Description
H
In + θ1Z1Z'1 + ... + θcZcZ'c
|H|
the determinant of H
H-1
the inverse of H
mi
the number of levels for the random term
the error variance component
In
the identity matrix with n rows and columns
Restricted Maximum Likelihood (REML) estimation
By default, Minitab calculates parameter estimates that maximize the restricted likelihood function, which is equivalent to minimizing the following function:
To minimize the function, Minitab differentiates the function with respect to β, σ2, and θi and sets the differentials equal to 0:
where
Algebraic rearrangement of the first two equations to solve for the estimated parameters with respect to the differentiation give the following equations:
The derivative with respect to cannot be explicitly solved for the . Minitab uses Newton's method to estimate with the following steps:
Use the Minimum Norm Quadratic Unbiased Estimates (MINQUE)12 of the variance components to construct the initial values of σ2 and θi.
Estimate β and σ2 with the equations for and .
Find θi with Newton's method to minimize L(β, σ2, θ).
Repeat steps 2 and 3 until convergence.
The converged solutions for are the variance ratio estimates. The variance component for the random term is as follows:
Notation
Term
Description
tr(·)
the trace of the matrix
X'
the transpose of X
1 Rao, C.R. (1971 a). Estimation of variance covariance components - MINQUE theory. Journal of Multivariate Analysis 1, 257–275.
2 Rao, C.R. (1971 b). Minimum variance quadratic unbiased estimation of variance components. Journal of Multivariate Analysis 1, 445–456.
By using this site you agree to the use of cookies for analytics and personalized content. Read our policy
random term in the model
)
)
, the ratio of the variance of the 
random term over the error variance
random term
cannot be explicitly solved for the 
. Minitab uses Newton's method to estimate 
with the following steps:
are the variance ratio estimates. The variance component for the 
random term is as follows:
