Methods for Fit Mixed Effects Model

Select the method or formula of your choice.

The mixed effects model and the log-likelihood

The general form of the mixed effects model

Mixed effects models contain both fixed and random effects. The general form of the mixed effects model is:

y =+ Z1μ1 + Z2μ2 + ... + Zcμc + ε

Notation

TermDescription
ythe n x 1 vector of response values
Xthe n x p design matrix for the fixed effect terms, pn
βa p x 1 vector of unknown parameters
the n x mi design matrix for the random term in the model
μian mi x 1 vector of independent variables from N(0, )
εan n x 1 vector of independent variables from N(0, )
nthe number of observations
pthe number of parameters in
cthe number of random terms in the model

Variance-covariance matrix

Based on the model assumption for the general form of the mixed effects model, the response vector, y, has a multivariate normal distribution with mean vector and the following variance-covariance matrix:

V(σ2) = V(σ2, σ21, ... , σ2c) = σ2In + σ21Z1Z'1 + ... + σ2cZcZ'c

where

σ2 = (σ2, σ21, ... , σ2c)'

σ2, σ21, ... , σ2c are called variance components.

By factoring from the variance, you can find a representation of H(θ), which is in the computation of the log-likelihood of mixed effects models.

V(σ2) = σ2H(θ) = σ2[In + θ1Z1Z'1 + ... + θcZcZ'c]

Notation

TermDescription
θi, the ratio of the variance of the random term over the error variance

Log-likelihood

When the model contains a random factor, by default the unknown parameter estimates come from minimizing twice the negative of the restricted log-likelihood function. The minimization is equivalent to maximizing the restricted log-likelihood function. Minitab uses an iterative algorithm to minimize the restricted log-likelihood function. The function to minimize is:

Notation

TermDescription
HIn + θ1Z1Z'1 + ... + θcZcZ'c
|H|the determinant of H
H-1the inverse of H
mithe number of levels for the random term
the error variance component
Inthe identity matrix with n rows and columns

Restricted Maximum Likelihood (REML) estimation

By default, Minitab calculates parameter estimates that maximize the restricted likelihood function, which is equivalent to minimizing the following function:
To minimize the function, Minitab differentiates the function with respect to β, σ2, and θi and sets the differentials equal to 0:

where

Algebraic rearrangement of the first two equations to solve for the estimated parameters with respect to the differentiation give the following equations:
The derivative with respect to cannot be explicitly solved for the . Minitab uses Newton's method to estimate with the following steps:
  1. Use the Minimum Norm Quadratic Unbiased Estimates (MINQUE)12 of the variance components to construct the initial values of σ2 and θi.
  2. Estimate β and σ2 with the equations for and .
  3. Find θi with Newton's method to minimize L(β, σ2, θ).
  4. Repeat steps 2 and 3 until convergence.
The converged solutions for are the variance ratio estimates. The variance component for the random term is as follows:

Notation

TermDescription
tr(·)the trace of the matrix
X'the transpose of X
1 Rao, C.R. (1971 a). Estimation of variance covariance components - MINQUE theory. Journal of Multivariate Analysis 1, 257–275.
2 Rao, C.R. (1971 b). Minimum variance quadratic unbiased estimation of variance components. Journal of Multivariate Analysis 1, 445–456.
    By using this site you agree to the use of cookies for analytics and personalized content.  Read our policy