Methods and formulas for the estimated equation in Binary Fitted Line Plot

Select the method or formula of your choice.

Coefficients

Two methods to find the maximum likelihood estimates of the coefficients exist. One method is to maximize directly the likelihood function with respect to the coefficients. These expressions are nonlinear in the coefficients. The alternative method is to use an iterative reweighted least squares approach, which is the method that Minitab uses to obtain the estimates of the coefficients. McCullagh and Nelder1 show that the two methods are equivalent. However, the iterative reweighted least squares method is easier to implement. For details, see 1.

[1] P. McCullagh and J. A. Nelder (1989). Generalized Linear Models, 2nd Ed., Chapman & Hall/CRC, London.

Standard error of coefficients

The standard error of the ith coefficient is the positive square root of the ith diagonal element of the variance-covariance matrix. The variance-covariance matrix has the following form:

W is a diagonal matrix where the diagonal elements are given by the following formula:

where

This variance-covariance matrix is based on the observed Hessian matrix as opposed to the Fisher's information matrix. Minitab uses the observed Hessian matrix because the model that results is more robust against any conditional mean misspecification.

If the canonical link is used then the observed Hessian matrix and the Fisher's information matrix are identical.

Notation

TermDescription
yithe response value for the ith row
the estimated mean response for the ith row
V(·)the variance function given in the table below
g(·)the link function
V '(·)the first derivative of the variance function
g'(·)the first derivative of the link function
g''(·)the second derivative of the link function

The variance function depends on the model:

Model Variance function
Binomial
Poisson

See [1] and [2] for more information.

[1] A. Agresti (1990). Categorical Data Analysis. John Wiley & Sons, Inc.

[2] P. McCullagh and J.A. Nelder (1992). Generalized Linear Model. Chapman & Hall.

Odds ratios for binary logistic regression

The odds ratio is provided only if you select the logit link function for a model with a binary response. In this case, the odds ratio is useful in interpreting the relationship between a predictor and a response.

The odds ratio (τ) can be any nonnegative number. The odds ratio = 1 serves as the baseline for comparison. If τ = 1, no association exists between the response and predictor. If τ < 1, the odds of the event are higher for the reference level of the factor (or for lower levels of a continuous predictor). If τ > 1, the odds of the event are less for the reference level of the factor (or for lower levels of a continuous predictor). Values farther from 1 represent stronger degrees of association.

Note

For the binary logistic regression model with one covariate or factor, the estimated odds of success are:

The exponential relationship provides an interpretation for β: The odds increase multiplicatively by eβ1 for every one-unit increase in x. The odds ratio is equivalent to exp(β1).

For example, if β is .75, the odd ratio is exp(.75), which is 2.11. This indicates that there is a 111% increase in the odds of success for every one unit increase in x.

Notation

TermDescription
the estimated probability of a success for the ith row in the data
the estimated intercept coefficient
the estimated coefficient for predictor x
the data point for the ith row

Variance-covariance matrix

A d x d matrix, where d is the number of predictors plus one. The variance of each coefficient is in the diagonal cell and the covariance of each pair of coefficients is in the appropriate off-diagonal cell. The variance is the standard error of the coefficient squared.

The variance-covariance matrix is from the final iteration of the inverse of the information matrix. The variance-covariance matrix has the following form:

W is a diagonal matrix where the diagonal elements are given by the following formula:

where

This variance-covariance matrix is based on the observed Hessian matrix as opposed to the Fisher's information matrix. Minitab uses the observed Hessian matrix because the model that results is more robust against any conditional mean misspecification.

If the canonical link is used then the observed Hessian matrix and the Fisher's information matrix are identical.

Notation

TermDescription
yi the response value for the ith row
the estimated mean response for the ith row
V(·)the variance function given in the table below
g(·)the link function
V '(·)the first derivative of the variance function
g'(·)the first derivative of the link function
g''(·)the second derivative of the link function

The variance function depends on the model:

Model Variance function
Binomial
Poisson

See [1] and [2] for more information.

[1] A. Agresti (1990). Categorical Data Analysis. John Wiley & Sons, Inc.

[2] P. McCullagh and J.A. Nelder (1992). Generalized Linear Model. Chapman & Hall.

By using this site you agree to the use of cookies for analytics and personalized content.  Read our policy