What is a generalized linear model?

Both generalized linear models and least squares regression investigate the relationship between a response variable and one or more predictors. A practical difference between them is that generalized linear model techniques are usually used with categorical response variables. Least squares regression is usually used with continuous response variables. For a thorough description of generalized linear models, see ¹

Both generalized linear model techniques and least squares regression techniques estimate parameters in the model so that the fit of the model is optimized. Least squares minimizes the sum of squared errors to obtain maximum likelihood estimates of the parameters. Generalized linear models obtain maximum likelihood estimates of the parameters using an iterative-reweighted least squares algorithm.

For example, you could use a generalized linear model to study the relationship between machinists' years of experience (a nonnegative continuous variable), and their participation in an optional training program (a binary variable: either yes or no), to predict whether their products meet specifications (a binary variable: either yes or no). The first two variables are the predictors; the third is the categorical response.

Types of logistic regression

Minitab Statistical Software provides four generalized linear model techniques that you can use to assess the relationship between one or more predictor variables and a response variable of the following types. The previous example uses binary logistic regression because the response variable has two levels.

Variable type	Number of categories	Characteristics	Examples
Binary	2	Two levels	Pass/Fail Yes/No High/Low
Ordinal	3 or more	Natural ordering of the levels	Taste (Mild, Medium, Hot) Medical condition (Critical, Serious, Stable, Good) Survey results (Disagree, Neutral, Agree)
Nominal	3 or more	No natural ordering of the levels	Taste (Bitter, Sweet, Sour) Color (Red, Blue, Black) School subject (Math, Science, Art)
Poisson	3 or more	The response variable describes the number of times an event occurs in a finite observation space.	0, 1, 2, ...

Note

For a model that has one continuous predictor and a binary response variable, Minitab provides a fifth technique. A Binary Fitted Line Plot quickly describes the relationship between the predictor and the response.

¹ P. McCullagh and J. A. Nelder (1992). Generalized Linear Models. Chapman & Hall.