Distance and discriminant functions for Discriminant Analysis

Squared distance

Squared Mahalanobis distance - General form

The squared distance (also called the Mahalanobis distance) of observation x to the center (mean) of group t for linear discriminant is given by the following general form:

Squared Mahalanobis distance - Quadratic function

The squared Mahalanobis distance from x to group t for the quadratic discriminant function is calculated as follows:

Generalized squared distance - Linear function

The generalized squared distance from x to group t for the linear discriminant function is calculated as follows:

Generalized squared distance - Quadratic function

The generalized squared distance from x to group t for the quadratic discriminant function is calculated as follows:

Posterior probability

The posterior probability for x belonging to group t is calculated as follows:

Linear discriminant scores

The linear discriminant scores are calculated as follows:

Notation

TermDescription
xcolumn vector of length p containing the values of the predictors for this observation (this column vector is stored as one row)
pnumber of predictors
ntotal number of observations
tgroup subscript
ntnumber of observations in group t
qtthe prior probability of group t , which equals nt/n
Sppooled covariance matrix for linear discriminant analysis
Si covariance matrix of group i for quadratic discriminant analysis
mtcolumn vector of length p containing the means of the predictors calculated from the data in group t
Stcovariance matrix of group t
|St|determinant of St

Linear discriminant function

The linear discriminant function corresponds to the regression coefficients in multiple regression and is calculated as follows:

For a given x, this rule allocates x to the group with largest linear discriminant function.

Notation

TermDescription
xcolumn vector of length p containing the values of the predictors for this observation (this column vector is stored as one row)
micolumn vector of length p containing the means of the predictors calculated from the data in group i
Sppooled covariance matrix
ln pinatural log of the prior probability for group i

Generalized squared distance

The generalized squared distance is used as the quadratic distance measure and is calculated as follows:

Notation

TermDescription
xcolumn vector of length p containing the values of the predictors for this observation (this column vector is stored as one row)
micolumn vector of length p containing the means of the predictors calculated from the data in group i
Sppooled covariance matrix f
ln pinatural log of the prior probability for group i

Posterior probability

The posterior probability is the probability of group i given the data and is calculated as follows:

The largest posterior probability is equivalent to the largest value of ln [pi fi (x)]

where (if the distribution is normal):
and

Notation

TermDescription
piprior probability of group i
fi(x)the joint density for the data in group i (with the population parameters replaced by the sample estimates)
By using this site you agree to the use of cookies for analytics and personalized content.  Read our policy