Distance and discriminant functions for Discriminant Analysis

In This Topic

Squared distance
Linear discriminant function
Generalized squared distance
Posterior probability

Squared distance

Squared Mahalanobis distance - General form

The squared distance (also called the Mahalanobis distance) of observation x to the center (mean) of group t for linear discriminant is given by the following general form:

Squared Mahalanobis distance - Quadratic function

The squared Mahalanobis distance from x to group t for the quadratic discriminant function is calculated as follows:

Generalized squared distance - Linear function

The generalized squared distance from x to group t for the linear discriminant function is calculated as follows:

Generalized squared distance - Quadratic function

The generalized squared distance from x to group t for the quadratic discriminant function is calculated as follows:

Posterior probability

The posterior probability for x belonging to group t is calculated as follows:

Linear discriminant scores

The linear discriminant scores are calculated as follows:

Notation

Term	Description
x	column vector of length p containing the values of the predictors for this observation (this column vector is stored as one row)
p	number of predictors
n	total number of observations
t	group subscript
n_t	number of observations in group t
q_t	the prior probability of group t, which equals n_t/n
S_p	pooled covariance matrix for linear discriminant analysis
S_i	covariance matrix of group i for quadratic discriminant analysis
m_t	column vector of length p containing the means of the predictors calculated from the data in group t
S_t	covariance matrix of group t
\|S_t\|	determinant of S_t

Linear discriminant function

The linear discriminant function corresponds to the regression coefficients in multiple regression and is calculated as follows:

For a given x, this rule allocates x to the group with largest linear discriminant function.

Notation

Term	Description
x	column vector of length p containing the values of the predictors for this observation (this column vector is stored as one row)
m_i	column vector of length p containing the means of the predictors calculated from the data in group i
S_p	pooled covariance matrix
ln p_i	natural log of the prior probability for group i

Generalized squared distance

The generalized squared distance is used as the quadratic distance measure and is calculated as follows:

Notation

Term	Description
x	column vector of length p containing the values of the predictors for this observation (this column vector is stored as one row)
m_i	column vector of length p containing the means of the predictors calculated from the data in group i
S_p	pooled covariance matrix f
ln p_i	natural log of the prior probability for group i

Posterior probability

The posterior probability is the probability of group i given the data and is calculated as follows:

The largest posterior probability is equivalent to the largest value of ln [p_i f_i(x)]

where (if the distribution is normal):

and

Notation

Term	Description
p_i	prior probability of group i
f_i(x)	the joint density for the data in group i (with the population parameters replaced by the sample estimates)