kth term. Each term can be a single predictor, a polynomial term, or an interaction term.
estimate of kth regression coefficient
Standard error of fitted value (SE Fit)
The standard error of the fitted value in a regression model with one
The standard error of the fitted value in a regression model with more than
one predictor is:
For weighted regression, include the weight matrix in the equation:
When the data have a test data set or K-fold cross validation, the formulas
are the same. The value of
s2 is from the training data. The design matrix and the
weight matrix are also from the training data.
value of the predictor
mean of the predictor
ith predictor value
vector of values
that produce the fitted values, one for each column in the design matrix,
beginning with a 1 for the constant term
transpose of the new vector of predictor
Confidence interval for a fitted value (CI)
For regression, the following formula gives the confidence bounds for a fitted value:
For weighted regression, the formula includes the weights:
where tv is the 1–α/2 quantile of the t distribution with v degrees of freedom for a two-sided interval. For a 1-sided bound, tv is the 1–α quantile of the t distribution with v degrees of freedom.
When you use a test data set or k-fold cross-validation, the degrees of freedom and the mean square error are from the training data set.
When you use a Box-Cox transformation, apply the inverse transformation to the confidence interval formula to find the bounds in the units of the original response. For example, if the Box-Cox transformation is the natural log, then the following formula gives the inverse transformation:
quantile from the t distribution
degrees of freedom
mean square error
leverage for the ith observation
weight for the ith observation
The residual is the difference between an observed value and the corresponding fitted value. This part of the observation is not explained by the model. The residual of an observation is:
ith observed response value
ith fitted value for the response
Standardized residual (Std Resid)
Standardized residuals are also called "internally Studentized residuals."
ith diagonal element of X(X'X)–1X'
mean square error
transpose of the design matrix
Standardized residual (Std Resid) with validation
For validation data, the denominator of the formula for the standardized residual adds the leverage instead of subtracting the leverage.
For weighted regression, the formula includes the weight:
ith residual in the validation data set
leverage for the ith validation row
mean square error for the training data set
weight for the ith observation in the validation data set
Deleted (Studentized) residuals
Also called the externally Studentized residuals. The formula is:
Another presentation of this formula is:
The model that estimates the ith observation omits the ith observation from the data set. Therefore, the ith observation cannot influence the estimate. Each deleted residual has a student's t-distribution with degrees of freedom.
mean square error calculated without the ith observation