Goodness-of-fit tests for Analyze Binary Response for Response Surface Design

Find definitions and interpretation guidance for every statistic in the Goodness-of-fit tests table.

Deviance Goodness-of-Fit Test

The deviance goodness-of-fit test assesses the discrepancy between the current model and the full model.

Interpretation

Use the goodness-of-fit tests to determine whether the predicted probabilities deviate from the observed probabilities in a way that the binomial distribution does not predict. If the p-value for the goodness-of-fit test is lower than your chosen significance level, the predicted probabilities deviate from the observed probabilities in a way that the binomial distribution does not predict. This list provides common reasons for the deviation:
  • Incorrect link function
  • Omitted higher-order term for variables in the model
  • Omitted predictor that is not in the model
  • Overdispersion

If the deviation is statistically significant, you can try a different link function or change the terms in the model.

Many of the goodness-of-fit statistics are affected by how the data are arranged in the worksheet and whether there is one trial per row or multiple trials per row. The p-value for the deviance test tends to be lower for data that are have a single trial per row arrangement compared to data that have multiple trials per row, and generally decreases as the number of trials per row decreases.

The Hosmer-Lemeshow test does not depend on the format of the data. When the data have few trials per row, the Hosmer-Lemeshow test is a more trustworthy indicator of how well the model fits the data. For more information, go to How data formats affect goodness-of-fit in binary logistic regression.

Pearson Goodness-of-Fit Test

The Pearson goodness-of-fit test assesses the discrepancy between the current model and the full model.

Interpretation

Use the goodness-of-fit tests to determine whether the predicted probabilities deviate from the observed probabilities in a way that the binomial distribution does not predict. If the p-value for the goodness-of-fit test is lower than your chosen significance level, the predicted probabilities deviate from the observed probabilities in a way that the binomial distribution does not predict. This list provides common reasons for the deviation:
  • Incorrect link function
  • Omitted higher-order term for variables in the model
  • Omitted predictor that is not in the model
  • Overdispersion

If the deviation is statistically significant, you can try a different link function or change the terms in the model.

Many of the goodness-of-fit statistics are affected by how the data are arranged in the worksheet and whether there is one trial per row or multiple trials per row. The approximation to the chi-square distribution that the Pearson test uses is inaccurate when the expected number of events per row in the data is small. Thus, the Pearson goodness-of-fit test is inaccurate when the data are in the single trial per row format.

The Hosmer-Lemeshow test does not depend on the format of the data. When the data have few trials per row, the Hosmer-Lemeshow test is a more trustworthy indicator of how well the model fits the data. For more information, go to How data formats affect goodness-of-fit in binary logistic regression.

Hosmer-Lemeshow

The Hosmer-Lemeshow goodness-of-fit test compares the observed and expected frequencies of events and non-events to assess how well the model fits the data.

Interpretation

Use the goodness-of-fit tests to determine whether the predicted probabilities deviate from the observed probabilities in a way that the binomial distribution does not predict. If the p-value for the goodness-of-fit test is lower than your chosen significance level, the predicted probabilities deviate from the observed probabilities in a way that the binomial distribution does not predict. This list provides common reasons for the deviation:
  • Incorrect link function
  • Omitted higher-order term for variables in the model
  • Omitted predictor that is not in the model
  • Overdispersion

If the deviation is statistically significant, you can try a different link function or change the terms in the model.

The Hosmer-Lemeshow test does not depend on the number of trials per row in the data as the other goodness-of-fit tests do. When the data have few trials per row, the Hosmer-Lemeshow test is a more trustworthy indicator of how well the model fits the data.

Observed and expected frequencies for Hosmer-Lemeshow test

The model predicts the expected frequencies for the Hosmer-Lemeshow test.

Interpretation

Use the observed and expected frequencies for the Hosmer-Lemeshow test to describe how well the model fits the data or to look for areas of poor fit.

For example, the model with the term X produces goodness-of-fit tests with small p-values, which indicates that the model fits the data poorly. In the table of observed and expected frequencies, the expected values were different by more than 10 events for all of the groups except for group 4, when the probability of the event is between 0.32 and 0.325.

When the model includes X and X*X, the goodness-of-fit tests have large p-values. The data do not provide evidence that the estimated probabilities deviate from the observed probabilities in a way that the binomial distribution does not predict. The largest difference between the observed and expected number of events is in group 4. This difference is approximately 7.

Model with X

Coefficients

TermCoefSE CoefZ-ValueP-ValueVIF
Constant-0.8000.167-4.790.000 
X0.000920.002710.340.7351.00

Goodness-of-Fit Tests

TestDFChi-SquareP-Value
Deviance378.500.000
Pearson374.960.000
Hosmer-Lemeshow374.960.000

Observed and Expected Frequencies for Hosmer-Lemeshow Test


Event
Probability
Range





EventNon-event
GroupObservedExpectedObservedExpected
1(0.000, 0.310)1031.09069.0
2(0.310, 0.315)4031.56068.5
3(0.315, 0.320)6032.04068.0
4(0.320, 0.325)3532.56567.5
5(0.325, 0.330)1533.08567.0

Model with X and X*X

Coefficients

TermCoefSE CoefZ-ValueP-ValueVIF
Constant-2.1070.282-7.460.000 
X0.09040.01217.460.00011.97
X*X-0.0008890.000115-7.750.00011.97

Goodness-of-Fit Tests

TestDFChi-SquareP-Value
Deviance23.780.151
Pearson23.760.152
Hosmer-Lemeshow33.760.288

Observed and Expected Frequencies for Hosmer-Lemeshow Test


Event
Probability
Range





EventNon-event
GroupObservedExpectedObservedExpected
1(0.000, 0.108)1010.89089.2
2(0.108, 0.124)1512.48587.6
3(0.124, 0.401)4040.16059.9
4(0.401, 0.419)3541.96558.1
5(0.419, 0.548)6054.84045.2