Model summary for CART^® Regression

R² is the percentage of variation in the response that the model explains. Outliers have a greater effect on R² than on MAD and MAPE.

When you use a validation method, the table includes an R² statistic for the training data set and an R² statistic for the test data set. When the validation method is k-fold cross-validation, the test data set is each fold when the tree building excludes that fold. The test R² statistic is typically a better measure of how the model works for new data.

Interpretation

Use R² to determine how well the model fits your data. The higher the R² value, the better the model fits your data. R² is always between 0% and 100%.

You can graphically illustrate the meaning of different R² values. The first plot illustrates a simple regression model that explains 85.5% of the variation in the response. The second plot illustrates a model that explains 22.6% of the variation in the response. The more variation that is explained by the model, the closer the data points fall to the fitted values. Theoretically, if a model can explain 100% of the variation, the fitted values would always equal the observed values and all of the data points would fall on the line y = x.

A test R² that is substantially less than the training R² indicates that the tree might not predict the response values for new cases as well as the tree fits the current data set.

The root mean square error (RMSE) measures the accuracy of the tree. Outliers have a greater effect on RMSE than on MAD and MAPE.

When you use a validation method, the table includes an RMSE statistic for the training data set and an RMSE statistic for the test data set. When the validation method is k-fold cross-validation, the test data set is each fold when the tree building excludes that fold. The test RMSE statistic is typically a better measure of how the model works for new data.

The mean square error (MSE) measures the accuracy of the tree. Outliers have a greater effect on MSE than on MAD and MAPE.

When you use a validation method, the table includes an MSE statistic for the training data set and an MSE statistic for the test data set. When the validation method is k-fold cross-validation, the test data set is each fold when the tree building excludes that fold. The test MSE statistic is typically a better measure of how the model works for new data.

The mean absolute deviation (MAD) expresses accuracy in the same units as the data, which helps conceptualize the amount of error. Outliers have less of an effect on MAD than on R², RMSE, and MSE.

When you use a validation method, the table includes an MAD statistic for the training data set and an MAD statistic for the test data set. When the validation method is k-fold cross-validation, the test data set is each fold when the tree building excludes that fold. The test MAD statistic is typically a better measure of how the model works for new data.

The mean absolute percent error (MAPE) expresses accuracy as a percentage of the error. Because the MAPE is a percentage, it can be easier to understand than the other accuracy measure statistics. For example, if the MAPE, on average, is 0.05, then the average ratio between the fitted error and the actual value across all cases is 5%. Outliers have less of an effect on MAPE than on R², RMSE, and MSE.

However, sometimes you may see a very large MAPE value even though the tree appears to fit the data well. Examine the fitted vs actual response value plot to see if any data values are close to 0. Because MAPE divides the absolute error by the actual data, values close to 0 can greatly inflate the MAPE.

When you use a validation method, the table includes an MAPE statistic for the training data set and an MAPE statistic for the test data set. When the validation method is k-fold cross-validation, the test data set is each fold when the tree building excludes that fold. The test MAPE statistic is typically a better measure of how the model works for new data.