A team of researchers
wants to use data about a borrower and the location of a
property to predict the amount of
a mortgage. Variables include the income, race, and gender of the borrower as well as the census tract location of the property, and other information about the borrower and the type of property.
After initial exploration with CART®
Regression to identify the important predictors, the team now considers TreeNet®
Regression as a necessary follow-up step. The researchers hope to gain more insight into the relationships between the response and the important predictors and predict for new observations with greater accuracy.
These data were adapted based on a public data set containing information
on federal home loan bank mortgages. Original data is from
fhfa.gov.
Choose Predictive Analytics
Module > TreeNet®
Regression > Fit
Model.
In Response, enter Loan Amount.
In Continuous predictors, enter Annual Income – Area Income.
In Categorical predictors, enter First Time Home Buyer – Core Based Statistical Area.
Click Validation.
In Validation
method, select K-fold cross-validation.
In Number of
folds (K), enter 3.
Click
OK in each dialog box.
Interpret the results
For this analysis, Minitab grows 300 trees and the optimal number of trees is 300. Because the optimal number of trees is close to the maximum number of trees that the model grows, the researchers repeat the analysis with more trees.
Example with 500 trees
After the model summary table, click Tune
Hyperparameters to Identify a Better Model.
In Number of
trees, enter 500.
Click
Display
Results.
Interpret the results
For this analysis, there were 500 trees grown and the optimal number of trees for the combination of hyperparameters with the best value of the accuracy criterion is 500. The subsample fraction changes to 0.7 instead of the 0.5 in the original analysis. The learning rate changes to 0.0437 instead of 0.04372 in the original analysis.
Examine both the Model summary table and the R-squared vs Number of Trees Plot. The R2 value when the number of trees is 500 is 86.79% for the test data and is 96.41% for the training data. These results show improvement over a traditional regression analysis and a CART®
Regression.