Optimization of hyperparameters for Fit Model and Discover Key Predictors with TreeNet^® Regression

The optimal number of trees usually differs at each step. When the optimal number is close to the maximum number of trees for the analysis, the model is more likely to improve if you increase the number of trees than a model with an optimal number of trees that is far from the maximum. You can consider whether to further explore an alternative model that seems likely to improve.

R² is the percentage of variation in the response that the model explains. Outliers have a greater effect on R² than on MAD.

When you use the squared error loss function or the Huber loss function, then the table includes the R² value for each model. The results that follow are for the model with the highest R² value.

The mean absolute deviation (MAD) is the average of the absolute value of the difference between a predicted value an actual value. The smaller the MAD, the better the model fits the data. The MAD expresses accuracy in the same units as the data, which helps conceptualize the amount of error. Outliers have less of an effect on MAD than on R².

When you use the absolute deviation loss function, then the table includes the MAD value for each model. The full results that follow the table are for the model with the least MAD value.

Low learning rates weigh each new tree in the model less than higher learning rates and sometimes produce more trees for the model. A model with a low learning rate has less chance of overfitting the training data set. Models with low learning rates generally use more trees to find the optimal number of trees.

The subsample fraction is the proportion of the data that the analysis uses to build each tree.

TreeNet^® Regression combines many small CART® trees into a powerful model. You can specify either the maximum number of terminal nodes or the maximum tree depth for these smaller CART® trees. Trees with more terminal nodes can model more complex interactions. In general, values above 12 could slow the analysis without much benefit to the model.

TreeNet^® Regression combines many small CART® trees into a powerful model. You can specify either the maximum number of terminal nodes or the maximum tree depth for these smaller CART® trees. Deeper trees can model more complex interactions. Values from 4 to 6 are adequate for many datasets.