Run Tune Hyperparameters to Identify a Better Model button after the Model Summary table. . Click the
This command is available with the Predictive Analytics Module. Click here for more information about how to activate the module.
The performance of TreeNet® models is generally sensitive to values of the learning rate, the subsample fraction, and the complexity of the individual trees that form the model. In the results for a model, click Tune Hyperparameters to Identify a Better Model to evaluate multiple values of these hyperparameters to learn which combination produces the best values of an accuracy criterion, such as the average –loglikelihood. Better values of these hyperparameters have the potential to significantly improve prediction accuracy, so the exploration of different values is a common step in the analysis.
You can also adjust the number of trees that the model includes. In general, 300 trees is enough to distinguish values of the hyperparameters. Generally, you increase the number of trees when the optimal number of trees for one or more models of interest is close to the maximum number of trees. If the number of trees is closer to the maximum number, an increase in the number of trees is more likely to improve the performance of the model.
Specify one or more values for each hyperparameter to evaluate. The analysis evaluates the hyperparameters to find the combination with the best value of the accuracy criterion. If you enter no values for a hyperparameter, the evaluation uses the value for that hyperparameter from the model in the results. If the response is binary and the original model specifies the proportion of events and nonevents to sample, the evaluation always uses the proportions from the original model.
Enter up to 10 values. Eligible values are from 0.0001 to 1.
Enter up to 10 values. Eligible values are greater than 0 and less than or equal to 1.
Subsample fraction is disabled when the original model specifies the proportion of events and nonevents to sample for a binary response.
Enter a value between 1 and 5000 to specify the maximum number of trees to build. The default value of 300 usually provides useful results for the evaluation of the hyperparameter values.
If one or more models of interest have a number of trees that is close to the number of trees that you specify, then consider whether to increase the number of trees. If the number of trees is closer to the maximum number, an increase in the number of trees is more likely to improve the performance of the model.
In this example, the analysis that does not evaluate the complete set of parameter combinations includes 8 models in the evaluation table. An analysis of all the parameter combinations has 3 × 3 × 2 = 18 combinations and takes longer to calculate.
After you specify the values to examine, click Display Results. In a new set of results, Minitab produces a table that compares the accuracy criterion for the hyperparameter combinations and the results for the model with the best value of the accuracy criterion.
Minitab recreates the same tables and graphs for the new model as for the original model. The tables and graphs for the new model are in a new set of results. Storage is the same as for the original analysis. The storage columns are in the same worksheet. For example, if the original analysis stored the fitted values in a column titled "Fit," then the new analysis titles an empty column "Fit_1" and stores the fitted values.