Methods and formulas for tune hyperparameters in Fit Model and Discover Key Predictors with TreeNet^® Regression

If you specify values for more than one hyperparameter, then the models in the evaluation table depend on whether you evaluate the complete combinations of the hyperparameters.

• If you select Evaluate complete parameter combinations, then the algorithm evaluates every combination of the hyperparameters. This option generally takes longer to calculate.
• Otherwise, the algorithm evaluates the hyperparameters in this order:
  1. Learning rate
  2. Subsample fraction
  3. Individual tree complexity parameter
  For example, suppose that the algorithm receives the following hyperparameters:

  • Learning rates: 0.001, 0.01, 0.1
  • Subsample fractions: 0.4, 0.5, 0.7
  • Maximum numbers of terminal nodes: 4, 6

  1. The algorithm sets the subsample proportion to 0.4 and the maximum number of terminal nodes to 4. Then, the algorithm evaluates the learning rates in order from least to greatest: 0.001, 0.01, 0.1.
  2. Suppose the algorithm identifies 0.01 as the best learning rate. Then the algorithm sets the learning rate to 0.01 and the maximum number of terminal nodes to 4. Then, the algorithm evaluates the subsample proportions of 0.4, 0.5, and 0.7.
  3. Suppose that the algorithm identifies 0.5 as the best subsample proportion. Then the algorithm sets the learning rate to 0.01, the subsample proportion to 0.5. Then, the algorithm evaluates the maximum numbers of nodes of 4 and 6.
  4. Suppose that the algorithm identifies 6 as the best maximum number of terminal nodes. Then Minitab produces the evaluation table and the results for the model with learning rate = 0.01, subsample proportion 0.5, and maximum number of terminal nodes 6.

In this example, the analysis that does not evaluate the complete set of parameter combinations includes 8 models in the evaluation table. An analysis of all the parameter combinations has 3 × 3 × 2 = 18 combinations and takes longer to calculate.

For details on the calculation of the accuracy criteria for an individual model, go to Methods and formulas for the model summary in Fit Model and Discover Key Predictors with TreeNet® Regression.