Example of Discover Key Predictors for TreeNet® Regression

Note

This command is available with the Predictive Analytics Module. Click here for more information about how to activate the module.

A team of researchers wants to use data from an injection molding process to study settings for machines that maximize one type of strength of a plastic part. The variables include controls on the machines, different plastic formulas, and the injection molding machines.

As part of the initial exploration of the data, the team decides to use Discover Key Predictors to compare models by sequentially removing unimportant predictors to identify key predictors. The researchers hope to identify key predictors that have the most effect on response and gain more insight into the relationships between the response and the key predictors.

  1. Open the sample data set InjectionProcess.MTW.
  2. Choose Predictive Analytics Module > TreeNet® Regression > Discover Key Predictors.
  3. In Response, enter Strength.
  4. In Continuous predictors, enter Injection PressureTemperature at Measurement.
  5. In Categorical predictors, enter Machine and Formula.
  6. Click OK.

Interpret the results

For this analysis, Minitab Statistical Software compares 20 models. The asterisk in the Model column of the Model Evaluation table shows that the model with the greatest value of the cross-validated R2 statistic is model 16. Model 16 contains 5 important predictors. The results that follow the model evaluation table are for model 16.

Although Model 16 has the greatest value of the cross-validated R2 statistic, other models have similar values. The team can click Select Alternative Model to produce results for other models from the Model Evaluation table.

In the results for Model 16, the R-squared vs. Number of Trees Plot shows that the optimal number of trees equals the number of trees in the analysis, 300. The team can click Tune Hyperparameters to increase the number of trees and to see whether changes to other hyperparameters improve the performance of the model.

The Relative Variable Importance graph plots the predictors in order of their effect on model improvement when splits are made on a predictor over the sequence of trees. The most important predictor variable is Mold Temperature. If the importance of the top predictor variable, Mold Temperature, is 100%, then the next important variable, Machine, has a contribution of 58.7%. This means the machine that injects is 58.7% as important as the temperature inside the mold.

Use the partial dependency plots to gain insight into how the important variables or pairs of variables affect the predicted response. The partial dependence plots show whether the relationship between the response and a variable is linear, monotonic, or more complex.

The one predictor partial dependence plots show that mold temperature, injection pressure, and cooling temperature all have a positive relationship with strength. The plot of the machines shows the differences between machines, with machine 1 making the weakest parts on average and machine 4 making the strongest parts on average. The team notices that the mold temperature and the machine have the strongest interaction in the data, so they look at the two-predictor partial dependence plot to further understand how these variables affect strength. The team can select One-Predictor Plots in the results to produce plots for other variables, such as Injection Temperature.

The two-predictor partial dependence plot of Mold Temperature and Machine gives some insight into the difference average strengths for the machines. One reason is that the data from machine 1 do not include as many observations at the highest mold temperatures as the other machines. The team could still decide to look for other reasons that the machines produce different strengths when other settings are the same. The team can click Two-Predictor Plots in the results to produce plots for other pairs of variables.

Method

Loss functionSquared error
Criterion for selecting optimal number of treesMaximum R-squared
Model validation3-fold cross-validation
Learning rate0.01408
Subsample fraction0.5
Maximum terminal nodes per tree6
Minimum terminal node size3
Number of predictors selected for node splittingTotal number of predictors = 21
Rows used1408

Response Information

MeanStDevMinimumQ1MedianQ3Maximum
485.247318.61141.2082301.099398.924562.4492569.04

Model Selection by Eliminating Unimportant Predictors

Test
ModelOptimal
Number
of Trees
R-squared
(%)
Number of
Predictors
Eliminated Predictors
130089.3221None
230089.3419Plastic Flow Rate, Change Position
330089.3918Drying Temperature
430089.4617Melt Temperature Zone 2
530089.5116Plastic Temperature
630089.5015Formula
730089.5914Hold Pressure
830089.5713Screw cushion
930089.6912Melt Temperature Zone 4
1030089.7011Back Pressure
1130089.8610Melt Temperature Zone 1
1230089.909Drying Time
1330089.928Temperature at Measurement
1430090.067Melt Temperature Zone 5
1530090.166Melt Temperature Zone 3
16*30090.235Screw Rotation Speed
1730089.964Injection Temperature
1829779.373Cooling Temperature
1924466.642Injection Pressure
2016446.191Machine
The algorithm removed one predictor and any predictors with 0 importance at each step.
* Selected model has maximum R-squared. Output for the selected model follows.

Model Summary

Total predictors5
Important predictors5
Number of trees grown300
Optimal number of trees300
StatisticsTrainingTest
R-squared92.23%90.23%
Root mean squared error (RMSE)88.804999.5673
Mean squared error (MSE)7886.31529913.6420
Mean absolute deviation (MAD)68.923174.4113
Mean absolute percent error (MAPE)0.20830.2175