This command is available with the Predictive Analytics Module. Click here for more information about how to activate the module.

TreeNet^{®} models are an approach to solving classification and
regression problems that are both more accurate and resistant to overfitting
than a single classification or regression tree. A broad, general description
of the process is that we begin with a small regression tree as an initial
model. From that tree come residuals for every row in the data which become the
response variable for the next regression tree. We build another small
regression tree to predict the residuals from the first tree and compute the
resulting residuals again. We repeat this sequence until an optimal number of
trees with minimum prediction error is identified using a validation method.
The resulting sequence of trees makes the TreeNet^{®} Classification
Model.

For the classification case, we can add some more mathematical detail for an analysis with a binary response and for an analysis with a multinomial response.

The creation of the model uses the following information:

- The response variable, , takes the following values: {-1, 1}.
- The initial fitted values for the calculation of the generalized residuals have the following form:

Where is the number of events and is the number of nonevents.

The model creation also uses the following inputs from the analyst:

Input | Symbol |
---|---|

learn rate | |

sampling rate | |

maximum number of terminal nodes per tree | |

number of trees |

The process has the following general steps for growing the
*j*^{th} tree,
*j*=1,...,J:

- Draw a random sample of size
*s***N*from the training data, where*N*is the number of rows in the training data. - Calculate the generalized
residuals,
*g*, for : where_{i, j}and is a vector that represents the

*i*^{th}row of the predictor values in the training data. - Fit a regression tree with at
most
*M*terminal nodes to the generalized residuals. The tree partitions the observations into at most*M*mutually exclusive groups. - For the
*m*^{th}terminal node in the regression tree, calculate the within-node updates for fitted values from the previous tree as follows:whereTerm Description number of events in terminal node *m*at tree*j*number of cases in terminal node *m*at tree*j*arithmetic mean of for all cases in terminal node *m*at tree*j* - Shrink the within-node updates
by the learning rate and apply the values to get the updated fitted values,
*f*(_{j}*x*):_{i} - Repeat steps 1-5 for each of
the
*J*trees in the analysis.

For a multinomial response with K levels, the analysis fits a tree to each
level of the response variable at each iteration. The initial fitted values for
the calculation of the generalized residuals for one of the trees has the
following form:

where
is the number of cases where the response value is
*k* and
*N* is the number of rows in the training data.

The model creation also uses the following inputs from the analyst:

Input | Symbol |
---|---|

learn rate | |

sampling rate | |

maximum number of terminal nodes per tree | |

number of trees |

The calculation of the probabilities from the fits accounts for the dependent nature of these trees. Otherwise, the process is substantially the same as for the binary case.

- Draw a random sample of size
*s***N*from the training data, where*N*is the number of rows in the training data set. - Calculate the generalized
residuals,
*g*for , , the number of trees in the analysis, and , the number of levels of the response variable:_{i, j, k}where

and is a vector that represents the

*i*^{th}row of the predictor values in the training data set.For example, the probability for an outcome coded as 1 from a multinomial response with 3 levels has the following form:where is the fit for the*i*^{th}row at the*j*–1 tree for the*k*^{th}level of the response variable. - Fit a regression tree with at
most
*M*terminal nodes to the generalized residuals. The tree partitions the observations into at most*M*mutually exclusive groups. - For the
*m*^{th}terminal node in the*j*^{th}regression tree, calculate the within-node updates for fitted values from the previous tree as follows:where

Term Description number of cases for outcome *k*in terminal node*m*at tree*j*number of cases in terminal node *m*at tree*j*arithmetic mean of for all cases in terminal node *m*at tree*j*. - Shrink the within-node updates
by the learning rate and apply the values to get the updated fitted values,
*f*(_{j, k, m}*x*):_{i} - Repeat steps 1-5 for each of
the
*J*trees in the analysis and for each of the*K*levels of the response variable.