Each deployment has a single champion model that is used to produce the predictions for the production environment. The deployment also allows up to two challenger models that can run the same data and provide performance metrics to your team. If and when a challenger model becomes the best model, you can easily promote it to be the new champion model.
Champion and challenger details
- Champion model
- The champion model is the best predictive model chosen from a group of models that meets your champion criteria. Upon deployment, the engineer selects the initial champion and sets the response variable and initial predictor variables.
- Challenger models
- Challenger models are competing models. Each challenger model must have the same response variable as the champion model. However, challenger models can have different predictors. The challenger model that generates the best results can become the new champion model if you decide to promote it.
- Name
- Displays the name of the variable. The production and baseline data sets must use identical names as the variables in the deployment.
- Type
- Displays the variable type, either continuous or categorical. The variables in the production and baseline data sets must use the same variable type as the model.
- Data Type
- Displays the data type, either text or numeric. Continuous variables will always have a numeric data type and categorical variables can use text or numeric values. The variables in the production and baseline data sets must use the same data type as the model.
- Classes
- Displays the class values of the variable. Only categorical variables have classes. Use the class values to make predictions.
- Models
- Displays which models contain the predictor variables.
Add challenger models to a deployment
The Models tab contains a list of the deployment’s competing models. Use the following steps to add up to two challenger models.
-
Open the Deployments
page.
- Select a deployment name to open the deployment details area.
- Open the Models tab.
- Select Add Model.
- Choose an available model from the repository. Challengers must have the same response variable and cannot be a challenger in any other deployment.
Upload baseline data for a challenger model
-
Open the Deployments
page.
- Select a deployment name to open the deployment details area.
- Open the Models tab.
-
Open the settings of the challenger model
.
- Select Upload baseline data or Replace baseline data.
Evaluate the performance of challenger models
Use the model performance metrics and graphs on the Performance tab to evaluate the champion and challenger models. Only the champion model is live at any given time. All the prediction requests are used by the current champion model in real time. Once a day, the same prediction requests are replayed against the challengers for analytical purposes.
By assessing the challenger models with the same production data as the champion model, you can easily compare the available models in the deployment.
Promote a challenger model
Review the performance metrics to determine whether one of the challenger models has better performance than the champion model. If a challenger model has better performance than the champion model, you can promote it to be the new champion model.
-
Open the Deployments
page.
- Select a deployment name to open the deployment details area.
- Open the Models tab.
-
Open the settings of the challenger model .
- Select Promote to Champion.
