CreateBoostedTreeModel — Analysis Tool Box

Introduction

Tabular data with lots of nonlinear interactions between features — the kind that trips up a plain linear model — is exactly where gradient boosted trees earn their reputation. CreateBoostedTreeModel runs the full XGBoost workflow in one call: it cleans nulls, splits train/test, fits an XGBClassifier or XGBRegressor depending on your outcome type, and produces three diagnostic plots — test-set performance, feature importance, and a training-vs-test comparison — so you can judge both accuracy and overfitting risk before trusting the model.

Teaching Note

Building boosted tree models is essential for:

Detecting fraudulent transactions or malicious activity (Classification)
Predicting customer churn or subscriber conversion (Classification)
Forecasting continuous targets like sales revenue or delivery times (Regression)
Identifying the most influential drivers of a specific business outcome
Modeling complex, non-linear interactions in high-dimensional tabular data
Optimizing tactical resource allocation based on predictive risk scoring
Rapidly prototyping machine learning baseline models for structured data

Parameters

Parameter	Default	Description
`dataframe`required	—	The input pandas DataFrame containing the training and testing data.
`outcome_variable`required	—	The name of the target column to be predicted.
`list_of_predictor_variables`required	—	A list of column names used as features for the model.
`maximum_depth`	`None`	Maximum tree depth for base learners. Increasing this value makes the model more complex and likely to overfit. Defaults to None (XGBoost's default depth).
`is_outcome_categorical`	`True`	If True, trains an XGBClassifier. If False, trains an XGBRegressor. Defaults to True.
`test_size`	`0.2`	The proportion of the dataset to include in the test split. Defaults to 0.2.
`random_seed`	`412`	Controls the randomness of the train-test split and the boosting process. Defaults to 412.
`filter_nulls`	`False`	Whether to drop rows containing any NaN values across the selected columns. Defaults to False.
`print_model_training_performance`	`False`	If True, prints standard evaluation metrics (MSE/R2 for regression, Classification Report for classification) to the console. Defaults to False.
`data_source_for_plot`	`None`	Source citation string displayed in the caption of all generated plots. Defaults to None.
`plot_model_test_performance`	`True`	Whether to generate a visualization of model performance on the test set (scatterplot/regplot for regression, heatmap for classification). Defaults to True.
`dot_fill_color`	`'#999999'`	Hex color for the scatterplot points in the regression performance plot. Defaults to '#999999'.
`line_color`	`None`	Hex color for the fitted reference line in the regression performance plot. Defaults to None (matplotlib default).
`heatmap_color_palette`	`'Blues'`	Colormap name for the classification confusion matrix heatmap. Defaults to 'Blues'.
`figure_size_for_model_test_performance_plot`	`(8, 6)`	Dimensions (width, height) for the performance plot. Defaults to (8, 6).
`title_for_model_test_performance_plot`	`'Model Performance'`	Title text for the test-set performance plot.
`subtitle_for_model_test_performance_plot`	`'The predicted values vs. the actual values in the test dataset.'`	Subtitle text for the test-set performance plot.
`caption_for_model_test_performance_plot`	`None`	Optional caption text for the test-set performance plot.
`title_y_indent_for_model_test_performance_plot`	`1.09`	Vertical position of the title on the test-set performance plot. Defaults to 1.09.
`subtitle_y_indent_for_model_test_performance_plot`	`1.05`	Vertical position of the subtitle on the test-set performance plot. Defaults to 1.05.
`caption_y_indent_for_model_test_performance_plot`	`-0.215`	Vertical position of the caption on the test-set performance plot. Defaults to -0.215.
`x_indent_for_model_test_performance_plot`	`-0.115`	Horizontal starting position for text on the test-set performance plot. Defaults to -0.115.
`plot_feature_importance`	`True`	Whether to generate a horizontal bar chart showing the predictive weight of each variable. Defaults to True.
`top_n_to_highlight`	`3`	The number of top features to color differently in the importance plot. Defaults to 3.
`highlight_color`	`'#b0170c'`	Hex color for the top-N feature bars in the importance plot. Defaults to '#b0170c'.
`fill_transparency`	`0.8`	Alpha transparency for the feature importance bars. Defaults to 0.8.
`figure_size_for_feature_importance_plot`	`(8, 6)`	Dimensions (width, height) for the importance plot. Defaults to (8, 6).
`title_for_feature_importance_plot`	`'Feature Importance'`	Title text for the feature importance plot.
`subtitle_for_feature_importance_plot`	`'Shows the predictive power of each feature in the model.'`	Subtitle text for the feature importance plot.
`caption_for_feature_importance_plot`	`None`	Optional caption text for the feature importance plot.
`title_y_indent_for_feature_importance_plot`	`1.15`	Vertical position of the title on the feature importance plot. Defaults to 1.15.
`subtitle_y_indent_for_feature_importance_plot`	`1.1`	Vertical position of the subtitle on the feature importance plot. Defaults to 1.1.
`caption_y_indent_for_feature_importance_plot`	`-0.15`	Vertical position of the caption on the feature importance plot. Defaults to -0.15.
`plot_training_and_test_performance`	`True`	Whether to render a bar chart comparing the training and test metric (MSE for regression, error rate for classification). Defaults to True.
`training_bar_color`	`'#3a86ff'`	Bar color for the training metric bar. Defaults to '#3a86ff'.
`test_bar_color`	`'#b0170c'`	Bar color for the test metric bar. Defaults to '#b0170c'.
`figure_size_for_performance_comparison_plot`	`(7, 5)`	Dimensions (width, height) for the comparison chart. Defaults to (7, 5).
`title_for_performance_comparison_plot`	`None`	Title text for the training/test comparison chart. Defaults to a sensible built-in title when None.
`subtitle_for_performance_comparison_plot`	`None`	Subtitle text for the training/test comparison chart. Defaults to a sensible built-in subtitle when None.
`caption_for_performance_comparison_plot`	`None`	Optional caption text for the training/test comparison chart.
`title_y_indent_for_performance_comparison_plot`	`1.1`	Vertical position of the title on the comparison chart. Defaults to 1.1.
`subtitle_y_indent_for_performance_comparison_plot`	`1.05`	Vertical position of the subtitle on the comparison chart. Defaults to 1.05.
`caption_y_indent_for_performance_comparison_plot`	`-0.15`	Vertical position of the caption on the comparison chart. Defaults to -0.15.
`x_indent_for_performance_comparison_plot`	`-0.115`	Horizontal starting position for text on the comparison chart. Defaults to -0.115.

Returns

The fitted xgboost.XGBModel object — either an XGBClassifier or XGBRegressor, depending on is_outcome_categorical.

Example

python

from analysistoolbox.predictive_analytics import CreateBoostedTreeModel

# Predict customer churn
model = CreateBoostedTreeModel(
    dataframe=churn_df,
    outcome_variable='churned',
    list_of_predictor_variables=[
        'tenure_months', 'monthly_charges', 'contract_type',
        'num_products', 'support_calls_last_90d'
    ],
    is_outcome_categorical=True,
    test_size=0.2,
    maximum_depth=4,   # constrain to reduce overfitting
)

# Score new customers
new_customers['churn_probability'] = model.predict_proba(new_customers[features])[:, 1]