Introduction
Before reaching for a heavier classification model, it's worth knowing how far a fast, interpretable probabilistic baseline gets you — and whether the added complexity is actually earning its keep. CreateBayesClassifier trains a Naive Bayes model end-to-end across five scikit-learn variants (Gaussian, multinomial, Bernoulli, complement, and categorical), reports error rates, and produces three diagnostic plots — a normalized confusion matrix, feature discriminability scores, and a training-vs-test error comparison — so you can judge fit quality and overfitting risk in one pass.
Teaching Note
The "naïve" in Naive Bayes refers to the assumption that all predictor variables are conditionally independent given the class label. In other words, knowing the value of one feature tells you nothing additional about any other feature, once you know the class. This assumption almost never holds exactly in practice — yet Naive Bayes classifiers are remarkably robust to its violation.
The key insight is that even when features are correlated, the model can still rank classes correctly (produce the right argmax prediction) as long as the conditional probability estimates are in the right relative order. Errors in the probability magnitudes cancel out across features more often than they accumulate.
Naive Bayes is a useful first model to reach for because:
- Speed: Training is a single pass over the data to compute class priors and per-feature conditional statistics. Scoring is O(n_features).
- Data efficiency: It performs well with small datasets where complex models would overfit.
- Interpretability: The class-conditional means (GaussianNB) or log-probability tables (MultinomialNB) are directly inspectable.
- Calibrated probabilities: When assumptions are met, the posterior probabilities are well-calibrated, which is useful for decision-making under uncertainty.
- Baseline value: A Naive Bayes model that outperforms a more complex one is a strong signal to investigate whether feature engineering can replace model complexity.
Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
dataframerequired | | — | The input pandas DataFrame containing training and testing data. |
outcome_variablerequired | | — | The name of the target column to be predicted (must be categorical). |
list_of_predictor_variablesrequired | | — | A list of column names used as features for the model. |
naive_bayes_type | | 'gaussian' | Which Naive Bayes variant to use. One of 'gaussian', 'multinomial', 'bernoulli', 'complement', or 'categorical'. Defaults to 'gaussian'. |
alpha | | 1.0 | Laplace/Lidstone smoothing parameter for 'multinomial', 'bernoulli', 'complement', and 'categorical' variants. Has no effect for 'gaussian'. Defaults to 1.0. |
var_smoothing | | 1e-09 | Portion of the largest variance across all features added to variances for numerical stability in 'gaussian' mode. Has no effect for other variants. Defaults to 1e-9. |
test_size | | 0.2 | The proportion of the dataset reserved for testing. Defaults to 0.2. |
random_seed | | 412 | Random state for reproducible train/test splits. Defaults to 412. |
filter_nulls | | False | If True, drops all rows containing any NaN values across the selected columns before splitting. Defaults to False. |
print_model_training_performance | | False | If True, prints Training Error Rate, Test Error Rate, and a full Classification Report to the console. Defaults to False. |
data_source_for_plot | | None | Optional source citation appended to every plot caption. Defaults to None. |
plot_model_test_performance | | True | If True, renders a normalized confusion matrix heatmap on the test set. Defaults to True. |
heatmap_color_palette | | 'Blues' | A seaborn/matplotlib colormap name for the confusion matrix. Defaults to 'Blues'. |
figure_size_for_model_test_performance_plot | | (8, 6) | Dimensions (width, height) for the confusion matrix plot. Defaults to (8, 6). |
title_for_model_test_performance_plot | | 'Model Performance' | Title text for the confusion matrix plot. |
subtitle_for_model_test_performance_plot | | 'The predicted values vs. the actual values in the test dataset.' | Subtitle text for the confusion matrix plot. |
caption_for_model_test_performance_plot | | None | Optional caption text for the confusion matrix plot. |
title_y_indent_for_model_test_performance_plot | | 1.09 | Vertical position of the title in axes-fraction coordinates. Defaults to 1.09. |
subtitle_y_indent_for_model_test_performance_plot | | 1.05 | Vertical position of the subtitle in axes-fraction coordinates. Defaults to 1.05. |
caption_y_indent_for_model_test_performance_plot | | -0.215 | Vertical position of the caption in axes-fraction coordinates. Defaults to -0.215. |
x_indent_for_model_test_performance_plot | | -0.115 | Horizontal starting position for the confusion matrix plot text. Defaults to -0.115. |
plot_feature_importance | | True | If True, renders a horizontal bar chart of feature discriminability scores. For 'gaussian', this is the standard deviation of class-conditional means across classes. For all other variants, this is the peak-to-peak range of log-probabilities across classes. Defaults to True. |
top_n_to_highlight | | 3 | Number of top features to emphasize in highlight_color. Defaults to 3. |
highlight_color | | '#b0170c' | Hex color for the top-N feature bars. Defaults to '#b0170c'. |
fill_transparency | | 0.8 | Alpha transparency for all feature bars. Defaults to 0.8. |
figure_size_for_feature_importance_plot | | (8, 6) | Dimensions (width, height) for the discriminability plot. Defaults to (8, 6). |
title_for_feature_importance_plot | | 'Feature Discriminability' | Title text for the discriminability plot. |
subtitle_for_feature_importance_plot | | 'Shows how useful each feature is for distinguishing between classes.' | Subtitle text for the discriminability plot. |
caption_for_feature_importance_plot | | None | Optional caption text for the discriminability plot. |
title_y_indent_for_feature_importance_plot | | 1.15 | Vertical position of the title. Defaults to 1.15. |
subtitle_y_indent_for_feature_importance_plot | | 1.1 | Vertical position of the subtitle. Defaults to 1.1. |
caption_y_indent_for_feature_importance_plot | | -0.15 | Vertical position of the caption. Defaults to -0.15. |
plot_training_and_test_performance | | True | If True, renders a bar chart comparing training vs. test error rates. Defaults to True. |
training_bar_color | | '#3a86ff' | Hex color for the training bar. Defaults to '#3a86ff'. |
test_bar_color | | '#b0170c' | Hex color for the test bar. Defaults to '#b0170c'. |
figure_size_for_performance_comparison_plot | | (7, 5) | Dimensions (width, height) for the comparison chart. Defaults to (7, 5). |
title_for_performance_comparison_plot | | None | Title text for the comparison chart. Defaults to 'Training vs. Test Error Rate'. |
subtitle_for_performance_comparison_plot | | None | Subtitle text for the comparison chart. |
caption_for_performance_comparison_plot | | None | Optional caption text for the comparison chart. |
title_y_indent_for_performance_comparison_plot | | 1.1 | Vertical position of the title. Defaults to 1.10. |
subtitle_y_indent_for_performance_comparison_plot | | 1.05 | Vertical position of the subtitle. Defaults to 1.05. |
caption_y_indent_for_performance_comparison_plot | | -0.15 | Vertical position of the caption. Defaults to -0.15. |
x_indent_for_performance_comparison_plot | | -0.115 | Horizontal starting position for comparison chart text. Defaults to -0.115. |
Returns
The fitted scikit-learn Naive Bayes model object (GaussianNB, MultinomialNB, BernoulliNB, ComplementNB, or CategoricalNB, depending on naive_bayes_type).
Example
python
from analysistoolbox.predictive_analytics import CreateBayesClassifier
from sklearn.datasets import load_iris
import pandas as pd
# Classify iris species using GaussianNB (continuous features)
iris = load_iris()
df = pd.DataFrame(iris.data, columns=iris.feature_names)
df['species'] = iris.target
model = CreateBayesClassifier(
df,
outcome_variable='species',
list_of_predictor_variables=iris.feature_names.tolist()
)