Introduction
"What happened to observations that looked like this one before?" is often a more natural predictive question than fitting an equation — and that's exactly what K-Nearest Neighbors answers. CreateKNearestNeighborsModel builds a KNN model end-to-end across regression, classification, and radius-based variants, handling data cleaning, optional (and strongly recommended) feature scaling, a train/test split, and diagnostic plots for test-set performance, permutation feature importance, and training-vs-test comparison.
KNN is a "lazy learner" — it memorizes the entire training dataset rather than learning an explicit mathematical formula. When asked to predict a new observation, it finds the k most similar (nearest) training examples and either averages their outcomes (regression) or takes a majority vote (classification).
This simplicity is both its strength and its limitation. KNN requires no assumptions about the underlying data distribution, making it flexible. But because it computes distances between observations to find neighbors, it is highly sensitive to two things:
-
Feature scale: A variable measured in thousands (e.g., income) will dominate distance calculations over a variable measured in ones (e.g., number of children). Always scale predictor variables before using KNN — this function enables
scale_variables=Trueby default for this reason. -
Number of predictors: As more variables are added, the geometric space becomes increasingly sparse. Every point starts to look equally far from every other point, so "nearest neighbors" become less meaningful. This is known as the "curse of dimensionality." In practice, 4 or fewer well-chosen predictors often outperform a model with many noisy or redundant ones.
Because KNN has no analytical formula (unlike linear regression), feature importance is measured via permutation importance: each predictor is randomly shuffled one at a time, and the drop in model accuracy shows how much the model depended on that feature.
Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
dataframerequired | | — | The input pandas DataFrame containing both predictor and outcome variables. |
outcome_variablerequired | | — | The name of the target column (dependent variable) to be predicted. |
list_of_predictor_variablesrequired | | — | A list of column names (independent variables) used to train the model. |
model_type | | 'regressor' | The KNN variant to use: 'regressor' (continuous outcome), 'classifier' (categorical outcome), 'radius_regressor' (regression using all neighbors within a fixed radius), or 'radius_classifier' (classification using a fixed radius). Defaults to 'regressor'. |
n_neighbors | | 5 | The number of nearest neighbors to use for prediction. Applies to 'regressor' and 'classifier' model types. Defaults to 5. |
weights | | 'uniform' | How neighbor contributions are weighted. 'uniform' gives equal weight to all neighbors; 'distance' gives closer neighbors more influence. Defaults to 'uniform'. |
algorithm | | 'auto' | The algorithm used to compute nearest neighbors. One of 'auto', 'ball_tree', 'kd_tree', or 'brute'. Defaults to 'auto'. |
leaf_size | | 30 | Leaf size passed to BallTree or KDTree, affecting query speed and memory. Defaults to 30. |
p | | 2 | Power parameter for the Minkowski distance metric. p=1 is Manhattan distance; p=2 (default) is Euclidean distance. |
radius | | 1.0 | The size of the neighborhood to search when using radius-based model types. Applies to 'radius_regressor' and 'radius_classifier'. Defaults to 1.0. |
scale_variables | | True | If True, standardizes predictor variables using StandardScaler before modeling. Strongly recommended for KNN because the algorithm is sensitive to feature scale. Defaults to True. |
test_size | | 0.2 | The proportion of the dataset used for testing. Defaults to 0.2. |
random_seed | | 412 | Controls the randomness of the train-test split for reproducibility. Defaults to 412. |
print_model_training_performance | | False | If True, prints evaluation metrics to the console. For regression: MSE and R²; for classification: error rate and classification report. Defaults to False. |
data_source_for_plot | | None | Source citation string displayed in the caption of all generated plots. Defaults to None. |
plot_model_test_performance | | True | Whether to generate a visualization of model performance on the test set (scatterplot for regression, confusion matrix heatmap for classification). Defaults to True. |
dot_fill_color | | '#999999' | Hex color for the scatterplot points in the regression performance plot. Defaults to '#999999'. |
line_color | | None | Hex color for the fitted reference line in the regression performance plot. Defaults to None (matplotlib default). |
heatmap_color_palette | | 'Blues' | Colormap name for the classification confusion matrix heatmap. Defaults to 'Blues'. |
figure_size_for_model_test_performance_plot | | (8, 6) | Dimensions (width, height) for the performance plot. Defaults to (8, 6). |
title_for_model_test_performance_plot | | 'Model Performance' | Title text for the test-set performance plot. |
subtitle_for_model_test_performance_plot | | 'The predicted values vs. the actual values in the test dataset.' | Subtitle text for the test-set performance plot. |
caption_for_model_test_performance_plot | | None | Optional caption text for the test-set performance plot. |
title_y_indent_for_model_test_performance_plot | | 1.09 | Vertical position of the title on the test-set performance plot. Defaults to 1.09. |
subtitle_y_indent_for_model_test_performance_plot | | 1.05 | Vertical position of the subtitle on the test-set performance plot. Defaults to 1.05. |
caption_y_indent_for_model_test_performance_plot | | -0.215 | Vertical position of the caption on the test-set performance plot. Defaults to -0.215. |
x_indent_for_model_test_performance_plot | | -0.115 | Horizontal starting position for text on the test-set performance plot. Defaults to -0.115. |
plot_feature_importance | | True | Whether to generate a horizontal bar chart of permutation feature importance computed on the test set. Defaults to True. |
top_n_to_highlight | | 3 | The number of top features to color differently in the importance chart. Defaults to 3. |
highlight_color | | '#b0170c' | Hex color for the top-N feature bars in the importance chart. Defaults to '#b0170c'. |
fill_transparency | | 0.8 | Alpha transparency for the feature importance bars. Defaults to 0.8. |
figure_size_for_feature_importance_plot | | (8, 6) | Dimensions (width, height) for the importance plot. Defaults to (8, 6). |
title_for_feature_importance_plot | | 'Feature Importance' | Title text for the feature importance plot. |
subtitle_for_feature_importance_plot | | 'Shows how much each predictor affects model accuracy when its values are randomly shuffled.' | Subtitle text for the feature importance plot. |
caption_for_feature_importance_plot | | None | Optional caption text for the feature importance plot. |
title_y_indent_for_feature_importance_plot | | 1.15 | Vertical position of the title on the feature importance plot. Defaults to 1.15. |
subtitle_y_indent_for_feature_importance_plot | | 1.1 | Vertical position of the subtitle on the feature importance plot. Defaults to 1.1. |
caption_y_indent_for_feature_importance_plot | | -0.15 | Vertical position of the caption on the feature importance plot. Defaults to -0.15. |
plot_training_and_test_performance | | True | Whether to generate a bar chart comparing training vs. test performance. Defaults to True. |
training_bar_color | | '#3a86ff' | Hex color for the training bar. Defaults to '#3a86ff'. |
test_bar_color | | '#b0170c' | Hex color for the test bar. Defaults to '#b0170c'. |
figure_size_for_performance_comparison_plot | | (7, 5) | Dimensions (width, height) for the comparison chart. Defaults to (7, 5). |
title_for_performance_comparison_plot | | None | Title text for the comparison chart. Defaults to a metric-appropriate string. |
subtitle_for_performance_comparison_plot | | None | Subtitle text for the comparison chart. Defaults to a metric-appropriate string. |
caption_for_performance_comparison_plot | | None | Optional caption text for the comparison chart. |
title_y_indent_for_performance_comparison_plot | | 1.1 | Vertical position of the title on the comparison chart. Defaults to 1.1. |
subtitle_y_indent_for_performance_comparison_plot | | 1.05 | Vertical position of the subtitle on the comparison chart. Defaults to 1.05. |
caption_y_indent_for_performance_comparison_plot | | -0.15 | Vertical position of the caption on the comparison chart. Defaults to -0.15. |
x_indent_for_performance_comparison_plot | | -0.115 | Horizontal starting position for text on the comparison chart. Defaults to -0.115. |
Returns
The fitted KNN estimator, or a sklearn.pipeline.Pipeline when scale_variables=True (bundling the StandardScaler and the estimator so new observations can be predicted without manual scaling).
Example
from analysistoolbox.predictive_analytics import CreateKNearestNeighborsModel
# Predict housing prices using comparable properties
model = CreateKNearestNeighborsModel(
df,
outcome_variable='SalePrice',
list_of_predictor_variables=['SquareFeet', 'Bedrooms', 'Age']
)