ImputeMissingValuesUsingNearestNeighbors

Introduction

Mean or median imputation fills every gap with the same number, ignoring how a missing value relates to everything else known about that row. This function answers "what value would this missing observation most plausibly have, given the other records most similar to it?" by using K-Nearest Neighbors to average the value from the most similar rows, preserving the relationships between variables that simple imputation destroys. Reach for it when you're preparing multivariate data for a model that can't handle missing values, and the missingness is likely correlated with other features rather than purely random.

Teaching Note

The function is particularly useful for:

Preparing datasets for machine learning models that do not support missing values
Recovering missing information in multivariate datasets (e.g., sensor data)
Handling non-ignorable missing values where the missingness is correlated with other features
Scientific research where data loss occurs sporadically across samples
Financial modeling where specific indicators might be missing for certain periods
Pre-processing clinical data where laboratory results may be incomplete

Parameters

Parameter	Default	Description
`dataframe`required	—	The pandas DataFrame containing the numeric records to be imputed.
`list_of_numeric_columns_to_impute`required	—	A list of column names containing numeric data with missing values. Similarity is calculated based on the values across all columns in this list.
`number_of_neighbors`	`3`	The number of nearby samples to use for calculating the imputed value. A larger 'k' provides a more global average; a smaller 'k' is more sensitive to local patterns.
`averaging_method`	`'uniform'`	The weight function used in prediction: 'uniform' (all neighbors weighted equally) or 'distance' (closer neighbors weighted more heavily).

Returns

The original DataFrame with additional columns added for each variable in list_of_numeric_columns_to_impute, labeled with the " - Imputed" suffix.

Example

python

from analysistoolbox.data_processing import ImputeMissingValuesUsingNearestNeighbors
import pandas as pd
import numpy as np

# Impute missing physiological data for patients
health_data = pd.DataFrame({
    'PatientID': [1, 2, 3, 4, 5],
    'Height_cm': [175, 160, 180, 165, 170],
    'Weight_kg': [80, 55, np.nan, 60, 72],
    'BMI': [26.1, 21.5, 27.2, np.nan, 24.9]
})
imputed_data = ImputeMissingValuesUsingNearestNeighbors(
    health_data,
    ['Height_cm', 'Weight_kg', 'BMI'],
    number_of_neighbors=2
)