ANALYSIS TOOL BOX

Data Processing

CleanTextColumns

Remove leading and trailing whitespace from all string columns in a DataFrame.

data-processingcleaningtransformationwrangling

Introduction

Whitespace that sneaks in from CSV exports, web forms, or scraped data — a stray leading space, a trailing tab — is invisible in a table view but breaks exact-match joins, group-bys, and deduplication silently. CleanTextColumns answers "are any of my text columns hiding whitespace that will break a join?" by stripping leading and trailing whitespace from every string column in a DataFrame in one call, leaving numeric and datetime columns untouched. Reach for it as a standard first step before merging datasets from different sources or matching on text keys.

Teaching Note

The function is particularly useful for:

  • Data import cleanup from CSV, Excel, or text files with formatting inconsistencies
  • Preprocessing data before joins or merges to avoid whitespace mismatch issues
  • Standardizing user-input data from web forms or surveys
  • Cleaning scraped web data with irregular spacing
  • Preparing data for string matching and deduplication
  • Database migration and ETL processes
  • General data quality improvement

Parameters

ParameterTypeDefaultDescription
dataframerequired—A pandas DataFrame containing one or more columns. String-type columns (object dtype) will be cleaned by removing leading and trailing whitespace; other column types are not modified.

Returns

The input DataFrame with all string columns cleaned — leading and trailing whitespace removed from all object dtype columns. The DataFrame is modified in place and also returned.

Example

python
from analysistoolbox.data_processing import CleanTextColumns
import pandas as pd

# Clean messy imported data with extra spaces
messy_data = pd.DataFrame({
    'name': ['  Alice  ', 'Bob   ', '  Charlie'],
    'city': ['New York  ', '  Boston', '  Seattle  '],
    'age': [25, 30, 35]
})
clean_data = CleanTextColumns(messy_data)
# name column becomes: ['Alice', 'Bob', 'Charlie']
# age column is unchanged (numeric type)