Geographic Analysis with Choropleth Maps and Spatial Hotspots

Most geographic analysis starts with a spreadsheet, not a shapefile: a column of place names next to a column of numbers. Turning that into a map usually means sourcing boundary files, matching names by hand, and dealing with the inevitable "USA" vs. "United States" mismatches. And once the map is built, a color ramp alone can't tell you whether a cluster of dark-colored regions is a real geographic pattern or just noise.

This tutorial covers both problems. We'll use CreateChoroplethMapFromNames to build a map directly from U.S. state names — no shapefile sourcing required — and then ConductSpatialAutocorrelation to test which high-value states form a statistically significant regional hotspot versus an isolated, one-off outlier.

Prerequisites

Python 3.9+
analysistoolbox installed (pip install analysistoolbox)
geopandas, folium, esda, and libpysal installed (pip install geopandas folium esda libpysal)

Step 1: Build the data

Imagine a SaaS company tracking customer churn rate by U.S. state. Most states hover around a baseline, but two things stand out: a regional cluster of four neighboring South-Central states with elevated churn (say, a shared billing-partner outage hit that whole region), and a single, geographically isolated state with high churn from an unrelated local cause (a competitor's regional launch).

python

import pandas as pd
import numpy as np

np.random.seed(42)

us_states = [
    'Alabama', 'Alaska', 'Arizona', 'Arkansas', 'California', 'Colorado',
    'Connecticut', 'Delaware', 'Florida', 'Georgia', 'Hawaii', 'Idaho',
    'Illinois', 'Indiana', 'Iowa', 'Kansas', 'Kentucky', 'Louisiana',
    'Maine', 'Maryland', 'Massachusetts', 'Michigan', 'Minnesota',
    'Mississippi', 'Missouri', 'Montana', 'Nebraska', 'Nevada',
    'New Hampshire', 'New Jersey', 'New Mexico', 'New York',
    'North Carolina', 'North Dakota', 'Ohio', 'Oklahoma', 'Oregon',
    'Pennsylvania', 'Rhode Island', 'South Carolina', 'South Dakota',
    'Tennessee', 'Texas', 'Utah', 'Vermont', 'Virginia', 'Washington',
    'West Virginia', 'Wisconsin', 'Wyoming', 'District of Columbia',
]

churn_df = pd.DataFrame({
    'state': us_states,
    'churn_rate': np.round(np.random.normal(6.0, 1.0, len(us_states)), 2),
})

# Regional hotspot: neighboring South-Central states hit by the same outage
regional_hotspot = ['Texas', 'Oklahoma', 'Arkansas', 'Louisiana']
churn_df.loc[churn_df['state'].isin(regional_hotspot), 'churn_rate'] = np.round(
    np.random.normal(12.5, 0.8, len(regional_hotspot)), 2
)

# Isolated outlier: one state with a local, unrelated cause
churn_df.loc[churn_df['state'] == 'Vermont', 'churn_rate'] = 13.2

print(churn_df.sort_values('churn_rate', ascending=False).head(8))

Step 2: Map it — from names alone

CreateChoroplethMapFromNames resolves the free-text state column to boundary polygons and builds the map for you. With boundary_source='census', it pulls U.S. state boundaries from the Census Bureau's TIGER database via the same machinery as FetchUSShapefile:

python

from analysistoolbox.geospatial_analysis import CreateChoroplethMapFromNames

matched, unmatched = CreateChoroplethMapFromNames(
    churn_df,
    location_column='state',
    value_column='churn_rate',
    geography_level='us_state',
    boundary_source='census',
    map_choropleth=True,
)

print(f"Matched: {len(matched)} states | Unmatched: {len(unmatched)}")

Warning

Always check unmatched, not just matched. Rows that don't clear the fuzzy-match threshold are returned separately with a best_fuzzy_score column rather than silently dropped — but only if you look. A map built from matched alone can quietly omit regions without anyone noticing.

python

if len(unmatched) > 0:
    print(unmatched[['state', 'best_fuzzy_score']])

At this point you have an interactive choropleth. Visually, the South-Central cluster and Vermont both stand out as dark cells. But color intensity alone can't tell you whether the South-Central cluster is a real regional pattern — states influencing or reflecting their neighbors — or whether all five high-churn states are just independently unlucky draws.

Step 3: Get a coordinate for each state

ConductSpatialAutocorrelation works on points, not polygons, so we need one representative coordinate per state. Compute each polygon's centroid — but reproject to an equal-area CRS first, since computing a centroid directly on geographic (longitude/latitude) coordinates distorts the result, especially for large or irregularly shaped states like Texas or Michigan:

python

albers = matched.to_crs('EPSG:5070')  # Equal-area projection for the contiguous US
centroids_geo = albers.geometry.centroid.to_crs('EPSG:4326')

matched['centroid_lon'] = centroids_geo.x
matched['centroid_lat'] = centroids_geo.y

Step 4: Test for statistically significant hotspots

ConductSpatialAutocorrelation computes Local Moran's I for each point against its k nearest neighbors, then classifies each one as a Hotspot (High-High), Coldspot (Low-Low), a spatial outlier, or Not Significant — based on a permutation test, not just eyeballing the color ramp.

python

from analysistoolbox.geospatial_analysis import ConductSpatialAutocorrelation

results = ConductSpatialAutocorrelation(
    matched,
    longitude_column='centroid_lon',
    latitude_column='centroid_lat',
    value_column='churn_rate',
    k=6,
    plot=True,
)

significant = results[results['cluster_category'] != 'Not Significant']
print(significant[['state', 'churn_rate', 'cluster_category']].sort_values('cluster_category'))

Teaching Note

This is the key distinction a choropleth alone can't give you. The four South-Central states should classify as Hotspot (High-High) — each one is high, and surrounded by neighbors that are also high, which is exactly the signature of a shared regional cause worth investigating as a region (a billing partner, a regional competitor, a shared onboarding flow). Vermont, by contrast, should classify as a Spatial Outlier (High-Low) — high on its own, but surrounded by ordinary neighbors. That's a local, state-specific problem, and the fix belongs at the state level, not the regional one. Same color on the map; different diagnosis, different response.

Common mistakes to avoid

Mistake 1: Reading a choropleth's color ramp as a significance test. Choropleth colors are just a linear (or binned) scale from the min to max value in your dataset — they say nothing about whether a pattern could be due to chance. Only the permutation-tested p_value and cluster_category from ConductSpatialAutocorrelation do that.

Mistake 2: Computing centroids on unprojected (EPSG:4326) geometry. Geographic coordinates aren't equal-area, so a centroid computed directly on them can land in a distorted position for large or irregular polygons. Reproject to an equal-area CRS (like EPSG:5070 for the contiguous U.S.) first, then convert the centroid back to longitude/latitude.

Mistake 3: Running nationwide KNN without accounting for Alaska and Hawaii. ConductSpatialAutocorrelation builds its k-nearest-neighbors weights matrix with ordinary (Euclidean) distance on longitude/latitude, not true geographic adjacency. For a contiguous-U.S. regional analysis, drop non-contiguous states first, or they can pull in "nearest neighbors" that are actually thousands of miles away.

Mistake 4: Skipping unmatched. A silently unmapped state isn't just a visual gap — it also means that state's centroid never enters the spatial autocorrelation step, so it can't be flagged as a hotspot, coldspot, or outlier even if it should be.

Full code

python

import pandas as pd
import numpy as np
from analysistoolbox.geospatial_analysis import (
    CreateChoroplethMapFromNames,
    ConductSpatialAutocorrelation,
)

np.random.seed(42)

us_states = [
    'Alabama', 'Alaska', 'Arizona', 'Arkansas', 'California', 'Colorado',
    'Connecticut', 'Delaware', 'Florida', 'Georgia', 'Hawaii', 'Idaho',
    'Illinois', 'Indiana', 'Iowa', 'Kansas', 'Kentucky', 'Louisiana',
    'Maine', 'Maryland', 'Massachusetts', 'Michigan', 'Minnesota',
    'Mississippi', 'Missouri', 'Montana', 'Nebraska', 'Nevada',
    'New Hampshire', 'New Jersey', 'New Mexico', 'New York',
    'North Carolina', 'North Dakota', 'Ohio', 'Oklahoma', 'Oregon',
    'Pennsylvania', 'Rhode Island', 'South Carolina', 'South Dakota',
    'Tennessee', 'Texas', 'Utah', 'Vermont', 'Virginia', 'Washington',
    'West Virginia', 'Wisconsin', 'Wyoming', 'District of Columbia',
]

churn_df = pd.DataFrame({
    'state': us_states,
    'churn_rate': np.round(np.random.normal(6.0, 1.0, len(us_states)), 2),
})
regional_hotspot = ['Texas', 'Oklahoma', 'Arkansas', 'Louisiana']
churn_df.loc[churn_df['state'].isin(regional_hotspot), 'churn_rate'] = np.round(
    np.random.normal(12.5, 0.8, len(regional_hotspot)), 2
)
churn_df.loc[churn_df['state'] == 'Vermont', 'churn_rate'] = 13.2

# 1. Resolve state names to boundaries and map churn rate
matched, unmatched = CreateChoroplethMapFromNames(
    churn_df,
    location_column='state',
    value_column='churn_rate',
    geography_level='us_state',
    boundary_source='census',
)
if len(unmatched) > 0:
    print(unmatched[['state', 'best_fuzzy_score']])

# 2. Get an equal-area centroid per state
albers = matched.to_crs('EPSG:5070')
centroids_geo = albers.geometry.centroid.to_crs('EPSG:4326')
matched['centroid_lon'] = centroids_geo.x
matched['centroid_lat'] = centroids_geo.y

# 3. Test for statistically significant hotspots vs. isolated outliers
results = ConductSpatialAutocorrelation(
    matched,
    longitude_column='centroid_lon',
    latitude_column='centroid_lat',
    value_column='churn_rate',
    k=6,
)

significant = results[results['cluster_category'] != 'Not Significant']
print(significant[['state', 'churn_rate', 'cluster_category']].sort_values('cluster_category'))