VerifyGranularity — Analysis Tool Box

Introduction

Before joining or aggregating a table, it pays to confirm an assumption that's easy to get wrong: does this combination of columns actually identify one row per record, the way you think it does? VerifyGranularity answers "is this the grain I believe it is, or are there hidden duplicates?" by building a composite key from the columns you specify and checking whether the row count matches the number of unique keys. Reach for it before any join or groupby where an unexpected duplicate would silently inflate your results.

Teaching Note

The function is particularly useful for:

Validating primary key assumptions in new datasets
Ensuring data integrity before performing table joins
Identifying unexpected duplicates in transactional logs
Documenting the grain/level of analysis for a dataset
Preparing DataFrames for indexing in specialized time-series or panel data
Debugging ETL pipelines where row duplication may have occurred

Parameters

Parameter	Default	Description
`dataframe`required	—	The pandas DataFrame to investigate.
`list_of_key_columns`required	—	A list of column names that are expected to form a unique identifier for each row. Columns will be concatenated with a ' -- ' separator.
`set_key_as_index`	`True`	If True, the calculated composite key will be set as the DataFrame's index, and the temporary 'Dataset Key' column will be removed.
`print_as_markdown`	`True`	If True, results and warnings are formatted using IPython Markdown for rich display in Jupyter Notebooks. If False, standard print statements are used.

Returns

If set_key_as_index is True, returns the DataFrame with the composite key as its index. Otherwise, returns the updated DataFrame (the original may be modified in-place).

Example

python

from analysistoolbox.data_processing import VerifyGranularity
import pandas as pd

# Verify granularity of sales data by Date and StoreID
sales = pd.DataFrame({
    'Date': ['2023-01-01', '2023-01-01', '2023-01-02'],
    'StoreID': [101, 102, 101],
    'Revenue': [5000, 6200, 4800]
})
sales_indexed = VerifyGranularity(
    sales,
    ['Date', 'StoreID'],
    set_key_as_index=True
)