TTestOfTwoMeansFromStats — Analysis Tool Box

Introduction

When two groups' results only exist as summary numbers — a published mean, standard deviation, and sample size for each side of an A/B test or study — you still need to know whether the gap between them is real. TTestOfTwoMeansFromStats runs an independent two-sample t-test directly from those summary statistics, with the option to assume equal variances (Student's t-test) or switch to Welch's correction when the groups' variances or sizes differ, and visualizes both groups' simulated sampling distributions side by side.

Teaching Note

A two-sample t-test from statistics is essential for:

Comparing A/B test results when only aggregate data is available
Benchmarking performance across different regions or departments
Validating experimental outcomes from published research papers
Testing the efficacy of a treatment group relative to a control group
Analyzing variations in service delivery times between two providers
Evaluating differences in customer lifetime value across segments
Assessing the impact of environmental factors on outcome distributions

Parameters

Parameter	Default	Description
`group1_mean`required	—	The mean value calculated from the first sample.
`group1_sd`required	—	The standard deviation calculated from the first sample.
`group1_sample_size`required	—	The number of observations in the first sample.
`group2_mean`required	—	The mean value calculated from the second sample.
`group2_sd`required	—	The standard deviation calculated from the second sample.
`group2_sample_size`required	—	The number of observations in the second sample.
`homogeneity_of_variance`	`True`	If True, assumes equality of variance between the two groups (Student's t-test). If False, performs Welch's t-test for unequal variances. Defaults to True.
`confidence_interval`	`0.95`	The confidence level used to determine statistical significance (e.g., 0.95 for a 5% significance level). Defaults to 0.95.
`alternative_hypothesis`	`'two-sided'`	Defines the alternative hypothesis for the test. Must be one of 'two-sided', 'less', or 'greater'. Defaults to 'two-sided'.
`plot_sample_distributions`	`True`	If True, displays a kernel density estimate of simulated sample means for both groups to visualize the comparison. Defaults to True.
`value_name`	`'Value'`	Descriptive name for the value being measured, used as the x-axis label in the visualization. Defaults to 'Value'.
`color_palette`	`'Set2'`	Seaborn color palette name or list of colors used for the group distributions. Defaults to 'Set2'.
`title_for_plot`	`'T-Test of Two Means'`	Main title text to display at the top of the plot. Defaults to 'T-Test of Two Means'.
`subtitle_for_plot`	`'Shows the distribution of the sample means for each group.'`	Subtitle text to display below the main title.
`caption_for_plot`	`None`	Caption text displayed at the bottom of the plot. Defaults to None.
`data_source_for_plot`	`None`	Optional text identifying the data source, displayed in the caption area. Defaults to None.
`show_y_axis`	`False`	If True, displays the y-axis (density scale) on the distribution plot. Defaults to False.
`title_y_indent`	`1.1`	Vertical position for the main title relative to the axes. Defaults to 1.1.
`subtitle_y_indent`	`1.05`	Vertical position for the subtitle relative to the axes. Defaults to 1.05.
`caption_y_indent`	`-0.15`	Vertical position for the caption relative to the axes. Defaults to -0.15.
`figure_size`	`(8, 6)`	Tuple specifying the (width, height) of the figure in inches. Defaults to (8, 6).

Returns

A scipy.stats.Ttest_indResult named tuple containing the t-statistic and p-value, accessible via result.statistic and result.pvalue.

Example

python

from analysistoolbox.hypothesis_testing import TTestOfTwoMeansFromStats

# Compare average spend between two marketing variants
result = TTestOfTwoMeansFromStats(
    group1_mean=45.2, group1_sd=12.5, group1_sample_size=150,
    group2_mean=48.7, group2_sd=13.2, group2_sample_size=150
)
print(f"P-value: {result.pvalue:.4f}")