CreateLinearRegressionModel — Analysis Tool Box

Introduction

Before reaching for anything more elaborate, most predictive questions about a continuous outcome deserve a linear regression first — it's fast, its coefficients are directly interpretable, and it tells you quickly whether the relationship you're chasing is even roughly linear. CreateLinearRegressionModel fits a scikit-learn multiple linear regression end-to-end: it cleans the data, optionally scales features, can run stepwise variable selection (forward, backward, or mixed), splits train/test, and produces diagnostic plots for predicted-vs-actual performance, feature importance from the beta coefficients, and a training-vs-test MSE comparison to catch overfitting.

Teaching Note

Linear regression is essential for:

Estimating the impact of marketing spend on sales revenue
Analyzing the relationship between macroeconomic indicators and asset prices
Predicting infrastructure or energy demand based on seasonal variables
Assessing the influence of demographic factors on social or health outcomes
Identifying key operational drivers of efficiency in manufacturing processes
Modeling the sensitivity of output variables to changes in input parameters
Establishing baseline predictive models for continuous data analysis

Parameters

Parameter	Default	Description
`dataframe`required	—	The input pandas DataFrame containing both predictor and outcome variables.
`outcome_variable`required	—	The name of the target column (dependent variable) to be predicted.
`list_of_predictor_variables`required	—	A list of column names (independent variables) used to train the model.
`scale_variables`	`False`	If True, scales the predictor variables using StandardScaler prior to modeling. Recommended for variables with vastly different units. Defaults to False.
`test_size`	`0.2`	The proportion of the dataset used for testing. Set to 0 to train on the full dataset. Defaults to 0.2.
`fit_intercept`	`True`	Whether to calculate the intercept for this model. If False, the intercept will be set to 0.0. Defaults to True.
`random_seed`	`412`	Controls the randomness of the train-test split for reproducibility. Defaults to 412.
`variable_selection`	`'none'`	Stepwise variable selection method applied to the training set before fitting the final model. 'none' uses all predictors. 'forward' greedily adds the predictor that most lowers AIC. 'backward' starts with all predictors and iteratively removes the one with the highest p-value until all remaining p-values fall below selection_p_threshold. 'mixed' combines forward addition with backward removal after each step. Defaults to 'none'.
`selection_limit`	`4`	Hard cap on the number of predictors kept by 'forward' or 'mixed' selection. Defaults to 4.
`selection_p_threshold`	`0.05`	P-value threshold used by 'backward' and 'mixed' selection. Predictors with p-value at or above this threshold are candidates for removal. Defaults to 0.05.
`print_peak_to_peak_range_of_each_predictor`	`False`	If True, prints the statistical range of each predictor column to the console. Defaults to False.
`print_model_training_performance`	`False`	If True, prints model accuracy metrics (MSE and R-squared Variance Score). Defaults to False.
`data_source_for_plot`	`None`	Source citation string displayed in the caption of all generated plots. Defaults to None.
`plot_model_test_performance`	`True`	Whether to generate a regression plot showing predicted vs. actual values on the test set. Defaults to True.
`dot_fill_color`	`'#999999'`	Hex color for the scatterplot points in the performance plot. Defaults to '#999999'.
`line_color`	`None`	Hex color for the fitted reference line in the performance plot. Defaults to None (matplotlib default).
`figure_size_for_model_test_performance_plot`	`(8, 6)`	Dimensions (width, height) for the performance visualization. Defaults to (8, 6).
`title_for_model_test_performance_plot`	`'Model Performance'`	Title text for the test-set performance plot.
`subtitle_for_model_test_performance_plot`	`'The predicted values vs. the actual values in the test dataset.'`	Subtitle text for the test-set performance plot.
`caption_for_model_test_performance_plot`	`None`	Optional caption text for the test-set performance plot.
`title_y_indent_for_model_test_performance_plot`	`1.1`	Vertical position of the title on the test-set performance plot. Defaults to 1.1.
`subtitle_y_indent_for_model_test_performance_plot`	`1.05`	Vertical position of the subtitle on the test-set performance plot. Defaults to 1.05.
`caption_y_indent_for_model_test_performance_plot`	`-0.215`	Vertical position of the caption on the test-set performance plot. Defaults to -0.215.
`x_indent_for_model_test_performance_plot`	`-0.115`	Horizontal starting position for text on the test-set performance plot. Defaults to -0.115.
`plot_feature_importance`	`True`	Whether to generate a horizontal bar chart showing the magnitude of calculated beta coefficients. Defaults to True.
`top_n_to_highlight`	`3`	The number of top influential features to color differently in the importance plot. Defaults to 3.
`highlight_color`	`'#b0170c'`	Hex color for the top-N feature bars in the importance plot. Defaults to '#b0170c'.
`fill_transparency`	`0.8`	Alpha transparency for the feature importance bars. Defaults to 0.8.
`figure_size_for_feature_importance_plot`	`(8, 6)`	Dimensions (width, height) for the importance visualization. Defaults to (8, 6).
`title_for_feature_importance_plot`	`'Feature Importance'`	Title text for the feature importance plot.
`subtitle_for_feature_importance_plot`	`'Shows the predictive power of each feature in the model.'`	Subtitle text for the feature importance plot.
`caption_for_feature_importance_plot`	`None`	Optional caption text for the feature importance plot.
`title_y_indent_for_feature_importance_plot`	`1.15`	Vertical position of the title on the feature importance plot. Defaults to 1.15.
`subtitle_y_indent_for_feature_importance_plot`	`1.1`	Vertical position of the subtitle on the feature importance plot. Defaults to 1.1.
`caption_y_indent_for_feature_importance_plot`	`-0.15`	Vertical position of the caption on the feature importance plot. Defaults to -0.15.
`plot_training_and_test_performance`	`True`	Whether to generate a bar chart comparing MSE on the training set versus the test set. A large gap indicates overfitting. Defaults to True.
`training_bar_color`	`'#3a86ff'`	Fill color for the training MSE bar. Defaults to '#3a86ff'.
`test_bar_color`	`'#b0170c'`	Fill color for the test MSE bar. Defaults to '#b0170c'.
`figure_size_for_performance_comparison_plot`	`(7, 5)`	Dimensions (width, height) for the MSE comparison chart. Defaults to (7, 5).
`title_for_performance_comparison_plot`	`'Training vs. Test MSE'`	Title text for the MSE comparison chart.
`subtitle_for_performance_comparison_plot`	`'Compares model error on the training and test datasets.'`	Subtitle text for the MSE comparison chart.
`caption_for_performance_comparison_plot`	`None`	Optional caption text for the MSE comparison chart.
`title_y_indent_for_performance_comparison_plot`	`1.1`	Vertical position of the title on the MSE comparison chart. Defaults to 1.1.
`subtitle_y_indent_for_performance_comparison_plot`	`1.05`	Vertical position of the subtitle on the MSE comparison chart. Defaults to 1.05.
`caption_y_indent_for_performance_comparison_plot`	`-0.15`	Vertical position of the caption on the MSE comparison chart. Defaults to -0.15.
`x_indent_for_performance_comparison_plot`	`-0.115`	Horizontal starting position for text on the MSE comparison chart. Defaults to -0.115.

Returns

The fitted sklearn.linear_model.LinearRegression model, or a sklearn.pipeline.Pipeline when scale_variables=True (bundling the scaler and the regressor).

Example

python

from analysistoolbox.predictive_analytics import CreateLinearRegressionModel

# Basic model to forecast revenue
model = CreateLinearRegressionModel(
    df,
    outcome_variable='Revenue',
    list_of_predictor_variables=['AdSpend', 'Followers', 'Season'],
)

# Scaled model with forward variable selection and performance reporting
model = CreateLinearRegressionModel(
    df,
    outcome_variable='HousePrice',
    list_of_predictor_variables=['SqFt', 'Bedrooms', 'Age', 'LotSize', 'Garage'],
    scale_variables=True,
    variable_selection='forward',
    selection_limit=3,
    print_model_training_performance=True,
)