Introduction
When a dataset has a dozen or more numeric columns, it's hard to tell just by scanning tables which observations are actually similar, which form natural groups, and which are outliers. ConductManifoldLearning answers that by applying t-distributed Stochastic Neighbor Embedding (t-SNE) to compress high-dimensional data into two or three dimensions while preserving local neighborhood structure — points that were close together in the original feature space stay close together in the reduced space. The result is a plot where clusters and patterns that were invisible across a dozen columns become visible at a glance.
Manifold learning with t-SNE is essential for:
- Exploratory data analysis and pattern discovery in high-dimensional datasets
- Visualizing complex data structures in 2D or 3D space
- Feature engineering for machine learning pipelines
- Identifying clusters and outliers in multivariate data
- Reducing computational complexity for downstream analysis
- Creating interpretable representations of complex relationships
- Data preprocessing for classification and clustering tasks
Parameters
| Parameter | Type | Default | Description |
|---|---|---|---|
dataframerequired | | — | A pandas DataFrame containing the data to transform. Must include numeric columns for the manifold learning algorithm to process. |
list_of_numeric_columns | | None | List of column names to use for manifold learning. If None, all numeric columns in the DataFrame will be automatically selected. Defaults to None. |
number_of_components | | 3 | Number of dimensions in the reduced manifold space. Common values are 2 or 3 for visualization purposes. Defaults to 3. |
random_seed | | 412 | Random seed for reproducibility of the t-SNE algorithm. Use the same seed to ensure consistent results across multiple runs. Defaults to 412. |
show_component_summary_plots | | True | Whether to display pair plots showing the relationship between original features and the learned manifold components using kernel density estimation. Defaults to True. |
summary_plot_size | | (20, 20) | Figure size for the summary pair plots as a tuple of (width, height) in inches. Defaults to (20, 20). |
Returns
A pandas DataFrame — the original data with additional columns for each manifold learning component, named MLC1, MLC2, ..., MLCn. Rows with missing values in the selected numeric columns are removed.
Example
from analysistoolbox.descriptive_analytics import ConductManifoldLearning
import pandas as pd
# Reduce high-dimensional customer data to 2D for visualization
customer_df = pd.DataFrame({
'age': [25, 34, 45, 23, 56],
'income': [50000, 75000, 90000, 45000, 120000],
'spending_score': [60, 81, 45, 72, 38],
'loyalty_years': [1, 5, 10, 2, 15]
})
result_df = ConductManifoldLearning(
customer_df,
number_of_components=2,
show_component_summary_plots=False
)
# Result includes MLC1, MLC2 columns for clustering analysis