t-SNE and UMAP

t-SNE (t-distributed Stochastic Neighbor Embedding) and UMAP (Uniform Manifold Approximation and Projection) are dimensionality reduction techniques that take data described by hundreds or thousands of variables and place each point on a two-dimensional map so that similar points land close together. They are the standard way data scientists look at high-dimensional data — especially embeddings from language and vision models — as a picture.^[1][2]

History

t-SNE was introduced by Laurens van der Maaten and Geoffrey Hinton in 2008 as an easier-to-optimise variant of Stochastic Neighbor Embedding, and remains among the most-cited methods for visualising high-dimensional structure.^[1] UMAP followed in 2018 from Leland McInnes, John Healy and James Melville, grounded in Riemannian geometry and algebraic topology — specifically fuzzy simplicial sets — and designed to match t-SNE's visual quality while running faster, scaling to larger datasets and preserving more global structure.^[2] UMAP became a default in bioinformatics after Becht and colleagues showed it could visualise millions of single cells, and both techniques ship inside scikit-learn, so they are a few lines of code away for any Python user.^[3]

Key Concepts and Technology

Both methods convert distances into similarities in the original space, then search for a low-dimensional layout whose similarity structure matches. t-SNE models high-dimensional similarities with a Gaussian whose bandwidth is tuned per point to a fixed perplexity — roughly an effective neighbour count — models the low-dimensional layout with a Student-t kernel, and minimises Kullback-Leibler divergence between the two. The heavy-tailed t-distribution is what fixes the crowding problem earlier methods suffered from.^[1] UMAP instead builds a fuzzy topological representation of the data and optimises a cross-entropy objective with negative sampling, which is where its speed advantage comes from.^[2]

Two properties matter when reading the output. First, both excel at preserving local neighbourhoods: points that were close in the original space stay close on the plot, so groups corresponding to genuine structure — product categories, cell types, fraud rings — become visible. Second, they distort: distances between clusters, cluster diameters and densities are not faithful measurements, and rerunning either method with different perplexity or neighbourhood settings can move points around. The accepted practice is to treat the plot as a qualitative map, then validate findings numerically in the original space.^[4]

Applications and Impact

Typical uses include visualising sentence or image embeddings to audit what a model groups together, exploratory analysis of customer segments, error analysis of classifier outputs, anomaly and fraud detection, and single-cell genomics, where UMAP plots of gene-expression profiles are a standard figure in research papers. Both also serve as a prelude to clustering: practitioners often run t-SNE or UMAP first, inspect the shape, then cluster in the reduced space.^[3][4]

>See Also

• Principal Component Analysis
• Embedding
• K-Means Clustering
• Unsupervised Learning
• Anomaly Detection
• Vector Database
• Explainable AI