1.6 Dimensionality reduction and sampling

Key points

1. Dimensionality reduction means representing data with fewer columns while keeping most of the information. PCA keeps the directions with the most variance. PCA means principal component analysis.

   What NVIDIA says (1)

   “PCA (Principal Component Analysis) is a fundamental dimensionality reduction technique used to combine features in X in linear combinations such that each new component captures the most”

   — cuML API: PCA

2. UMAP builds a nearest-neighbor graph and embeds the data into fewer dimensions. It keeps local neighborhoods, which makes clusters visible. UMAP means Uniform Manifold Approximation and Projection.

   What NVIDIA says (2)

   “UMAP is a popular dimension reduction algorithm used in fields like bioinformatics, NLP topic modeling, and ML preprocessing.”

   — Even Faster and More Scalable UMAP on the GPU with RAPIDS cuML

   “It works by creating a k-nearest neighbors (k-NN) graph, which is known in literature as an all-neighbors graph, to build a fuzzy topological representation of the data, which is used to embed high-dimensional data into lower dimensions.”

   — Even Faster and More Scalable UMAP on the GPU with RAPIDS cuML

3. Sampling picks a subset of rows to work with. A fixed random_state gives the same sample every time.

   What NVIDIA says (2)

   “This function will always produce the same sample given an identical random_state .”

   — cuDF API: DataFrame.sample

   “frac float, optional Fraction of axis items to return.”

   — cuDF API: DataFrame.sample

4. Random projection multiplies data by a random matrix to get fewer dimensions. The Johnson-Lindenstrauss lemma bounds how many dimensions keep distances roughly intact. PCA means principal component analysis.

   What NVIDIA says (2)

   “Reduce dimensionality through Gaussian random projection.”

   — cuML API: GaussianRandomProjection

   “n_components can be automatically adjusted according to the number of samples in the dataset and the bound given by the Johnson-Lindenstrauss lemma.”

   — cuML API: GaussianRandomProjection

Key terms

• Dimensionality reduction: Representing data with fewer columns while keeping most of its structure.
• Principal component analysis: A method that combines features into a few linear components that keep the most variance.
• UMAP: A non-linear method that maps high-dimensional data to two or three dimensions for plotting.
• Sampling: Taking a random subset of rows, for example for quick exploration.

Sample question

What does PCA do?