✦ScienexySCIENCE KNOWLEDGE LIBRARY
Science reference · SCI-CS-003-088

Dimensionality Reduction

Dimensionality reduction decreases the number of variables or dimensions used to represent data while attempting to preserve important information.

Explanation

Dimensionality reduction is the process of representing data using fewer dimensions than the original feature space. It can be performed through feature selection, which keeps a subset of existing variables, or feature extraction, which creates new variables from the original data. Linear techniques include Principal Component Analysis and Linear Discriminant Analysis, while nonlinear techniques include t-distributed Stochastic Neighbor Embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP). Dimensionality reduction can lower computational requirements, reduce redundancy, support visualization, and sometimes improve generalization by removing noisy information. However, reducing dimensions may also remove useful information. Some methods preserve global structure better than others, while visualization-oriented methods may primarily preserve local neighborhood relationships. The appropriate technique depends on the dataset and the purpose of the analysis.

Example or application

A dataset containing 200 numerical features may be transformed into 20 principal components. The smaller representation can make visualization and some downstream analyses more manageable.

Text

High-dimensional datasets can be simplified by selecting fewer features or transforming them into a smaller representation.

Key Points

  • Dimensionality reduction decreases the number of data dimensions.

  • Feature selection retains a subset of original variables.

  • Feature extraction creates new lower-dimensional representations.

  • PCA is a common linear dimensionality-reduction method.

  • t-SNE and UMAP are often used for nonlinear visualization.

  • Reducing dimensions can also remove useful information.

Applications

  • High-dimensional data visualization

  • Image processing

  • Text analysis

  • Noise reduction

  • Computationally efficient machine learning

Learning Outcomes

  • Define dimensionality reduction.

  • Differentiate feature selection and feature extraction.

  • Identify common linear and nonlinear techniques.

  • Explain the benefits and limitations of reducing dimensions.

Practice Questions

  • What is dimensionality reduction?

  • How does feature selection differ from feature extraction?

  • What is PCA commonly used for?

  • Why should a reduced representation be evaluated before use?

Related Concepts

  • Feature Selection

  • Feature Extraction

  • Principal Component Analysis

  • t-SNE

  • UMAP

Common Mistakes

  • Assuming every dimensionality-reduction method preserves all information.

  • Interpreting t-SNE distances as reliable global distances.

  • Assuming PCA components are directly interpretable.

  • Reducing dimensions without checking downstream performance.

Sources and verification

Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.

  • Jolliffe, I. T., and Cadima, J. (2016). Principal Component Analysis: A Review and Recent Developments.
  • van der Maaten, L., and Hinton, G. (2008). Visualizing Data using t-SNE.

Suggest a correction or source · More in Artificial Intelligence & Machine Learning