✦ScienexySCIENCE KNOWLEDGE LIBRARY
Science reference · SCI-CS-003-058

Semi-Supervised Learning

Semi-supervised learning combines a relatively small amount of labeled data with a larger amount of unlabeled data to train a machine learning model.

D = DL ∪ DU

Explanation

Semi-Supervised Learning is a machine learning paradigm that uses both labeled and unlabeled training examples. It is particularly useful when obtaining accurate labels requires expert knowledge, expensive experiments, or substantial human effort. A small labeled dataset provides direct information about the desired target, while a larger unlabeled dataset may reveal the structure and distribution of the input data. Common semi-supervised methods include self-training, pseudo-labeling, consistency regularization, and graph-based approaches. In self-training, a model is initially trained on labeled examples and then uses its predictions to assign pseudo-labels to selected unlabeled examples. These pseudo-labeled examples may be added to the training set for further learning. Consistency-based methods encourage a model to produce similar predictions for different valid transformations of the same input. Semi-supervised learning can improve performance when the unlabeled data contains useful information and the assumptions of the method are appropriate. However, incorrect pseudo-labels, distribution differences, or unsuitable assumptions can introduce errors and reduce performance. It is important to distinguish semi-supervised learning from self-supervised learning: semi-supervised learning uses some human-provided or externally supplied target labels alongside unlabeled data, whereas self-supervised learning creates its own training targets from the data.

Variables

SymbolMeaning

D

The combined dataset used for semi-supervised learning.

DL

The labeled portion of the dataset.

DU

The unlabeled portion of the dataset.

Pseudo-Label

A target label generated by a model for an unlabeled example.

Consistency

The expectation that predictions remain similar under suitable transformations of an input.

Example or application

A medical image dataset may contain 1,000 images labeled by specialists and 20,000 additional images without labels. A semi-supervised model can learn from the labeled images while using suitable methods to benefit from the larger unlabeled collection.

Text

Semi-supervised learning is useful when collecting labels is expensive or time-consuming. It allows a model to learn from a limited number of labeled examples while also using information from a larger unlabeled dataset.

Key Points

  • Semi-supervised learning uses both labeled and unlabeled data.

  • It is useful when obtaining labels is expensive.

  • Pseudo-labeling can extend the labeled training set.

  • Consistency-based methods encourage stable predictions under suitable input transformations.

  • Incorrect pseudo-labels can negatively affect training.

  • The unlabeled data should be relevant to the target problem.

Applications

  • Medical image classification

  • Speech recognition

  • Document classification

  • Web content categorization

  • Industrial image inspection

  • Large-scale image analysis

Learning Outcomes

  • Define semi-supervised learning.

  • Explain why unlabeled data can be useful alongside labeled data.

  • Describe pseudo-labeling and self-training.

  • Distinguish semi-supervised learning from supervised and self-supervised learning.

Practice Questions

  • What is semi-supervised learning?

  • Why is it useful when labeling data is expensive?

  • What is pseudo-labeling?

  • How does semi-supervised learning differ from self-supervised learning?

Related Concepts

  • Supervised Learning

  • Unsupervised Learning

  • Self-Supervised Learning

  • Pseudo-Labeling

  • Consistency Regularization

Common Mistakes

  • Assuming semi-supervised learning uses only unlabeled data.

  • Confusing pseudo-labels with verified human annotations.

  • Assuming adding more unlabeled data always improves model performance.

  • Treating semi-supervised and self-supervised learning as identical.

Sources and verification

Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.

  • Chapelle, O., Schölkopf, B., and Zien, A. Semi-Supervised Learning.
  • Zhu, X. Semi-Supervised Learning Literature Survey, 2005.
  • Van Engelen, J. E. and Hoos, H. H. A Survey on Semi-Supervised Learning, Machine Learning, 2020.

Suggest a correction or source · More in Artificial Intelligence & Machine Learning