Semi-Supervised Learning is a machine learning paradigm that uses both labeled and unlabeled training examples. It is particularly useful when obtaining accurate labels requires expert knowledge, expensive experiments, or substantial human effort. A small labeled dataset provides direct information about the desired target, while a larger unlabeled dataset may reveal the structure and distribution of the input data. Common semi-supervised methods include self-training, pseudo-labeling, consistency regularization, and graph-based approaches. In self-training, a model is initially trained on labeled examples and then uses its predictions to assign pseudo-labels to selected unlabeled examples. These pseudo-labeled examples may be added to the training set for further learning. Consistency-based methods encourage a model to produce similar predictions for different valid transformations of the same input. Semi-supervised learning can improve performance when the unlabeled data contains useful information and the assumptions of the method are appropriate. However, incorrect pseudo-labels, distribution differences, or unsuitable assumptions can introduce errors and reduce performance. It is important to distinguish semi-supervised learning from self-supervised learning: semi-supervised learning uses some human-provided or externally supplied target labels alongside unlabeled data, whereas self-supervised learning creates its own training targets from the data.
Semi-Supervised Learning
Semi-supervised learning combines a relatively small amount of labeled data with a larger amount of unlabeled data to train a machine learning model.
D = DL ∪ DU
Explanation
Variables
| Symbol | Meaning |
|---|---|
D | The combined dataset used for semi-supervised learning. |
DL | The labeled portion of the dataset. |
DU | The unlabeled portion of the dataset. |
Pseudo-Label | A target label generated by a model for an unlabeled example. |
Consistency | The expectation that predictions remain similar under suitable transformations of an input. |
Example or application
A medical image dataset may contain 1,000 images labeled by specialists and 20,000 additional images without labels. A semi-supervised model can learn from the labeled images while using suitable methods to benefit from the larger unlabeled collection.
Text
Semi-supervised learning is useful when collecting labels is expensive or time-consuming. It allows a model to learn from a limited number of labeled examples while also using information from a larger unlabeled dataset.
Key Points
Semi-supervised learning uses both labeled and unlabeled data.
It is useful when obtaining labels is expensive.
Pseudo-labeling can extend the labeled training set.
Consistency-based methods encourage stable predictions under suitable input transformations.
Incorrect pseudo-labels can negatively affect training.
The unlabeled data should be relevant to the target problem.
Applications
Medical image classification
Speech recognition
Document classification
Web content categorization
Industrial image inspection
Large-scale image analysis
Learning Outcomes
Define semi-supervised learning.
Explain why unlabeled data can be useful alongside labeled data.
Describe pseudo-labeling and self-training.
Distinguish semi-supervised learning from supervised and self-supervised learning.
Practice Questions
What is semi-supervised learning?
Why is it useful when labeling data is expensive?
What is pseudo-labeling?
How does semi-supervised learning differ from self-supervised learning?
Related Concepts
Supervised Learning
Unsupervised Learning
Self-Supervised Learning
Pseudo-Labeling
Consistency Regularization
Common Mistakes
Assuming semi-supervised learning uses only unlabeled data.
Confusing pseudo-labels with verified human annotations.
Assuming adding more unlabeled data always improves model performance.
Treating semi-supervised and self-supervised learning as identical.
Sources and verification
Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.
- Chapelle, O., Schölkopf, B., and Zien, A. Semi-Supervised Learning.
- Zhu, X. Semi-Supervised Learning Literature Survey, 2005.
- Van Engelen, J. E. and Hoos, H. H. A Survey on Semi-Supervised Learning, Machine Learning, 2020.
Suggest a correction or source · More in Artificial Intelligence & Machine Learning