A support vector machine is commonly used for classification, although support vector regression is also available. In a linear SVM, the algorithm searches for a hyperplane that separates classes while maximizing the margin between the boundary and the closest observations. These closest observations are called support vectors because they determine the position of the decision boundary. When data cannot be separated adequately by a linear boundary, kernel functions can represent nonlinear decision boundaries through an implicit higher-dimensional feature space. The regularization parameter C controls the trade-off between a wider margin and classification errors on training data. Feature scaling is generally important because SVMs are sensitive to the numerical scales of their input features.
Support Vector Machines (SVM)
Support vector machines are supervised learning algorithms that construct decision boundaries by maximizing the margin between classes.
Decision boundary: wᵀx + b = 0; Margin width for a canonical linear SVM: 2 / ||w||
Explanation
Variables
| Symbol | Meaning |
|---|---|
w | Weight vector defining the decision boundary |
x | Input feature vector |
b | Bias or intercept term |
C | Regularization parameter controlling the margin-error trade-off |
||w|| | Euclidean norm of the weight vector |
Example or application
An SVM classifier can separate two categories of flowers using measurements such as petal length and petal width. The model finds a boundary that maximizes the margin between the categories.
Text
SVM identifies a separating boundary that aims to keep the largest possible distance from the nearest training observations of different classes.
Key Points
SVM constructs decision boundaries called hyperplanes.
Support vectors are the closest influential training observations.
The margin measures the distance from the boundary to the closest observations.
Kernel functions support nonlinear decision boundaries.
The C parameter controls regularization.
Feature scaling is generally important.
Applications
Image classification
Text categorization
Handwriting recognition
Bioinformatics
Support vector regression
Learning Outcomes
Explain the purpose of support vector machines.
Define a hyperplane and margin.
Identify the role of support vectors.
Understand how kernels support nonlinear classification.
Practice Questions
What is a support vector?
What is the margin in an SVM?
How does a kernel function help an SVM?
What effect does the C parameter have?
Related Concepts
Hyperplanes
Support Vectors
Kernel Trick
Margin Maximization
Regularization
Common Mistakes
Assuming SVM is limited to linear classification.
Ignoring feature scaling.
Assuming every training observation becomes a support vector.
Selecting kernel and regularization settings without validation.
Sources and verification
Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.
- Cortes, C., and Vapnik, V. (1995). Support-Vector Networks.
- Bishop, C. M. (2006). Pattern Recognition and Machine Learning.
Suggest a correction or source · More in Artificial Intelligence & Machine Learning