✦ScienexySCIENCE KNOWLEDGE LIBRARY
Science reference · SCI-CS-003-078

Support Vector Machines (SVM)

Support vector machines are supervised learning algorithms that construct decision boundaries by maximizing the margin between classes.

Decision boundary: wᵀx + b = 0; Margin width for a canonical linear SVM: 2 / ||w||

Explanation

A support vector machine is commonly used for classification, although support vector regression is also available. In a linear SVM, the algorithm searches for a hyperplane that separates classes while maximizing the margin between the boundary and the closest observations. These closest observations are called support vectors because they determine the position of the decision boundary. When data cannot be separated adequately by a linear boundary, kernel functions can represent nonlinear decision boundaries through an implicit higher-dimensional feature space. The regularization parameter C controls the trade-off between a wider margin and classification errors on training data. Feature scaling is generally important because SVMs are sensitive to the numerical scales of their input features.

Variables

SymbolMeaning

w

Weight vector defining the decision boundary

x

Input feature vector

b

Bias or intercept term

C

Regularization parameter controlling the margin-error trade-off

||w||

Euclidean norm of the weight vector

Example or application

An SVM classifier can separate two categories of flowers using measurements such as petal length and petal width. The model finds a boundary that maximizes the margin between the categories.

Text

SVM identifies a separating boundary that aims to keep the largest possible distance from the nearest training observations of different classes.

Key Points

  • SVM constructs decision boundaries called hyperplanes.

  • Support vectors are the closest influential training observations.

  • The margin measures the distance from the boundary to the closest observations.

  • Kernel functions support nonlinear decision boundaries.

  • The C parameter controls regularization.

  • Feature scaling is generally important.

Applications

  • Image classification

  • Text categorization

  • Handwriting recognition

  • Bioinformatics

  • Support vector regression

Learning Outcomes

  • Explain the purpose of support vector machines.

  • Define a hyperplane and margin.

  • Identify the role of support vectors.

  • Understand how kernels support nonlinear classification.

Practice Questions

  • What is a support vector?

  • What is the margin in an SVM?

  • How does a kernel function help an SVM?

  • What effect does the C parameter have?

Related Concepts

  • Hyperplanes

  • Support Vectors

  • Kernel Trick

  • Margin Maximization

  • Regularization

Common Mistakes

  • Assuming SVM is limited to linear classification.

  • Ignoring feature scaling.

  • Assuming every training observation becomes a support vector.

  • Selecting kernel and regularization settings without validation.

Sources and verification

Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.

  • Cortes, C., and Vapnik, V. (1995). Support-Vector Networks.
  • Bishop, C. M. (2006). Pattern Recognition and Machine Learning.

Suggest a correction or source · More in Artificial Intelligence & Machine Learning