✦ScienexySCIENCE KNOWLEDGE LIBRARY
Science reference · SCI-CS-003-065

Data Normalization and Standardization

Normalization and standardization are feature-scaling techniques that place numerical variables on comparable scales.

Min-max normalization: x' = (x - x_min) / (x_max - x_min); Standardization: z = (x - μ) / σ

Explanation

Normalization and standardization are commonly used to transform numerical features before model training. Min-max normalization maps values into a selected range, often zero to one, using the minimum and maximum values. Standardization transforms values based on the training-set mean and standard deviation, producing features with a mean near zero and a standard deviation near one. These methods are useful for algorithms sensitive to feature scales, including k-nearest neighbors, support vector machines, and many gradient-based models. Tree-based algorithms are generally less sensitive to feature scaling. Scaling parameters should be calculated from training data and then reused for validation, test, and future data. Extreme outliers can strongly affect min-max normalization, while standardization does not guarantee that all transformed values fall within a fixed range.

Variables

SymbolMeaning

x

Original feature value

x'

Normalized feature value

x_min

Minimum value in the training feature

x_max

Maximum value in the training feature

μ

Training-set mean

σ

Training-set standard deviation

z

Standardized feature value

Example or application

Suppose a training feature has values ranging from 10 to 50. A value of 30 becomes 0.5 after min-max normalization. If the training mean is 30 and standard deviation is 10, the same value has a standardized score of 0.

Text

Feature scaling helps prevent variables with large numerical ranges from dominating certain machine learning algorithms.

Key Points

  • Normalization commonly maps values into a fixed range.

  • Standardization uses the mean and standard deviation.

  • Scaling is important for many distance-based and gradient-based algorithms.

  • Standardization does not restrict all values to zero and one.

  • Scaling parameters must be learned from training data only.

Applications

  • K-nearest neighbors

  • Support vector machines

  • Neural networks

  • Clustering algorithms

  • Gradient-based optimization

Learning Outcomes

  • Explain why feature scaling is useful.

  • Differentiate normalization from standardization.

  • Apply basic feature-scaling formulas.

  • Identify algorithms that are sensitive to feature scales.

Practice Questions

  • What is the purpose of feature scaling?

  • How does min-max normalization differ from standardization?

  • Does standardization guarantee values between zero and one?

  • Why must scaling parameters be calculated from training data?

Related Concepts

  • Feature Scaling

  • Z-Score

  • Min-Max Scaling

  • Outliers

  • Data Leakage

Common Mistakes

  • Treating normalization and standardization as identical methods.

  • Assuming standardized values must lie between zero and one.

  • Fitting a scaler using the entire dataset before splitting.

  • Ignoring outlier effects on min-max scaling.

Sources and verification

Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.

  • Géron, A. (2022). Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow.
  • Bishop, C. M. (2006). Pattern Recognition and Machine Learning.

Suggest a correction or source · More in Artificial Intelligence & Machine Learning