Normalization and standardization are commonly used to transform numerical features before model training. Min-max normalization maps values into a selected range, often zero to one, using the minimum and maximum values. Standardization transforms values based on the training-set mean and standard deviation, producing features with a mean near zero and a standard deviation near one. These methods are useful for algorithms sensitive to feature scales, including k-nearest neighbors, support vector machines, and many gradient-based models. Tree-based algorithms are generally less sensitive to feature scaling. Scaling parameters should be calculated from training data and then reused for validation, test, and future data. Extreme outliers can strongly affect min-max normalization, while standardization does not guarantee that all transformed values fall within a fixed range.
Data Normalization and Standardization
Normalization and standardization are feature-scaling techniques that place numerical variables on comparable scales.
Min-max normalization: x' = (x - x_min) / (x_max - x_min); Standardization: z = (x - μ) / σ
Explanation
Variables
| Symbol | Meaning |
|---|---|
x | Original feature value |
x' | Normalized feature value |
x_min | Minimum value in the training feature |
x_max | Maximum value in the training feature |
μ | Training-set mean |
σ | Training-set standard deviation |
z | Standardized feature value |
Example or application
Suppose a training feature has values ranging from 10 to 50. A value of 30 becomes 0.5 after min-max normalization. If the training mean is 30 and standard deviation is 10, the same value has a standardized score of 0.
Text
Feature scaling helps prevent variables with large numerical ranges from dominating certain machine learning algorithms.
Key Points
Normalization commonly maps values into a fixed range.
Standardization uses the mean and standard deviation.
Scaling is important for many distance-based and gradient-based algorithms.
Standardization does not restrict all values to zero and one.
Scaling parameters must be learned from training data only.
Applications
K-nearest neighbors
Support vector machines
Neural networks
Clustering algorithms
Gradient-based optimization
Learning Outcomes
Explain why feature scaling is useful.
Differentiate normalization from standardization.
Apply basic feature-scaling formulas.
Identify algorithms that are sensitive to feature scales.
Practice Questions
What is the purpose of feature scaling?
How does min-max normalization differ from standardization?
Does standardization guarantee values between zero and one?
Why must scaling parameters be calculated from training data?
Related Concepts
Feature Scaling
Z-Score
Min-Max Scaling
Outliers
Data Leakage
Common Mistakes
Treating normalization and standardization as identical methods.
Assuming standardized values must lie between zero and one.
Fitting a scaler using the entire dataset before splitting.
Ignoring outlier effects on min-max scaling.
Sources and verification
Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.
- Géron, A. (2022). Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow.
- Bishop, C. M. (2006). Pattern Recognition and Machine Learning.
Suggest a correction or source · More in Artificial Intelligence & Machine Learning