Categorical variables contain values such as colors, product types, countries, or subscription plans. Many machine learning algorithms require numerical input, so categorical values must be represented appropriately. One-hot encoding creates a separate binary feature for each category. Ordinal encoding assigns integer values to categories with a meaningful order, such as low, medium, and high. Label encoding assigns numbers to labels, but those numbers can incorrectly imply an order when categories are nominal. Frequency encoding and target encoding are additional techniques used in particular situations. Target encoding must be designed carefully, usually with cross-validation or out-of-fold calculations, because it can cause target leakage. Encoders should handle previously unseen categories during prediction.
Categorical Data Encoding
Categorical encoding converts non-numerical categories into numerical representations that machine learning algorithms can process.
Explanation
Example or application
A color feature containing Red, Blue, and Green can be one-hot encoded into three binary columns: IsRed, IsBlue, and IsGreen. A Red observation becomes (1, 0, 0).
Text
Categorical features describe groups or labels rather than directly measured numerical quantities.
Key Points
Categorical variables represent labels or groups.
One-hot encoding represents categories using binary columns.
Ordinal encoding is appropriate when a meaningful order exists.
Arbitrary integer labels can create misleading relationships.
Target encoding requires leakage-aware validation.
Applications
Customer segmentation
Product recommendation
Credit risk prediction
Churn prediction
Text and document classification
Learning Outcomes
Identify nominal and ordinal categorical variables.
Explain one-hot and ordinal encoding.
Select encoding methods according to feature meaning.
Recognize risks associated with target encoding.
Practice Questions
What is categorical data encoding?
When should one-hot encoding be used?
Why is arbitrary integer encoding risky for nominal categories?
What is the main leakage risk of target encoding?
Related Concepts
Nominal Variables
Ordinal Variables
One-Hot Encoding
Ordinal Encoding
Target Encoding
Common Mistakes
Assigning arbitrary numerical order to nominal categories.
Using one-hot encoding without considering very high cardinality.
Ignoring unseen categories during inference.
Calculating target encodings using validation or test targets.
Sources and verification
Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.
- Kuhn, M., and Johnson, K. (2019). Feature Engineering and Selection: A Practical Approach for Predictive Models.
- Géron, A. (2022). Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow.
Suggest a correction or source · More in Artificial Intelligence & Machine Learning