✦ScienexySCIENCE KNOWLEDGE LIBRARY
Science reference · SCI-CS-003-066

Categorical Data Encoding

Categorical encoding converts non-numerical categories into numerical representations that machine learning algorithms can process.

Explanation

Categorical variables contain values such as colors, product types, countries, or subscription plans. Many machine learning algorithms require numerical input, so categorical values must be represented appropriately. One-hot encoding creates a separate binary feature for each category. Ordinal encoding assigns integer values to categories with a meaningful order, such as low, medium, and high. Label encoding assigns numbers to labels, but those numbers can incorrectly imply an order when categories are nominal. Frequency encoding and target encoding are additional techniques used in particular situations. Target encoding must be designed carefully, usually with cross-validation or out-of-fold calculations, because it can cause target leakage. Encoders should handle previously unseen categories during prediction.

Example or application

A color feature containing Red, Blue, and Green can be one-hot encoded into three binary columns: IsRed, IsBlue, and IsGreen. A Red observation becomes (1, 0, 0).

Text

Categorical features describe groups or labels rather than directly measured numerical quantities.

Key Points

  • Categorical variables represent labels or groups.

  • One-hot encoding represents categories using binary columns.

  • Ordinal encoding is appropriate when a meaningful order exists.

  • Arbitrary integer labels can create misleading relationships.

  • Target encoding requires leakage-aware validation.

Applications

  • Customer segmentation

  • Product recommendation

  • Credit risk prediction

  • Churn prediction

  • Text and document classification

Learning Outcomes

  • Identify nominal and ordinal categorical variables.

  • Explain one-hot and ordinal encoding.

  • Select encoding methods according to feature meaning.

  • Recognize risks associated with target encoding.

Practice Questions

  • What is categorical data encoding?

  • When should one-hot encoding be used?

  • Why is arbitrary integer encoding risky for nominal categories?

  • What is the main leakage risk of target encoding?

Related Concepts

  • Nominal Variables

  • Ordinal Variables

  • One-Hot Encoding

  • Ordinal Encoding

  • Target Encoding

Common Mistakes

  • Assigning arbitrary numerical order to nominal categories.

  • Using one-hot encoding without considering very high cardinality.

  • Ignoring unseen categories during inference.

  • Calculating target encodings using validation or test targets.

Sources and verification

Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.

  • Kuhn, M., and Johnson, K. (2019). Feature Engineering and Selection: A Practical Approach for Predictive Models.
  • Géron, A. (2022). Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow.

Suggest a correction or source · More in Artificial Intelligence & Machine Learning