Feature engineering uses domain knowledge, statistical reasoning, and experimentation to construct useful model inputs. Techniques include creating ratios, aggregations, interaction terms, date-time components, text statistics, and transformations such as logarithms. For example, a transaction dataset may be enriched with a customer's average purchase amount, number of transactions in the last month, or time since the previous purchase. Feature engineering differs from feature selection because it can create new variables, while feature selection chooses among existing ones. It also differs from feature extraction, although the terms can overlap in practice. Good features should be meaningful, reproducible, available at prediction time, and calculated without using future information or target leakage. Feature engineering is often an iterative process involving model evaluation and refinement.
Feature Engineering
Feature engineering is the process of creating, transforming, and combining input variables to make useful patterns more accessible to machine learning models.
Explanation
Example or application
A housing dataset contains house price and floor area. A new feature called price per square meter can be created by dividing the price by the floor area, helping comparisons between properties of different sizes.
Text
Well-designed features can help a model learn relationships that are difficult to identify from raw data alone.
Key Points
Feature engineering creates or transforms model inputs.
Domain knowledge is valuable when designing features.
Ratios, aggregations, and date-time components are common techniques.
Features must be available at the time predictions are made.
Feature engineering should avoid future-information leakage.
Applications
Fraud detection
Customer churn prediction
Real estate price prediction
Demand forecasting
Recommendation systems
Learning Outcomes
Define feature engineering.
Identify common feature construction techniques.
Explain the role of domain knowledge.
Distinguish feature engineering from feature selection and extraction.
Practice Questions
What is feature engineering?
Give three examples of engineered features.
How does feature engineering differ from feature selection?
Why should future information not be used when creating prediction features?
Related Concepts
Feature Selection
Feature Extraction
Feature Transformation
Domain Knowledge
Data Leakage
Common Mistakes
Creating features that depend on information unavailable at prediction time.
Adding many features without evaluating their usefulness.
Ignoring domain knowledge.
Confusing feature engineering with feature selection.
Sources and verification
Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.
- Kuhn, M., and Johnson, K. (2019). Feature Engineering and Selection: A Practical Approach for Predictive Models.
- Zheng, A., and Casari, A. (2018). Feature Engineering for Machine Learning.
Suggest a correction or source · More in Artificial Intelligence & Machine Learning