Linear regression is one of the most widely used supervised machine learning algorithms for predicting continuous numerical values. It assumes that the target variable can be approximated by a linear relationship with one or more input features. In simple linear regression, one independent variable is used to predict one dependent variable. The algorithm estimates the intercept and slope by minimizing the sum of squared differences between actual and predicted values, commonly known as the Ordinary Least Squares (OLS) method. After training, the fitted model can estimate outputs for previously unseen inputs. Linear regression is relatively simple and interpretable, but its performance may be limited when relationships are strongly nonlinear or when important assumptions are violated.
Linear Regression
Linear regression is a supervised learning algorithm that models the relationship between an input variable and a continuous numerical target using a linear equation.
y = β₀ + β₁x + ε; Predicted value: ŷ = β₀ + β₁x
Explanation
Variables
| Symbol | Meaning |
|---|---|
y | Actual dependent variable |
ŷ | Predicted dependent variable |
x | Independent variable or input feature |
β₀ | Intercept of the regression line |
β₁ | Regression coefficient or slope |
ε | Random error term |
Example or application
A model learns the relationship between house size and selling price from historical housing data. If the fitted equation is ŷ = 50,000 + 2,000x, where x represents house size in square feet, it predicts a price of 250,000 for a 100-square-foot property.
Text
Linear regression predicts numerical values by fitting a straight line that best represents the relationship between input and output data.
Key Points
Linear regression is a supervised learning algorithm.
It predicts continuous numerical values.
Simple linear regression uses one independent variable.
Ordinary Least Squares minimizes the sum of squared residuals.
The slope describes how the predicted target changes with the input.
The model assumes a linear relationship in its parameters.
Applications
House price prediction
Sales forecasting
Demand estimation
Economic analysis
Trend prediction
Learning Outcomes
Define linear regression and its purpose.
Identify the intercept and slope.
Understand the role of residuals in model fitting.
Recognize situations where linear regression is appropriate.
Practice Questions
What is linear regression?
What is the difference between an intercept and a slope?
What does Ordinary Least Squares minimize?
Why is linear regression suitable for predicting house prices?
Related Concepts
Supervised Learning
Regression
Ordinary Least Squares
Residuals
Mean Squared Error
Common Mistakes
Using linear regression for categorical targets without an appropriate modeling approach.
Assuming correlation proves causation.
Ignoring influential outliers.
Confusing predicted values with actual observations.
Sources and verification
Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.
- James, G., Witten, D., Hastie, T., Tibshirani, R., and Taylor, J. (2023). An Introduction to Statistical Learning.
- Bishop, C. M. (2006). Pattern Recognition and Machine Learning.
Suggest a correction or source · More in Artificial Intelligence & Machine Learning