✦ScienexySCIENCE KNOWLEDGE LIBRARY
Science reference · SCI-CS-003-071

Linear Regression

Linear regression is a supervised learning algorithm that models the relationship between an input variable and a continuous numerical target using a linear equation.

y = β₀ + β₁x + ε; Predicted value: ŷ = β₀ + β₁x

Explanation

Linear regression is one of the most widely used supervised machine learning algorithms for predicting continuous numerical values. It assumes that the target variable can be approximated by a linear relationship with one or more input features. In simple linear regression, one independent variable is used to predict one dependent variable. The algorithm estimates the intercept and slope by minimizing the sum of squared differences between actual and predicted values, commonly known as the Ordinary Least Squares (OLS) method. After training, the fitted model can estimate outputs for previously unseen inputs. Linear regression is relatively simple and interpretable, but its performance may be limited when relationships are strongly nonlinear or when important assumptions are violated.

Variables

SymbolMeaning

y

Actual dependent variable

ŷ

Predicted dependent variable

x

Independent variable or input feature

β₀

Intercept of the regression line

β₁

Regression coefficient or slope

ε

Random error term

Example or application

A model learns the relationship between house size and selling price from historical housing data. If the fitted equation is ŷ = 50,000 + 2,000x, where x represents house size in square feet, it predicts a price of 250,000 for a 100-square-foot property.

Text

Linear regression predicts numerical values by fitting a straight line that best represents the relationship between input and output data.

Key Points

  • Linear regression is a supervised learning algorithm.

  • It predicts continuous numerical values.

  • Simple linear regression uses one independent variable.

  • Ordinary Least Squares minimizes the sum of squared residuals.

  • The slope describes how the predicted target changes with the input.

  • The model assumes a linear relationship in its parameters.

Applications

  • House price prediction

  • Sales forecasting

  • Demand estimation

  • Economic analysis

  • Trend prediction

Learning Outcomes

  • Define linear regression and its purpose.

  • Identify the intercept and slope.

  • Understand the role of residuals in model fitting.

  • Recognize situations where linear regression is appropriate.

Practice Questions

  • What is linear regression?

  • What is the difference between an intercept and a slope?

  • What does Ordinary Least Squares minimize?

  • Why is linear regression suitable for predicting house prices?

Related Concepts

  • Supervised Learning

  • Regression

  • Ordinary Least Squares

  • Residuals

  • Mean Squared Error

Common Mistakes

  • Using linear regression for categorical targets without an appropriate modeling approach.

  • Assuming correlation proves causation.

  • Ignoring influential outliers.

  • Confusing predicted values with actual observations.

Sources and verification

Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.

  • James, G., Witten, D., Hastie, T., Tibshirani, R., and Taylor, J. (2023). An Introduction to Statistical Learning.
  • Bishop, C. M. (2006). Pattern Recognition and Machine Learning.

Suggest a correction or source · More in Artificial Intelligence & Machine Learning