✦ScienexySCIENCE KNOWLEDGE LIBRARY
Science reference · SCI-CS-003-074

Logistic Regression

Logistic regression is a supervised learning algorithm that estimates the probability of a categorical outcome, commonly for binary classification.

p(y = 1 | x) = 1 / (1 + e^(-z)); z = β₀ + β₁x₁ + ... + βₙxₙ

Explanation

Logistic regression estimates the probability that an observation belongs to a particular class. In binary classification, it first calculates a linear combination of input features and then applies the sigmoid function to convert the result into a probability between zero and one. A classification threshold, often 0.5, can convert this probability into a predicted class. Model parameters are commonly estimated by maximizing likelihood or minimizing binary cross-entropy loss. Logistic regression is relatively efficient and interpretable, and its coefficients can be interpreted in terms of changes in log-odds. Multinomial and ordinal extensions can handle other categorical prediction settings. Its decision boundary is linear in the original feature space unless features are transformed.

Variables

SymbolMeaning

p

Estimated probability of the positive class

z

Linear combination of input features

e

Base of the natural logarithm

β₀

Intercept

βᵢ

Coefficient for feature i

xᵢ

Input feature i

Example or application

An email classification system estimates a 0.92 probability that a message is spam. If the classification threshold is 0.5, the model labels the message as spam.

Text

Despite its name, logistic regression is primarily used for classification rather than predicting unrestricted continuous values.

Key Points

  • Logistic regression is commonly used for binary classification.

  • The sigmoid function maps model scores to probabilities.

  • Predictions can be converted into classes using a threshold.

  • Parameters are commonly learned using maximum likelihood.

  • Coefficients describe changes in log-odds.

  • Multinomial logistic regression supports multiple classes.

Applications

  • Spam detection

  • Disease risk classification

  • Customer churn prediction

  • Credit default prediction

  • Fraud detection

Learning Outcomes

  • Explain why logistic regression is used for classification.

  • Describe the role of the sigmoid function.

  • Interpret predicted probabilities and classification thresholds.

  • Understand the meaning of logistic regression coefficients.

Practice Questions

  • Why is logistic regression considered a classification algorithm?

  • What is the purpose of the sigmoid function?

  • How does a classification threshold affect predictions?

  • What does a logistic regression coefficient represent?

Related Concepts

  • Binary Classification

  • Sigmoid Function

  • Log-Odds

  • Cross-Entropy Loss

  • Classification Threshold

Common Mistakes

  • Assuming logistic regression predicts unrestricted continuous values.

  • Treating a probability as a guaranteed outcome.

  • Assuming 0.5 is the best threshold for every application.

  • Ignoring class imbalance when evaluating performance.

Sources and verification

Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.

  • James, G., Witten, D., Hastie, T., Tibshirani, R., and Taylor, J. (2023). An Introduction to Statistical Learning.
  • Hosmer, D. W., Lemeshow, S., and Sturdivant, R. X. (2013). Applied Logistic Regression.

Suggest a correction or source · More in Artificial Intelligence & Machine Learning