Optimization problems are classified according to whether their decision variables are subject to constraints. In unconstrained optimization, the objective function is optimized over its specified domain without additional equality or inequality restrictions. For differentiable functions, stationary points are often identified by setting the gradient equal to zero. In constrained optimization, decisions must also satisfy conditions such as resource limits, capacity restrictions, or required relationships among variables. These restrictions can change the location of the optimum and may cause the solution to occur on a boundary. Constrained methods include Lagrange multipliers, Karush–Kuhn–Tucker conditions, penalty methods, and specialized linear or nonlinear programming algorithms. Even an apparently unconstrained problem has an implicit domain in which its variables must be defined.
Constrained and Unconstrained Optimization
Constrained optimization searches for the best solution while satisfying specified restrictions, whereas unconstrained optimization has no explicit constraints beyond the variable domain.
Unconstrained: minimize f(x)
First-order stationary condition: ∇f(x*) = 0
Constrained: minimize f(x) subject to gᵢ(x) ≤ 0 and hⱼ(x) = 0
Explanation
Variables
f(x): objective function
x: decision vector
∇f(x): gradient of the objective
gᵢ(x): inequality constraints
hⱼ(x): equality constraints
Example or application
Finding the minimum of f(x) = (x − 2)² without additional restrictions is unconstrained over the real line. If x must satisfy x ≥ 5, the problem becomes constrained, and the minimum over the permitted domain occurs at x = 5.
Definition
Unconstrained optimization minimizes or maximizes an objective without explicit additional constraints, while constrained optimization performs the same task subject to equality, inequality, or domain restrictions.
Purpose
The purpose is to classify a problem according to its restrictions and select a suitable mathematical solution method.
Importance
Constraints often represent the most important practical limitations. Understanding their effect is necessary for producing feasible and useful decisions.
Learning Objectives
Define constrained and unconstrained optimization.
Recognize equality and inequality restrictions.
Interpret gradient-based stationary conditions.
Understand why constrained optima may occur at boundaries.
Identify suitable methods for each problem type.
Key Concepts
Unconstrained optimization
Constrained optimization
Gradient
Stationary point
Boundary optimum
Lagrange multiplier
KKT conditions
Working Principle
Unconstrained methods analyze the objective over its domain, while constrained methods must also preserve all specified restrictions. The solution method accounts for the mathematical relationship between the objective and the constraints.
Process
Identify the objective function.
Determine the variable domain and all additional restrictions.
Classify the problem as constrained or unconstrained.
Select an appropriate analytical or numerical method.
Find candidate solutions.
Check the candidates against all constraints.
Compare objective values to establish optimality.
Applications
Unrestricted mathematical function minimization
Engineering design with material limits
Portfolio allocation with budget constraints
Production planning with capacity restrictions
Energy optimization with operating limits
Worked Example
Minimize f(x) = (x − 2)². Without additional constraints, differentiating gives f'(x) = 2(x − 2). Setting f'(x) = 0 gives x = 2, with minimum value 0. Now impose x ≥ 5. The feasible domain begins at 5, and since the function increases for x greater than 2, its constrained minimum occurs at x = 5, giving f(5) = 9.
Common Misconceptions
An unconstrained problem still has a mathematical domain.
A constrained optimum does not have to satisfy the unconstrained stationary condition.
A boundary point can be optimal even when the gradient is not zero.
Lagrange multipliers and KKT conditions are not interchangeable in every situation; their applicability depends on problem assumptions.
Practice Questions
Define constrained and unconstrained optimization.
Find the unconstrained minimum of f(x) = (x − 4)².
How does the restriction x ≥ 6 change the previous problem?
What is a boundary optimum?
Name two methods used in constrained optimization.
Key Takeaway
The presence or absence of constraints changes the mathematical structure of an optimization problem and often changes its optimal solution.
Prerequisites
Differentiation
Objective Functions
Constraints and Feasible Solutions
Reference
Boyd, S., and Vandenberghe, L. Convex Optimization.
Nocedal, J., and Wright, S. J. Numerical Optimization.
Related Topics
Continuous and Discrete Optimization
Local and Global Optima
Optimization Problem Formulation
Optimization Algorithms and Solution Methods
Sources and verification
Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.
- Source verification pending; check an authoritative textbook, university, standards body, or peer-reviewed source.
Suggest a correction or source · More in Optimization & Operations Research