✦ScienexySCIENCE KNOWLEDGE LIBRARY
Science reference · SCI-CS-003-060

Reinforcement Learning

Reinforcement Learning (RL) is a machine learning paradigm in which an agent learns to make decisions by interacting with an environment and receiving rewards or penalties.

Gₜ = Rₜ₊₁ + γRₜ₊₂ + γ²Rₜ₊₃ + ... = Σₖ₌₀∞ γᵏRₜ₊ₖ₊₁

Explanation

Reinforcement Learning is a machine learning approach concerned with how an agent should act in an environment to achieve long-term objectives. The agent observes the current state, selects an action according to its policy, and receives a reward and a new state from the environment. The learning process is repeated over time, allowing the agent to improve its decisions based on experience. Unlike traditional supervised learning, reinforcement learning does not normally receive a correct target action for every situation. Instead, it receives reward feedback that may be delayed or incomplete. The agent therefore needs to balance exploration, which involves trying actions to discover their consequences, and exploitation, which involves choosing actions that are already believed to produce good outcomes. A common mathematical framework is the Markov Decision Process (MDP), defined using states, actions, transition probabilities, rewards, and a discount factor. The agent aims to learn a policy that maximizes expected cumulative reward. Important approaches include value-based methods such as Q-learning, policy-based methods, and actor-critic methods. Reinforcement learning is useful for sequential decision-making, but training can be computationally demanding and sensitive to reward design, exploration strategy, and environment complexity.

Variables

SymbolMeaning

Agent

The learner or decision-maker that interacts with the environment.

Environment

The external system with which the agent interacts.

State (s)

The current situation observed by the agent.

Action (a)

A decision or operation selected by the agent.

Reward (r)

Numerical feedback received after an action.

Policy (π)

A strategy that determines how the agent selects actions.

γ

The discount factor, usually between 0 and 1, controlling the importance of future rewards.

Gₜ

The discounted cumulative reward from time step t.

Example or application

A simulated robot learns to navigate a room. It receives positive rewards for reaching a target and penalties for hitting obstacles or taking inefficient routes. By trying different actions and learning from the resulting feedback, it gradually develops a more effective navigation policy.

Text

Reinforcement learning focuses on learning a sequence of actions that produces high cumulative reward. An agent observes the environment, chooses an action, receives feedback, and updates its behavior over time.

Key Points

  • Reinforcement learning uses an agent and an environment.

  • The agent observes states and selects actions.

  • Rewards provide feedback about the consequences of actions.

  • The objective is generally to maximize expected cumulative reward.

  • Exploration and exploitation are important parts of learning.

  • Markov Decision Processes provide a common mathematical framework.

  • Q-learning, policy-gradient methods, and actor-critic methods are important RL approaches.

Applications

  • Robotics and autonomous control

  • Game-playing agents

  • Traffic signal optimization

  • Resource allocation

  • Industrial process control

  • Adaptive recommendation and decision systems

Learning Outcomes

  • Define reinforcement learning.

  • Identify the agent, environment, state, action, and reward.

  • Explain the exploration-exploitation trade-off.

  • Describe the role of a policy and cumulative reward.

  • Recognize common reinforcement learning algorithms.

Practice Questions

  • What is reinforcement learning?

  • What are the main components of a reinforcement learning system?

  • What is the difference between exploration and exploitation?

  • What is the purpose of the discount factor?

  • How does reinforcement learning differ from supervised learning?

Related Concepts

  • Types of Machine Learning

  • Markov Decision Processes

  • Q-Learning

  • Policy Gradient Methods

  • Exploration and Exploitation

Common Mistakes

  • Assuming the agent receives the correct action as a label at every step.

  • Confusing immediate reward with cumulative reward.

  • Ignoring the exploration-exploitation trade-off.

  • Assuming a high immediate reward always leads to the best long-term outcome.

  • Designing rewards without considering unintended behavior.

Sources and verification

Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.

  • Sutton, R. S. and Barto, A. G. Reinforcement Learning: An Introduction, Second Edition.
  • Szepesvári, C. Algorithms for Reinforcement Learning.
  • Sutton, R. S. Introduction to Reinforcement Learning, course materials.

Suggest a correction or source · More in Artificial Intelligence & Machine Learning