✦ScienexySCIENCE KNOWLEDGE LIBRARY
Science reference · SCI-CS-003-152

Agents, States, Actions, and Rewards

Agents, states, actions, and rewards are the fundamental components of reinforcement learning. They describe who makes decisions, what information is available, which choices can be made, and how outcomes are evaluated.

S_t, A_t → R_{t+1}, S_{t+1}

Explanation

A reinforcement learning system can be described using four central components. The agent is the learner or decision-making system that attempts to achieve a goal. The environment is the external system with which the agent interacts. A state represents the information used to describe the current situation. Depending on the problem, a state may include an object's position, velocity, available resources, or other relevant information. An action is a decision available to the agent, such as moving in a direction, selecting a command, or adjusting a control value. The environment responds to the action by changing its state and providing a reward. A reward is a numerical signal that indicates how desirable an outcome is according to the task's reward design. Positive rewards may encourage useful behavior, while negative rewards may discourage undesirable behavior. The reward is not necessarily the same as the agent's final objective; poorly designed rewards can lead to unintended strategies. The agent uses experience to learn a policy, which specifies how actions should be selected in different states. In partially observable environments, the agent may not know the complete state and must instead work with observations or a belief about the current situation.

Variables

SymbolMeaning

S_t

State at time step t

A_t

Action selected at time step t

R_{t+1}

Reward received after the action

S_{t+1}

Next state after the environment responds

Example or application

In a simulated delivery robot, the agent is the robot's decision system, the state includes its location and nearby obstacles, an action moves it in a selected direction, and the reward encourages reaching the delivery point efficiently while avoiding obstacles.

Text

An agent is the decision-maker, a state describes the current situation, an action is a choice made by the agent, and a reward is numerical feedback from the environment.

Key Points

  • The agent makes decisions and learns from experience.

  • The environment is the system being interacted with.

  • A state describes the current situation.

  • An action changes or influences the environment.

  • A reward provides numerical feedback.

  • A policy determines how the agent selects actions.

Applications

  • Robot movement and control

  • Game-playing agents

  • Autonomous vehicles in simulation

  • Industrial automation

  • Resource management

  • Adaptive scheduling systems

Learning Outcomes

  • Identify the main components of an RL system.

  • Distinguish an agent from its environment.

  • Explain the relationship between states and actions.

  • Describe how rewards influence learning.

  • Understand the role of a policy.

Practice Questions

  • What is an agent in reinforcement learning?

  • What information does a state represent?

  • How is an action different from a reward?

  • What happens after an agent takes an action?

  • What is a policy?

Related Concepts

  • Reinforcement Learning

  • Markov Decision Processes

  • Reward Design

  • State Space

  • Action Space

  • Policy

Common Mistakes

  • Confusing the agent with the environment.

  • Treating a reward as an action.

  • Assuming every state must contain complete information about the environment.

  • Designing rewards without considering unintended behavior.

  • Assuming that a reward directly specifies the exact action the agent must take.

Sources and verification

Sources are shown as provided in the project data. Not every source has yet been independently verified. Please report errors or missing citations.

  • Sutton, R. S., and Barto, A. G. (2018). Reinforcement Learning: An Introduction, Second Edition. MIT Press.
  • Puterman, M. L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley.

Suggest a correction or source · More in Artificial Intelligence & Machine Learning