Simple reflex agents select actions on the basis of the current percept, ignoring the rest of the percept history. They have no state: the same percept always produces the same action, through condition–action rules. This is the architecture embodied by the classic subsumption architecture line of work and by rule-based reactive systems.
The subsumption architecture is the best-known architecture for reactive agents, developed by Rodney Brooks. The agent’s decision-making is realized through a set of task-accomplishing behaviours arranged into hierarchical layers:
When the layers are combined, higher layers subsume the lower ones when appropriate, giving reactivity with a hierarchical organization of behaviours.
The general family of layered agent architectures organizes behaviours, or more generally reasoning capabilities, into layers that interact through defined interfaces — the subsumption architecture being the reactive member of the family.
Model-based reflex agents overcome the statelessness of simple reflex agents: the agent has an internal state that depends on the percept history, used to reflect at least some of the unobserved aspects of the current state. The internal state is updated as time goes by, using two kinds of knowledge:
Together, the transition model and the sensor model allow an agent to keep track of the state of the world, to the extent possible given the limitations of its sensors.
A model-based goal-based agent has an explicit knowledge and representation of the goal/task to accomplish. The agent program combines this with the model (the same information used by the model-based reflex agent) to choose actions that achieve the goal. Search and planning are the subfields of AI devoted to finding action sequences that achieve the agent’s goals.
Decision making of this kind is fundamentally different from condition–action rules: it involves consideration of the future. The classic realization is the Sense-Plan-Act architecture: sense the environment, plan a course of action that achieves the goal, act on it; then repeat.
Reactive and deliberative layers each have weaknesses: pure reactivity cannot pursue long-term goals; pure deliberation is too slow for dynamic environments. Hybrid architectures combine both in layers, with a control mechanism deciding which layer drives behaviour at each moment. Two classic examples:
The layering tension — react while deliberating, deliberate while reacting — is resolved in the BDI architecture by interleaving plan selection and action execution. The BDI reasoning cycle of Chapter 14 is, in a precise sense, the mature answer to the hybrid-architecture problem.
In many real-world scenarios, goals alone are not enough to generate high-quality behaviour: there may be different action sequences achieving the goal, some evaluated better than others with respect to the performance measure — goals provide only a crude binary distinction between “happy” and “unhappy” states. A more general performance measure allows a comparison of different world states according to exactly how happy they would make the agent. Because “happy” does not sound scientific, economists and computer scientists use the term utility.
The central question: how do the agent programs come into being? In his famous early paper, Turing (1950) considers programming his intelligent machines by hand, estimates the work, and concludes that “some more expeditious method seems desirable” — the method he proposes is to build learning machines and then to teach them. In many areas of AI this is now the preferred method for creating state-of-the-art systems; any type of agent (model-based, goal-based, utility-based, etc.) can be built as a learning agent or not.
A learning agent can be divided into four conceptual components:
Reinforcement Learning (RL) agents instantiate the four components concretely. The policy π(s) selects the action; a state estimator may be needed if percepts ≠ states. Typical implementation:
The four components map onto RL machinery:
AI agents / “language agents” — also called LLM-based autonomous agents or “generative agents” — are the “AI Agents” of Agentic AI: agent systems whose reasoning core is a large language model. The reference model used by the module is Sumers, Yao, Narasimhan and Griffiths, Cognitive Architectures for Language Agents (TMLR, 02/2024), which explicitly frames language agents as a species of cognitive architecture — the connection to Chapter 14.
Their architecture is based on prompt-engineering techniques and runs a continuous cycle of perception, reasoning, planning, action and reflection (see Plate 13.8).
Several in-progress technologies (libraries, frameworks, platforms) support building such agents. Main examples:
The programming style is declarative configuration around an LLM core, with tools, instructions and input schemas. The course’s example comes from Google ADK (2025):
Note the architectural echoes: instruction is a goal in natural language, tools are the action repertoire, and inputSchema defines the agent’s interface with the world — the vocabulary of Chapter 12 reappears in the agentic wave.
Two emerging “standards” frame the agentic wave:
MCP and A2A are complementary: MCP connects an agent to its tools and data sources (the agent–environment interface), while A2A connects agents to each other (the agent–agent interaction). Chapter 15 will revisit this complementarity from the MAS side, contrasting “tool”/MCP with artifacts (A&A) and A2A with agent communication languages.
Simple reflex agents select actions on the basis of the current percept, ignoring percept history, with no state — condition–action rules. The subsumption architecture (Brooks) realizes reactive decision-making through task-accomplishing behaviours arranged in hierarchical layers; each behaviour maps perceptual input to an action, higher layers (lower priority) are more abstract, and low-level layers carry high priority (e.g. avoid obstacles).
They maintain an internal state that depends on the percept history, reflecting unobserved aspects of the current state. The state is updated over time using two kinds of knowledge: a model of the world (how the world changes, effects of the agent’s actions, how it evolves independently) and a sensor model (how the world state is reflected in percepts). Together they let the agent track the world to the extent its sensors allow.
Goal-based agents have an explicit representation of the goal/task and combine it with the world model to choose actions that achieve it; decision making involves consideration of the future. Search and planning are the AI subfields that find action sequences achieving goals. The sense-plan-act cycle realizes this: sense the environment, plan a course of action, act, and repeat.
Goals give only a crude binary distinction between “happy” and “unhappy” states, while different action sequences achieving a goal may be evaluated differently by the performance measure. The utility function internalizes the performance measure, ranking states by how good they are, and is essential for decision making under uncertainty (partial observability, conflicting goals). A rational utility-based agent chooses the action maximizing expected utility of outcomes.
Performance element — selects external actions (what we previously called the whole agent). Critic — tells the learning element how well the agent is doing against a fixed performance standard (percepts alone do not indicate success). Learning element — uses critic feedback to modify the performance element; its design depends on the performance element’s design. Problem generator — suggests exploratory actions leading to new, informative experiences, balancing short-run suboptimality against long-run discovery.
The policy π(s) is the performance element, selecting actions (exploit: argmaxa Q(s,a)). The critic receives (state, action, reward, next state) and produces an evaluation signal — the TD error δ, positive meaning “better than expected”, negative “worse”. The learning element uses δ to update internal parameters (Q-table or network weights), e.g. tabular Q-learning. The problem generator injects exploration into the policy (ε-greedy, softmax, UCB, curiosity bonuses), balancing exploration vs exploitation.
AI agents (LLM-based autonomous agents, generative agents) are agent systems whose reasoning core is a large language model, framed as cognitive architectures for language agents (Sumers et al., TMLR 2024). Their architecture is based on prompt-engineering techniques and runs a continuous cycle: perception (gather information from sensors, databases, UIs), reasoning (LLM analyzes the data and formulates solutions), planning (set goals, break them into steps), action (perform tasks, interact with systems), and reflection (evaluate results and adjust future plans).
MCP (Model Context Protocol, Anthropic, Nov 2024) is an open standard for how AI systems integrate and share data with external tools, systems and data sources — the agent–environment interface. A2A (Agent to Agent, Google) is an open standard for communication and collaboration between AI agents — the agent–agent interface. They are complementary because they standardize two orthogonal axes of the same system: tools/data access and inter-agent communication.
Because the same abstraction holds: an explicit goal (the instruction), an environment (tools, data sources, input schema), autonomy (the LLM decides the steps), perception (tools/sensors), and action (tool calls). The declarative LlmAgent example from Google ADK shows goals, action repertoires and interfaces reappearing in the agentic wave — the vocabulary of Chapter 12 remains the vocabulary of AI agents.