Why Agentic Systems Need a Theory of State Beyond the Reward Function

Much of the evaluation machinery around agentic systems still concentrates on a familiar question: did the system produce an acceptable result? That remains necessary. A system that produces wrong answers, unsafe actions, or unusable artifacts has failed in an obvious sense. The problem is that output quality becomes an increasingly incomplete description of reliability as an agent operates for longer periods, accumulates memory, calls tools, modifies external systems, and carries decisions from one step into the next.

In those settings, two agents can produce the same satisfactory output at one checkpoint while occupying very different internal conditions. One may still be exploring alternatives, preserving uncertainty, and retaining enough context to change course. The other may have accumulated commitments, discarded contrary evidence, exhausted much of its tool budget, or written assumptions into memory that will influence later decisions. A point-in-time output test can treat those cases as equivalent even though their future behavior may diverge sharply.

That creates an architectural problem. Reliability over a long horizon depends partly on the states through which the system moves, the pressures acting on those states, and the constraints that determine which future transitions remain available. An evaluation regime centered on outputs observes the visible result of this process. It does not necessarily explain how the system arrived there or what the current condition makes likely next.

For architects, the useful unit of reasoning therefore expands from action quality to behavioral trajectory. Internal state becomes a first-class concern because memory contents, unresolved goals, prior tool results, resource consumption, accumulated commitments, and active constraints all influence what the agent can sensibly do next.

The familiar policy framing is useful because it compresses a decision: given a represented state, select an action. It is not inherently static. In formal sequential decision models, state can encode history and a policy can operate over a dynamical process. The architectural problem in production agent systems is different: their effective state is often distributed across model context, memory stores, orchestration state, tool side effects, budgets, external commitments, and the surrounding environment, while evaluation still concentrates on the action or output observed at a checkpoint.

Consider an agent that has spent twenty steps pursuing one implementation path. Its next decision depends on more than the current task description. It may carry evidence gathered through tools, assumptions written into working memory, a plan that has already consumed significant compute, and prior actions that changed the environment. Some alternatives that were cheap at the beginning may now be expensive. Others may have disappeared because the agent has altered files, created tickets, sent messages, or committed external resources.

The architectural questions are temporal. How strongly should a plan persist once the system has invested effort in it? What happens when objectives pull in different directions? Which constraints dominate when time, tool calls, context capacity, and task requirements compete? Does the behavior converge toward a stable operating condition, oscillate between alternatives, or drift as accumulated state changes the effective decision surface?

A reward or scoring function can express preference over outcomes or actions, and a sufficiently rich state representation can encode path dependence. The problem is treating either abstraction as a sufficient architecture model when the variables that actually drive behavior are fragmented, hidden, or governed by different components. The same apparent objective can produce very different behavior depending on memory design, resource limits, commitment mechanisms, tool effects, and what the architecture permits the system to reconsider. Those mechanisms belong in the behavioral model rather than being treated as incidental implementation detail.

An intellectual catalyst for this framing comes from Jon Doyle's 2006 book Extending Mechanics to Minds: The Mechanical Foundations of Psychology and Economics. Written long before today's wave of LLM-based agents, the book's documented project investigates whether mechanics can be generalized beyond its traditional physical domain to the treatment of minds. For this series, the value of that project is the provocation rather than a ready-made agent architecture: behavioral systems can be examined in terms of state, persistence, pressures, constraints, and change over time.

Saying that an agent has "momentum" is easy metaphor. The stronger architectural move is to ask what in the system actually persists, what changes it, what resists change, and what limits the reachable next states. Those questions become useful when they determine what the architecture represents, observes, governs, and tests.

The framework developed in this series is an independent synthesis for contemporary tool-using agents: treat the agent as a dynamical system whose effective state changes over time under multiple pressures and constraints. Doyle supplies an intellectual precedent for mechanics-style reasoning about minds; the state, commitment, competing-objective, and trajectory model that follows is developed here for agent architecture.

Once the system is viewed this way, familiar components take on a different role. Memory is part of the mechanism through which prior states affect later ones. Planning introduces commitments whose persistence needs to be governed. Tool use couples internal decisions to an external environment that can change irreversibly. Objective hierarchies create simultaneous pressures rather than a single scalar instruction. Context management determines which parts of prior state remain available to influence the next decision.

This does not require importing physical equations into an agent runtime. The architectural gain comes earlier: state transitions become objects of design. The team can ask which state variables matter, which should be observable, which effects should persist, what conditions should trigger reconsideration, and which constraints should be represented explicitly rather than discovered through failure.

The mechanics framing becomes especially useful around bounded rationality. Doyle's account places limited time, information, attention, and computational capacity inside the description of the agent. For an agentic AI system, the corresponding engineering constraints are familiar: finite context windows, inference and compute budgets, latency requirements, tool-call limits, incomplete observations, and finite opportunities to search or revise.

Architectures often treat these limits as degradations relative to an imagined ideal agent. That assumption can distort design. A production agent will always operate under some combination of finite time, finite compute, partial information, access restrictions, and external deadlines. Increasing one resource changes the operating envelope; it does not eliminate the need to decide how the system behaves under constraints.

A context limit, for example, affects which history remains causally available to the next decision. A tool-call budget changes the cost of further verification and may force a choice between additional evidence and task completion. A latency target can make exhaustive reconsideration impossible. Compute allocation influences how much search, critique, or replanning can occur before an action must be selected. These are behavioral parameters because they affect the trajectory the agent can follow.

The architectural response is to design the resource regime deliberately. Teams need to know which decisions deserve additional search, which state must survive compression, how uncertainty is represented when verification stops, and what the agent should do when its remaining resources are inconsistent with the plan it is pursuing. Resource exhaustion should have defined behavioral consequences rather than arriving as an accidental edge condition.

This also changes evaluation. A useful test does more than ask whether an agent completed a task under one generous configuration. It examines how behavior changes as available context, time, tools, or compute tighten. The object of interest is the transition pattern: whether the agent degrades predictably, becomes trapped in an old commitment, abandons goals too readily, or begins making actions whose apparent local quality hides a deteriorating internal condition.

Treating agent behavior as a trajectory brings three design questions into sharper focus. Each concerns how a system changes over time rather than whether one isolated action was acceptable.

Goal persistence and inertia. An agent can keep repairing a failing implementation because prior planning, memory, and accumulated work continue to reinforce the existing path even after evidence favors abandoning it. The design problem is deciding what persistence should survive new evidence and what conditions should make a plan easier to leave.

Competing objectives as forces. An agent asked to complete a task quickly, verify its work, minimize tool usage, respect policy, and avoid unnecessary changes operates under pressures that may point toward different actions. The architecture must determine how those pressures interact over time, especially when satisfying one objective changes the state relevant to the others.

Stability of trajectories. An agent can produce individually plausible steps and still enter an undesirable pattern: repeated replanning, oscillation between strategies, progressive context degradation, or increasing commitment to a mistaken premise. Reliability then depends on whether the system tends toward useful operating conditions and whether departures from them are detectable and recoverable.

These questions require different mechanisms and deserve separate treatment. Their common foundation is a state-based view of agent behavior. Once internal trajectories are treated as architectural objects, persistence, competing pressures, and stability become concrete design concerns that can be specified, observed, and tested before surprising behavior appears in production.

Organizations building agentic systems across multiple teams or platforms can easily end up with separate approaches to memory, planning, tool governance, resource limits, evaluation, and recovery, even when those mechanisms jointly determine system behavior. Architecture leadership in this setting means establishing a coherent technical model for those cross-domain decisions and a governance route for resolving them when local design choices interact. DUNNIXER's architecture-leadership work focuses on that kind of technical coherence and decision governance: making the assumptions that shape agent behavior explicit enough to be reviewed consistently across architectures, delivery teams, and platforms.

This is part 1 of 4 in the series The Mechanics of Agentic Behavior.

• Part 2: Goal Persistence as Inertia: When an Agent Should Abandon a Failing Plan
• Part 3: Competing Objectives as Forces: Designing Agents Under Multiple Pressures
• Part 4: Correct Actions, Unstable Trajectories: Reasoning About Agent Behavior Over Time

Browse the full Architectural Insights series →

• Jon Doyle, Extending Mechanics to Minds: The Mechanical Foundations of Psychology and Economics, Cambridge University Press, 2006.