Authoritative State Must Live Outside the Model Context

Few teams explicitly decide to use a model’s context window as their system of record. The architecture usually drifts there. A workflow begins with a prompt containing the current task, recent messages, tool results, and a compact summary of prior activity. The model then produces an updated plan or status description. That output is appended to the next call. Before long, the only complete account of what the system believes is true exists inside a chain of prompts and generated summaries.

The design can appear coherent while one agent, one team, and one execution path control the workflow. It breaks once the system must answer harder questions: What is the current state independently of the next model call? Which update superseded an earlier one? What did another agent observe? Which facts were retrieved, which were inferred, and which were generated? Can a human or service query the state without reconstructing a conversation?

If those questions can only be answered by replaying prompts, parsing model output, or asking another model to summarize the history, the context window has quietly become the database. The system lacks an authoritative state boundary.

State and context are related, but they do different jobs. A community-maintained overview in Awesome-LLM-based-AI-Agents-Knowledge describes state as the persistent and transient data an agent maintains over time, while context represents information relevant to the immediate conversation or task. That distinction is useful, but a production architecture needs a stricter definition.

Authoritative state is the system’s accepted account of entities, decisions, commitments, task status, permissions, versions, and material events. To function as authority, it must have four properties.

• Durable: it survives model calls, process restarts, retries, and changes of model or orchestration framework
• Canonical: the system has an explicit rule for which representation wins when observations or updates conflict
• Independently queryable: services, agents, operators, and auditors can inspect it without invoking a model or reconstructing a prompt
• Lifecycle-independent: it outlives any one context assembly, conversation turn, agent handoff, or generated summary

Model context has almost the opposite characteristics. It is selected for one operation, constrained by relevance and token budget, transformed into a model-compatible representation, and discarded or rebuilt after the call. It may contain authoritative facts, but containing a fact does not make the context authoritative. It is a view assembled from sources of record, instructions, recent events, retrieved knowledge, and temporary working material.

This leads to a clean architectural rule: the model may reason over a representation of state, but the representation in the prompt must never become the only place where that state exists.

State fragments across teams and tools

When each application composes its own prompts and retains its own conversation history, each application also develops its own version of reality. One agent may hold a summary saying a task is approved. Another may retrieve an earlier record saying it remains under review. A third may infer completion from a tool response that was never committed to shared storage.

DataHub’s discussion of enterprise context management makes the organizational consequence clear: application-specific context engineering does not by itself create shared, governed infrastructure. Relevance, provenance, access controls, and retention have to operate across applications. Otherwise, every team builds a local context pipeline around a local interpretation of the underlying information.

The duplicated engineering creates multiple operational truths. Teams can change prompt construction, summarization, or retrieval logic independently and thereby alter what their agents believe, even though no authoritative business record has changed.

Generated summaries replace accountable history

A summary is a lossy projection. It can be useful as model input, but it cannot safely substitute for the events from which it was derived. Summaries omit details, collapse uncertainty, and often erase the distinction between source facts and model inference. Repeated summarization compounds those losses.

Once a generated summary is treated as the current state, later decisions inherit its omissions. The system may still produce plausible outputs, but it cannot reliably explain why a particular fact was considered current or which source established it. Auditability then depends on recovering meaning from prompt traces rather than querying structured records and versioned events.

Multi-agent work exposes the transactional problem

The risk becomes sharper when several agents make interdependent changes. A workflow may reserve resources, update a plan, notify another service, and commit a financial or operational decision. If each agent carries its own prompt-resident state, no common mechanism ensures that all participants observe the same committed status or compensate coherently after a partial failure.

SagaLLM, by Edward Y. Chang and Longling Geng, addresses this class of problem by combining persistent memory, compensable execution, state tracking, and independent validation agents. Its significance here is architectural rather than framework-specific: distributed agent workflows require explicit mechanisms for consistency, recovery, constraint validation, and inter-agent coordination. A collection of context windows cannot supply those guarantees merely by carrying more history.

More tokens can preserve additional narrative, but narrative continuity is not transactional coherence. A longer prompt does not establish which operation committed, which update must be rolled back, or which version another agent is entitled to act upon.

The correct pattern has two layers: durable state outside the model and a controlled process that compiles a temporary context for the current operation.

The external state layer stores canonical entities, workflow status, event history, decisions, permissions, versions, source references, and other facts that must remain available beyond one invocation. The storage substrate depends on the workload, but its responsibilities do not. It must support the required query, concurrency, governance, and recovery semantics.

An Oracle Developers article comparing file systems and databases for agent memory distinguishes the interface presented to an agent from the storage substrate beneath it. Files may offer a convenient working interface, while shared state introduces requirements such as concurrency control, structured querying, auditability, and transactions. The vendor-specific recommendations are secondary to the general point: once memory becomes shared, durable, and operationally consequential, storage guarantees matter more than prompt convenience.

Above that store sits a retrieval and context-assembly layer. Its job is to select the smallest complete state slice needed for the current operation, not to reproduce the database inside the prompt — it applies authorization and provenance rules, resolves versions, transforms records into a model-usable form, and records what was supplied.

Google’s Agent Development Kit architecture describes this relationship as a separation between durable Session state and per-call working context. In the Google Developers Blog’s production architecture, sessions, memory, and artifacts act as state sources, while processors compile a working context for one invocation. The working context is recomputed and disposable; the underlying state remains independently available.

This pattern creates several important freedoms. Storage schemas can evolve without treating prompt text as a migration format. Context formatting can change without rewriting history. A different model can receive a different representation of the same canonical facts. Retrieval decisions can be tested and observed separately from model reasoning. Most importantly, state changes can be committed through explicit services and validation rules rather than accepted because a model said they occurred.

The model’s output should therefore be treated as a proposed action, interpretation, or state transition. A deterministic or separately governed component validates that proposal, applies the update to authoritative storage, records the event, and makes the new state available to subsequent operations. The next context is then assembled from the committed record, not from confidence in the previous model’s prose.

The simplest test is operational: stop the agent immediately after a model call and discard its working context. Can the system still determine the current state, explain how it arrived there, and construct the next valid operation from durable records?

A system has probably collapsed state into context when any of the following is true:

• The latest prompt or generated summary is the only complete description of workflow status
• Another service must call a model to discover what the system currently believes
• Retries depend on resending the full conversation because committed state cannot be queried directly
• Two agents can hold contradictory status without a canonical version or conflict-resolution rule
• Important updates are accepted by appending model output rather than validating and committing a defined state transition
• An audit requires interpreting prompt logs instead of inspecting versioned records, events, and source provenance
• Changing summarization logic can retroactively change the system’s effective memory

When these signals appear, the first correction is not a larger context window or a more sophisticated memory prompt. Identify the minimum set of facts and events that the system must be able to defend as true. Assign ownership for those records. Define their canonical representation, versioning, write path, validation rules, and query interface. Then rebuild context assembly as a consumer of that state rather than its custodian.

The final test is equally direct: every material fact supplied to the model should have a traceable origin, and every consequential state change proposed by the model should pass through an explicit commit path. Once both conditions hold, context becomes what it should have been all along—a temporary reasoning surface over a system whose truth exists elsewhere.

• Part 1: Architectural Principles for Token-Efficient Agentic Systems
• Part 2: Minimal and Complete Model Context Design
• Part 4: Normalized Entities and Context Slices Beyond Chunk Retrieval
• Part 5: Preserving Exact Source Wording in LLM Systems
• Part 1: Governing the Digital Investigation Lifecycle Across Systems and Teams
• Part 2: From UAE and Dubai Cybersecurity Controls to Operational Investigation Evidence
• Part 3: A Vendor-Neutral Event Architecture for Digital Investigation Governance
• Part 4: Human Authority in AI-Assisted Digital Investigation Governance

Browse the full Architectural Insights series →

• What is State and Context in LLM-Based AI Agents?
• Context Management: The Missing Piece for Agentic AI
• Comparing File Systems and Databases for Effective AI Agent Memory Management
• Architecting Efficient Context-Aware Multi-Agent Framework for Production
• SagaLLM: Context Management, Validation, and Transaction Guarantees for Multi-Agent LLM Planning