Architectural Principles for Token-Efficient Agentic Systems

In an isolated model call, excess context can look like a cost problem. In an agentic system, it becomes a control problem. Every additional instruction, retrieved passage, tool result, message, entity record, and intermediate observation changes the informational environment in which the model chooses its next action. The consequences compound across an agent trajectory: irrelevant material competes with relevant evidence, stale state can survive into later decisions, and provenance becomes harder to trace when facts and interpretations are repeatedly compressed or copied.

This is why context design can no longer be treated as prompt-writing hygiene. LangChain describes context engineering as filling the context window with the right information at each step of an agent trajectory. Anthropic defines the discipline more broadly as curating and maintaining the optimal set of tokens used during inference. Both formulations shift attention from the wording of one instruction to the system that selects, structures, updates, isolates, and preserves information over time.

For architects, the practical question is whether the system can justify each token presented for the next operation. A context window is capacity. Context architecture determines what occupies that capacity, where the material came from, which version is authoritative, and what the model is expected to do with it.

A larger window removes one hard constraint: the system can submit a longer input without truncating it. It does not establish that the model will use every part of that input consistently. The distinction matters because many agent architectures still behave as though availability and effective use were equivalent. They retain an expanding transcript, append retrieved material, add tool outputs, and assume that important information remains equally accessible because it remains technically present.

The evidence does not support that assumption. In Lost in the Middle, Nelson Liu and his co-authors found that model performance could change substantially according to the position of relevant information. Performance was often strongest when that information appeared near the beginning or end of the input and weaker when it appeared in the middle. The resulting U-shaped pattern appeared even in models designed for long contexts.

Chroma’s context-rot research extends the concern beyond positional bias. Across controlled experiments, performance degraded as input length increased, while semantic similarity, distractors, context structure, and model family affected the rate and shape of that degradation. The impact was non-uniform: extra tokens did not impose a predictable accuracy tax that could be handled through a simple threshold or safety margin.

Those findings change the architectural interpretation of context length. The maximum supported window is an upper operating limit, not a target payload size and not a guarantee of uniform attention. Filling the window because capacity exists is comparable to sending every available table, log, and configuration object into a transaction because the network can carry them. The system may remain within its technical limits while becoming less reliable.

The failure mode is often described as context rot, but the architectural issue begins earlier. Rot becomes possible when context grows through accumulation rather than deliberate assembly. Once a working context contains current state, superseded state, source evidence, summaries, speculative notes, tool errors, and prior model interpretations in one undifferentiated stream, the system has already surrendered important control over what the model treats as relevant or authoritative.

The correct unit of design is the next operation, not the conversation or the agent as a whole. A planning step, an entity-resolution step, a tool-selection step, and a final-response step may belong to one workflow, but they do not require identical information. Each should receive a context artifact assembled for its own decision.

That artifact should have an explicit contract. The contract identifies the operation’s objective, required instructions, authoritative state, relevant evidence, available tools, output constraints, and provenance requirements. It also defines what must be excluded. Exclusion prevents unrelated or outdated information from acquiring accidental influence simply because it was nearby in the agent's history, which makes it more than an optimization technique.

Anthropic’s guidance on effective context engineering emphasizes finding the smallest set of high-signal tokens that maximizes the likelihood of the desired outcome. That is an architectural objective rather than an instruction to make every prompt short. Some operations legitimately require extensive evidence. The requirement is that the evidence be necessary, well-structured, and distinguishable from instructions, state, and prior interpretation.

This reframe separates the system of record from the model’s working view. The full workflow history may remain available to orchestration, observability, audit, and recovery components. The model does not need that entire history on every turn. A context builder can instead produce a controlled slice from durable state, selected evidence, and operation-specific constraints. The context is then a compiled execution artifact, not a serialized dump of everything the system remembers.

The quality of that artifact can be reviewed directly. Vera Vishnyakova’s context-engineering framework proposes five useful criteria: relevance, sufficiency, isolation, economy, and provenance. Together they expose the trade-off more accurately than token count alone. Context must be economical without becoming incomplete, isolated without losing necessary dependencies, and relevant without obscuring where its claims originated.

Minimal and complete model context

Minimal context is the smallest context that still preserves every dependency needed for a reliable operation — shorter than that produces an answer only by accident. Completeness therefore constrains reduction. A context builder should remove material that does not affect the decision while retaining the instructions, facts, relationships, constraints, and provenance the model needs to act correctly. The design task is to reduce noise without creating hidden informational gaps.

Authoritative state and model context separation

The model’s context should not become the authoritative store for workflow state. Conversations and summaries are lossy, mutable representations. Durable state belongs in controlled system components with explicit schemas, versioning, ownership, and update rules. The model receives a projection of that state for the current operation and returns a proposed result or state transition. This separation limits the damage from stale context, hallucinated state, or an earlier summary that silently omitted a critical field.

Normalized entities and context slices

Agent workflows frequently refer to the same customer, policy, task, document, or decision through multiple tools and messages. Without normalized identity, the context accumulates duplicate descriptions and inconsistent versions. A normalized entity model gives the control plane stable identifiers and structured relationships. Context slices can then be generated from those canonical entities according to the operation, rather than reconstructed from prose each time the agent needs them.

Preservation of exact source wording

Summaries are useful for orientation, but they should not replace exact source text when wording carries legal, technical, policy, or evidentiary significance. Repeated summarization can erase qualifications, change modality, and detach a claim from its source. The architecture should preserve exact passages alongside derived summaries and expose the original wording whenever the operation depends on interpretation, verification, quotation, or compliance with a stated constraint.

Once context is treated as a designed artifact, several responsibilities move out of the prompt and into the system architecture. The control plane must know which state is authoritative, which operation is executing, what evidence it requires, what context policy applies, and which transformations produced the model-visible representation. Retrieval becomes one input to context construction rather than the whole solution. Memory becomes a governed source, not an ever-growing transcript. Summarization becomes a traceable transformation, not a substitute for retained evidence.

This also changes observability. Token counts remain useful, but they are not sufficient. Teams need to inspect context composition: which sources were selected, which versions were used, what was omitted, how much of the payload consisted of instructions versus evidence, and whether a failure originated in reasoning or in the context supplied to the model. Without that distinction, evaluation results collapse model behavior and architecture behavior into one opaque metric.

The same principle applies to multi-agent handoffs. Passing a full originating context to every downstream agent preserves volume rather than intent. A handoff should define the receiving operation, the state transition already completed, the evidence that remains relevant, unresolved constraints, and the provenance of any derived conclusion. That design creates explicit interfaces between agents instead of relying on conversational continuity as an integration mechanism.

Architects should therefore review context handling before expanding agent count, workflow length, or tool access. More capable models and larger windows may increase the system’s operating range, but they do not replace decisions about state ownership, information selection, transformation, and control. Scaling an agent architecture without those decisions increases the number of places where ambiguous context can influence an action.

• Part 2: Minimal and Complete Model Context Design
• Part 3: Authoritative State Must Live Outside the Model Context
• Part 4: Normalized Entities and Context Slices Beyond Chunk Retrieval
• Part 5: Preserving Exact Source Wording in LLM Systems
• Part 1: Governing the Digital Investigation Lifecycle Across Systems and Teams
• Part 2: From UAE and Dubai Cybersecurity Controls to Operational Investigation Evidence
• Part 3: A Vendor-Neutral Event Architecture for Digital Investigation Governance
• Part 4: Human Authority in AI-Assisted Digital Investigation Governance

Browse the full Architectural Insights series →

• Anthropic, Effective context engineering for AI agents
• Chroma Research, Context Rot: How Increasing Input Tokens Impacts LLM Performance
• LangChain, Context Engineering
• Liu et al., Lost in the Middle: How Language Models Use Long Contexts
• Vishnyakova, Context Engineering: From Prompts to Corporate Multi-Agent Architecture