The world of AI development is moving quickly. In fact, we have gone from simple chatbots to advanced, autonomous agents in just a few years. As a result, these new agents can now tackle complex, long-term jobs—they automate workflows, perform deep research, and manage intricate systems. However, this ambition brings a major challenge: managing context effectively.
Takeaways 🚀
✅AI evolves beyond simple chatbots.
✅Context management ensures scalable systems.
✅ADK separates storage from presentation.
✅Memory retrieves knowledge on demand.
✅Multi-agent systems require scoped context.
As AI agents run for longer, the amount of information they must track explodes. For example, this includes chat histories, tool outputs, and external documents. Previously, the solution was simply to give AI models bigger context windows. Nonetheless, this is not a sustainable strategy for scaling up. In order to build reliable and efficient AI for production, we need a new approach called context engineering.
In this post, we will explore how a structured approach to context management is key to building next-generation AI. Specifically, we will cover:
- The main bottlenecks that limit current AI systems.
- A new way to think about context as a structured system.
- How to build efficient, context-aware multi-agent frameworks.
The Bottleneck of AI Context
At first glance, simply giving an AI a large context window can help. Yet, it doesn’t solve every problem. The common pattern of dumping all information into one giant prompt creates significant issues. This, in turn, leads to a three-way pressure that causes systems to collapse.
1. Spiraling Costs and Latency
To begin with, model costs and response times grow with the size of the context. Whenever you feed raw history and detailed tool data into the window, agents become slow and expensive. Thus, they become impractical for many real-world uses.
2. Signal Degradation
Next, a cluttered context window can distract the model. Irrelevant logs or old information may cause the AI to focus on the wrong things. Consequently, to ensure good decision-making, we must provide the model with dense, relevant information. This phenomenon is often called the “lost in the middle” problem.
3. Physical Limitations
Finally, real-world tasks can quickly overwhelm even the largest context windows. In many cases, information from research, long conversations, and other data sources eventually exceeds the fixed limits of the model.
While throwing more tokens at the problem is a temporary fix, it’s not enough. To truly scale, we must change how we represent and manage context, not just how much we can fit into a single call.
A New Thesis: Context as a Compiled View
In the past, older AI frameworks often treated context like a simple string of text. However, the Google Agent Development Kit (ADK) introduces a new idea: context is a compiled view of a richer, stateful system.
In this improved model:
- Sources: Sessions, memory, and files serve as the original sources of information. They hold the full, structured state of an interaction.
- Compiler Pipeline: Flows and processors act like a compiler, transforming the raw state into a refined output.
- Compiled View: The working context is the final, optimized view sent to the Large Language Model (LLM) for a specific task.
By shifting context engineering from simple prompt writing to systems engineering, we are forced to ask important questions about how to process, compact, and observe transformations. Furthermore, this approach delivers much greater flexibility and control.
The Tiered Model for Structure
Early AI systems often used a single window for all context. In contrast, ADK uses a multi-layered approach. It separates storage from presentation and organizes context into four distinct layers. Each layer plays a specific role.
- Working Context: This is the immediate prompt for a single model call. It contains system instructions, agent identity, and selected history. Since it’s ephemeral, configurable, and model-agnostic, it supports future changes easily.
- Session: This is the permanent log of the interaction. It captures every user message, agent reply, and tool call as structured
Eventobjects, providing a ground truth for the conversation. - Memory: This layer holds long-term, searchable knowledge. For instance, it includes user preferences and facts from past conversations that outlive a single session.
- Artifacts: This is where large data files, such as PDFs or logs, are referenced by name rather than pasted directly into the prompt.
Through this tiered structure, you gain a strong foundation for managing context efficiently.
Using Pipelines to Process Context
Once you separate storage from presentation, you need a way to “compile” one into the other. For this reason, ADK uses LLM Flows, which are ordered lists of processors that build the final context.
Importantly, each processor builds on the output of the previous one. This design creates natural points to insert custom filters, compaction strategies, or caching. Rather than rewriting a giant prompt template each time, you can simply add or reorder these processors.
Making Context Relevant
With a structured system in place, the next challenge is relevance. In other words, we need to decide what specific information belongs in the model’s active window at any given moment.
ADK addresses this by combining human knowledge with agent decision-making. Engineers define the architecture and where data lives. Then, the AI agent intelligently decides when to retrieve specific memories or files to handle the current user request.
Managing Large Data with Artifacts
A common mistake is “context dumping,” where large files are placed directly into the chat history. As a result, this inflates costs and buries important instructions.
To manage this, ADK treats large data as Artifacts—named objects handled by a separate service. The agent sees only a lightweight reference by default. When the agent needs the raw data, it uses a tool to load the artifact temporarily into the working context. Afterwards, the artifact is removed, keeping the context window lean.
Retrieving Long-Term Knowledge from Memory
While Artifacts handle large files, the Memory layer manages long-term knowledge. This information, such as user preferences or domain facts, must always remain searchable and retrievable on demand.
An agent can access this knowledge in two ways:
- Reactive Recall: The agent recognizes it needs information and calls a tool to search its memory.
- Proactive Recall: The system automatically searches for relevant information based on user input and injects it into the context.
Through this “memory-based” workflow, agents can access exactly what they need without being weighed down by all past conversations.
Managing Context in Multi-Agent Systems
Although single-agent systems can struggle with context bloat, multi-agent systems make the problem even worse. For example, if a main agent passes its full history to a sub-agent, the context can quickly explode.
ADK addresses this by allowing you to explicitly scope what a sub-agent sees. Specifically, you can choose to pass only the latest user query and a specific file, while hiding the rest of the history. Needless to say, this keeps sub-agents focused and efficient.
Additionally, when control passes from one agent to another, ADK can reframe the conversation. It ensures the new agent sees a coherent history from its own point of view. In doing so, it prevents confusion and allows each agent to operate correctly within its role.
Ultimately, as we ask AI agents to handle more complex and long-running tasks, we can no longer treat context management as simple string manipulation. Instead, it must become a core architectural concern.
By adopting a structured approach—with tiered storage, compiled views, and strict scoping—you can overcome these challenges. In turn, this type of rigorous systems engineering will move AI agents from interesting prototypes to scalable, reliable production systems that can solve real-world problems.
