Is This the End of MCP for AI Agents?

Detailed Summary

Introduction to MCP Problems (0:00 - 2:08)

• MCP, despite its theoretical appeal, creates practical issues for AI agents, primarily due to poor context management.
• Connecting just two MCPs can consume nearly 20,000 tokens (about 10% of the context window) for tool definitions alone, even before any messages are sent.
• Intermediate results from tool calls further bloat the LLM's context, making management difficult.
• Anthropic highlights these issues: tool definitions overload the context window, and intermediate results consume additional tokens.
• For example, a two-hour sales meeting transcript returned by a tool could add approximately 50,000 tokens, which may not be entirely necessary for the agent's immediate task.

Code-First Agents as a Solution (2:08 - 4:45)

• The proposed solution is to build code agents that interact with tools independently through code, rather than directly using tools via MCP.
• This concept aligns with earlier research on executable code actions for better LLM agents and Cloudflare's "Code Mode."
• Cloudflare's approach involves converting MCP tools into TypeScript APIs, then having the LLM write code to call these APIs.
• This method allows agents to handle more numerous and complex tools, especially when chaining multiple tools together.
• Tools are treated as a directory structure, with each tool implemented in a separate file, functioning as an API call.
• Agents are equipped with bash and grep commands to discover relevant tools on demand, avoiding the need to load all tool definitions into the LLM's context.

Workflow and Architecture of Code Mode (4:45 - 7:48)

• The workflow begins with a user request, followed by the agent searching the directory structure to find relevant tools using code execution.
• The agent then writes code to execute these tools, allowing for concatenation of multiple tool calls without directly exposing intermediate results to the LLM's context.
• Data retrieval and further processing occur within this code execution environment.
• Only the final results and the steps taken are passed back to the LLM, preserving its context from noise.
• Traditional MCP servers provide tool schemas and definitions directly to the agent, which then decides and invokes the tool, polluting the context.
• In "Code Mode," the tool schema is used to create a list of API calls; the agent searches for needed tools, writes code for API calls, and an agent worker executes this code in a separate, dynamic isolate sandbox.
• This sandbox execution prevents the main agent's context from being polluted by execution details.

Benefits of Code-First Agents (7:48 - 9:17)

• Progressive Disclosure: Tool definitions are loaded on demand, leading to more efficient tool calls by not stuffing everything into the model's context.
• Privacy Preserving Operations: Security is enhanced as tool execution happens in an isolated sandbox, allowing for data anonymization (e.g., masking PII) before results are sent to the LLM.
• State Persistence and Skills: This approach supports new concepts like Anthropic's "skills," which also progressively load information, enabling code agents to interact directly with them.

The Evolving Role of MCP (9:17 - 12:07)

• Cloudflare suggests that MCP remains useful because it provides a uniform way to connect to and learn about APIs, even if not used for direct tool calling.
• MCP's primary benefit is offering a standardized RPC interface with attached documentation, allowing AI agents to interact with APIs uniformly, regardless of the specific server or agent developer.
• This uniform interface enables agentic frameworks to implement sandboxed execution, handling connectivity and authorization in a standard way, independent of the AI code.
• The speaker notes a personal shift towards using MCP less due to context bloat but acknowledges that the code-first approach can mitigate this issue.

Notable Quotes

"tool definitions overload the context window and then intermediate tool results consume additional tokens."
— [The presenter]

"the solution is not to directly use tools through MCP but rather build code agents that are going to interact with these tools independently through code."
— [The presenter]

"instead of putting all of the tool definitions in the context of the LLM, you basically search for the tools that are going to be needed, but you do that through code execution."
— [The presenter]

"all the execution and everything is happening within this dynamic isolate sandbox which is not rotting the context of your main agent."
— [The presenter]

"MCP is still useful because it's it is uniform. So MCP is designed for tool calling but it doesn't actually have to be used that way."
— [The presenter]

"It provides a uniform way to connect to and learn about an API."
— [The presenter]