An LLM does not need to understand MCP

I think MCP has legs well beyond just the LLM / agent world. Just like USB went from "how I connect my mouse" to "how I charge my beard trimmer."

In fact, I imagine it's going to go full-duplex with all our systems, becoming a more standard way for systems to communicate with each other.

Under the hood, MCP is just JSON RPC, which is a fine format for communicating between systems.

MCP layers on some useful things like authentication and discovery. Both are critical to any kind of communication between systems built by different authors (e.g. various apps and services). Discovery, especially, is the fascinating part. Rather than hoping an OpenAPI spec exists and hoping it's right, MCP has this exchange of capabilities baked in.

I spent the last 9 years building integration technology, and from that perspective, the discovery-documentation-implementation problem is the core issue.

Right now, LLMs basically "solve" the integration problem because they can do the mapping between external tools/resources/formats and internal ones.

But there's nothing that strictly "requires" an LLM to be involved at all. That's just the primary reason to develop MCP. But you could just as well use this as a way for integrating systems, making some bets on interface stability (and using LLMs for cases only when your prior expectations no longer hold and you need a new mapping).

The comparison is perhaps imperfect and overused, but I feel like we're witnessing the birth of a new USB-like standard. There's something right now that it was designed to do, but it's a decent enough standard that can actually handle many things.

I wouldn't be surprised if in some period of time we see enterprise apps shift from REST to MCP for bi-directional integrations.

For the OP, I'm not sure if you're working on an MCP proxy (A) as a commercial offering, (B) as something for your team to use, closed source, or (C) as something open source for fun. But we just built and started selling an MCP proxy/gateway. It handles identities for humans & bots, tool allowlists, and policy setting for an org.

If you don't want to build something on your own because of option B above, get in touch.

MCP is a means of communicating information about externally-defined tools to the “application wrapper” (and your examples of “generic chatbots” are also application wrappers). Well, between the application wrapper and servers; “application wrappers” for LLMs are pretty much the motivating (but not sole) case of MCP Clients.

Without something like MCP, each application wrapper is left do do its own ad hoc wrappers for external tools (tools internal to the wrapper don’t use MCP.) With MCP, it just integrates an MCP client library, and then it can use any tool, resource, or prompt provided by any MCP server available to it.

In my opinion, I get the desire to create some sort of specification for an LLM to interface with [everything else], but I don't really see the point at doing it on an inference level by smashing JSON into the context.

These models are usually very decent at parsing out stuff like that anyway; we don't need the MCP spec, everyone can just specify the available tools in natural language and then we can expect large param models to just "figure it out".

If MCP had been a specification for _training_ models to support tool use on an architectural level, not just training it to ask to use a tool with a special token as they do now.

It's an interesting topic because it's the exact same as the boundary between humans (sloppy, organic, analog messes) and traditional programs (rigid types, structures, formats).

To be fair if we can build tool use in architecturally and solve the boundary between these two areas then it also works for things like objective facts. LLMs are just statistical machines and data in the context doesn't really mean all that much, we just hope it is statistically relevant given some input and it is often enough that it works, but not guaranteed.

Personally I find LLMs functionally useless without any external data besides than which I write in the prompt.

One MCP that I use is as simple as todays date and time - how else would LLMs know what day of the week it is?

MCP sucks because it has to be connected to a desktop client. I'd love to build some MCP-like integrations but no one on my team can use them. We use LLM's via - as you noted - other means like via Notion, via web UI, via our own API integrations. Until there is a more central way to connect these things - yeah they won't reach mass adoption.

What about LangChain makes more sense? It’s one of the most prematurely complex libs I’ve seen. I’m calling it right now, LangChain is going to run a mind fuck on everyone and convince people that’s actually how complicated orchestrating LLM control flow should be. The community needs to fight this framework off.

That’s besides the point. MCP servers let you discover function interfaces that you’ll have to implement yourself (in which case, yeah, what’s the point of this? I want the whole function body).

[dead]

MCP is just packaging. It's the ideal abstraction for building AI applications.

I think it provides the similar benefits of decoupling the front and back end of a standard app.

I can pick my favorite AI "front end"- whether that's in my IDE as a dev, a desktop app as a business user, or on a server if I'm running an agentic workflow.

MCP allows you to package tools, prompts, etc. in a way that works across any of those front ends.

Even if you don't plan on leveraging the MCP across multiple tools in that way- I do think it has some benefits in de-coupling the lifecycle of the tool development from the model/ UI.

The auth story for MCPs is a complete mess right now, though, which is why people make ones to run locally.

This is 100% a problem with the MCP spec: it does not currently provide a way to narrow what tools, and therefore context, flow into the LLM.

I don't really think there's an easy solution at the protocol level, since you can't just make the LLM say what tools it wants upfront. There's a whole discovery process during the handshake:

LLM(Host): Hi, I'm Claude Desktop, what do you offer?

MCP Server: Hi, I'm Salesforce MCP, I offer all these things: {...tools, prompts, resources, etc.}

Discoverability is one of the reasons MCP has a leg up on traditional APIs. (Sure, OpenAPI helps, but it's not quite the same thing.)

I'd be interested in hearing other recommendations or ideas, but when I saw this, I realized that the spec effectively necessitates a whole new layer exist: the gateway plane.

Basically, you need a place where the MCPs can connect & expose everything they offer. Then, via composability and settings, you can select what you want to pass through to the LLM (host), given the specific job it has.

I basically pivoted my company to start building one of these, and we're getting inundated right now.

This whole thing reminds me of the early web days, where the protocols and standards were super basic and loose, and we all just built systems and tools to fill those gaps. Just because MCP isn't "complete" doesn't mean it's not valuable. In fact, I think leaving some things to the community & commercial offerings is a great way for this tech to keep winning.

Came here to say this: people present MCP’s verbosity as all the context the LLM needs. But almost always, this isn’t the case.

I wrote recently, “ Connecting your model to random MCPs and then giving it a task is like giving someone a drill and teaching them how it works, then asking them to fix your sink. Is the drill relevant in this scenario? If it’s not, why was it given to me? It’s a classic case of context confusion.”

https://www.dbreunig.com/2025/07/30/how-kimi-was-post-traine...

Perhaps tools trained into the model rather than exposed through prompting would mitigate the performance hit (but might affect model quality?).

Can you elaborate on how the agents degrades from more tools? By paralysis or overuse? Isn’t this both ways a function of direction on correctly instructing which to use when? Tnx

How can an LLM “talk” to another LLM, except by emitting tokens in its output stream?

You can name the mechanism whatever you want, but the models don’t have hands. Tool calling conventions (as a concept, or as a spec) is what gives the model hands!

> writing bash commands into my terminal

This is what the author means by "knowing how to use the tool". The LLM alone is effectively a function that outputs text, it has no other capabilities, it cannot "connect to" or "use" anything by itself. The closest it can come is outputting an unambiguous, structured text request that can be interpreted by the application code that wraps it and does something on its behalf.

The author's point hinges on the architectural distinction between the LLM itself and that application code, which is increasingly irrelevant and invisible to most people (even developers) because the application code that knows how to do things like call MCP servers is already baked in to most LLM-driven products and services. No one is "talking directly to" an LLM, it's all mediated by multiple layers, including layers that perform tool calling.

I think what your quote is trying to say essentially boils down to: LLMs can be given facts in the context, we _hope_ that the statistical model picks up on that information/tool calls but it isn't _guaranteed_.

Unlike human beings such as yourself (presumably), LLMs do not have agency, they do not have conscious or active thought. All they do is predict the next token.

I've thought about the above a lot, these models are certainly capable of a lot, but they do not in any form or fashion emulate the consciousness that we have. Not yet.

A lot of people resist the idea that programming is intrinsically mathematical, but this is one of the places it pops out. The power of programming lies precisely in the way it brings together text that "represents" something with text that "does" something. That is, at the core, the source of its power. You can still draw the distinction philosophically, as you just did, but at the same time there is also a profound way in which there is in fact no difference between "using" computers and "representing" your use of computers.

I think you might be missing the point of this quote, which is that you don't have to introduce additional code into the model to support MCP.

MCP happens at a different layer. You have to run the MCP commands. Or use a client that does it for you:

> But the LLM will never know you are using MCP, unless you are letting it know in the system prompt of tool definitions. You, the developer, is responsible for calling the tools. The LLM only generates a snippet of what tool(s) to call with which input parameters.

The article is describing how MCP works, not making an argument about what it means to "understand" something.

10000% this

Agreed.

Context engineering is "just" prompt engineering for LLMs with tool use: it extends the concerns of prompt engineering with the concern of setting up an environment in which tools can be used, and how the LLM can most effectively interact with the environment.

I think you're mixing up tool calling and structured outputs.

You can have both of those or either without MCP.

MCP just standardizes the tool calling and only makes sense if you want to share your tools across the org. I wouldn't use it for simple functions like getting current date for e.g.

I don’t understand. LLMs cannot connect to MCP servers directly they would always need a client (like a chat app or agent) to call the servers. Where are you calling your LLMs from?

It shouldn't care about the format, true. But the LLM needs a mechanism to be able to connect to that tool from a sandboxed environment. MCP is the glue between the LLM and the actual tool. Technically you can expose a full HTTP proxy via an MCP so that your LLM has access to the whole Internet.

What's your preferred way, based on your experience?