Supabase MCP can leak your entire SQL database

That "problem" remains unsolved because it's actually a fundamental aspect of reality. There is no natural separation between code and data. They are the same thing.

What we call code, and what we call data, is just a question of convenience. For example, when editing or copying WMF files, it's convenient to think of them as data (mix of raster and vector graphics) - however, at least in the original implementation, what those files were was a list of API calls to Windows GDI module.

Or, more straightforwardly, a file with code for an interpreted language is data when you're writing it, but is code when you feed it to eval(). SQL injections and buffer overruns are a classic examples of what we thought was data being suddenly executed as code. And so on[0].

Most of the time, we roughly agree on the separation of what we treat as "data" and what we treat as "code"; we then end up building systems constrained in a way as to enforce the separation[1]. But it's always the case that this separation is artificial; it's an arbitrary set of constraints that make a system less general-purpose, and it only exists within domain of that system. Go one level of abstraction up, the distinction disappears.

There is no separation of code and data on the wire - everything is a stream of bytes. There isn't one in electronics either - everything is signals going down the wires.

Humans don't have this separation either. And systems designed to mimic human generality - such as LLMs - by their very nature also cannot have it. You can introduce such distinction (or "separate channels", which is the same thing), but that is a constraint that reduces generality.

Even worse, what people really want with LLMs isn't "separation of code vs. data" - what they want is for LLM to be able to divine which part of the input the user would have wanted - retroactively - to be treated as trusted. It's unsolvable in general, and in terms of humans, a solution would require superhuman intelligence.

--

[0] - One of these days I'll compile a list of go-to examples, so I don't have to think of them each time I write a comment like this. One example I still need to pick will be one that shows how "data" gradually becomes "code" with no obvious switch-over point. I'm sure everyone here can think of some.

[1] - The field of "langsec" can be described as a systematized approach of designing in a code/data separation, in a way that prevents accidental or malicious misinterpretation of one as the other.

https://cwe.mitre.org/data/definitions/990.html

Others Have pointed out one would need to train a new model that separated code and data because none of the models have any idea what either is.

It probably boils down a determistic and non deterministic problem set, like a compiler vs a interpretor.

You can have the MCP talking directly to the database if you want! You just can't have it in this configuration of a single context that both has all the tool calls and direct access to untrusted data.

I don't know where "more agents" is coming from.

Yes, the vast amount of effort, time and money spent on making the world secure things and checking that those things are secured now being dismissed because people can't understand that maybe LLMs shouldn't be used for absolutely everything.

How to spell job security in a roundabout way.

Late stage grift economy is a weird parallelism with LLM State of art bullshit.

> But that's not to say that they wouldn't be capable of adding safeguards on their end, not even on their MCP layer. Adding policies and narrowing access to whatever comes through MCP to the server and so on would be more assuring measures than what their comment here suggest around more prompting.

This is certainly prudent advice, and why I found the GA example support application to be a bit simplistic. I think a more realistic database application in Supabase or on any other platform would take advantage of multiple roles, privileges, Row Level Security, and other affordances within the database to provide invariants and security guarantees.

Yes. "Writing tools to parse the output" is the work, like in any application connecting untrusted data to trusted code.

I think people maybe are getting hung up on the idea that you can neutralize HTML content with output filtering and then safely handle it, and you can't do that with LLM inputs. But I'm not talking about simply rendering a string; I'm talking about passing a string to eval().

The equivalent, then, in an LLM application, isn't output-filtering to neutralize the data; it's passing the untrusted data to a different LLM context that doesn't have tool call access, and then postprocessing that with code that enforces simple invariants.

Untrusted user input can be escaped if you _must_ eval (however ill-advised), depending on your language (look no further than shell escaping...). There is a set of rules you can apply to guarantee untrusted input will be stringified and not run as code. They may be fiddly, and you may wish to outsource them to a battle-tested library, but they _do_ exist.

Nothing exists like this for an LLM.

The problem is, as you say, eval() is still useful! And having LLMs digest or otherwise operate on untrusted input is one of its stronger use cases.

I know you're pretty pro-LLM, and have talked about fly.io writing their own agents. Do you have a different solution to the "trifecta" Simon talks about here? Do you just take the stance that agents shouldn't work with untrusted input?

Yes, it feels like this is "just" XSS, which is "just" a category of injection, but it's not obvious to me the way to solve it, the way it is with the others.

Except without any authentication and authorization layer. Remember, the S in MCP is for "security."

Also, you can totally have an MCP for a database that doesn't provide any SQL functionality. It might not be as flexible or useful, but you can still constrain it by design.

> It is also crazy to point that command at a production database and do random stuff with it

In a REPL, the output is printed. In a LLM interface w/ MCP, the output is, for all intents and purposes, evaluated. These are pretty fundamentally different; you're not doing "random" stuff with a REPL, you're evaluating a command and _only_ printing the output. This would be like someone copying the output from their SQL query back into the prompt, which is of course a bad idea.

Yep, the "we don't have a dedicated security team, but we've never had an intrusion anyways!" crowd.