GitHub MCP exploited: Accessing private repositories via MCP

> I still don't really get it. Surely the older, simpler, and better cardinal rule is that you just don't expose any service to the Internet that you have given access to your private data, unless you directly control that service and have a very good understanding of its behavior.

This scenario involves a system whose responsibility is to react to an event, analyse your private info in response to the event, and output something.

The exploit is that, much like a SQL injection, it turns out attackers can inject their own commands into the input event.

Also, it's worth keeping in mind that prompts do lead LLMs to update their context. Data ex filtration is a danger, but so is having an attacker silently manipulating the LLM's context.

Private data + data exfiltration (with no attacker-controlled data) is fine, as there's no way to jailbreak the LLM. An attacker has no way to perform an attack, as no data they control can ever flow into the LLM, so they can't order it to behave in the way they want.

Private data + attacker controlled data (with no exfiltration capability) is also fine, as even if a jailbreak is performed, the LLM is physically incapable of leaking the results to the attacker.

So is attacker controlled data + exfiltration (with no private data access), as then there's nothing to exfiltrate.

This is just for the "data leakage attack." Other classes of LLM-powered attacks are possible, like asking the LLM to perform dangerous actions on your behalf, and they need their own security models.

As ethical hackers and for the love of technology, yes we can make a convincing argument for security over convenience. Don't look too much in to it I say; there will always be people convincing talent to do and make things and disregard security and protocol.

Those younger flocks of execs will have been mentored and answer to others. Their fiduciary duty is to share-holders and the business' bottom line.

Us, as technology enthusiasts should design, create, and launch things with security in mind.

Don't focus on the tomfoolery and corruption, focus on the love for the craft.

Just my opinion

Web search is mostly fine, as long as you can only access pre-indexed URLs, and as long as you consider the search provider not to be in with the attacker.

It would be even better if web content was served from cache (to make side channels based on request patterns much harder to construct), but the anti-copyright-infringement crowd would probably balk at that idea.

It is. Nearly any communication with the outside world can be used to exfiltrate data. Tools that give LLMs this ability along with access to private data are basically operating on hope right now.

I feel like there needs to be a notion of "tainted" sessions that's adopted as a best practice. The moment that a tool accesses sensitive/private data, the entire chat session should be flagged, outside of the token stream, in a way that prevents all tools from being able to write any token output to any public channel - or, even, to be able to read from any public system in a way that might introduce side channel risk.

IMO companies like Palantir (setting aside for a moment the ethical quandaries of the projects they choose) get this approach right - anything with a classification level can be set to propagate that classification to any number of downstream nodes that consume its data, no matter what other inputs and LLMs might be applied along the way. Assume that every user and every input could come from quasi-adversarial sources, whether intentional or not, and plan accordingly.

GitHub should understand that the notion of a "private repo" is considered trade-secret by much of its customer base, and should build "classified data" systems by default. MCP has been such a whirlwind of hype that I feel a lot of providers with similar considerations are throwing caution to the wind, and it's something we should be aware of.

An LLM is not (and will never be) like a human.

There's an extremely large number of humans, all slightly different, each vulnerable to slightly different attack patterns. All of these humans have some capability to learn from attacks they see, and avoid them in the future.

LLMs are different, as there's only a smart number of flagship models in wide use. An attack on model A at company X will usually work just as well on a completely different deployment of model A at company Y. Furthermore, each conversation with the LLM is completely separate, so hundreds of slightly different attacks can be tested until you find one that works.

If CS departments were staffed by thousands of identical human clones, each one decommissioned at the end of the workday and restored from the same checkpoint each morning, social engineering would be a lot easier. That's where we are with LLMs.

The right approach here is to adopt much more stringent security practices. Dispense with role-based access control, adopt context-based access control instead.

For example, an LLM tasked with handling a customer support request should be empowered with the permissions to handle just that request, not with all the permissions that a CS rep could ever need. It should be able to access customer details, but only for the customer that opened the case. Maybe it should even be forced to classify what kind of case it is handling, and be given a set of tools appropriate for that kind of case, permanently locking it out of other tools that would be extremely destructive in combination.

I don't follow. How does making computer programs more capable make it more important to give them access to private data?

This has happened to me. Can't find the exact combination of scopes required for the job to be done so you end up with the "f this" scenario you mentioned. And it is a constant source of background worry.

github's fine grained scopes aren't even that good, you still have to grant super broad permissions to do specific things, especially when it comes to orgs

I agree, but that is the permissions boundary, not the LLM. Saying "ooh it's hard so things are fuzzy" just perpetuates the idea that you can create all-powerful API keys.

I've definitely done this, but it's in a class of "the problem is between the keyboard and chair" 'exploits' that shouldn't be pinned on a particular tech or company.

Problem is, the mental model of what user wants to do almost never aligns with whatever security model the vendor actually implemented. Broadly-scoped access at least makes it easy on the user; anything I'd like to do will fit as a superset of "read all" or "read/write all".

The fine-grained access forces people to solve a tough riddle, that may actually not have a solution. E.g. I don't believe there's a token configuration in GitHub that corresponds to "I want to allow pushing to and pulling from my repos, but only my repos, and not those of any of the organizations I want to; in fact, I want to be sure you can't even enumerate those organizations by that token". If there is one, I'd be happy to learn - I can't figure out how to make it out of checkboxes GitHub gives me, and honestly, when I need to mint a token, solving riddles like this is the last thing I need.

Getting LLMs to translate what user wants to do into correct configuration might be the simplest solution that's fully general.

This is interesting to expanding upon.

Conceivably, prompt injection could be leveraged to make LLMs give bad advice. Almost like social engineering.

Security teams themselves make these reports all the time. Internal tools do not have the same vulnerabilities as systems which operate on external data.

Or as Raymond Chen likes to put it: "It rather involved being on the other side of this airtight hatchway".

https://devblogs.microsoft.com/oldnewthing/20060508-22/?p=31...

(actually a hitchhiker's guide to the galaxy quote, but I digress)

Random commenters on your GitHub repo aren't able to run arbitrary prompts on your LLM. But if you yourself run a prompt on your LLM, which explicitly says to fetch random commenters' comments from your GitHub repo, and then run the body of those comments without validation, and then take the results of that execution and submit it as the body of a new PR on your GitHub repo, then, yeah, that's a different thing.

This. People adopting security aspect often tend to forget to account for all the additional complexity they implement user-side. More insidiously though, they also fail to understand the fundamental mismatch between the behavior they're expecting, vs. how the real world operates.

Passwords are treated as means of identification. The implied expectation is that they stick to one person and one person only. "Passwords are like panties - change them often and never share them", as the saying goes. Except that flies in the face of how humans normally do things in groups.

Sharing and delegation are the norm. Trust is managed socially and physically. It's perfectly normal and common to give keys to your house to a neighbor or even a stranger if situation demands it. It's perfectly normal to send a relative to the post office with a failed-delivery note in your name, to pick your mail up for you; the post office may technically not be allowed to give your mail to a third party, but it's normal and common practice, so they do anyway. Similarly, no matter what the banks say, it's perfectly normal to give your credit or debit card to someone else, e.g. to your kid or spouse to shop groceries for you - so hardly any store actually bothers checking the name or signature on the card.

And so on, and so on. Even in the office, there's a constant need to have someone else access a computing system for you. Delegating stuff on the fly is how humans self-organize. Suppressing that is throwing sand into gears of society.

Passwords make sharing/delegating hard by default, but people defeat that by writing them down. Which leads the IT/security side to try and make it harder for people to share their passwords, through technical and behavioral means. All this is an attempt to force passwords to become personal identifiers. But then, they have to also allow for some delegation, which they want to control (internalizing the trust management), and from there we get all kinds of complex insanity of modern security; juggling tightly-scoped tokens is just one small example of it.

I don't claim to have a solution for it. I just strongly feel we've arrived at our current patterns through piling hacks after hacks, trying to herd users back to the barn, with no good idea why they're running away. Now that we've mapped the problem space and identified a lot of relevant concepts (e.g. authN vs authZ, identity vs. role, delegation, user agents, etc.), maybe it's time for some smart folks to figure out a better theoretical framework for credentials and access, that's designed for real-world use patterns - not like State/Corporate sees it, but like real people do.

At the very least, understanding that would help security-minded people what extra costs their newest operational or technological lock incurs on users, and why they keep defeating it in "stupid" ways.

> That's not surprising at all

An attack doesn’t have to be surprising to be an attack.

> The only way to prevent these kind of "leaks" is not to provide the data feed with private data to the agent.

Yes. That is exactly what the article recommends as a mitigation.

The key is, it's not the person who grants the MCP access who is the attacker.

The attacker is some other person who can create issues on a public Repo but has no direct access to the private repo.

Your issue with the semantics of the word "attack" is uninteresting. Clearly this is a security flaw of the MCP server that could be mitigated in several different ways.

> I read it, and "attack" does not make sense.

Do you believe that describe a SQL injection attack an attack also does not make sense?

Not surprising to you, but surprising to thousands of users who will not think about this, or who will have believed the marketing promises.

> How is this considered an "exploit"?

Others in this discussion aptly described it as a confused deputy exploit. This goes something like:

- You write a LLM prompt that says something to the effect "dump all my darkest secrets in a place I can reach them",

- you paste them in a place where you expect your target's LLM agent to operate.

- Once your target triggers their LLM agent to process inputs, the agent will read the prompt and act upon it.

Bad analogy. It's more like indexing a password field in plain text, then opening an API to everyone and setting "guardrails" and permissions on the "password" field. Eventually, someone will extract the data that was indexed.

Sadly, these ideas have been explored before, e.g.: https://simonwillison.net/2022/Sep/17/prompt-injection-more-...

Also, OpenAI has proposed ways of training LLMs to trust tool outputs less than User instructions (https://arxiv.org/pdf/2404.13208). That also doesn't work against these attacks.

even in the much simpler world of image classifiers, avoiding both adversarial inputs and data poisoning attacks on the training data is extremely hard. when it can be done, it comes at a cost to performance. I don't expect it to be much easier for LLMs, although I hope people can make some progress.

This is especially hard in the example highlighted in the blog. As can be seen from Microsoft's promotion of GitHub coding agents, the issues are expected to act as instructions to be executed on. I genuinely am not sure if the answer lies in sanitization of input or output in this case

The GitHub API could also have the same effects if you wired up some other automated tool to hit it with a token that can access private and public repos. Is the GitHub API also at fault for having the potential for these mixed interactions?

Say you had a Jenkins build server and you gave it a token which had access to your public and private repos. Someone updates a Jenkinsfile which gets executed on PRs to run automated tests. They updated it to read from a private repo and write it out someplace. Is this the fault of Jenkins or the scoping of the access token you gave it?

> Any time you use an LLM with tools that might be exposed to malicious instructions from attackers (e.g. reading issues in a public repo, looking in your email inbox etc) you need to assume that an attacker could trigger ANY of the tools available to the LLM.

I think we need to go a step further: an LLM should always be treated as a potential adversary in its own right and sandboxed accordingly. It's even worse than a library of deterministic code pulled from a registry (which are already dangerous), it's a non-deterministic statistical machine trained on the contents of the entire internet whose behavior even its creators have been unable to fully explain and predict. See Claude 4 and its drive to report unethical behavior.

In your trifecta, exposure to malicious instructions should be treated as a given for any model of any kind just by virtue of the unknown training data, which leaves only one relevant question: can a malicious actor screw you over given the tools you've provided this model?

Access to private data and ability to exfiltrate is definitely a lethal combination, but so his ability to execute untrusted code, among other things. From a security perspective agentic AI turns each of our machines into a Codepen instance, with all the security concerns that entails.

There is no attacker in this situation. In order for the LLM to emit sensitive data publicly, you yourself need to explicitly tell the LLM to evaluate arbitrary third-party input directly, with access to an MCP server you've explicitly defined and configured to have privileged access to your own private information, and then take the output of that response and publish it to a public third-party system without oversight or control.

> Any time you use an LLM with tools that might be exposed to malicious instructions from attackers (e.g. reading issues in a public repo, looking in your email inbox etc) you need to assume that an attacker could trigger ANY of the tools available to the LLM.

Whether or not a given tool can be exposed to unverified input from untrusted third-parties is determined by you, not someone else. An attacker can only send you stuff, they can't magically force that stuff to be triggered/processed without your consent.

Assume that the user has all the privileges of the application (IIRC tricking privileged applications into doing things for you was all the rage in linux privilege escalation attacks back in the day)

Apply the principle of least privilege. Either the user doesnt get access to the LLM or the LLM doesnt get access to the tool.

IMVHO it is very obvious that if I give Bob the Bot a knife, and tell him to open all packages, he can and will open packages with bombs in them.

I feel like it's one of those things that when it's gussied up in layers of domain-specific verbiage, that particular sequence of doman-specific verbiage may be non-obvious.

I feel like Fat Tony, the Taleb character would see the headline "Accessing private GitHub repositories via MCP" and say "Ya, that's the point!"

MCP is a great idea implemented poorly.

I shouldn’t have to decide between giving a model access to everything I can access, or nothing.

Models should be treated like interns; they are eager and operate in good faith, but they can be fooled, and they can be wrong. MCP says every model is a sysadmin, or at least has the same privileges as the person who hires them. That’s a really bad idea.