Popular GitHub Action tj-actions/changed-files is compromised
118 comments
·March 15, 2025mubou
ashishb
> Can you really say you trust all of these?
We need better capabilities. E.g. when I run `fd`, `rg` or similar such tool, why should it have Internet access?
IMHO, just eliminating Internet access for all tools (e.g. in a power mode), might fix this.
The second problem is that we have merged CI and CD. The production/release tokens should ideally not be on the same system as the ones doing regular CI. More users need access to CI (especially in the public case) than CD. For example, a similar one from a few months back https://blog.yossarian.net/2024/12/06/zizmor-ultralytics-inj...
redserk
I’ve been doing all of my dev work in a virtual machine as a way to clamp things down. I’ve even started using a browser in a VM as a primary browser.
Computers are fast enough where the overhead doesn’t feel like it’s there for what I do.
For development, I think Vagrant should make a comeback as one of the first things to setup in a repo/group of repos.
bombcar
https://www.qubes-os.org/ is the extension of this.
CamJN
You also need to block write access, so they can’t encrypt all your files with an embedded public key. And read access so they can’t use a timing side channel to read a sensitive file and pass that info to another process with internet privileges to report the secret info back to the bad guy. You get the picture, I’m sure.
ashishb
> You also need to block write access, so they can’t encrypt all your files with an embedded public key. And read access so they can’t use a timing side channel to read a sensitive file and pass that info to another process with internet privileges to report the secret info back to the bad guy. You get the picture, I’m sure.
Indeed.
One can think of a few broad capabilities that will drastically reduce the attack surface.
1. Read-only access vs read-write 2. Access to only current directory and its sub-directories 3. Configurable Internet access
Docker mostly gets it right. I wish there was an easy way to run commands under Docker.
E.g.
If I am running `fd`
1. Mount current read-only directory to Docker without Internet access (and without access to local network or other processes) 2. Run `fd` 3. Print the results 4. Destroy the container
hypeatei
OpenBSDs pledge[0] system call is aimed at helping with this. Although, it's more of a defense-in-depth measure on the maintainers part and not the user.
> The pledge() system call forces the current process into a restricted-service operating mode. A few subsets are available, roughly described as computation, memory management, read-write operations on file descriptors, opening of files, networking (and notably separate, DNS resolution). In general, these modes were selected by studying the operation of many programs using libc and other such interfaces, and setting promises or execpromises.
yencabulator
Pledge is for self-isolating, it helps with mistakes but not against intentional supply chain attacks.
h4ck_th3_pl4n3t
But that's what firejail and docker/podman are for. I never run any build pipeline on my host system, and neither should you. Build containers are pretty good for these kind of mitigations of security risks.
usef-
Yes. Same with browser plugins. I've heard multiple free-plugin authors say they're receiving regular offers to purchase their projects. I'm sure some must take up the offer.
ronjouch
For an example of a scary list of such offers, see https://github.com/extesy/hoverzoom/discussions/670
remram
This is cool but useless because they redacted all the company names. The opposite of a name and shame, because no name and no shame.
Gigachad
I have long since stopped using any extension that doesn’t belong to an actual company (password managers for example). Even if they aren’t malware when you installed them, they will be after they get sold.
fluidcruft
Actual companies also get sold and churned into shit. See LastPass for example.
from-nibly
I got an outreach for an extension I made as a joke. It had like maybe 5000 downloads ever.
from-nibly
This is the death of fun. Like when you had to use SSL for buying things online.
Adding SSL was not bad, don't get me wrong. It's good that it's the default now. However. At one point it was sorta risky, and then it became required.
Like when your city becomes crime ridden enough that you have to lock your car when you go into the grocery store. Yeah you probably should have been locking it the whole time. what would it have really cost? But now you have to, because if you don't your car gets jacked. And that's not a great feeling.
ycombiredd
Yes, this...
I hope the irony is not completely lost on the fine folks at semgrep that the admittedly "overkill" suggested semgrep solution is exactly the type of pattern that leads to this sort of vulnerability: that of executing arbitrary code that is modifiable completely outside of one's own control.
scrapcode
Are there examples of these types of actions in other circles outside of the .NET ecosystem? I knew about the FluentAssertions ordeal, but the Moq thing was news to me. I guess I've just missed it all.
do_not_redeem
node-ipc is a recent example from the Node ecosystem. The author released an update with some code that made a request to a geolocation webservice to decide whether to wipe the local filesystem.
sanex
Missed them too. Always was annoyed by FluentAssertions anyway, some contractor added it to a project that we took over couldn't see the value add.
mh-
> eslint's dependency tree
And if you turn on devDependencies (top right), it goes from 85 to 1263.
Terr_
I'd also emphasize out that there's nothing safe about it being "only dev", given how many attacks use employee computers (non-prod) as a springboard elsewhere.
kubectl_h
npm supply chain attacks are the lone thing that keeps me up at night, so to speak. I shudder thinking about the attack surface.
I go out of my way to advocate for removing dependencies and pushing against small dependency introductions in a large ruby codebase. Some dependencies that suck and impose all sorts of costs, from funky ass idiosyncratic behavior or absurd file sizes (looking at you any google produced ruby library, especially the protocol buffer dependent libraries) are unavoidable, but I try to keep fellow engineers honest about introducing libraries that do things like determine the underlying os or whatever and push towards them just figuring that out themselves or, at the least, taking "inspiration" from the code in those libraries and reproducing behavior.
A nice side effect of AI agents and copilots is they can sometimes write "organic" code that does the same thing as third party libraries. Whether that's ethical, I don't know, but it works for me.
h4ck_th3_pl4n3t
The alternative would be to find a sustainable funding model for open source, which is the source of betrayals due to almost all of the maintainers having to sell their projects to make a living in the first place.
The problem you're describing is an economical and a social one.
Currently, companies exploit maintainers of open source projects. There are rarely projects that make it due to their popularity, like webpack, when it comes to funding...but the actual state is that everyone that webpack is based on as a dependency didn't get a single buck for it, which is unfair, don't you think?
On top of sustainable funding, we need to change our workflows to reproducible build ecosystems that can also revert independent of git repositories. GitHub has become the almost single source of code for the planet, which is insane to even bet on from a risk assessment standpoint. But it's almost impossible to maintain your own registry or mirror of code in most ecosystems due to the sheer amount of transitive dependencies.
Take go mod vendor, for example. It's great to stick your dependencies but it comes with a lot of overhead work in case something like OPs scenario happens to its supply chain. And we need to account for that in our workflows.
from-nibly
It's not going to happen. If buying a forever license of unlimited usage for an open source library cost $1 I'd skip it. Not be cause I don't want to give money to people who deserve it, but because of the absolute monstrous bureaucratic nightmare that comes from trying to purchase anything at a company larger than 10 people.
Don't even talk about when the company gets a lawyer who knows what a software license is.
gbraad
Am I seeing this correctly, that a (fake/impersonation?) Renovate bot actually proposed the fix... and then other repositories trickled that fix in, also suggested by Renovate or Dependabot, as the dependency updated?
I usually fork (or create my own) actions, as I do not trust the whole chain on GitHub. The marketplace does no enforcement. It is really based on trust you have in the 3rd-party... and I do not have this; as many actions have side-effects, or only operate on a specific runner OS, etc.
themgt
This is hilarious, the maven-lockfile project "Lockfiles for Maven. Pin your dependencies. Build with integrity" appears to have auto-merged a PR for the compromised action commit. So the real renovate bot immediately took the exfiltration commit from the fake renovate bot and started auto-merging it into other projects:
sureIy
The fun part is that they used commits specifically for security, but then add an auto-updater. Might as well use tags.
mdaniel
heh, timing is everything https://github.com/chains-project/maven-lockfile/issues/1085...
> After some cleanup the changed-files (https://github.com/tj-actions/changed-files) action seems to be more work to remove. It would be awesome if it could be added to the allowlist
> Done. Allowed all versions of this action. Should I pin it to one version in the allowlist (won't be convenient if renovate updates this dependency)?
harrisi
It's always been shocking to me that the way people run CI/CD is just listing a random repository on GitHub. I know they're auditable and you pin versions, but it's crazy to me that the recommended way to ssh to a server is to just give a random package from a random GitHub user your ssh keys, for example.
This is especially problematic with the rise of LLMs, I think. It's the kind of common task which is annoying enough, unique enough, and important enough that I'm sure there are a ton of GitHub actions that are generated from "I need to build and deploy this project from GitHub actions to production". I know, and do, know to manually run important things in actions related to ssh, keys, etc., but not everyone does.
remram
People don't pin versions. Referencing a tag is not pinning a version, those can be updated, and they are even with the official actions from GitHub.
harrisi
Aren't GitHub action "packages" designate by a single major version? Something like checkout@v4, for example. I thought that that designated a single release as v4 which will not be updated?
I'm quite possibly wrong, since I try to avoid them as much as I can, but I mean.. wow I hope I'm not.
remram
No the "v4" tag gets updated from v4.1 to v4.2 etc as those minor versions are released. They are branches, functionally.
sestep
The crazier part is, people typically don't even pin versions! It's possible to list a commit hash, but usually people just use a tag or branch name, and those can easily be changed (and often are, e.g. `v3` being updated from `v3.5.1` to `v3.5.2`).
nextts
Fuck. Insecure defaults again. I argue that a version specifier should be only a hash. Nothing else is acceptable. Forget semantic versions. (Have some other method for determining upgrade compatibility you do out of band. You need to security audit every upgrade anyway). Process: old hash, new hash, diff code, security audit, compatibility audit (semver can be metadata), run tests, upgrade to new hash.
harrisi
You and someone else pointed this out. I only use GitHub-org actions, and I just thought that surely there would be a "one version to rule them all" type rule.. how else can you audit things?
I've never seen anything recommending specifying a specific commit hash or anything for GitHub actions. It's always just v1, v2, etc.
mcpherrinm
OpenSSF scorecard flags dependencies (including GitHub actions) which aren’t pinned by hash
https://github.com/ossf/scorecard/blob/main/docs/checks.md#p...
sieabahlpark
[dead]
dan_manges
GitHub Actions should use a lockfile for dependencies. Without it, compromised Actions propagate instantly. While it'd still be an issue even with locking, it would slow down the rollout and reduce the impact.
Semver notation rather than branches or tags is a great solution to this problem. Specify the version that want, let the package manager resolve it, and then periodically update all of your packages. It would also improve build stability.
nextts
Also don't het GH actions to do anything other than build and upload artifacts somewhere. Ideally a write only role. Network level security too no open internet.
Use a seperate system for deployments. That system must be hygienic.
This isn't foolproof but would make secrets dumping not too useful. Obviously an attack could still inject crap into your artefact. But you have more time and they need to target you. A general purpose exploit probably won't hurt as much.
cmckn
I always use commit hashes for action versions. Dependabot handles it, it’s a no brainer.
Terr_
> commit hashes
There is some latent concern that most git installations use SHA-1 hashes, as opposed to SHA-256. [0]
Also the trick of creating a branch that happens to be named the same as a revision, which then takes precedence for certain commands.
password4321
creating a branch that happens to be named the same as a revision, which then takes precedence for certain commands
TIL; yikes! (and thanks)
mceachen
GitHub actions supports version numbers, version ranges, and even commit hashes.
frenchtoast8
The version numbers aren't immutable, so an attacker can just update the versions to point to the compromised code, which is what happened here. Commit hashes are a great idea, but you still need to be careful: lots of people use bots like Renovate to update your pinned hashes whenever a new version is published, which runs into the same problem.
werrett
Only commit hashes are safe. In this case the bad actor changed all of the version tags to point to their malicious commit. See https://github.com/tj-actions/changed-files/tags
All the tags point to commit `^0e58ed8` https://github.com/tj-actions/changed-files/commit/0e58ed867...
mixologic
All the version tags got relabled to point to a compromised hash. Semver does nothing to help with this.
your build should always use hashes and not version tags of GHA's
jasonthorsness
Since they edited old tags here … maybe GitHub should have some kind of security setting a repo owner can make that locks-down things like old tags so after a certain time they can't be changed.
CaliforniaKarl
In your GitHub Actions YAML, instead of referencing a specific tag, you can reference a specific commit. So, instead of …
uses: actions/checkout@v4
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683OptionOfT
That still doesn't help when the action is a docker action only marked with a tag.
So you need to check the action.yml itself to see if it has a sha256 pinned (in the case it uses Docker).
eddythompson80
You can always just fork it and reference your own fork.
postalrat
Or just write your own.
netvarun
@dang: The original URL (from Step Security, the company that discovered this flaw) is a better source for this:
https://www.stepsecurity.io/blog/harden-runner-detection-tj-...
moyer
Yeah maybe we can merge with https://news.ycombinator.com/item?id=43367987
captn3m0
Helpful update: The gist author has deleted the gist, so https://gist.githubusercontent.com/nikitastupin/30e525b776c4... now results in a 404, and stops the action from any further secrets being leaked. This means you're impacted only if you used the action, and had a build triggered in the last 6 hours or so.
samschooler
https://gist.github.com/gmatuz/7186f583df5f28196cc1f402af3bf...
This gist is pretty much the exact code, from the base64 encoded stuff. Looks like who ever put this in at least neafed the shell script.
Sytten
I am surprised nobody here mentionned immutable github actions that are coming [1]. Been waiting for them since the issue was open in 2022. This would have significantly reduce impact and hopefully github will get it over the finish line.
I always fork my actions or at least use a commit hash.
jasonthorsness
I think this article from earlier today was the discoverer who opened the issue
nomilk
I'd used GitHub Actions for at least 6-12 months before even realising <thing>/<thing> was not something that got parsed by the action (like a namespace and method/function), but was simply a reference to a github user name and repo. That whole object should really have been called a 'repo', because that's what it is, and that would alert users to use extreme caution whenever using one that wasn't created by themselves.
oefrha
> https://github.com/tj-actions/changed-files/pull/2460
This kind of auto dependency bump bots are more trouble than their worth. If your app works today, bumping random deps won’t make it work better in any meaningful sense in 95% of cases. With such a small upside, the downside of introducing larger attack surfaces, subtle breakages (despite semver), major breakages, and in the worst cases, compromises (whether it’s a compromised dep, or fake bot commits that people are trained to ignore) just completely outweighs the upside. You’re on the fast lane to compromises by using this kind of crap.
People should really learn from Go’s minimum version selection strategy.
In recent years, it's started to feel like you can't trust third-party dependencies and extensions at all anymore. I no longer install npm packages that have more than a few transitive dependencies, and I've started to refrain from installing vscode or chrome extensions altogether.
Time and time again, they either get hijacked and malicious code added, or the dev themselves suddenly decides to betray everyone's trust and inject malicious code (see: Moq), or they sell out to some company that changes the license to one where you have to pay hundreds of dollars to keep using it (e.g. the recent FluentAssertions debacle), or one of those happens to any of the packages' hundreds of dependencies.
Just take a look at eslint's dependency tree: https://npmgraph.js.org/?q=eslint
Can you really say you trust all of these?