Docker limits unauthenticated pulls to 10/HR/IP from Docker Hub, from March 1

I get why this is annoying.

However, if you are using docker's registry without authentication and you don't want to go through the effort of adding the credentials you already have, you are essentially relying on a free service for production already, which may be pulled any time without prior notice. You are already taking the risk of a production outage. Now it's just formalized that your limit is 10 pulls per IP per hour. I don't really get how this can shift your evaluation from using (and paying for) docker's registry to paying for your own registry. It seems orthogonal to the evaluation itself.

> take the risk of a production outage in case we do too many pulls at once.

And the exact time you have some production emergency is probably the exact time you have a lot of containers being pulled as every node rolls forward/back rapidly...

And then docker.io rate limits you and suddenly your 10 minute outage becomes a 1 hour outage whilst someone plays a wild goose chase trying to track down every docker hub reference and point it at some local mirror/cache.

> If we had instead simply mirrored everything into a registry at a big cloud provider, we would never have paid docker a cent for the privilege of having unplanned work foisted upon us.

Indeed, you’d be paying the big cloud provider instead, most likely more than you pay today. Go figure.

Devsec/ops guy here, the fact that you were pulling public images at all ever is the thing that is insane to me.

They should have provided more notice. Your case is simply prioritizing work that you would have wanted to complete anyway. As a paying customer you could check if your unauthenticated requests can go via specific outbound IP addresses that they can then whitelist? I’m not sure but they may be inclined to provide exceptions for paying customers - hopefully.

So it goes. You're a business, pay to make the changes. It's a business expense. Docker ain't doing anything that their agreements/licenses say they can't do.

It's not fair, people shout. Neither are second homes when people don't even have their first but that doesn't seem to be a popular opinion on here.

> mirrored everything into a registry at a big cloud provider

https://cloud.google.com/artifact-registry/docs/pull-cached-...

You might try complaining and see if they give you an extension.

You can always rebuild the dockerfile, or create an account increasing your pull limit, or just host all the free infrastructure they’ve already built, or use competitors providing the same service with the risk of them doing exactly the same in the future. The difference with docker hub and just mirroring a linux repo is docker made it really easy for people so they don’t need it to get into infra weeds but the complexity was always there.

"foo.com will always be there and privide free, unlimited use" is the primary example of "clouded" thinking.

> Countless other tools and workflows and experiences have been built on that free assumption of availability

Cannot help but notice that, had Microsoft offered such a sweet deal, this place would've been ablaze with cries of "Embrace, extend, extinguish" and suchlike. (This still regularly happens, e.g., when new Github features are announced). Perhaps even justifiably so, but the community has failed to apply that kind of critical thinking to any other company involved in open source. If your workflow is not agnostic wrt where you pull images from, it is kind of silly to blame it on Docker Inc.

Having said that, it is definitely a problem for many. I work at a technical university and I am sure colleges/research institutes will hit the limit repeatedly and easily.

[dead]

This wouldn't be much of a problem for hobbyists if this was 240 per day or 1680 per week, but hitting 10 is easy.

This means there is a market for a docker proxy. Just install it, in a Docker container of course, and it caches the most common containers you use locally!

Docker Personal is free and does 40 pulls an hour. Why is everyone stuck on the 10 an hour number?

That's business, baby. We're all involved in it, like it or not. And especially American tech/developer culture seems to thrive on fyigm gated community stuff.

I couldn't give 2 shits whatever Docker does. They're a service, if I wanna use it I'll pay, if not then I'll use something else. Ez pz

> Does this commercial company expect volunteers to give them images for free which give their paid subscriptions value?

Yes, to an extent, because it costs money to store and serve data, no matter what kind of data it is or it's associated IP rights/licensing/ownership. Regardless, this isn't requiring people to buy a subscription or otherwise charging anyone to access the data. It's not even preventing unauthenticated users from accessing the data. It's reducing the rate at which that data can be ingested without ID/Auth to reduce the operational expense of making that data freely (as in money) and publicly available. Given the explosion in traffic (demand) and the ability to make those demands thanks to automation and AI relative to the operational expense of supplying it, rate limiting access to free and public data egress is not in and of itself unreasonable. Especially if those that are responsible for that increased OpEx aren't respecting fair use (legally or conceptually) and even potentially abusing the IP rights/licensing of "images [given] for free" to the "Library built on the back of volunteers".

To what extent that's happening, how relevant it is to docker, and how effective/reasonable Docker's response to it are all perfectly reasonable discussions to have. The entitlement is referring to those that explicitly or implicitly expect or demand such a service should be provided for free.

Note: you mentioned you don't use docker. a single docker pull can easily be 100's of MB's (official psql image is ~150MB for example) or even in some cases over a GB worth of network transfer depending on the image. Additionally, there is no restriction by docker/dockerhub that prevents or discourages people from linking to source code or alternative hosts of the data. Furthermore you don't have to do a pull everytime you wish to use an image, and caching/redistributing them within your LAN/Cluster is easy. Should also be mentioned Docker Hub is more than just a publicly accessible storage endpoint for a specific kind of data, and their subscription services provide more that just hosting/serving that data.

> Does this commercial company expect volunteers to give them images for free which give their paid subscriptions value?

If you're only looking at Docker Hub as a host of public images, you're only seeing the tip of the iceberg.

Docker Hub subscriptions are primarily for hosting private images, which you can't see from the outside.

IMO, hosting public images with 10 pulls per hour is plenty generous, given how much bandwidth it uses.

It's like GitHub limiting the number of checkout you can do each hour on public repos. Unless you pay a sub to get rid of the limit.

So, yeah, they kind of taking advantage of people putting their work on DH to try&sell subs.

But nobody have to put their images on DH. And to be honest, I don't think the discoverability factor is as important on DH that it is on GitHub.

So if people want to pay for they own registry to make it available for free for everyone, it's less an issue than hosting your repo on your own GitLab/Gitea instance.

> Is the Docker Library built on the back of volunteers which is then used to sell paid subscriptions?

Yes.

> Does this commercial company expect volunteers to give them images for free which give their paid subscriptions value?

Yes.

[dead]

Good news, Docker Hub supports IPv6.

https://www.docker.com/blog/docker-hub-registry-ipv6-support...

Docker Personal is free unless I am missing something?

> All I can do

You could also, you know, pay Docker for the resources you're using.

Artifactory is a tire fire though.

Someone (maybe the podman folks?) should do what every Linux distribution has done, and set up a network of signed mirrors that can be rsynced.

This is what I've seen (and done) at every place that used containers at any kind of scale. I'm frankly impressed with the people who can sleep at night with their production hosts pointed directly at docker hub.

One of the first things I did was move to Quay.io which is unlimited everything for OSS projects. I was reaching a point where I had 1M+ pulls a month (I suspect some kind of DDoS, accidental or otherwise, for a project with just 1.7k stars) - and not having to even think about the bandwidth or anything was wonderful. It's nice to be supported by Red Hat which I generally consider more benevolent towards OSS as opposed to Docker Hub.

Does GitHub's container registry still require authentication for pulls?

It’s like playing Plague Inc. (reverse version of Pandemic the board game where you play as the disease): to win, develop all possible methods of spreading first; only then develop symptoms, and do it fast before anyone has time to react

There is no network effect here though. I can host my image wherever and still distribute it.

I feel that dockerhub no longer can be the steward for the default docker repo because of this and the limitations they previously have implemented. It is time for them to hand over the baton stick to someone else, or that the notion of a default repo is removed all together

I like Podman's solution: all images must be fully qualified, with even Docker Hub images requiring a docker.io/ prefix.

That's why it is imperative that you use sha256 digests.

Remember, tags are mutable. `latest` today can be different tomorrow.

And it expands to all other tags. Nothing prevents me from pushing a new 'version' of a container to an existing `v1.0.1` tag.

Tags are not a way of uniquely identifying a container.

To elaborate on this, podman allows specifying mirrors, as described in https://github.com/containers/image/blob/main/docs/container...

So the source of the image can be decided on pull. Some more on this https://www.redhat.com/en/blog/manage-container-registries

> It just sucks if it wasn't part of your earlier accounting.

I generally agree with you, but to be fair to the complainers, what sucks is that Docker didn't make it clear up front that it should be part of your accounting. I don't know if they always intended to monetize this way (if so, we'd call that a bait and switch) or if they sincerely had other plans that just didn't pan out, but either way the problem is the same: There's a trend across all of technology of giving your stuff away for free until you become the obvious choice for everything, then suddenly altering the deal and raising prices.

That kind of behavior has in the past been deemed anticompetitive and outlawed, because it prevents fair competition between solutions on their merits and turns it into a competition for who has the deepest war chest to spend on customer-acquisition-by-free-stuff.

> authentication, validation and security

Those are generally solved using SSL, no need for centralized storage.

Forcing the client to default to a centralized repo only provides value to docker though, not to the customers.

if i start an eks cluster in a NAT environment with 10 nodes and 4 daemon sets. I need 40 pulls by default. Lots of tutorials out there to do this that will no longer work as well.

The way Dockerfiles work, yes I think it does need to do this. It wouldn't be a matter of "conflicts" but rather of assembling containers with the wrong data. Imagine a Dockerfile like so:

    RUN echo 1 > A
    RUN echo "$(cat A) + 1" | bc > B

If, in a later version, the first command changes to `echo 3 > A` then the contents of B should become "4", even though the second command didn't change. That is, neither layer can be reused because the layers depend on each other.

But maybe there's no dependency. If your Dockerfile is like this:

    RUN echo 1 > A
    RUN echo 2 > B

    RUN echo 3 > A  # new
    RUN echo 2 > B  # no dependency on layer 1, can be reused

Build tools like nix and bazel require more of the user. You can't just run commands all willy nilly, you have to tell them more info about which things depend on which other things. But the consequence is that instead of a list of layers you end up with a richer representation of how dependency works in your project (I guess it's a DAG). Armed with this, when you try to build the next version of something, you only have to rebuild the parts that actually depend on the changes.

Whether the juice is worth the squeeze is an open question. I think it is.

Having gcr.io and ghcr.io both be a thing is kinda funny to me, almost like google typo-squatting github.

More details: https://cloud.google.com/artifact-registry/docs/pull-cached-...

TIL! Can you link to a source for this? I am curious!

Which means GitHub is going subscription-only in the near or mid-term future. Will we repeat the same mistakes?

Is there any licence constraints?

I.e Docker terms of service restrict distribution in this way?

Is there any technical restraints?

I.e Docker specify no-cache

I expect Docker don't want their images cached and would want you to use their service and transform you in to a paying subscriber through limitations on free tier.

These platforms do cache quite a bit. It's just that there is a very high volume of traffic and a lot of it does update pretty frequently (or has to check for updates)

Gitlab does this, and it works nicely

Yeah I don't get why I have to setup caching myself for this kind of thing. Like wouldn't it be more efficient to do it lower down in their infra anyway?

Yes rude and lazy on their part.

Disclaimer: I work for Amazon, but not on ECR or ECR Public.

The rate limit for unauthenticated pulls is 1/second/IP, source: https://docs.aws.amazon.com/general/latest/gr/ecr-public.htm...

Yes, but in this case it's not the problem. It's more about not accepting `?ns=docker.io` as a query parameter on an endpoint, so a rather small and isolated technical issue.