Redis is open source again

Interestingly their CEO states that AWS an Google forking redis and maintaining it separately was their "goal" all along. Because fragmentation is apparently good?

This keeps happening:

1. People put a lot of work into building databases. The license choice is OSS / FOSS.

2. Some people in the community (original authors, community leads) make a company around the database and continue developing it for years on end. They sometimes raise venture capital to expand the business.

3. Amazon / Google / Microsoft offer managed versions of the database and make bank on it. Easily millions in revenue. Original creator / company doesn't get anything, and the hyperscaler isn't obliged to pay.

4. The company decides to change the license to force Amazon / Google / Microsoft to pitch in and pay a fee.

5. Amazon / Google / Microsoft fork the database. The community revolts. Sometimes the people revolting are employees of the hyperscalers, other times these are just FOSS fans that hate "source available" licenses or relicensing.

6. Database company is forced to walk back the changes. Still no revenue.

---

The solution is clear: start your new database with an "equitable source / source available" license from day one. Nobody will complain about a relicense since your license will handle the hyperscalers right off the bat.

Basically your license needs one of a few things if you want to prevent Amazon from walking off with your money:

- A hyperscaler clause such that any managed offering has to (1) be fully open source, (2) has to pay a fee, or (3) is prevented outright.

- A MAU / ARR clause such that hyperscalers are in the blast radius. Note that this also hits your customers.

This seems to be accelerating. I guess the era of lighting investment money on fire and pretending that the flames equal success is coming to an end.

Unless of course you are an AI startup.

Good decision, I hope the best for Redis!

Not quite. Company can easily change license at any time. No string attached.

Nothing wrong in checking other alternatives, but Redis the company didn't call me to rejoin. I approached them to do something like an evangelist and bring back some kind of community vision inside. Then... if you can code, you end coding often times, and instead of doing the evangelist I wrote the Vector Set data type :D Just to clarify that me rejoining was not some kind of "winning back the community plan". I wrote at large about all that, even clearly stating that even the paycheck is modest (to avoid that kind of conflict of interest of the economical motivation).

the problem is the definition of Open Source is controlled by the Open Source Initative, which has been captured by the hyperscalers

which is sort of funny because the term "Open Source" was itself coined to make it possible and easier for people to fund, build, and grow companies based on software with its source code available

20 years later, the structure of the industry has now changed, "Open Source" now exists to feed Microsoft, Google and Amazon.

if it's not possible to alter the definition to include licenses that include terms that allow sustainable value creation for businesses other than the hyperscalers then the term is no longer fit for purpose, and we need a new one

"Fair Software"?

Wow. First time hearing about Garnet. MS should package and deploy it as a service in the Azure SAAS offerings.

Garnet fascinates me. Their benchmarks even claim that it is better than Redis and also Dragonfly. Are there any papers or write ups explaining what makes Garnet fast? (I do know its based on FASTER)

Nice. I've been using Memurai (https://www.memurai.com/) for development on Windows (native, no WSL or Docker - for reasons), but this looks much better.

EDIT: Weird that being a program from Microsoft (well, it's Microsoft Research, so that probably explains it?) it has no installer and doesn't run as a service on its own.

Yup, it’s a complete reimplementation in pure C#. It’s built on top of FASTER KV / Tsavorite project from MSR.

You do see how that's even worse, right?

A copyleft license like the AGPL didn't stop MongoDB from rugpulling. I'd argue that the AGPL, and the copyright assignment that tends to go with it, makes it easier to rugpull because forking entities would be at an extreme disadvantage in keeping the lights on compared to the closed-sourcing company. A non-copyleft license, on the other hand, makes it much easier for a forking company to cover all the same niches as the original company, making a rugpull that much more difficult.

You do realize the owners of the copyright can relicense it under any terms they want, even if it's a copyleft license like GPL, right?

[dead]

All advantage accrues to hyperscaler "managed" versions. That's so much more fucked than a rug pull.

Amazon gets to make millions off of the thing you built.

"Equitable source" licenses with MAU / ARR limits, hyperscaler resale limits, and AGPL-like "entire stack must be open" clauses is the way to go. It's a "fuck you" to Amazon, Google, and Microsoft in particular and leaves you untouched.

Open source today is hyperscaler serfdom. Very few orgs are running Redis on bare metal, and a equitable source license can be made to always support the bare metal case.

There are good legal reasons to avoid the GPL; there are open legal questions about whether the GPL and its variants are enforceable.

It's a win for the community over and against the corporations that are Redis and Elastic. They're not the good guys for giving in to the pressure. They tried to ride FOSS to prominence and then extract wealth on the backs of the community and found that the community mattered more than they did.

So sure, let's celebrate, but celebrate the community, not those who tried to pull the rug out from under them.

I imagine there is quite a large, quiet fraction (majority) of users who were the same way.

Not to say it’s not an important discussion!

From the blog post it seems like existing users kept using Redis but new users adopted alternatives instead.

Same here. The response from the community was valid, but basically didn't affect us.

Yeah, there has been a lot of stuff like performance [1] and efficiency improvements [2]. A lot of the contributors, that didn't work for Redis labs but worked on Redis OSS before the fork, moved to Valkey and they continued to contribute.

[1] https://valkey.io/blog/unlock-one-million-rps-part2/ [2] https://valkey.io/blog/new-hash-table/

Well Valkey has more commits to their repository then Redis does, and more contributors. So it appears to be active.

https://github.com/valkey-io/valkey

https://github.com/redis/redis

Valkey has RDMA support, which offers significant performance improvements.

The team of sales was, AFAIK, rebuilt from scratch recently. Please if this happened recently tell me, and I'll make sure to report back. Thanks.

Amazon really encourages valkey in the elasticache dashboard. There's a banner advertising lower prices and it's listed first in the dropdown when you go to create one. Default settings do have power.

If you use the latest versions of Redis, you are benefiting from the continued efforts of the Valkey development community. [1]

This is Open Source working well.

Unfortunately, the reverse flow does not work.

[1] https://github.com/redis/redis/pull/13638

That kind of assertion really needs some backup or it's just noise. I'll be honest and say that I have no idea what the usage stats for Valkey are -- and it may be that it's a drop in the bucket compared to Redis. But I don't know. Can you back this up or is this just your gut feeling?

I've been using Valkey simply because after I updated to the latest Fedora version, it dropped redis and pointed me to Valkey instead. I assume as more distros do this and more people update their systems, the Valkey user base will grow. But perhaps with the AGPL redis that will no longer be the case.

Are there usage stats available? How do you know this?

Yeah, all the open source distributions and most open source projects switching to valkey must be "nobody".

Yet. It's a drop-in replacement, and both faster and cheaper.

I don't know about valkey but I got word Nvidia was switching away from Redis.

Why would PaaS providers switch back to offering Redis? They've clearly all already invested a lot in Valkey (AWS, GCP, Heroku).

I can easily see this for a midsize company.

While it's likely an easy process to drop in valkey, creating the new instances, migrating apps to those new instances, and making sure there's not some hidden regression (even though it's "drop in") all takes time.

At a minimum, 1 or 2 hours per app optimistically.

My company has hundreds of apps (hurray microservices). That's where "hundreds of hours" seems pretty reasonable to me.

We don't have a lot of redis use in the company, but if we did it'd have taken a bit of time to switch over.

Edit: Dead before I could respond but I figured it was worthwhile to respond.

> It's literally just redis with a different name, what is there to test?

I've seen this happen quite a bit in opensource where a "x just named y" also happens to include tiny changes that actually conflict with the way we use it. For example, maybe some api doesn't guarantee order but our app (in a silly manor) relied on the order anyways. A bug on us, for sure, but not something that would surface until an update of redis or this switch over.

It can also be the case that we were relying on an older version of redis, the switchover to valkey necessitates that we now bring in the new changes to redis that we may not have tested.

These things certainly are unlikely (which is why 1 or 2 hours as an estimate, it'd take more if these are more common problems). Yet, they do and have happened to me with other dependency updates.

At a minimum, simply making sure someone didn't fat finger the new valkey addresses or mess up the terraform for the deployment will take time to test and verify.

I believe it. There are companies that invested hundreds of engineering hours to rename master to main.

At the very least you have to validate everything that touches redis, which means finding everything that touches redis. Internal tools and docs need to be updated.

And who knows if someone adopted post-fork features?

If this is a production system that supports the core business, hundreds of hours seems pretty reasonable. For a small operation that can afford to YOLO it, sure, it should be pretty easy.

By switch I mean that all new projects use Valkey instead of Redis, and we've invested hundreds of hours into those new projects.

There are companies using many thousands of Redis instances storing petabytes of data with millions of users.

Now consider a no-down-time migration. How long do you think that'll take to engineer and execute?

Even the infrastructure switch and testing should take a lot of time, yet the application level tests etc.

I think you might underestimate how little time hundreds of hours is. It's very, very easy to reach your first hundred hours in a task, e.g. taking a 40 hour week, 3 engineers = 120 hours.

If valkey is working, why spend that time reverting to redis, when you could be spending it on things that are actually going to provide value?

My company is relatively small. With probably 6 separate redis instances deployed in various places (k8s, bare metal, staging and prod environments) and dozens of (micro)services using them it's probably at least 40 hours (one person-week) to migrate everything at this point. Also there are things like documentation, legacy apps that keep working but nobody wants to spend time updating them, naming problems everywhere (renaming "redis" everywhere with zero downtime would be a huge pain), outdated documentation, possibly updates in CI, CD, and e2e tests, and probably more problems that ight become apparent in scale.

And we're honestly not large. For a mid size company, hundreds hours sound reasonable. For a big company the amount of work must be truly staggering.

Hundreds could be 200 which, at 10 hours a day 5 days a week, is like a week and a half for a team of 3. It seems quite possible if you had to do testing/benchmarking, config changes, deploy the system, watch metrics, etc.

By switch I mean that all new projects use Valkey instead of Redis, and we've invested hundreds of hours into those new projects. We've also tried stuff with the Valkey Glide client.

Elastic announced they were switching back a few months ago, we should be able to measure the results by now.