Why is everything so scalable?

THANK YOU. People look at me like I’m insane when I tell them that their overly-complicated pipeline could be easily handled by a couple of beefy servers. Or at best, they’ll argue that “this way, they don’t have to manage infrastructure.” Except you do - you absolutely do. It’s just been partially abstracted away, and some parts like OS maintenance are handled (not that that was ever the difficult part of managing servers), but you absolutely need to configure and monitor your specific XaaS you’re renting.

Exactly this! The educational product I work on is used by hundreds of thousands of students a day, and the secret to our success is how simple our architecture is. PHP monoliths + Cache (Redis/Memcached) scale super wide basically for free. We don't really think about scalability, it just happens.

I have a friend whose startup had a super complicated architecture that was falling apart at 20 requests per second. I used to be his boss a lifetime ago and he brought me in for a meeting with his team to talk about it. I was just there flabbergasted at "Why is any of this so complicated?!" It was hundreds of microservices, many of them black boxes they'd paid for but had no access to the source. Your app is essentially an async chat app, a fancy forum. It could have been a simple CRUD app.

I basically told my friend I couldn't help, if I can't get to the source of the problematic nodes. They'll need to talk to the vendor. I explained that I'd probably rewrite it from the ground up. They ran out of runway and shut down. He's an AI influencer now...

> The general setup for the largest players in that space was haproxy in front of nginx in front of several PHP servers in front of a MySQL database that had one primary r/w with one read only replica.

You'd be surprised that the most stable setups today are run this way. The problem is that this way it's hard to attract investors; they'll assume you are running on old or outdated tech. Everything should be serverless, agentic and, at least on paper, hyperscalable, because that sells further.

> Today at AWS, it is easily possible for people to spend a multiple of the cost of that hardware setup every month for far less compute power and storage.

That is actually the goal of hyperscalers: they are charging you premium for way inferior results. Also, the article stated a very cold truth: "every engineer wants a fashionable CV that will help her get the next job" and you won't definitely get a job if you said: "I moved everything from AWS and put it behind haproxy on one bare-metal box for $100/mo infra bill".

Are those over engineered systems even actually scalable? I know teams who designed a CQRS architecture using messages queues and a distributed NoSQL database and fail to sustain 10req/s for a read in something that is basically a CRUD application. Heck once someone literally said "But we use Kafka, why aren't we fast?!".

I thought i knew about scaled deployments before i started working where i do now. After staring here, i realized i had no idea what an environment of huuuuge scale actually was. Id been part of multi site deployments and scaled infra, but it was basically potatoes comparatively. We have a team whose platform we, on IT, call the DoS'er of the company. Its responsible for processing hundreds of thousands of test runs a day, and data is fed to a plethora of services after. The scale is so large that they are able to take down critical services, or deeply impact them, purely due to throughput, if a developer goes too far (like say uploading a million small logs to an s3 bucket every minute).

We also have been contacted by AWS having them ask us what the hell we are doing, for a specific set of operations. We do a huge prep for some operations, and the prep feeds massive amounts of data through some AWS services, so much so, they thought we were under attack or had been compromised. Nope, just doin data ingestion!

The architecture you describe is ok because in the end it is a fairly simple website. Little user interaction, limited amount of content (at most a few million records), few content changes per day. The most complex part is probably to have some kind of search engine but even with 10 million videos an ElasticSearch index is probably no larger than 1GB.

The only problem is that there is a lot of video data.

Unfortunately, you can't really get experience from solving hypothetical problems. The actual problems you'll encounter are different, and while you can get experience in a particular "scalable" stack, it won't be worth its maintenance cost for a company that doesn't need it.

I guess the work is deterministic, but it often (unintentionally) makes the systems being developed non-deterministic!

>“dress for the job you want, not the job you have”

I don't think I should dress down any further :>

> 1000TPS and 10K daily users

I absolutely agree with your point, but I want to point out, like other commenters here, that the numbers should be much larger. We think that, because 10k daily users is a big deal for a product, they're also a big deal for a small server, but they really aren't.

It's fantastic that our servers nowadays can easily handle multiple tens of thousands of daily users on $100/mo.

In my opinion, if those influential programmers actually architected around some concrete metrics like 1,000 TPS and 10K daily users, they would end up with much simpler systems.

The problem I see is much more about extremely vague notions of scalability, trends, best practices, clean code, and so on. For example we need Kafka, because Kafka is for the big boys like us. Not because the alternatives couldn’t handle the actual numbers.

CV-driven development is a much bigger issue than people picking overly ambitious target numbers.

I've seen senior engineers get fired and the business suffer a setback because they didn't have any way to scale beyond a single low spec VPS from a budget provider, and their system crashed when a hall full of students tried to sign up together during a demo and each triggered 200ms of bcrypt CPU activity.

1000TPS isn't that much? Engineer for low latency and with a 10ms budget that'd be 10 cores if it were CPU-bound, less in practice since usually part of the time is spent in IO wait.

Clearly this project failed for either

  1. scaling for a very specific use case, or because
  2. it hasn't even found product-market fit 

> 1000TPS and 10K daily users

That is not a lot. You can host that on a Raspberry Pi.

> unless you are doing something silly, it's difficult to run into CPU speed limitations.

Yes, but it's not difficult to do something silly without even noticing until too late. Implicitly (and unintentionally) calling something with the wrong big-O, for example.

That said, anyone know what's up with the slow deletion of Safari history? Clearly O(n), but as shown in this blog post still only deleted at a rate of 22 items in 10 seconds: https://benwheatley.github.io/blog/2025/06/19-15.56.44.html

One of the silliest things you can do to cripple your performance is build something that is artificially over distributed, injecting lots of network delays between components, all of which have to be transited to fulfill a single user request. Monoliths are fast. Yes, sometimes you absolutely have to break something into a standalone service, but that’s rare.

For how many users, and at what transaction rate?

Not disagreeing that you can do a lot on a lot less than in the old days, but your story would be much more impactful with that information. :)

Scalability isn't just about CPU.

It's just as much about storage and IO and memory and bandwidth.

Different types of sites have completely different resource profiles.

> If you have a simple job to do that fits in an AWS Lambda, why not deploy it that way, scalability is essentially free. But the real advantage is that by writing it as a Lambda you are forced to think of it in stateless terms.

What you are describing is already the example of premature optimization. The moment you are thinking of a job in terms of "fits in an AWS Lambda" you are automatically stuck with "Use S3 to store the results" and "use a queue to manage the jobs" decisions.

You don't even know if that job is the bottleneck that needs to scale. For all you know, writing a simple monolithic script to deploy onto a VM/server would be a lot simpler deployment. Just use the ram/filesystem as the cache. Write the results to the filesystem/database. When the time comes to scale you know exactly which parts of your monolith are the bottleneck that need to be split. For all you know - you can simply replicate your monolith, shard the inputs and the scaling is already done. Or just use the DB's replication functionality.

To put things into perspective, even a cheap raspberry pi/entry level cloud VM gives you thousands of postgres queries per second. Most startups I worked at NEVER hit that number. Yet their deployment stories started off with "let's use lambdas, s3, etc..". That's just added complexity. And a lot of bills - if it weren't for the "free cloud credits".

> It's of course great to have a modular architecture, but whether or not they run in the same process should be an implementation detail

It should be, but I think "microservices" somehow screwed up that. Many developers think "modular architecture == separate services communicating via HTTP/network that can be swapped", failing to realize you can do exactly what you're talking about. It doesn't really matter what the barrier is, as long as it's clear, and more often than not, network seems to be the default barrier when it doesn't have to be.

The complexity that makes money is all the essential complexity of the problem domain. The "complexity in the architecture" can only add to that (and often does).

This is the part that is about math as a language for patterns as well as research for finding counter-examples. It’s not an engineering problem yet.

Once you have product market fit, then it becomes and engineering problem.

> Are there really people out there who still reach for Meta-scale by default? Who start with microservices?

Anecdotally, the last three greenfield projects I was a part of, the Architects (distinct people in every case) began the project along the lines of "let us define the microservices to handle our domains".

Every one of those projects failed, in my opinion not primarily owing to bad technical decisions - but they surely didn't help either by making things harder to pivot, extend and change.

Clean Code ruined a generation of engineers IMO.