Multi-threading is always the wrong design (2023)

> A CPU with 4 cores is going to have the capacity of executing 4 seconds of CPU-time per second. It does not matter how much “background idle threading” you do or don’t. The CPU doesn’t care. You always have 4 seconds of CPU-time per second. That’s an important concept to understand.

> If you write a program in the design of Node.js — isolating a portion of the problem, pinning it to 1 thread on one CPU core, letting it access an isolated portion of RAM with no data sharing, then you have a design that is making as optimal use of CPU-time as possible

This is .. not true as written? You get one second of CPU time per second, not four. Now, it may be quite hard to reach your full four seconds of CPU time per second, usually because of RAM bandwidth issues despite all the caching, and a hyperthreading fake "core" absolutely does not count the same as a separate die core, but the difference is real.

Author does have a point that slicing the work too small has significant overheads. But they've overstated it.

And this is before we get into the real source of parallel FLOPS, the GPU.

(edit: note that there may also be thermal issues and CPU frequency scaling going on; it is usually impossible to run all cores of a modern CPU at their max rated frequency for more than a very short time! But if you've bought a 64-core Ryzen and are only using one core, there's a huge gap there which you're not using)

> The irony is that, since you are splitting the problem in a way that requires synchronization between cores, you are actually introducing more work to be executed in the same CPU-time budget. So you are spending more time on overhead due to synchronization, which does the opposite of what you probably hoped for — it makes your code even slower, not faster.

That is certainly not universally true for every scenario and if you need to sync state between cpu cores very often then your tasks simply don't lend themselves to parallelization. That doesn't mean that multi-threading is inheritely the wrong design choice. Of course it will always be a trade-off between performance gains and the code complexity of your job control.

> If you write a program in the design of Node.js — isolating a portion of the problem, pinning it to 1 thread on one CPU core, letting it access an isolated portion of RAM with no data sharing, then you have a design that is making as optimal use of CPU-time as possible. It is how you optimize for NUMA systems and CPU cache locality. Even a SMP system is going to perform better if treated as NUMA.

Well that's... just wrong.

I'm perfectly happy to accept that multi-threading isn't the best solution to certain classes of problems, and the author makes compelling arguments to this end, but they haven't said what classes of problems shouldn't be solved with multi-threading.

That's the fundamental weakness of all these 'best-practice' blog posts: authors are much more willing to pluck a tool from their toolbox and tell us to use that one than they are to give advice on how to pick the right tool for the right job.

Not sure why this is on the HN front page. Post is riddled with errors, although it mixes in a few in-some-cases truths as well.

Oh, and [2023].

And the reason given is:

"it brings complexity very few developers understand"

Someone tell this guy about GPUs ...

For modern large-scale production systems I agree except in cases where performance is critical. To choose a wild example - a microservice serving geospatial queries over millions of objects that can exist anywhere in the world has plenty of parallelism at the level of each query, but handling multiple queries can be done by scaling horizontally with multiple instances of the service.

Instead of "just throw it on a thread and forget about it" - in a production environment, use the job queue. You gain isolation and observability - you can see the job parameters and know nothing else came across, except data from the DB etc.

Using something you don't actually need is wasteful. When you don't need threads, don't use them. Trivially parallelizable problems should be trivially parallelized.

Sometimes you want to do something complex in a short time though. If latency matters, sometimes you don't have a choice. Running 10 instances of the same game at 10fps each is not the answer.

Bit dizzy from the praise of Node.js and complaints about cache invalidation in one rant.

This guy's patronizing rants has made me avoid his software (µWebSockets) even though it seems good. Stop trying to be Torvalds. Even the docs used to be riddled with it:

> In the 1970s, programming was an elite's task. Today programming is done by uneducated "farmers" and as a result, the care for smart algorithms, memory usage, CPU-time usage and the like has dwindled in comparison.

It is 2025, multithread scalability is a well understood, if not easy, problem.

The reality is that hardware-provided cache coherence is an extremely powerful paradigm. Building your application on top of message passing not only gives away some performance, but it means that if you have any sort of cross thread logical shared state that needs to be kept in sync, you have to implement cache coherence yourself, which is an extremely hard problem.

With my apologies to Greenspun, any sufficiently complicated distributed system contains an ad-hoc, informally-specified, bug-ridden, slow implementation of MESI.

But of course, if you have a trivially parallel problem, rejoice! You do not need much communication and shared memory is not as useful. But not all, or even most, problems are trivially parallel.

Every time I try to increase the performance of my software by using multiple cores, I need a lot of cores to compensate for the loss of per-core efficiency. Like, it might run 2-3 times as fast on 8 cores.

I'm sure I've been doing it wrong. I just had better luck optimizing the performance per core rather than trying to spread the load over multiple cores.