Skip to content(if available)orjump to list(if available)

Building a message queue with only two UNIX signals

otterley

Before y'all go nuts with the criticisms...

"Yes, we built a message broker using nothing but UNIX signals and a bit of Ruby magic. Sure, it’s not production-ready, and you definitely shouldn’t use this in your next startup (please don’t), but that was never the point.

"The real takeaway here isn’t the broker itself: it’s understanding how the fundamentals work. We explored binary operations, UNIX signals, and IPC in a hands-on way that most people never bother with.

"We took something “useless” and made it work, just for fun. So next time someone asks you about message brokers, you can casually mention that you once built (or saw) one using just two signals. And if they look at you weird, well, that’s their problem. Now go build something equally useless and amazing. The world needs more hackers who experiment just for the fun of it."

SchwKatze

Unfortunately I bet that 90% won't even reach at that part and just ragebait based on the title. The golden rule of modern age is always do the disclaimer as soon as possible.

cortesoft

Or you could skip the rage bait title entirely?

SchwKatze

Maybe, but I know Leandro, it was more a joke than anything else. People just don't chill, the post is cool

dataflow

What would a better title be?

kalterdev

The later the disclaimer, the funnier.

gldrk

UNIX signals *do not* queue. If two or more signals with the same number are sent faster than the receiving thread handles them (due to the signal being blocked and/or the thread not being scheduled), all but the last will be lost irrevocably. There is no mechanism to prevent this.

https://ldpreload.com/blog/signalfd-is-useless

gldrk

Come to think of it, I think the original idea can be salvaged with an acknowledgment signal. Send bit, wait for acknowledgment, send next bit or retransmit accordingly.

jmalicki

RT signals do get queued... that is one of the major differences (and yes, the essay is not using them, so your point stands as it is written, but using RT signals is a mechanism to prevent it).

https://davmac.org/davpage/linux/rtsignals.html

kscarlet

Are they guaranteed to be delivered in order?

jmalicki

Yes as long as you use sigqueue with SA_SIGINFO

nakamoto_damacy

Naively asking: what prevents RT Unix/Linux from being used in place of non-RT mainstream versions? Seems like a superset.

jmalicki

RT signals are an extended API of POSIX you don't need actual RT Linux to use them.

RedShift1

That's easily solved, just create a queue for your signals.

foofoo12

This has nothing to do with Kafka and it's not not really a functioning message queue except for theoretically speaking.

The article is fine, but call it what it is: abusing the Unix signal system for shit and giggles. Nothing wrong with that.

m-hodges

This was a really fun article and many in the comments seem to have forgotten you're allowed to have fun with the computer.

sheepscreek

I challenged ChatGPT the other day to design a bidirectional process interop in *nix and this was one of the suggestions. Until then I had only ever thought of pipes as unidirectional. I still thought it was bonkers. This looks like a neat prototype though.

bcrl

This is the kind of article that deserves to be posted on April 1st, preferably with an accompanying RFC published at the IETF.

bradleybuda

This is awesome. Does POSIX guarantee the order of signal delivery? And I'm dying to see what the bandwidth / throughput of this channel is...

toast0

I don't know if POSIX has a position on signal order. But I'm pretty sure it allows signals to be coallesced... if a process is sent the same signal several times before the handler is invoked, it's in spec to only invoke it once.

o11c

Real-time signals have guaranteed order: first by number (lowest first, i.e. `SIGRTMIN`), then by the order in which they are sent.

Signals are generally the slowest IPC method, unless you're doing something stupid with a different method.

bradleybuda

Answering both of my questions, from the post:

  sleep 0.001 # Delay to allow the receiver to process the signal

cannonpalms

For standard signals--no, but for real-time signals, yes. The latter are still a portability issue, though.

cortesoft

Wow, I didn't know! I will work on replacing my Kafka cluster handling 10 million msg/sec with this right away!

ok123456

Why use signals when POSIX defines a complete message queue interface: https://www.opensourceforu.com/2023/06/a-deep-dive-into-posi...

dbacar

Such titles should be flagged and banned to protect the innocent.

hshdhdhehd

Can those two Unix signals run Doom?

nialv7

you can literally just attach data[1] to signals... you don't need to do this.

[1]: https://man7.org/linux/man-pages/man3/sigqueue.3.html