Show HN: Boulette - Protect you from yourself (even as root).
77 comments
·January 10, 2025jiscariot
franktankbank
Double deep environment protection. True prod servers can't be deployed to except from special terminal which only appears visible to the ProdGlasses(TM). If you are wearing ProdGlass(TM) you receive mild brain wave stimulation which is collected for "product control purposes"(TM) future product development(patent pending).
oslem
I’ve tried using iTerm2’s automatic profile switching feature to adjust the theme depending on the connection, but I’ve never been able to get it to work reliably.
reddit_clone
I too use iTerm's profiles to change background colors, fonts etc. to indicate where I am.
Reduces number of fat fingering disasters.
jean_dupont
I think such a thing can be achieved easily with the starship prompt!
binwiederhier
I've used this successfully in the past for Terminator: https://github.com/GratefulTony/TerminatorHostWatch
drcongo
I have iTerm set up to change the background colour of a shell based on whether it's local, docker, staging or production, and on top of that prompt colours change depending on how privileged the user is. Without all this, I'm sure I'd be making terrible mistakes on a regular basis.
jiscariot
I'm always running in tmux panes so often iterm and those settings seem to conflict. It would be great to go a step further and include a background color change--will have to give it another shot. Thanks for the tip.
dmd
What I really want is a way to get a read only root shell. I do a lot of work on a "traditional" multi-user unix host, where hundreds of scientists share a powerful computer. I often need to become root to look at files. I want to be able to do that without the ability to screw anything up.
jcalvinowens
If it's Linux, you can bind-mount the rootfs read-only and chroot into it:
mahler ~ # mkdir /roroot
mahler ~ # mount -o bind,ro / /roroot
mahler ~ # chroot /roroot
mahler / # passwd
Enter new password:
Re-type new password:
passwd: Authentication token lock busy
passwd: password unchanged
dmd
Oh, that's a good idea.
jeroenhd
I've used this command to achieve that in the past:
sudo systemd-nspawn --directory=/ --read-only --ephemeral --volatile=yes
[systemd-nspawn](https://man7.org/linux/man-pages/man1/systemd-nspawn.1.html) would normally be used to run a command inside of a container (a directory), but in some modes you can specify the system root as the "container" path.I believe this specific set of options relies on (BTRFS) file system snapshots for performance. It's possible that you can get it to work on non-BTRFS systems by providing another combination of command line variables, but the default setting is to copy the file system tree to a temporary path.
You can also pass parameters like --volatile=state (so you can write to /var) and --volatile=overlay (so you can "write" to state, but all changes are discarded after the container exits). --volatile=state is useful for extracting data from a temporary read-only system, --volatile=overlay is useful for running tools that crash if they run on a read-only filesystem.
ronsor
> I believe this specific set of options relies on (BTRFS) file system snapshots for performance. It's possible that you can get it to work on non-BTRFS systems by providing another combination of command line variables, but the default setting is to copy the file system tree to a temporary path.
You can use overlayfs + chroot.
ape4
--read-only would be a nice option for `run0`
makemyworkforme
Can you create a group and set a sticky bit for ACLs for your root for the filesystem with read permissions for that group? That way all files created under the filesystem root will have read permissions for that user, but you cannot modify anything.
dmd
Not really, there are tons of different filesystems mounted from all over the place. It's a mess.
spease
Can you create a second read-only mount of root?
Maybe this would help:
https://serverfault.com/questions/136515/read-only-bind-moun...
6r17
MicroOS has something in that spirit I think ? Maybe I'm wrong tough, don't even know if MicroOS is usable in host
Rygian
Why should my user have the right to shut down a production box?
Having a tool to warn me that I'm about to do something I shouldn't be allowed to do is okay. Having proper access control would be better.
(Fun: 'boulette' is French for 'dumb mistake')
deathanatos
> Having proper access control would be better.
Some of us live on the "granted" side of that access control. I'm an SRE; I have access to a fair number of things. I worry about making dumb mistakes like that in the article due to sleep deprivation. The warning here might very well be a warning that you're about to do something that you are allowed to do.
In the specific example … sometimes rebooting a production machine is what needs doing. Ideally, there's a second VM, ideally, the services on the machine would gracefully drain, ideally, it wouldn't matter. Ideally. But the reality is that often they don't, whether because of developer laziness, incompetence, some ugly confluence of bullshit bugs that conspire to prevent sanity, etc., more often than not I want to be careful even when the process/theory would like to say "this is a fully automated & completely safe operation" — and half-asleep is not careful, and accidental allegedly safe commands while half asleep is where Murphy likes to rear his head.
I've definitely used techniques like this. Red prompts that scream "You're SSH'd into PROD dummy" also come to mind. I keep my kubectl context fixed to "local laptop" so that I am forced to always type out --context the_production_cluster when I want that.
Temet nosce.
GuB-42
The idea is to protect you from yourself. You already have root access to the production server, because you are the admin and you need it to do your job.
Fun followup: the image is from the French movie "Le dîner de cons", where the character depicted says "Oh la boulette", after making a gaffe.
ramon156
When you work at a bank and have multiple production boxes, this is pretty typical
roblabla
I mean, you're probably going to need to give the reboot rights to someone. For instance, the people in charge of applying kernel updates. And those people will, at this point, be prone to making a boulette.
f1shy
We used to have special user for such tasks, with long names and password that should make clear what you want to do. For example shutdown/youwillshutdownproductionXXXXXXX (with Xs being a “real” password) because the random part of the password and sometimes the “fix” part was changed, there was no way that muscle memory would take over.
wiether
I guess the name and Pignon's picture only resonate with a French audience, thus why so many serious comments here.
Moi ça m'a fait sourire !
hn92726819
Isn't this a little over engineered? You can accomplish the same thing with a 5-line bash script. put protect.sh somewhere in your path:
#!/bin/bash
if [[ $SSH_TTY ]]; then
read -p 'You are in SSH. Are you sure (enter hostname for yes)? '
[[ $REPLY == $(hostname) ]] || exit 2
fi
exec "$@"
Then in your bashrc or zshrc: alias shutdown='protect.sh shutdown'
alias reboot='protect.sh'
alias sudo='sudo ' # Don't allow sudo to bypass the protection. Can do the same with doas
theamk
It's mentioned in the bottom of the page, but I'd like to highlight that "molly-guard" [0] provides a very similar functionality for at least 10 years.
Since it's so old, it is present in all the systems - "sudo apt install molly-guard" on your server, and your shutdown, reboot, etc.. are all protected, no need for 3rd party tools.
tptacek
I like the idea, but my immediate comment here, in a spirit of thinking about what would get a tool like this wider adoption, is that "boulette" is a clunky thing to type on a terminal. You can be more assertive about the importance of your tool, and give it a shorter, punchier (command-line) name; it might see more use if you do.
wiether
You're not really supposed to type it everytime. Just declare an alias prefixed with boulette replacing the actual risky commands.
If you're in a mental state where you can shutdown a prod server while thinking you're on a dev one, you certainly won't think to type "wtf command", no matter "wtf" actually is.
Fnoord
ZFS snapshots with bootloader support (other CoW filesystem should also work).
For the specific purpose of shutdown, there has been a solution for decades: molly-guard.
rafram
Why does it take the command as a single string argument? Seems like it would make more sense to take it as an argument list, like other "wrapper" commands like sudo, nohup, etc., do.
hk__2
Probably to avoid mistakes such as `boulette some command || some other command`, which would not happen with `boulette "some command || some other command"`.
jean_dupont
It just didn't cross my mind! You've got a point here! I'll rethink the argument parsing.
ncoco
Loving the name "boulette" here, which means "blinder" or "slip-up".
samsk
I've a shell alias for this, requiring valid hostname for reboot, halt etc...
tln
Running "shutdown" on remote hosts isn't something I have ever needed to do routinely.
Is this for ephemeral dev boxes? Does shutdown suspend billing on AWS/cloud type hosts?
jean_dupont
Boulette isn't only for safeguarding "shutdown", it is for safeguarding what needs to be. This is to prevent yourself from some harmeful reflexes.
I my case, I use atuin a lot and automatically retype long commands and sometimes as root.
The regularity of this actions tend to lower my attention, but the fact that many users depend on what I do on big machines and with lots of priviledges isn't less true.
Danger comes from getting used to it.
frizlab
This for the case where you think you type it locally but are actually typing it remotely. But I’d personally never type this locally so this use case is not convincing at all for me.
jean_dupont
This! Another use case: I use it on nixos-rebuild to update the system, when I don't want to rebuild from tiny hosts but rather from a powerfull machine and then send binaries,because it would take to much time to compile otherwise.
voidUpdate
I have a laptop that just runs a bare ubuntu terminal, no DE, and I've accidentally shut down a home server before when I tried to shut down the laptop. Quite annoying, since I was away from home at the time
hk__2
Why do you shut your laptop down instead of just closing the lid?
For me, what is more common is the likelihood of doing something in the wrong environment (e.g. lab, dev, stage, prod). To help make things a bit more clear, our images now override the PS1 with a `(environment)` at the beginning, which is a different color, lab=green, dev=purple, prod=red. If it saves me once, it was worth it.