Show HN: Interactive systemd – a better way to work with systemd units
107 comments
·January 18, 2025owyn
This looks neat. I have to look up the very fiddly and unintuitive systemd commands all the time. service start? service.foo start? start foo.service? Oh right, sudo systemctl start service.foo
And the feedback is so bad. It should know everything in its own config dir and tell me how to do what I want to do. Was it enabled? I forget. How do I look at logs? Oh right journalctl. Also the layout of things with lots of symlinks and weird directories in places that annoy my 90's linux sysadmin brain. Why am I looking at /lib/systemd/system
I am annoyed by the redundant "systemd/system" directory name every time I have to go there. At this point, just promote it to /etc/systemd and build a better CLI.
As a very occasional linux sysadmin just trying to make things work, the "typing at a console" systemd interfaces are not fun to work with. Maybe nobody should be doing that. In an enterprise, sure that's different. I think interfaces should be human, and linux should still be fun.
pkkm
> I have to look up the very fiddly and unintuitive systemd commands all the time. service start? service.foo start? start foo.service? Oh right, sudo systemctl start service.foo
I don't get this complaint. It's the same order as almost every other command-line utility that has subcommands: <command> <subcommand> <thing to operate on>. To me, that kind of consistency is very intuitive.
systemctl stop my-service
systemctl status my-service
git add my-file
git remote remove upstream
apt install my-package
docker run my-container
adb push local-file remote-file
OlympusMonds
If I had to guess it's because of the 'service' command, which goes 'service foo start'.
It took me ages to unlearn that pattern for using systemctl, even though as you say: it's far more consistent
godelski
I feel the same way. The big part for me is that it tells us that owyn doesn't use tab completion if they're forgetting about the ".service" part. Sure, I don't remember either, I don't have to.
I'll add the abstraction for anyone confused
program [command [subcommand]] [flags] [object]
e.g.s
systemctl status sshd.service
systemctl enable --now sshd.service
touch -c test.sh
echo 'Hello World'
echo "Hello ${USER}"
Anything in brackets is optional and might not appear or be available. By command I mean a category of commands. Such as 'pip install' vs 'pip uninstall', which are sub-programs inside the main program. But this can have layers such as 'uv pip install'. Often flags can be used in any order because you'll just loop over all the arguments but this is still the standard order.There's also the two actor pattern
program [command [subcommand]] [flags] source destination
e.g.s
cp /foo/bar/baz.txt "${HOME%/}/"
scp -i "${HOME}/.ssh/foo" ${HOME}/to_upload.sh user@remote:~/
dd if=/dev/urandom of=/dev/diskToDestroy
rsync /mnt/hdd1/ /mnt/hdd2/
kstrauser
For me:
* /etc/init.d/my-service stop
and Ubuntu’s:
* service my-service stop
both lurk in my brain.
pkkm
Sure, it's different from the old way, but I don't think "unintuitive" is the right word for that. systemd forced people to change their habits so that it could be more intuitive. Of course, people are going to disagree about whether it was worth it - it's the age-old question about breaking backwards compatibility for the sake of minor improvement. Personally, I got used to it pretty quickly and I like it more than the old commands now.
IgorPartola
When it comes to starting and stopping services I want the verb to go last. Way easier to press up, backspace backspace backspace o p to change service ssh start to service ssh top. This is a frequent pattern I follow as I start/stop/restart/reload. Having to go back at least one word adds keystrokes that aren’t necessary.
brontitall
Doing it the way it does allows specifying multiple services.
systemctl status myapp mydb
emmelaich
It actually should be possible to switch them around. No-ones going to call their service 'stop' or 'service' right?
kjellsbells
I agree, but here's a handy Bashism:
^art^op
Converts the "start" in "foo start bar" to "stop", ie runs "foo stop bar". Append :p to do the substitution but print the command instead of running it.
johnchristopher
Do you use your space bar to handle heating ?
gf000
Press alt+B twice?
ripitout
Plus, you could always write a quick shell script to swap the arguments if it detects this specific failure. Inability to remember or learn which commands expect which arguments is, at some point, not the responsibility of the software to fix. Unfortunately, this field does necessitate some amount of memorization and ability to problem-solve.
zamadatix
My biggest annoyance is "systemctl status" gives you just enough of the service's log to make the output take up most of the terminal each time you run it but never enough of the service's log to get a useful picture of what's actually happened with the service lately.
Not to mention unless the problem with the service completely prevented it from running (it advises some commands to run in that case) you're supposed to just always remember "journalctl -xeu $SERVICE" was the incantation, less you want to go look up the flags again or manually parse the entire "journalctl" output.
Overall I generally like systemd though. The syntax can just be a burden sometimes.
kristopolous
it's the same mentality that brought us the git design - the easiest, least typing options are rarely, if ever the thing you want to do.
Instead, these invocations give cryptic messages, throw errors, or sometimes, even break things.
The most common and helpful things are hidden deep behind multiple flags and command line arguments in manuals that read like dictionaries more than guides.
I'm always at a complete loss as to how such decisions are made. For instance, "git branch -vv" is the useful output you would like to see, every time, that should be "git branch". Why not make the current output, "git branch -qq"? Is a humane interface too much to ask for? Apparently...
I know people defend this stuff, but as a senior engineer in programming pits for 30 years, they're wrong. Needless mistakes and confusions are the norm. We can do better.
We need to stop conflating elitism with fucked up design.
godelski
> Why not make the current output, "git branch -qq"? Is a humane interface too much to ask for? Apparently...
Yes, it is too much.You have the wrong mentality, and I hope this can help make your life easier. Programs are made so that the simplest option is the base option. This is because there is a high expectation that things will be scripted AND understanding that there is a wide breadth of user preference. There's an important rule
DON'T TRY TO MAKE A ONE SIZE FITS ALL PROGRAM
Customization is at the root of everything. We have aliases that solve most of the problems and small functions for everything else. You default to an non-noisy output, showing only __essentials__ and nothing more unless asked. Similarly, you do no filtering other than hidden files. This way, everyone can get what they want. Btw, this is why so many people are upset with default options on things like fdfind and ripgrep.For your problem with git, there are 2 solutions you have.
# alias git branch using git
git config --global alias.branch 'branch -vv'
# write a simple function and add to .bashrc or .zshrc or .*rc
git() {
case "$1" in
branch)
shift
command git branch -vv "$@"
;;
*)
command git "$@"
;;
esac
}
> We need to stop conflating elitism with fucked up design.
The design isn't fucked up, it is that you don't understand the model. This is okay. You aren't going to learn it unless you read docs or books on linux. If you learn the normal way, by usage, then it is really confusing at first. But there is a method to the madness. Things will start making more sense if you understand the reason for the design choices. (In a sibling comment I wrote the abstraction to command patterns that makes the gp's confusion odd. Because systemd follows the standard)Side note: if you try to design something that works for everyone or works for the average person, you end up designing something that is BAD for most people. This is because people's preference is not uniformly distributed, meaning the average person is not representative of any person in the distribution. This is because anything that is normally distributed has its density along the shell while a uniform distribution has a uniform density all throughout it.
Denvercoder9
In case you don't know, you can use the `-n` argument to `systemctl status` to tweak the log output, e.g. `-n0` to disable the log output and `-n40` to get more than the default 10 lines.
zamadatix
That's a great option to tweak the behavior and I hadn't known about it (or if I ever had, I'd well forgotten). Thanks!
From the man page it ?looks like? if you want reverse or full then it's still off to the journalctl command and arguments but at least "-n9999" is better than "always 10 lines".
kai-tub
Nice! I didn't know that was an option. Definitely something I should make configurable in `isd` :+1:
SJC_Hacker
> My biggest annoyance is "systemctl status" gives you just enough of the service's log to make the output take up most of the terminal each time you run it but never enough of the service's log to get a useful picture of what's actually happened with the service lately.
How about
systemctl status foo | tail
Denvercoder9
> I am annoyed by the redundant "systemd/system" directory name
It's not redundant, you also have /etc/systemd/user (and /lib/systemd/user) where units that run in the user context (as opposed to system-wide context) are stored.
godelski
It's also worth noting that this is a fairly standard pattern in /etc
/
| etc
| | fail2ban
| | | action.d
| | | fail2band.d
| | | filter.d
| | | jail.d
| | ssh
| | | ssh_config
| | | sshd_config
| | systemd
| | | network # network wide context
| | | nspawn # containers
| | | system # system wide context
| | | user # user wide context
Personally I like it more than /
| etc
| | cron.d
| | cron.daily
| | cron.hourly
| | cron.monthly
| | cron.weekly
| | ...
| | firewall
| | firewalld
Keeps things less cluttered. Hierarchical categorization is >> than lateralgreenavocado
This is why I have instated a policy of using systemd --user services whenever possible.
If you don't need elevated permissions this is ideal.
All you have to do is enable linger using loginctl if you want your service to auto start as the user on boot unattended.
Your user services live in ~/.config/systemd/user
bityard
I used to do this but frankly it's easier to run a system-level unit as whatever user you want and keep all the files in /etc instead of scattered around /home.
The user-level units are most useful when running an actual multi-user system. If you trust your users to not abuse them, anyway.
rcxdude
I find that at least systemd means that it's consistent across distros. I spent way more time looking up this kind of thing when every distro rolled their own init system.
PhilipRoman
I just wish systemd allowed abbreviations for the subcommands, like "ip" (and git, for long options).
kai-tub
Author here: Yeah, I agree.
It is a bit weird. On the one hand, I understand that it makes sense to have [command] [verb] [object] on a "logical" level and that viewing logs should be a separate command (`journalctl`), but it is definitely not ergonomic. Especially if you frequently have to switch between start/stop/restart.
> As a very occasional linux sysadmin just trying to make things work, the "typing at a console" systemd interfaces are not fun to work with. Maybe nobody should be doing that. In an enterprise, sure that's different. I think interfaces should be human, and linux should still be fun.
This was precisely the case for me. I "enjoy" playing around with systemd and am super interested in better understanding it, but the feedback loop just felt sooo slow. So hopefully this TUI can make it "fun" again :)
kjkjadksj
It is a bit crazy to me how everyone says “dont use cron systemd is in now” but cron just does what it says on the tin with no problems. I have lines that work fine ran in script or on my crontab but when wrapped in a launchd command no longer work (log says things work until the db is to be updated which tells me launchd ran processes lack sufficient permissions perhaps to update my db but its not clear why this is the case or how I can elevate launchd sufficiently.
MrDrMcCoy
> cron just does what it says on the tin with no problems.
I can name a rather large problem with cron that systemd timers solve handily: long-running job duplication. When jobs take longer to run than the space between their triggering times, duplicates start piling up. I've had to rescue numerous systems from such states, which are difficult to detect until things have gotten quite bad. Sure, you can write a bunch of boilerplate to handle this yourself with cron, but with systemd timers it's all handled for you along with other niceties like capturing all output in journald and ensuring that the next run starts at soon as possible.
viraptor
Launchd is not used in systemd - why would anything be wrapped in it? The config is simple - User for the user and Group for the group. You can print it the result of "id" if you're not sure what the result is.
kjkjadksj
Sorry I conflate the two since they share a lot of similarity. Launchd is what I use as its a macos system.
ww520
Looks nice.
One thing I found systemd really confusing was its treatment of ExecStop in a service script. ExecStart is the command to run when systemd starts the service at system boot up (or when a user tells systemd to start the service). However, ExecStop is run when the starting command has finished running. You have to set RemainAfterExit=yes to have the desired function of running the stop command on system shutdown or on user stopping the service. ExecStop is basically the "on-cleanup" event rather than "to-shutdown-the-service" event.
Cyph0n
I think about them as “on start” and “on stop”.
It is important to keep in mind that systemd is tailored towards daemons. So if your service just runs a command that eventually exits, you need to explicitly tell systemd to treat it differently than a daemon.
Edit: As others noted, you’re probably looking for oneshot + RemainAfterExit.
rcxdude
It is a little asymmetric, because 'ExecStart' is actually normally 'Executable that is the service', not just script that starts the service, but I think that's a hangover from the self-daemonizing approach to init scripts.
Cyph0n
True, but it still makes sense to reason about them this way. Say you have an HTTP server:
- on start: start the server
- on stop: do nothing, because you are already terminating the server
But suppose you need to perform an additional task when the server is terminated. That is where you would add a ExecStop command or script.
wasted_intel
Love this. I use raw CLI commands until it hurts, and have recently embraced tools like lazygit/lazydocker to get visibility into otherwise opaque system/tree states, and it’s been a huge level-up.
I have several user and system level services I manage, but debugging them is tedious. Your opening line that lists common commands and their pain points really resonated with me.
I’m on NixOS, so editing immutable unit files directly won’t work, but the service discovery, visibility, and management will be really helpful. Nice work!
kai-tub
I am also a NixOS user and this is exactly what motivated me to work on this project!
I am planning on adding some "guides" in the documentation but in short: You should check out `systemctl edit --runtime` for debugging units on NixOS. It makes debugging sooo much easier.
mahoro
So cool that with uv it becomes so easy to install such tools.
What's missing in the install routine is uv installing this tool ignoring the Python dependency. My box has 3.10 and isd won't work with it. Fixed with `-p 3.13` option. May be worth mention in the docs.
Fnoord
At long last, systemd-client: merry meet! Next step is such a TUI for non-Linux such as macOS, FreeBSD, Windows. For macOS I use LaunchControl.app but it isn't a TUI.
Just one thing: I had to do
$ uv tool install git+https://github.com/isd-project/isd
instead of $ uv tool install https://github.com/isd-project/isd/isd@latest
and uvx wouldn't work at all, version: uv 0.5.21. That said, uv is way more quick than pip(x) so I just switched.kai-tub
Yeah, silly me. It is fixed now, thanks for letting me know!
elric
Looks great, well done. It's a shame that it's needed at all. The vast majority of my interactions with systemd are trivial: (re)starting a service, looking at a log file to figure out what's wrong, and making sure a service starts on boot. I find it baffling that the ergonomics of systemd for those common tasks are so lacking. But the TUI seems to help, so thanks.
And sure, systemd it's more deterministic and includes the kitchen sink, unlike initd.
Thankfully these days I can automate most of such interactions out of existence, so I no longer feel the burning hatred that I once did. More like a smoldering ember.
kai-tub
I mean, I get why they don't want to "bloat" systemd with a complex/opinionated TUI, but I would've also really liked a more "upstreamed" interface. Though, I guess making systemd more beginner-friendly/accessible is not really that important for their funding?
ripley12
This looks very good, thanks for sharing! I maintain a similar project and working with the systemd/dbus APIs has been pretty painful; eager to try this and see what I can learn from it.
gurgeous
This is incredible! I will use this a ton. Only thing missing is a deb package...
amelius
Is there any way I can run a service before a given other service AND as late as possible (without rewriting the other service's unit file)?
throeurir
I can not install this stuff on remote servers and docker images. I would like multiple backbends to execute commands and gather informations (local, ssh, docker).
It should be installable locally, and run commands on remote machine via ssh! And via 'docker exec' commands.
johnchristopher
> If you ever became frustrated while typing:
Hey, that's me ! (And I love systemd !)
I haven't installed it yet so quick question: can it connect to remote host ? I often use systemctl --host <hostname> status foo.service (status, timers, logs etc. )
kai-tub
Dang. I have never heard of `systemctl --host`. Sadly not. It is more or less a fancy wrapper around (the local) `systemctl`. But there is also an appimage that should make it (hopefully) easy to run it on remote servers.
Either way, feel free to open an issue and I will have a look at it.
diggan
> Dang. I have never heard of `systemctl --host`. Sadly not. It is more or less a fancy wrapper around (the local) `systemctl`.
Sounds like you could easily support it by letting users pass in $REMOTE_HOST and when you use your `systemctl` wrapper, add `$CMD --host=$REMOTE_HOST`, after that everything should work as before.
kai-tub
I will investigate it!
gchamonlive
How's security handled? Not in terms of system permissions which Linux handles well, but in terms of guarantees that it can't be hijacked and remotely controlled by an external attacker.
kai-tub
Author here: I also find this an important thing to ask yourself when you are running applications/scripts that do anything with sudo and which is why I have written a fairly in-depth "Security" section on the isd documentation page:
https://isd-project.github.io/isd/security/
Let me know if anything is missing!
yonatan8070
Looks super cool, I've been working quite a bit with systemd recently, and typing systemctl and journalctl + their flags gets old rather fast.
Can it connect to remote hosts like you can with systemctl --host?
I created a TUI for systemd/systemctl called isd (interactive systemd).
It provides a fuzzy search for units, auto-refreshing previews, smart sudo handling, and a fully customizable, keyboard-focused interface for power users and newcomers alike.
It is a more powerful (but heavier) version of sysz, which was the inspiration for the project.
This should be a huge timesaver for anybody who frequently interacts with or edits systemd units/services. And if not, please let me know why! :)