Skip to content(if available)orjump to list(if available)

Debian Technical Committee overrides systemd change

upofadown

This part seems fairly editorial:

>Debian Policy still cites the FHS, even though the FHS has gone unmaintained for more than a decade.

What ongoing maintenance would a file system standard require? A successful standard of that type would have to remain static unless there was a serious issue to address. Regular changes are what the standard was intended to combat in the first place.

>The specification was not so much finished as abandoned after FHS 3.0 was released...

OK.

>...though there is a slow-moving effort to revive and revise the standard as FHS 4.0, it has not yet produced any results.

So it is not abandoned then. A slow moving process is exactly what you would want for the maintenance of a file system standard.

>Meanwhile, in the absence of a current standard, systemd has spun off its file-hierarchy documentation to the Linux Userspace API (UAPI) Group as a specification. LWN covered that development in August, related to Fedora's search for an FHS successor.

Ah. Systemd/Fedora want a standard that they can directly control without interference from others.

jzb

Author here: It was abandoned. I linked to one of the former maintainers who said as much. The current effort is by a few people who asked the LF to take out over, and have (so far) done little after an initial flurry of activity. That, too, is covered in the other article I wrote about the FHS recently.

Prior to the group who started an update effort, it had not been touched in about a decade. That’s not slow-moving: that’s abandoned.

upofadown

The FHS ultimately belongs to the users collectively, not those maintaining it. I am old enough to remember the horror that existed before the influence of the FHS. It exists in the fact that it is to some extent respected, not because there is a file somewhere that says it is the FHS standard. If you want changes, then sure, do the politics required to develop support for those changes. You can't just declare a new standard and then do whatever you want.

Developers have this thing where they will think of a standard as a specification. Instead it is a statement of political will. Saying that a standard is "abandoned" due to lack of "maintenance" seems like an example of thinking of a standard as the instantation of a specification; an actual program.

dijit

I know it's not the same, but imagine thinking a law is not longer meant to be followed because it hasn't been updated in 10 years.

Hendrikto

Counterpoint: Modern distro’s needs have evolved past the FHS in some cases, and everybody deviates from it slightly but incompatibly.

A standard does no good if it does not reflect reality. I think it is a worthwhile effort to try to bring it back in line with actual real world usage.

dathinab

> What ongoing maintenance would a file system standard require?

adaption to _a lot_ of subtle changes to requirements

- very different security related requirements today

- very different performance related requirements/characteristics

- very different need for various edge cases

and lastly adapt based on what turned out to work well and what didn't

so some examples not already mentioned in the article

- /boot -- dead or at least differently used if you use efistub booting

- /etc/X11 -- half dead on wayland

- /etc/xml, /etc/sgml -- dead, should IMHO never have existed

- also why was /etc/{X11,xml,sgml} every explicit part of the standard when the spec for `/etc` already implies them as long as e.g X11 is used ??

- `/media` -- dead/half dead depending on distro, replaced by `/run/media/{username}/{mount}`

- `/sbin` -- "controversial"; frequent reoccurring discussions that it isn't needed anymore, didn't work out as intended etc. It was useful for very old style thin clients as `/sbin` was in storage but `/bin` was mounted. And there are still some edge cases where it can makes sense today but most fall under "workaround for a different kind of problem which is better fixed properly".

- `/tmp` -- "controversial", long history of security issues, `/tmp` dir per program fixes the security issues (e.g. systemd service PrivateTmp option) but requires having a concept of "programs" instead of just "running processes" (e.g. by systemd services or flatpack programs). Also `tmpfiles.d` can help here.

- `/usr/libexec` -- dead, nice idea but introduces unneeded complexity and can be very misleading in combination swith suid and similar

- `/usr/sbin` see `/sbin`

- `/usr/share/{color,dict,man,misc,ppd,sgml,xml}` -- should never have been in the standard they are implied by the definition of `/usr/share`; at least sqml,xml are dead. dict was for spell check/auto completion, except that neither works anymore like dict expects

- `/var/account` -- to specific to some subset of partially dead programs, shouldn't be in the standard

- `/var/crash` -- distro specific mess

- `/var/games` -- basically dead/security mess, I mean 99% of games today are user per-user installed (e.g. Steam) and even for such which are packed any variable download data is per user, making it shared creates a permission/security mess

- `/var/lock` -- as mentioned there are better technical solutions by now, e.g. using `flock` instead of "presence of file" and some other techniques. Tend to also avoid issues of crashed programs not cleaning up "lock files" leading to dead locks and needing manual intervention.

- `/var/mail` assumes a quite outdated form of managing mail which is quite specific to the mailing program, as it's very program specific it IMHO shouldn't be in the standard

- various legacy program specific, non "generic" file system requirements e.g. that `/usr/lib/sendmail` must exist and be a link to a sendmail compatible program and similar.

also missing parts:

- `/run/user/{uid}`

- `/var/run/user/{uid}`

- `/proc`

- `/sys`

- user side versions (e.g. from the XDG spec which is also somewhat in a zombie state from my personal experience with it , e.g. .config, .local/{bin,share})

- references to light weight sandboxing, e.g. per-program /temp etc.

- factory reset stuff (`/usr/share/factory`) needed for having a uniform way for devices sold with Linux and device specific distro customization(e.g. steam deck)

so yes, it's quite outdated

Starlevel004

> `/usr/libexec` -- dead,

Definitely not dead, the XDG portals and Polkit agents live here.

rascul

The /usr merge is an example of something that a modern FHS might reflect.

curt15

OS X doesn't have a merged /usr. On my Mac I see /bin as a separate directory, not a symlink into `/usr`. Does Linux have a compelling reason that OS X lacks?

dathinab

it makes things more complicated without any benefits (at least not anymore)

this doesn't matter for OS X which main changes mostly tend to be diverging away from it's roots into a fully proprietary direction

but it does matter if you build image based Linux distros which might be the future of Linux

jcgl

Merged /usr is one (increasingly accepted?) means of implementing image-based distros. New OS version, new /usr parition.

mariusor

The reason for having a separate /usr has long slid into obsolescence. Our storage is no longer constrained to require us to mount a remote partition which holds most of the binaries and other sundry required to boot a distribution.

blueflow

I'm very happy that the "independent standard" facade of the UAPI group fell, and its actions are now directly attributed to systemd's interests.

meltyness

Well, it probably depends on which software's concern will be implementing a policy to prevent users from having permission to fill critical directories and prevent the system from operating normally, which is discussed in the article. Which is also a coordination problem because the most common user of disk is software itself, I think.

FHS seems to specifically imbue the user with the responsibility and consequences of filling up the disk.

paulddraper

A more neutral phrasing would be.

> Debian Policy still cites the FHS, and FHS has remained static for over a decade.

palmotea

>> Debian Policy still cites the FHS, even though the FHS has gone unmaintained for more than a decade.

> What ongoing maintenance would a file system standard require? A successful standard of that type would have to remain static unless there was a serious issue to address. Regular changes are what the standard was intended to combat in the first place.

It's 2025, anything that wants to be considered modern (and everything should want that), needs to be undergoing constant change and delivering regular "improvements."

>>...though there is a slow-moving effort to revive and revise the standard as FHS 4.0, it has not yet produced any results.

> So it is not abandoned then. A slow moving process is exactly what you would want for the maintenance of a file system standard.

The FHS people to get off their butts. There's no excuse for that pace now that we have such well-developed AI assistants. They should be pushing quarterly updates at a minimum, and a breaking change at least every year or two. It's been obvious for decades that "etc" is in urgent need of renaming to "config", "home" to "user", and "usr" to "Program Files" to keep up with modern UX trends.

cweagans

I genuinely can't tell if this is satire.

palmotea

I thought it would have been obvious by the "Program Files" at the end :).

Anyway, Linux community as a whole has an antiquated development process, and needs to modernize and follow the best practices of an industry-leading trend-setter, like MS Teams.

Hackbraten

> The FHS 3.0 is clearly reaching the end of its useful life, if not actually expired.

Interesting take.

I think that the FHS is still extremely helpful for packagers, sysadmins and others so they won't stomp on each other's feet constantly. It helps set expectations and prevents unnecessary surprises.

Just the fact that one particular FHS rule might be outdated or even harmful doesn't mean that the FHS as a whole has outlived its usefulness.

dathinab

yes but also no

every distro has defined their own new file system layout standard

sure they all started out with the common ancestor of FHS 3.0, but diverged since then in various degrees

and some modern competing standards try to fix it (mainly UAPI Group)

(And yes some people will go one and one about how UAPI is just a way for systemd to force their ideas on others, but if you don't update a standard for 10+[1] years and aren't okay with others taking over this work either, idk. how you can complain for them making their own standard).

[1]: It's more like 20 years, but 10 years ago the Linux Fundation took over it's ownership.

Steltek

Standards are a double edged sword though. They are great for getting everyone to agree to the "most correct" answer. But they also freeze evolution in place. What happens when your standard doesn't support contemporary use cases? What if it's at direct odds with, say, modern security practices?

FHS hasn't changed in years. Since then, sandboxing, containers, novel package schemes, and more are the zeitgeist. What does the FHS say about them?

lukeschlather

Looking at this specific use case, someone is saying /var/lock being world-writable is an unacceptable security risk, but that's very dependent on what your world/users look like. If anything it sounds to me like the maintainer is trying to make the FHS smaller and remove support for a lot of use cases. (Use cases that sound pretty valid to me, without digging in.)

Hackbraten

> What does the FHS say about them?

Nothing keeps you from following the FHS inside your container or sandbox.

Are you referring to the location where container images live? Then `/var/lib/containers/` and `/var/lib/containers/storage/` would be perfectly FHS compliant.

Steltek

The idea though is when you don't want to follow the FHS anymore, like systemd is doing.

Systemd frustrates and angers people with Poettering's complete disregard for bug reports, tradition, and basic common courtesy. At the same time, change needed to happen and change is gonna hurt. And big changes can't wait until they're just as stable as the old system: does anyone develop software like that in their own careers? I try not to ship complete crap but "just as stable as v1" is never a goal.

baobun

Debian systemd maintainer Luca Boccassi has recently pushed through and dismissed several problematic and undesired breakages as "niche cases" in a way I personally find antithetical to what I expect from Debian.

I hope they have a change of mind in their approach.

blueflow

This is usual systemd maintainer behavior. Poetterings "I don't consider it much of a problem" is legend: https://github.com/systemd/systemd/issues/5644

rpcope1

I don't know of anyone that's been doing this for a while that hasn't been touched by systemd stupidity in some way. I still loathe the default behavior around the stub-resolver with unqualified names that "just worked" before Lennart decided he knew best.

deaux

Interesting coincidence that both are @Microsoft.

kragen

Do we really want Microsoft employees setting standards for Debian?

crest

The fool doesn't know how globbing works, but considers his uninformed guess good enough without testing it or reading (and understanding) the spec.

Y_Y

There is Devuan, if you want to Debian but without systemd. I suspect though that "natively" non-systemd distros will be more consistent, personally I've found happiness with Guix.

zh3

Debian works fine without systemd (at least for now) though Devuan does indeed make like easier.

mx7zysuj4xew

[flagged]

bayindirh

Can anyone tell why systemd developers run fast and loose with what they believe and bully everyone with a stick made out of their ideas?

gwd

In this case, I think the upstream maintainer's response -- "Upstream systemd will do X, distros who want to are free to do Y" -- is legitimate. Consider the reverse: If systemd requires a writable /run/lock, then distros who want to be more safe won't really be able to (or will have to implement a much more intrusive patch).

Looking from the outside, it looks more that this is a failure of the Debian systemd package maintainer to follow Debian's rules. (Though since I'm not a part of that community, I recognize that there may be cultural expectations I'm not aware of.)

bayindirh

> "Upstream systemd will do X, distros who want to are free to do Y"

Yes this is a good response from upstream. I can work with that, but in that case, even this response didn't get reflected to mailing list discussion, or drowned out instantly.

My question was more general though, questioning systemd developers' behavior collectively (hence the projects' behavior) through time.

bluGill

The systemd developers have a long history of reinventing the wheel and trying to force it on everyone. We only put up with them because they do some difficult work that nobody wants to do.

advisedwang

Linux is 34 years old, and some of the Unix-ism borrowed are even older. There is genuine cruft that has downsides. Different relative priorities of backwards comparability, maintainability and the various issues the legacy issues cause are reasonable.

Systemd basically arose out of a frustration at the legacy issues so the whole project exists as a modernizing effort. No wonder they consider backwards compatibility low priority.

dsr_

That has been their method since the beginning; why would they change from a tactic which works for them?

The central problem with systemd is that they don't want to let you go about your business, they want you to conform to their rule.

toast0

Because that's what they've always done, and it continues to work for them?

Systemd doesn't work for me, but it has taken over most Linux distributions, so clearly it's got something people want that I don't understand. That was the case for PulseAudio too.

ho_schi

Did you read the same article?

   * Their is an option for the old behavior.
   * It is a security issue and better solutions to replace exist.
   * FHS isn't maintained.
I think everyone involved would prefer updates to the applications, which fix the issue. Debian opted - for now - for reliability for its users, which fits in their mission statement. On Arch /run/lock is only writeable for the superusers, which improves security. As user I value reliability and security and that legacy tools remain usable (sometimes by default, sometimes by a switch).

cogman10

> It is a security issue

The "security issue" expressed is that someone creates 4 billion lock files. The entire reason an application would have a path to create these lock files is because it's dealing with a shared resource. It's pretty likely that lock files wouldn't be the only route for an application to kill a system. Which is a reason why this "security issue" isn't something anyone has taken seriously.

The reason is much more transparent if you read between the lines. Systemd wants to own the "/run" folder and they don't like the idea of user space applications being able to play in their pool. Notice they don't have the same security concerns for /var/tmp, for example.

em-bee

they don't like the idea of user space applications being able to play in their pool

i think that is somewhat reasonable. but then systemd should have its own space, independent of a shared space: /var/systemd/run or /run/systemd/ ?

gwd

> On Arch /run/lock is only writeable for the superusers, which improves security.

Does it? That means anyone who needs a lock gets superuser, which seems like overkill. Having a group with write permissions would seem to improve security more?

bayindirh

No, I didn't read the whole article. I follow debian-devel directly. Watched all of it unravel, step by step. I know the resolution since the day it posted to debian-devel.

This was a general question to begin with.

> Their is an option for the old behavior.

The discussion never centered on an option for keeping old behavior for any legitimate reason. The general tone was "systemd wants it this way, so Debian shall oblige". It was a borderline flame-war between more reasonable people and another party which yelled "we say so!"

> It is a security issue and modern solutions to replace exist.

I'm a Linux newbie. Using Linux for 23 years and managing them professionally for 20+ years. I have yet to see an attack involving /var/lock folder being world-writeable. /dev/shm is a much bigger attack surface from my experience.

Migration to flock(2) is not a bad idea, but acting like Nero and setting mailing lists ablaze is not the way to do this. People can cooperate, yet some people love to rain on others and make their life miserable because they think their demands require immediate obedience.

> FHS isn't maintained.

Isn't maintained or not improved fast enough to please systemd devs? IDK. There are standards and RFCs which underpin a ton of things which are not updated.

We tend to call them mature, not unmaintained/abandoned.

> On Arch /run/lock is only writeable for the superusers. As user I value reliability and the legacy tools are usable.

I also value the reliability and agree that legacy tools shall continue working. This is why I use Debian primarily, for the same last 20+ years.

crest

In this case I assume their "fear" is that unprivileged users can exhaust resources (inodes, filesystem space) in an important tmpfs breaking the system. The proper solution for backward compatibility would probably be something like make /run/lock its own mountpoint, but they fixed it in their system (Fedora) so now it's no longer their problem. Just be thankful their software is portable to such strange niche operating systems like Debian. /s

ishouldbework

Funny part is that it used to be a separate file system, before Luca decided to kill it. https://lwn.net/Articles/1041948/

Suzuran

[flagged]

bayindirh

Oh, so can we hit people on the nose just because what we say is right?

That's a new one.

What if they are also right, and we start fighting?

Maybe people can discuss instead of bullying each other?

Being able to code doesn't give license to being rude.

jcalvinowens

The "somebody might mailicuously exhaust memory via /run” argument seems silly IMHO... you can trivially put a hard limit on its size via the tmpfs mount option. I guess if you bend over backwards that's still a DOS in that new files can't be created in /run once it is full, but come on...

There is support for quotas in tmpfs! /me runs and hides under desk to avoid fruit being thrown at me

rwmj

It's been a very long time since I heard about uucico (Unix-to-Unix Copy-In Copy-Out program, part of the UUCP suite). Glad to see it's still being shipped! I wonder if any network uses it.

phoronixrly

Non-clickbait title -- Debian Technical Committee overrides /run/lock permission change

amiga386

More-clickbaity title -- Debian Technical Committee tells its doofus maintainers to stop worshipping Poettering so much

lukeschlather

Why is that non-clickbait? Honestly "Debian Technical Committee overrides systemd /run/lock permission change" might be a better title than either, I don't know whether the thing or the actors are more interesting here. But you can only say so much in a title.

okanat

Yeah. I expected better from LWN.

ongy

IMO the interesting bit here isn't the specific technical change but the interpersonal one of overriding the maintainer(s) decision.

Thus the title reflects the most interesting bit of the story.

pjdesno

Is anyone surprised that the systemd folks basically said "fuck you, we'll do what we want to" to everyone else?

scottlamb

> Is anyone surprised that the systemd folks basically said "fuck you, we'll do what we want to" to everyone else?

I don't think that's an accurate paraphrase of "Consider this more a passing of the baton from upstream systemd to downstreams: if your distro wants this kind of legacy interface, then just add this via a distro-specific tmpfiles drop-in. But there's no point really in forcing anyone who has a more forward-looking view of the world to still carry that dir."

That kind of drop-in is pretty routine, so I don't know why this became a big thing we're all discussing now.

ishouldbework

Well yes, but the complication is that Luca is both systemd developer and debian developer, so the passing of the baton did not really happen here.

Barrin92

"everyone else" in this case is pretty much only the debian ecosystem because they insist on enforcing a serial lock policy from the 1980s. It's fine if Debian wants to move at the speed of a Soviet committee but I don't think it should be expected (or would be healthy) for systemd to move at the same pace.

A software developer's primary job is to develop software for their users, not to comply with a third party distributor that repackages their software.

ZeroConcerns

Somehow I feel that if all the time that has been invested in debating and discussing this had been spent on patching the affected apps, the problem would be properly solved.

I mean, yeah, I get it, systemd bad, democracy good, but these world-writable lock folders are actually a huge pain, and adding some shim code to upgrade to a more secure solution seems achievable?

kees99

Genuinely curious - why would world-writeable directory be bad for security? Assuming of course, it's on a separate filesystem mounted with sensible options. Here's what I see from "grep /run/lock /proc/mounts" in sid:

  rw,nosuid,nodev,noexec,size=5120k

seanhunter

The classic is say you know a root process will write a file called foo.lock in /run/lock, and you (a bad person) have write access to that directory. Then you make foo.lock a symlink to some file (/bin/init or /bin/sh or ld.so for example would be very inconvenient choices) and when the root process writes its lock it destroys that file.

Now obviously people these days generally know about that so hopefully don’t use predictable file names but that’s one way.

amiga386

> and when the root process writes its lock it destroys that file.

Unless you do open("/run/lock/foo.lock", O_WRONLY|O_CREAT|O_EXCL|O_NOFOLLOW)

null

[deleted]

mschuster91

> Now obviously people these days generally know about that so hopefully don’t use predictable file names but that’s one way.

Annoying side effect: now you gotta guess which process created the darn lockfile.

A more sensible approach is to do sanity checking on the lockfile and its contents (i.e. does the contained PID match one's own binary).

albertzeyer

The argument is also that you could effectively DoS the system by exhausting space or inodes.

pjc50

Hmm - I see there's now "lockdev" for managing access to things like serial lines, but what's the preferred method of expressing "only one instance of this program should run at any one time"?

jcgl

I don't know what the preferred method is. But so far, flocking on my own executable works for me.

raverbashing

Debian discussions make political discussions seem quick and fast acting by comparison

> He said that he uses cu ""almost constantly for interacting with embedded serial consoles on devices a USB connection away from my laptop""

Whyyyyyyyyyyyyyyy

There are a million better ways of doing this.

kees99

I use GNU screen for that. Looking at cu, it looks to be just as tiny and has ssh-like tilda-escapes, including "~." to disconnect. Nice, gotta try it out, thanks!

ExoticPearTree

I got used to using minicom way way back. Has a nice TUI and can do stuff.

ta1243

I only moved to using screen about 2 years ago after over 2 decades of minicom

raverbashing

cu might be good for anything that you have a fixed config, otherwise I'll just go with minicom

MobiusHorizons

What would you suggest? Personally I find cu the closest to ssh in ergonomics to any of the tui serial terminal programs, and it’s easily available across unixes like FreeBSD or Mac OS, which is a bonus.

munchlax

cu is the regular way of doing it

I don't see the problem. Minicom and even picocom are bloated compared to cu

Hackbraten

One comment [0] highlights a point in favor of the current implementation:

> create a lock file for every dial-in line to prevent its use by programs looking for a dial-out line.

[0]: https://lwn.net/Articles/1042594/

null

[deleted]

tuhgdetzhh

As always, it will just take another decade until debian has figured it out.

bayindirh

Considering I never had to reinstall a Debian system because it got bloated or broke one day, I can accept this slow-cooking approach. Even support them on this regard.

JohnFen

Yes, the slow-cooking approach is one of the main reasons why I prefer Debian.

lousken

once it's ready they'll push it, that's how it should be

gtsop

I am in no rush. I like something being stable.

pengaru

Wow, talk about making a mountain of a mole hill.

Letting upstream systemd single-handedly define what directories exist with what modes in your distro has never been the intended Modus Operandi.

Debian has a huge selection of packages available for it and clearly is going to have more headaches when it comes to preserving compatibility with all that software.

This is a trivial matter for Debian to handle appropriately, while systemd stays focused on its current priorities. I'm surprised this is being talked about at all outside the appropriate mailing lists... slow week for linux news?