C stdlib isn't threadsafe and even safe Rust didn't save us

Indeed, environment variables should be used to configure child processes, not to configure the current process, for non-shell programs, IMHO.

Note that Java, and the JVM, doesn't allow changing environment variables. It was the right choice, even if painful at times.

You can’t convince me that there is EVER a reason to call setenv() after program init as part of a regular program, outside needing to hack around something specific.

Environmental variables are not a replacement for your config. It’s not a place to store your variables.

Even if the env var API is fully concurrent, it is not convention to write code that expects an env var to change. There isn’t even a mechanism for it. You’d have to write something to poll for changes and that should feel wrong.

Environment variables are a gigantic, decades-old hack that nobody should be using... but instead everyone has rejected file-based configuration management and everyone is abusing environment variables to inject config into "immutable" docker containers...

> As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv.

This holds for a lot of programs, but what if you're writing a shell?

The underlying problem isn't just setenv, because the string returned by getenv can be invalidated by another call to getenv. ISO C says:

"The getenv function returns a pointer to a string associated with the matched list member. The string pointed to shall not be modified by the program, but can be overwritten by a subsequent call to the getenv function."

In a single threaded virtual machine, you can immediately duplicate the string returned by getenv and stop using it, right there.

Under threads, getenv is not required to be safe.

I think that with some care, it may be; an environment implementation could guarantee that a non-mutating operation like getenv doesn't invalidate any previously returned strings.

I think POSIX does that. It allows getenv to reallocate the environ array, but not the strings themselves:

"Applications can change the entire environment in a single operation by assigning the environ variable to point to an array of character pointers to the new environment strings. After assigning a new value to environ, applications should not rely on the new environment strings remaining part of the environment, as a call to getenv(), secure_getenv(), [XSI] [Option Start] putenv(), [Option End] setenv(), unsetenv(), or any function that is dependent on an environment variable may, on noticing that environ has changed, copy the environment strings to a new array and assign environ to point to it."

environ is documented together with the exec family of functions; that's where this is found.

So whereas there are things not to like about environ, it can be the basis for thread safety of getenv in an application that doesn't mutate the environment.

> As far as I'm concerned, the environment is a read-only input parameter set on process creation like argv.

Mutating argv is actually quite popular, or at least it used to be.

This is actually not that hard to fix.

Getenv() could keep several copies of the value around: one internal copy protected by a mutex, that it never returns, and one copy per thread that it stores in thread local storage. When you call getenv(), it locks the mutex, checks if the current thread's value exists, populates it from the internal copy if not, and returns it. It will also install a new setenv-specific signal handler on this thread and store info about this thread having a copy.

Setenv() will then take the same mutex as getenv(), check if the internal copy is different from the new value; if it is, it will modify the internal copy, modify the local thread's copy if that has one, and then signal each other thread in the process that has a copy in TLS. The setenv signal handler will modify the local copy that thread holds.

It's gonna be slow for a large multi-threaded program, but since setenv() used to corrupt memory for such programs, they probably don't care. And for single-threaded programs, or even for programs that don't access getenv()/setenv() on multiple threads, there should be no extra overhead other than the mutex and the bookkeeping.

The only issues that would remain are programs which send the pointer they get from getenv() to other threads without ensuring locking access, and programs which rely on modifying the pointer from getenv() directly as a way to set an env var, and expect this to be visible across threads. Those are just hopelessly broken and can't use the same API - but aren't more broken then they are today.

Of course, in addition to this complex work to make the old API (mostly) thread safe, it should also offer a new API that simply returns a copy every time, doesn't promise to show modifications to your copy when setenv() gets called (you need to call getenv() again), and puts the onus on you to free that copy explicitly.

According to ISO C, getenv returns a pointer to storage that can be overwritten by another call to getenv! Only POSIX slightly fixes it: the string comes from the environ array, and operations on environ by the library preserve the strings themselves (when not replacing or deleting them), just not the array. A program that calls nothing but getenv is okay on POSIX, not necessarily on ISO C.

C could provide functions to lock/unlock a mutex and require that any attempt to access the environment has to be done holding the mutex. This would still leave the correctness in the hands of the user, but at least it would provide a standard API to secure the environment in a multi threaded application that library and application developers could adopt.

They can have copy on write of course.

The process's current directory is mutable global state as well, and yet chdir(2) is thread-safe.

Welcome to the C standard library, the application of mutable global state to literally everything in it has to be the most consistent and predictable feature of the language standard.

I mean, it's only threadsafe in the sense that opening a file in cwd without being able to actually know what cwd is is "safe"

That will break if any code which is not aware of the mutex calls getenv, even for a variable not related to the communication.

This is neither an OS nor a machine scope problem. The environment is provided by the OS at startup. What the process does with it from there on is its own concern.

Agree about performance, but wouldn't there need to be >1 mutex to risk a deadlock?

The problem (with get/set/putenv as they are) was isn't the non-use of a mutex. It's the "meaning" of the pointer returned to by getenv(). It returns a char*. Nevermind the persistance of that value - you can work around that by deliberately leaking memory - but it's writeable. Whether it's a good idea to do so ... well. But simply locking "inside" these funcs doesn't solve all the / your issues.

I've only really seen dangerouslySetInnerHTML used while transitioning from certain kinds of server side rendering to React. There is still lots of really old internal tools in ancient html out there.

Code syntax highlighting libraries for react use dangerouslySetInnerHTML.

React doesn't have a tag and attribute sanitizer built in, so having non-js-programmers edit JSX isn't especially safe anyways, as an img or a href could exfiltrate data. If it were they could just block out an innerHTML attribute. A js programmer can get around it by setting up a ref and then using the reference to set innerHTML without the word dangerously appearing.

But the standard implementation could copy the environment at startup, and only uses its copy.

And the library's use of setenv is clearly a bug as setenv is documented to be not threadsafe in the C standard library. So that would take care of that problem.

Is it possible to skip libc completely or would this introduce too many portability concerns?

libc does do locking, but it's insufficient. The semantics of getenv/setenv/putenv just aren't safe for multi-threaded mutation, period, because the addresses are exposed. It's not really even a C language issue; were you to design a thread-safe env API, for C or Rust, it would look much different, likely relying on string copying even on reads rather than passing strings by reference (reference counted immutable strings would work, too, but is probably too heavy handed), and definitely not exposing the environ array.

The closest libc can get to MT safety is to never deallocate an environment string or an environ array. Solaris does this--if you continually add new variables with setenv it just leaks environ array memory, or if you continually overwrite a key it just leaks the old value. (IIRC, glibc is halfway there.) But even then it still requires the application to abstain from doing crazy stuff, like modifying the strings you get back from getenv. NetBSD tried adding safer interfaces, like getenv_r, but it's ultimately insufficient to meaningfully address the problem.

The right answer for safe, portable programs is to not mutate the environment once you go multi-threaded, or even better just treat process environment as immutable once you enter your main loop or otherwise finish with initial process setup. glibc could (and maybe should) fully adopt the Solaris solution (currently, IIRC, glibc leaks env strings but not environ arrays), but if applications are using the environment variable table as a global, shared, mutable key-value store, then leaking memory probably isn't what they want, either. Either way, the best solution is to stop treating it as mutable.

The real problem is that getenv() and setenv() were created before threads were really a thing.

Well it was better in the short term but is worse in the long term. In particular, the error handling situation is generally atrocious, which is fine for interactive/sysadmin use but much worse for serious production use.

That still doesn't mean getenv would be safe. Unless you know nothing uses **environ (e.g. by breaking the ABI, which no-one will do because it'll break everything), you can't rely on getenv being safe.

Yeah I don't think I've ever seen a single use of it. However I just checked on grep.app and at least a few big softwares use it - git, nginx, Postgresql, neovim, etc, which suggests that setenv/getenv is not sufficient.

That isn't thread safe, it's safER.

I am also quite technical, thanks.

Considering this is a libc issue, not a Linux specific one, I wonder how thread safe other libc implementations like musl and Bionic are. How do the BSDs stack up? Humorously, illumos also ships with glibc...

I think you would have to change the API to return a copy of the string as the get_env result which the caller is responsible for free-ing or the env implementation would have to ensure returned values from get_env are stable and never change which is effectively a memory leak.

> Yup, it's mostly just the story and tools we used to get ourselves out of a mess that was made harder by some decisions made earlier -- the tests were running in a container with stripped symbols (we're going to ship symbols after this, no reason to over-optimize)

It's worth noting here that you can also build your binaries and keep debug symbols separately.

You don't need to ship them with the binary (although it will make many scenarios a bit simpler if you do, since you'll always have the right ones available).

Some info that might help: https://www.tweag.io/blog/2023-11-23-debug-fission/ https://undo.io/resources/gdb-watchpoint/reduce-binary-size-...

> we're going to ship symbols after this, no reason to over-optimize

You might want to look into debuginfod.

It really does not look like a good idea to setenv() . The very notion is quite terrifying. Messing with a bunch of globals, that other code knows about as well? Nuh-uh.

The thing is, the OP people weren't doing that at all, it was some irresponsible library maintainers. If your code does that, you have to include something like the "surgeon general's warning" everywhere: "CAREFUL: USING THIS LIBRARY MAY CAUSE TERMINAL CRASHES".

There are other ways that could be achieved, like cloud storage constantly mirroring local but encrypted with local keys or keys controlled by the user.

This is the iCloud model and it works. Imagine a more open version with competing storage providers.

This, however, would hand control back to the user, which would be bad for the software industry with its addiction to lock in and recurring revenue.

The problem is that applications sometimes need to set environment variables which will be read by libraries in the same process. This is safe to do during startup, but at no later times.

Ideally all libraries which use environment variables should have APIs allowing you to override the env variables without calling setenv(), but that isn't always the case.

I guess the usecase is

    dlopen()

    DEBUG=verbose

Don't threaten me with a good time.

Ah yes, the cloud where we all happily share compute resources without any restrictions to avoid stomping on each others toes.

The universe, you mean.

just because someone makes a window doesn't mean you gotta jump out of it. there are good and bad uses for things, and the bad ones should be avoided lest one hurt themselves?

Yes, neither were memory or thread safe

Sure, I’m speaking about platforms that are relevant today, not historical ones. Windows, MacOS, {Free,Open,Net}BSD, Solaris, illumos, none of these do.

Sure, there's a broader concept here, which doesn't require any operating system. But any alternate string->string map you define won't answer to C code calling getenv, won't be passed to child processes created with fork, won't be visible through /proc/$PID/environ, etc.

Well, it's used by the OS when exec-ing a new process, but at least the Linux syscall for that takes the environment as an explicit parameter. So it could be managed in whatever way by the runtime until execve() is called.

Environment variables are not just technical, they're social. You need to get everyone on board with your scheme.

> Why not use Eyra?

Well, that's a lot of caveats. As I said, it would take years to complete. And it looks like it's well on its way but not near complete.

If I understand it correctly this still doesnt help with downstream dependencies.

That's quite a trade-off

That only works on Linux though right?