NixOS Is Not Reproducible
68 comments
·October 26, 2024poincaredisk
uecker
make and compilers are usually reproducible. Most Debian package are reproducible, so this is perfectly feasible.
Foxboron
You would think so, but after close to 10 years as a package maintainer it's clear to me that people don't know how make works when you do start doing anything else than `-j1`.
The gcc compiler also fails to reproduce binaries with the LTO streamer backend if you use certain macros. This has the consequence of making cgo binaries unreproducible.
As for Debian, we don't really know how many packages are reproducible. The CI they show is for fuzzing out issues, there are no system currently reproducing distributed packages.
petterroea
It is completely fine that NixOS is not reproducable for me. I personally switched(from arch) because my old setup was very finicky(eGPU + optimus on the go for iGPU) and i was sick of random updates bricking stuff. I'm becoming too old to enjoy spending an hour+ tinkering with stuff that suddenly broke. Now it is hard to imagine going back. I just enjoy that applying all my customizations is a single action.
But Linderud has a good point, and I think a community like HN is rightfully interested in busting myths like this. I didn't know how the hashes worked, I haven't needed to have a relationship to them. But now I do!
kstenerud
I'm having a hard time thinking of a case where 100% bit-for-bit reproducible builds would be a necessary thing. As long as I can rebuild and get a system that works the same way as before, I'm happy.
And I don't even need Nix to do that; there are much simpler ways to do so.
dwb
It's a fine point to make, but I get the feeling some people's perceptions (that Nix introductory docs are unclear) are out of date. From nixos.org, I click "Learn", "How Nix works", and it says in the first section that the hashes are "a cryptographic hash of the package’s build dependency graph".
BimJeam
I once tried to use it but it simple was not able to use Android-SDK /adb in it. Then I never used it again and fell back to arch linux.
spease
I have tried to use it for C++, iOS, Python, Flutter, Docker, MacOS binaries, an Ubuntu container, MacOS, and NixOS. In every case it became a time sink that failed to work due to broken packages and convoluted code.
Core issues are that:
(1) To meet nix’s goal of declarative package management, everything in nix wraps software to create a bespoke interface for nix. But the documentation for the nix interface is extremely spotty and inconsistent, where it exists at all.
(2) The language is a mix between functional and shell code, to create a declarative spec. This means many mental gear shifts while reading. This is made even worse by the nixpkgs API being inconsistent, with case-specific variants of functions (eg to set a property on a package you might need to call a language-specific property setter, which is very confusing when a package is built with multiple languages)
(3) Many packages are broken or unsupported on one OS or the other.
Because of these, it means that you need to be a coder AND have deep Linux knowledge AND deep Nix knowledge to use it for an extended period of time.
And then:
(4) Nix does not cleanly integrate with the packaging for a language, or expects running an AOT tool to generate a bespoke Nix derivation from the standard tooling
I and others have complained about Nix being too difficult to use for years, but the Nix community instead is more excited about flakes, which are even more convoluted than existing derivations and metastasize the existing architectural issues in nixpkgs into countless decentralized packages that will now need to be refactored if someone tries to overhaul the design of nixpkgs.
From my perspective, nix flakes are like if Linux was struggling with adoption because its internal and user APIs were undocumented and frequently broken, and the kernel developers got really excited about moving all the drivers out-of-tree into separate repositories because that’s “best practices”.
A killer cross-platform and project / system package management system with hermetic build environments would be a godsend. But nix’s practical implementation of that is too badly done for me to have ever been able to use it for anything but the tiniest most specialized of projects, or as a partial package manager for macOS (where I still have to install things manually or use homebrew).
There are a great many things that are theoretically possible with Nix, but I don’t know anything that I could practically recommend it for.
viraptor
> Many packages are broken or unsupported on one OS or the other.
Report them. There's thousands of packages. Efforts like Zero Hydra Fails help, but there's still a lot to fix. If you raise an issue, it will be prioritised, because we'll know someone actually cares about it.
Keep in mind that's compared to a small percent of those packages available in other systems where you're on your own - not much different than on nixos with a broken package.
exe34
$ nix-shell -p android-tools
[nix-shell:~]$ adb --version
Android Debug Bridge version 1.0.41
Version 35.0.1-android-tools
Installed as /nix/store/dxdygi06ixq7y419hmc4skz4cdzbcnwf- android-tools-35.0.1/bin/adb
Running on Linux 6.10.3 (x86_64)
pjmlp
I know Linux since Slackware 2, have lost count of how many distributions I have tried since that 1995's Summer, and still see no point on bothering with NixOS.
EDIT: As usual in some Linux forums, I got a bunch of marketing replies about all the cool ideas NixOS is supposed to bring as if I haven't read about them myself in LWN and other Linux sources I routinely check on.
wolletd
I still haven't found time to work with Nix or NixOS, but I like it's concept.
At work, I'm maintaining a bunch of custom configured Linux images for our products. While building those images through Dockerfiles is pretty declarative, my pain starts with keeping the configuration of systems in the field up to date: half of the build instructions is duplicated in Ansible playbooks.
Conceptually, with Nix(OS), I could maintain a single, declarative configuration that I would use to create new images and simply push to existing machines an re-configure them.
But I don't see getting something like that production-ready in the near future. I'm currently thinking of only building base images with docker and doing all product-specific configuration through ansible.
matrss
> While building those images through Dockerfiles is pretty declarative [...]
There is nothing declarative about Dockerfiles, they are a list of imperative steps. Same for ansible playbooks, which are also often called declarative when they really aren't.
h0l0cube
They are imperative steps with aspirational idempotency, and specifically Ansible is painfully slow to boot. I've noticed that order often becomes necessary in practice, so I'm not sure if this imperative nature is avoidable.
SvenL
Did you try using „immutable“ infrastructure? I mean instead of updating existing container, you just spawn a new one with the new configuration, add it to the load balancer and decommission the old one?
wolletd
Our products are hardware, I use docker to build and manage bootable images.
siwatanejo
its
SuperSandro2000
Most others just follow the same concept of running imperative commands and not much about them is revolutionary. NixOS gives that a new spin.
pdimitar
We know the beautiful theory. Show me something easy to onboard and use. I've learned no less than 500 separate tech pieces in my life, Nix and any others will have to make it worth my while in the first 10 minutes.
CapeTheory
I like the idea of it, but the Nix people seem to have really screwed the pooch. On the social front we have tedious community discussions about whether it is OK to be funded by a defence contractor, and on the technical front the switch to new Nix commands and flakes has left the documentation in a horrible mess. I think if things had gone differently over the last 18 months we could've seen some really significant adoption, but I have no hope of that any more.
agubelu
I think one of the issues with Nix's community is that they see using Nix as a goal on its own, not as a means to an end.
99.5% percent of people don't use Docker because they have a passion for writing dockerfiles, or because they find the ideas behind Docker elegant. They use it as a tool to help build other things. Meanwhile, the Nix community keeps pushing out these ideas about how Nix _works_ ("reproducible builds!", "lazy evaluation!") but they don't seem to particularly care about making Nix easy to use for the majority of the population out there who may want to use Nix as tool and don't care about its technical merits.
On top, I think most projects would benefit from having more functional programming advocates in them. Nix suffers from the opposite problem: it would benefit from having _less_ passionate FP people in it. Nix the language is pretty inscrutable for lots of people.
poincaredisk
>on the technical front the switch to new Nix commands and flakes has left the documentation in a horrible mess
Did it ever happen? Last I've checked flakes were still experimental.
arianvanp
Our contributor graph is still growing exponentially and seems mostly unaffected. I'm not too scared about it. There is a lot of work to do though. Especially if we want to go mainstream
exe34
Once I started using NixOS, I can't see the point of all the other distributions - it's so easy to modify/remove/overwrite something and mess up your install, whereas in NixOS, I tell it what I want the system to look like and it will bring it back. After every upgrade, my system is exactly the way it was before - nothing's moved, nothing's changed, I don't have to go reset some settings, etc. In a way, it's taken the adventure out of upgrades.
null
anthk
With Guix and Nix you often you don't run IDE's in the traditional way. You write a declarative file with all the processes, dependencies and so on. I know it would suck for GUI programming with Java/C#/Vala, as often some packages are not granted as dependencies, such as gcc/autohell/libtool if you install some C++ iIDE, but that's it.
PhilipRoman
Good article. I was really surprised that the hash is not based on built artifacts. In hindsight it makes sense, since making every package reproducible upfront would be a ton of work.
SuperSandro2000
That will change with content addressed derivations but they are progressing slowly exactly because of this.
Aleksana
In fact, reproducing artifacts will have more uncertain factors without a correctly designed tool chain. Accidentally introducing timestamps is just one common case, others include the order of multi-threaded builds, Haskell's GHC compiler has had this problem for a long time, and sometimes Rust seems to be affected as well. There are also some tricky situations, which are related to the features of the file system.
It is misleading to say that NixOS has complete reproducibility. However, NixOS is indeed more advanced in some aspects. You can reproduce a working development environment, you can easily test the build process you will deploy to CI locally, move your favorite configurations between computers, or share certain parts between them. This is the benefits of declarative package management (or package build system).
Unreproducible situations do occur during the building process of packages (derivation), but these situations usually have nothing to do with functionality, and it is easy to see where the problem is with diffscope. We have specifically tracked and dealt with these issues. At present, they are still within a relatively controllable range.
See
https://reproducible.nixos.org/
https://github.com/NixOS/nixpkgs/issues?q=sort%3Aupdated-des...
I would also thank @anthk for introducing people to Guix. As a late sibling to Nix (the ecosystem), Guix is cleaner in handling the basic toolchain and does a better job in full source bootstrapping, which we are working hard to learn and improve. However, Guix and Nix have the same constraints, they can only reproduce the way the system is built and composed, not the artifact itself. We both track reproducibility problems, and there are interested maintainers actively addressing these issues.
See
https://issues.guix.gnu.org/search?query=reproducible
When it comes to whether you should choose NixOS (or Guix), I think you should weigh your workflow, usage habits, and learning costs. Given the significant difference in the way of thinking, we can't make it painless for people to migrate from distributions like Debian/Arch or Ubuntu to NixOS. But if you really need it, I believe you will still come one day
szundi
Probably much better repdoducibility than embedded developers trying to maintain thet while upgrading their ubuntus thy work on.
blacklion
> A build is reproducible if given the same source code, build environment and build instructions
Same bits into input - some bits out of output.
Edit: formatting.
tazjin
You're either assuming that "build environment" includes more things than the author does (it's not exactly a well-defined term), or you're not aware of the sources of non-reproducibility.
It can be simple things (something includes timestamps in binaries, which can be worked around by pinning the time), and complex things (concurrent linking of binaries is done FIFO-style, and different compilation units finish at different times in different runs because of varying processor load).
rnhmjoj
> It is very strange definition. To me it is tautological, as computers are deterministic and if you have same build environment you will get same results by necessity.
There is a lot of randomness in a typical build system: randomly ordered lists, race conditions in multithreaded coded, timestamps in the filesystem, etc, The worse kind of build systems can even include fetching files the internet. So, no, for all practical purposes actual computers are not deterministic and this is not tautological at all.
You can try to emulate determinism using some kind of virtual machine and run your build there (for example, see Hermit[1]), but it will never be perfect determinism and there are many practical downsides.
Foxboron
> as computers are deterministic
I have some bad news.
SuperHeavy256
Why is every comment here negative?
elikoga
The NixCon is in session right now so all the core contributors are chatting in person.
0points
the "i dont get it" crowd found this thread
Paria_Stark
Hacker news showing its skill issues in force today I guess.
RandomThoughts3
The issue has more to do with Nix poor onboarding, dreadful documentation and difficulty articulating its value proposition.
I personally think that the idea of Nix is fine but the execution is not there.
Look even Bazel which is built on fairly similar concepts but only targets built where it makes more sense and has a far better configuration language still struggle with adoption. It’s clear that Nix which wants to make immutable something people actually mostly never tinker will have to fight an uphill battle.
agubelu
This is one of my main gripes with Nix. When something it's hard or unintuitive it's not Nix's fault, it's a skill issue on your part because you just "don't get it". It gets tiring quick.
IshKebab
Saying it's a skill issue just shows lack of imagination and ambition to make it better.
Alifatisk
So if it's not reproducable, what is it good for? To recreate similar environment easily?
ranguna
I've never tried nixos, but I'm very close to trying. For me the good points are what you said + having the full configuration of my system in a single place (no more looking at the bash history to understand what I did) and easy rollbacks (no more having to boot into a live usb to fix my broken system).
xterminator
Ever since I started using Zig, making everything reproducible has never been as easy as just using the Zig build system.
Good points in the OP. Instead of joining the distro war in the comments, I'll just note that NixOs builds may not be reproducible - because they often reuse non-reproducible building blocks, like makefiles or compilers which are also not reproducible - but NixOs maintainers spend a lot of time helping to improve the overall ecosystem and make builds more reproducible for everyone.