The chroot Technique – a Swiss army multitool for Linux systems

Here is an example of preparing a debian ARM image on x86 with debootstrap, qemu and chroot:

  ~# sudo apt install qemu-user-static debootstrap
  ~# mkdir /tmp/arm
  ~# debootstrap --foreign --arch=armhf buster /tmp/arm http://deb.debian.org/debian
  ~# cp /usr/bin/qemu-arm-static /tmp/arm/usr/bin/
  ~# chroot /tmp/arm   # from that point, you're running ARM!
  ~# /debootstrap/debootstrap --second-stage

I've done this to build custom RPi images. Way faster than trying to build on a low power ARM platform, and way less fragile than cross compilers.

Yeah, that’s how we customize the NVidia Orin BSP before flashing it. Untar the rootfs onto a fast x86-64 machine, chroot into it, and use qemu-user to run a bunch of Arm commands and build some stuff. Way easier than trying to set up a cross-compiler toolchain.

I've done this on some tinker boards we use at work.

systemd-nspawn is a great tool for this.

chroot can be a alternative to setting up fussy cross-compilers. It'll be slower, but that slowness pays off as long as you spend less time on extra CPU cycles than you do on getting cross compilers to work.

It's also useful for reverse-engineering router/IoT firmware.

I used it with Debian on my phone, but yes.

yeah exactly that. your laptop is x86 and your phone or raspberry pi or other hardware is not.

I use my Gentoo installation usb when this comes up

> a read only root with nothing in it

A lot of things break if there's no /proc/self. A lot more things break if the terminfo database is absent. More things break if there's no timezone database. Finally, almost everything breaks if the root file system has no libc.so.6.

When you write Dockerfiles, you can easily do it FROM scratch. You can then easily observe whether the thing you are sandboxing actually works.

> no users

Now you are breaking something as fundamental as getuid.

That is pretty much what jails are in FreeBSD, especially thin jails.

> Instead there is this elaborate ceremony where the principal process basically spawns another version of itself endowed with all the same privileges and then gives them up

The flags to unshare are copies of clone3 args, so you're actually free to do this. There's some song and dance though, because it's not actually possible to exec an arbitrary binary will access to nothing.

But I think the big discrepancy is that there is inherently a two step process to "spawn a new process with a new executable." Doesn't work that way - you clone3/fork into a new child process, inheriting what you will from the parent based on the clone args/flags (which could be everything, could be nothing), do some setup work, and then exec.

> There seems to be a fundamental mismatch between how sane people think about sandboxing, and how linux manages namespaces.

What bothers me most about sandboxing with linux namespaces is that edge cases keep turning up that allow them to trick the kernel into granting more privileges than it should.

I wonder if Landlock can/will bring something more like FreeBSD jails to the table. (I haven't made time to read about it in detail yet.)

I believe this is because on POSIX systems the only way to create a new process is fork().

yup! FreeBSD jails are essentially what OP wants with chroot++.

I was pretty puzzled when Docker and LXC came around as this whole new thing believed to have "never been done before"; FreeBSD had supported a very similar concept for years before security groups were added in Linux.

Jails and ezjail were stellar to make mini no-overhead containers when running various services on a server. Being able to archive them and expand them on a new machine was also pretty cool (as long as the BSD version was the same.)

Or DragonFlyBSD with vkernels

https://www.dragonflybsd.org/docs/handbook/vkernel/

*podman, etc

It has nothing to do with weirdness; Unix itself was plenty weird for its time. The relevant difference between Unix™ and Plan 9 is that Unix source code was given away (or cheaply licensed) to hardware companies which all wrote their own operating systems on top (SunOS, Ultrix, HP-UX, etc. etc.). This made Unix the common factor of very many commercial workstation environments. Plan 9? It was sold directly as a commercial product, for no hardware platform in particular. Nobody wanted to buy it.

People liked Unix because it was free – either really free, via BSD, or as a Unix derivative provided at no cost when people bought their workstations. A new revolutionary operating system had absolutely no reason for anybody to buy it: No commercial developers wanted to develop to a platform without users, and no users wanted a platform without software.

Plan 9 only changed their license many years later, when it was too late for anybody to care, and Unix had become the established standard.

rfork it's easy :D

I remember first installing Gentoo…printed the whole manual and followed along. After a failed attempt and second successful install I learned enough to know that I could use a LiveUSB of my choosing and chroot into the install. Good times!

Debian's debootstrap would also like to participate in this discussion

i was very happy when that arch-chroot + pacstrap simplified my arch setup not having to run those mounts (i remember doin the same setup process for arch for centos/rhel just to not use anaconda and removing bloat, or rather installing what i wanted)

for btrfs, if you use a consistent volume mapping on your systems, its pretty easy. in arch setup i typically enable ssh and have a pretty simple set of bashisms for target device, compression, and mount points. then its copy-pasta since its a boiler plate for when i need to recover or fresh install

not glanced at manjaro so not familiar with its install-methods