Resistance to Rust abstractions for DMA mapping

Additionally regardless of what upstream decides to do, Microsoft and Google are pretty much down on having Rust on their Linux forks, e.g. Sphere OS, Azure Linux, Android, ChromeOS.

I wonder why that C file which maps a more abstract Rust-friendly C-API on top of the existing API can't live inside the Rust directory and build structure. If I would be one of the Rust maintainers, I would try to be as non-intrusive to the established project structure as possible, ideally completely invisible. It's also cleaner to keep all the Rust-related changes on the 'Rust side' of the project, even if they happen to be written in C (I'm also still wondering why Rust-for-Linux cannot live completely downstream in a temporary fork).

I also can fully understand that a C programmer doesn't want to deal with such a 'Rust idiomatic' C API.

AFAIK, the merge was for the sake of change, not like he fully agreed with the adoption of Rust nor prioritization of Rust. The point is to have actual discussions, rather than pushing things with 10-feet poles just for being new and different. Anything that make sense will become the outcome of those discussions.

> that ship has already sailed when Linus decided to merge R4L into the kernel

Of course not. Merging the basic infra for Rust didn't come with "it can be used in every single part of the kernel, regardless of what the people maintaining that think"-kind of guarantee. One way to use Rust is to only use it in drivers for example, and it seems Hellwig is not against that: "do that in your driver so that you have to do it".

Yeah pretty much this

And I think Linus should be very clear that rust is there to stay

My daily soap opera fix

Nobody is asking Christoph to maintain Rust code. Just work with the R4L developers occasionally when a breaking change happens, the same way he likely would need to do so with every C-based consumer of those APIs. But he doesn't want to do that, and he also doesn't want to let anyone co-maintain the DMA subsystem. At some point this does need to be called out as gatekeepy and counterproductive behavior by a maintainer who doesn't want to work with others.

But working with others is kind of, like, necessary, in a project as large and critical as the Linux project. Developing new kernel talent was one of Linus' express reasons for doing the project in the first place.

https://www.youtube.com/watch?v=OvuEYtkOH88&t=6m07s

IIUC, I think it is kernel policy that if you change some kernel API and break its consumers, you are responsible for fixing those consumers. (correct me if this is wrong.) So in theory DMA devs may need to touch Rust code if they make changes to the DMA subsystem.

Edit: this is wrong/inaccurate, see comments below.

In Rust, unsafe (unchecked) C interop is almost automatic.

The hard part is in translating C code's high-level safety requirements into Rust APIs that enforce them. I'm talking about requirements that aren't expressed in the C syntax, but are arbitrary domain-specific rules defined in English by the C code's authors ("this function can be called only on Wednesdays"). These turn out to be hard, because they may not be precisely defined, and/or the conditions are complex and implementation-specific. That's less of language inteop problem, and more in creating formal definitions of informal documentation.

Having said that, there are a couple of things that Rust made harder for itself:

• Rust allows moving objects to a different address safely. C code often assumes objects never move and their addresses are unique and meaningful. If Rust had built-in support for immovable types, it wouldn't need Pin and macro hacks.

• Rust's reference types require strict immutability or strict pointer aliasing (exclusive access), while C allows memory to be mutated from anywhere, and pointers to const don't mean the data is immutable. This prevents Rust from using its nice safe reference and slice syntax on memory externally mutated in surprising/clever ways (by memory-mapped hardware, shared mem, MMU trickery).

> However, I must ask: is there a world in which interop didn't have to be so bad without hugely sacrificing memory safety?

You can't really make interop nicer than the lowest common denominator, and when one of the sides is C then that lowest common denominator is very low. Rust is one of very few languages that can potentially interop with memory-safe languages without having to completely buy into a common runtime - e.g. it's increasingly popular to write Python extensions in Rust and I believe there are now libraries for doing that without having to go via C - and when you have richer abstractions available on both sides of the line then you can make use of them. But to work with C you have to work like C, at least at the boundary.

Zig has some features to make C interop easier (for instance string literals are both ptr/length-slices and zero-terminated), but in general the problems are quite similar to Rust's.

There is a subset of Zig types that is compatible with C APIs, but most higher-level Zig types (like slices, odd-width integer types, optionals, error unions...) can't be tunneled through or mapped to a C API...

The main thing may be that the Zig compiler doesn't track ownership/lifetimes like the Rust compiler does, so it doesn't matter that this information would be lost at the C-API barrier.

> I like Rust, but my worst experiences with Rust are interop with C.

Could you elaborate? I find the interop to be quite nice. I mean, obviously, interop between any two languages cannot ever be "better" than the least good language at any one thing.

Zug does not attempt to solve memory safety. It aims to be a better C

Actually the interoperability is very good. C can directly call into Rust and Rust can directly call into C. Rust can construct any C type and can expose opaque pointers to C world.

> Maybe be an upstream for Android and the various corporate kernels.

> A soft fork might clarify things like the old gcc fork did.

Nice idea but in practice there is absolutely no chance that this would be a successful strategy for those people in favor of Rust.

Being out of mainline tree would perpetually put Rust in the "future" (like Nuclear Fusion -- always just "20 years away"). Meanwhile Linux would continue to evolve in ways that could be very disruptive for Rust integration. For example, Rust interop needs to think very carefully about locks, memory order etc. Any change in the mainline kernel could really mess the assumptions made in the Rust fork. A year or two of evolution of the slightly independent evolution of the two branches may make it almost impossible to merge the two.

Only companies as large as Google could maintain the Android fork. The RFL team is small and I don't think large vendors would like to invest their time and energy on forks. Who would pick up and package the fork ? Red Hat ? Debian ? Arch ? They are already complaining about compilation and maintainability of Rust when it is in the mainline kernel. No chance that they start using the down stream rust kernel in their main kernels.

The only way Rust for Linux succeeds is if it has the blessing and push from Greg/Linus.

> The scenario where someone wants to drive by dump some code, then you reject it, then they offer to maintain it, kind of sucks.

That's not what happened here.

This was a maintainer that was CCd on a Rust wrapper to a subsystem he maintains as a courtesy, immediately replying that that code shouldn't live in that subsystem and blocking the patch set from being merged by NAking, but the change was already not in the subsystem but rather the rust subdirectory, which gives the impression he didn't read the patch to begin with. After he was told this, he said that having multiple languages is a cancer that would kill the maintainability of Linux and he would block it any way he could.

That's a very different situation to the one you describe.

> doesn't this violate RAII?

It's C, you don't get any RAII.

This is similar to `Box::into_raw` in Rust `std`. It destructs the RAII struct into a raw pointer. After that it's pretty much expected that you are responsible for cleanup.

The method is called `CoherentAllocation::into_parts`. And it takes self by value, which means the struct is consumed (this is implied by the into_ prefix). Of course you would either reconstruct the allocation or free the allocation yourself after this.

See Vec::into_raw_parts or Box::into_raw for an stdlib analogy.

The uncharitable take is that it's just job protection. That is current maintainer doesn't want to be replaced with some new Rust-writing folks.

A more charitable take is that the current maintainer is worried not about extra work now, but later when more things use Rust.

For example, say the Rust changes were merged and maintained by RfL folks. Then say NVIDIA replaces their current GPU drivers with a new Rust-based drivers.

Later a change to the DMA layer kernel code breaks the Rust code in a way that's not trivially fixable, and with that no NVIDIA GPU drivers.

Would the DMA changes still be allowed to be merged without any additional work by the current DMA layer maintainer?

This seems like something that could happen down the line, if the RfL project continues.

> Personally, I'm curious if there's hard technical blockers.

That LWN or the kernel devs are keeping secret? I would be amazed why, if Christoph Hellwig had technical blockers, he wouldn't raise them in the LKML, right now, of all times.

The author has been professionally writing about the Linux kernel for close to 30 years. He's also maintaining the Linux kernel documentation.

> Personally, I'm curious if there's hard technical blockers.

Not really, apart from the sheer size of Linux.

The kernel maintains in-tree drivers that break, so it does impact kernel maintenance. This isn't currently true for RfL, but in the case of a C driver anyone who wanted to go and make a breaking change would have to go and fix up breakages to merge.

If it’s a common abstraction then wouldn’t it be more work to maintain the same piece of code in many different drivers?