Tiny JITs for a Faster FFI

Speaking from zero experience, the FFI stories of both Python and Java to C seems much better. Wouldn't going connecting them via a little C bridge a general solution?

Also see: cgo is not Go

  Go code and C code have to agree on how resources like address space, signal handlers, and thread TLS slots are to be shared — and when I say agree, I actually mean Go has to work around the C code's assumption. C code that can assume it always runs on one thread, or blithely be unprepared to work in a multi threaded environment at all.

  C doesn't know anything about Go's calling convention or growable stacks, so a call down to C code must record all the details of the goroutine stack, switch to the C stack, and run C code which has no knowledge of how it was invoked, or the larger Go runtime in charge of the program.

  It doesn't matter which language you’re writing bindings or wrapping C code with; Python, Java with JNI, some language using libFFI, or Go via cgo; it is C's world, you're just living in it.

How IPC methods would fit such cases?

Like, talk over some queue, file, http, etc

Is it? To me it seems like Ruby is declining [1]. It's still popular for a specific niche of applications, but to me it seems like it's well past its days of glory. Recent improvements are nice, but is a JIT really that exciting technologically in 2025?

[1]: https://www.tiobe.com/tiobe-index/ruby/

There was a little drama that played out as Java was getting a proper JIT.

In one major release, there was a bunch of Java code responsible for handling some UI element activities. It was found to be a bottleneck, and rewritten in C code for the next major release.

Then the JIT became properly useful, and the FFI overhead was more than the difference between the hand-tuned C code and what the JIT would spit out on its own. So in the next major release, they rolled back to the all-Java implementation.

Java had a fairly reasonably fast FFI for that generation of programming language, but they swapped for a better one a few releases after that. And by then I wasn't doing a lot of Java UI code so I had stopped paying attention. But around the same time they were also making a cleaner interface between the platform-specific and the general Java code for UI, so I'm not entirely sure how that played out.

But that's exactly the sort of see-sawing you need to at least keep an eye out for when doing this sort of work. Would you be better off waiting a couple milestones and saving yourself a bunch of hand-tuning work, or do you need it right now for political or technical reasons?

That is where a JIT enters the picture, ideally a JIT can re-optimize to an ideal state.

While this is suboptimal when doing one shot execution, when an application is long lived, mostly desktop or server workloads, this work pays off versus the overall application.

For example, Dalvik had a pretty lame JIT, thus it was faster calling into C for math functions, eventually with ART this was no longer needed, JIT could outperform the cost of calling into C.

https://developer.android.com/reference/android/util/FloatMa...

When dealing with a managed language that has a JIT or AOT compiler it's often ideal to write lots of stuff in the managed language, because that enables inlining and other optimizations that aren't possible when calling into C.

This is sometimes referred to as "self-hosting", and browsers do it a lot by moving things into privileged JavaScript that might normally have been written in C/C++. Surprisingly large amounts of the standard library end up being not written in native code.

FFI presents an opaque, unoptimizable boundary of code. Having chatty code like this is going to cost a lot. To the point where this is even a factor in much faster languages with zero-cost-ish interop like C# - you still have to make a call, sometimes paying the cost of modifying state flags for VM (GC transition).

If Ruby YJIT is starting to become a measurable factor (after all, it was slower than other, purely interpreted, languages until recently), then the same rule as above will become more relevant.

There's a phenomenal breakdown by JPCamara (https://jpcamara.com/2024/12/01/speeding-up-ruby.html) on why the Ruby#each method was rewritten in Ruby (https://bugs.ruby-lang.org/issues/20182). And bonus content from tender love: https://railsatscale.com/2023-08-29-ruby-outperforms-c/.

TL;dr - JIT rules.

If FFI calls are slow (even slower than Ruby -> Ruby calls) then informs the way you use native code. You look for workflows whereby frequent calls to a FFI function are avoided: e.g. large number of calls in some inner loop. Suppose such a situation cannot be avoided. Then you may have no recourse than to move that loop out of Ruby into C: create a custom FFI for that use case which you can call once and have it execute the loop, calling many times the function you really wanted to call.

If the FFI call can be made faster, maybe you can keep the loop in Ruby.

Of course that is attractive to people writing an application in Ruby.

That's how I interpret keeping as much code Ruby as possible.

Nobody in their right mind wants to add additional application-specific jigs written in C just to use some C piece.

Once you start doing that, why even have FFI; you can just create a module.

One attractive point about FFI is that you can take some C library and use it in a higher level language without writing a line of C.

It's been fast for a while now.

The totally safe and sane approach is to write C code that gets passed data via the command line during execution, then vomits results to the command line or just into a memory page.

Then just execute the c program with your flags or data in the terminal using ruby and viola, ruby can run C code.

If what could be written in C? The FFI library allows for dynamic binding of library methods for execution from Ruby without the need to write a native extension. That's a huge productivity boost and makes for code that can be shared across CRuby, JRuby, and TruffleRuby.

I suppose if you could statically determine all of the bindings at boot up you could write a stub and insert into the method table. But, that still would happen at runtime, making it JIT. And it wouldn't be able to adapt to the types flowing through the system, so it'd have to be conservative in what it accepts or what it optimizes, which is what libffi already does today. The AOT approach is to write a native extension.

libffi is slow; it doesn't JIT as far as I know.

In libffi you built up descriptor objects for functions. These are run-time data structures which indicate the arguments and return value types.

When making a FFI call, you must pass in an array of pointers to the values you want to pass, and the descriptor.

Inside libffi there is likely a loop which walks the loop of values, while at the same time traversing the descriptor, and places those values onto the stack in the right way according to the type indicated in the descriptor. When the function is done, it then pulls out the return according to its type. It's probably switching on type for all these pieces.

Even if the libffi call mechanism were JITted, the preparation of the argument array for it would still be slow. It's less direct than a FFI jit that directly accesses the arguments without going through an intermediate array.

FFI JIT code will directly take the argument values, convert them from the Ruby (or whatever) type to the C type, and stick it into the right place on the stack or register, and do that with inline code for each value. Then call the function, and convert the return value to the Ruby type. Basically as if you wrote extension code by hand:

  // Pseudo-code

  RubyValue *__Generated_Jit_Code(RubyValue *arg1, RubyValue *arg2)
  {
     return CStringToRuby(__targetCFunction(RubyToCString(arg1), RubyToCInt(arg2));

  }
 

The JIT code doesn't have to reflect over anything other than at worst the Ruby types. It doesn't have to have any array or list related to the arguments. It knows which C types are being converted to and form, and that is hard-coded: there is no data structure describing the C side that has to be walked at run0-time.

libffi can't know how to unwrap Ruby types (since it doesn't know what Ruby is). The advantage presented in this post is that the code for type unboxing is basically "cached" in the generated machine code based on the information the user passes when calling `attach_function`.

libffi doesn't JIT for FFI calls; and it still requires you to lay out argument values yourself, i.e. for a string argument you'd still need to write code that converts a Ruby string object to a C string pointer. And libffi is rather slow.

(the tramp.c linked in a sibling comment is for "reverse-FFI", i.e. exposing some dynamic custom operation as a function pointer; and its JITting there amounts to a total of 3 instructions to call into precompiled code)

you know i thought i knew what libffi was doing (i thought it was playing tricks with GOT or something like that) but i think you're right

https://github.com/libffi/libffi/blob/master/src/tramp.c

I think tenderworks wrote this post.

> the C code is still going to be waaay faster than the Ruby code even with YJIT.

I can't find it but I remember seeing a talk where they showed examples of Ruby + YJIT hitting the same speed and in some cases a bit more than C. The downside was though that it required to some warmup time.