ML needs a new programming language – Interview with Chris Lattner

Who uses this Exla in production?

>This guy is worried about GPU kernels

Then the title should be "why GPU kernel programming needs a new programming language." I can get behind that; I've written CUDA C and it was not fun (though this was over a decade ago and things may have since improved, not to mention that the code I wrote then could today be replaced by a couple lines of PyTorch). That said, GPU kernel programming is fairly niche: for the vast majority of ML applications, the high-level API functions in PyTorch/TensorFlow/JAX/etc. provide optimal GPU performance. It's pretty rare that one would need to implement custom kernels.

>which are never, ever written in Python.

Not true! Triton is a Python API for writing kernels, which are JIT compiled.

> That may be true, but some of us are still bitter that all that grew up around an at-least-averagely-annoying language rather than something nicer.

Don't worry. If you stick around this industry long enough you'll see this happen several more times.

> This guy is worried about GPU kernels, which are never, ever written in Python. As you point out, Python is a glue language for ML.

That's kind of the point of Mojo, they're trying to solve the so-called "two language problem" in this space. Why should you need two languages to write your glue code and kernel code? Why can't there be a language which is both as easy to write as Python, but can still express GPU kernels for ML applications? That's what Mojo is trying to be through clever use of LLVM MLIR.

Yeah the OP here listed a bunch of Python stuff that all ends up shelling out to C++. C++ is rich enough, period, but people find it unpleasant to work in (which I agree with).

It's not about "richness," it's about giving a language ecosystem for people who don't really want to do the messy, low-level parts of software, and which can encapsulate the performance-critical parts with easy glue

The problem I've seen is this: in order to get good performance, no matter what language you use, you need to understand the hardware and how to use the instructions you want to use. It's not enough to know that you want to use tensor cores or whatever, you also need to understand the myriad low level requirements they have.

Most people that know this kind of thing don't get much value out of using a high level language to do it, and it's a huge risk because if the language fails to generate something that you want, you're stuck until a compiler team fixes and ships a patch which could take weeks or months. Even extremely fast bug fixes are still extremely slow on the timescales people want to work on.

I've spent a lot of my career trying to make high level languages for performance work well, and I've basically decided that the sweet spot for me is C++ templates: I can get the compiler to generate a lot of good code concisely, and when it fails the escape hatch of just writing some architecture specific intrinsics is right there whenever it is needed.

> First of all some people really like Julia, regardless of how it gets discussed on HN, its commercial use has been steadily growing

Got any sources on that? I've been interested in learning Julia for a while but don't because it feels useless compared to Python, especially now with 3.13

Here's some benefits it might try to offer as differentiators:

1. Easy packaging into one executable. Then, making sure that can be reproducible across versions. Getting code from prior, AI papers to rub can be hard.

2. Predictability vs Python runtime. Think concurrent, low-latency GC's or low/zero-overhead abstractions.

3. Metaprogramming. There have been macro proposals for Python. Mojo could borrow from D or Rust here.

4. Extensibility in a way where extensions don't get too tied into the internal state of Mojo like they do Python. I've considered Python to C++, Rust, or parallelized Python schemes many times. The extension interplay is harder to deal with than either Python or C++ itself.

5. Write once, run anywhere, to effortlessly move code across different accelerators. Several frameworks are doing this.

6. Heterogenous, hot-swappable, vendor-neutral acceleration. That's what I'm calling it when you can use the same code in a cluster with a combination of Nvidia GPU', AMD GPU's, Gaudi3's, NPU's, SIMD chips, etc.

Actually in julia you can write kernels with a subset of the julia language:

https://cuda.juliagpu.org/stable/tutorials/introduction/#Wri...

With KernelAbstractions.jl you can actually target CUDA and ROCm:

https://juliagpu.github.io/KernelAbstractions.jl/stable/kern...

For python (or rather python-like), there is also triton (and probably others):

https://pytorch.org/blog/triton-kernel-compilation-stages/

I think Julia aspires to be performant enough that you can write the kernels in Julia, so Julia is more like Mojo + Python together.

Although I have my doubts that Julia is actually willing to make the compromises which would allow Julia to go that low level. I.e. semantic guarantees about allocations and inference, guarantees about certain optimizations, and more.

You can write kernels with Python using CUDA and Open API SDKs in 2025, that is one of the adoption problems regarding Mojo.

It is released but not open-source. Modular was aiming to open-source the compiler by Q4 2026; however, Chris now says they could be able to do that considerably faster, perhaps early 2026[1].

If you're interested, they think the language will be ready for open source after completing phase 1 of the roadmap[2].

1.https://youtu.be/I0_XvXXlG5w?si=KlHAGsFl5y1yhXnm&t=943

2. https://docs.modular.com/mojo/roadmap

I get the impression that most of the comments in this thread don't understand what a GPU kernel is. These high-level languages like Python and Julia are not running on the kernel, they are calling into other kernels usually written in C++. The goal is different with Mojo, it says at the top of the article:

> write state of the art kernels

You don't write kernels in Julia.

No, you can write the kernels directly in Julia using KernelAbstractions.jl [1].

[1] https://juliagpu.github.io/KernelAbstractions.jl/stable/

Yes; in particular see sections 2-4 of [0].

They say they'll open source in 2026 [1]. But until that has happened I'm operating under the assumption that it won't happen.

[0]: https://www.modular.com/legal/community

[1]: https://docs.modular.com/mojo/faq/#will-mojo-be-open-sourced

I tend to agree, which is why I can't recommend Mojo, despite thinking their tech is pretty innovative. If they're willing to lie about something that basic, I can't trust any of their other claims.

I hope that’s not what it was, that makes them seem very manipulative and dishonest. I was under the impression it was a goal, but they dropped it when it became apparent it was too hard. That’s much more reasonable to understand.

with juliac.jl and --trim, hello world is now 1MB and compiles in a second.

more realistic examples of compiling a Julia package into .so: https://indico.cern.ch/event/1515852/contributions/6599313/a...

julia> @time begin

       using Plots

       display(plot(rand(8)))

       end

  1.074321 seconds

improving, slowly. 5 steps forward 3 steps back.

1.9 and 1.10 made huge gains in package precompilation and native code caching. then attentions shifted and there were some regressions in compile times due to unrelated things in 1.11 and the upcoming 1.12. but at the same time, 1.12 will contain an experimental new feature `--trim` as well as some further standardization around entry points to run packages as programs, which is a big step towards generating self-contained small binaries. also nearly all efforts in improving tooling are focused on providing static analysis and helping developers make their program more easily compilable.

it's also important a bit to distinguish between a few similar but related needs. most of what I just described applies to generating binaries for arbitrary programs. but for the example you stated "time to first plot" of existing packages, this is already much improved in 1.10 and users (aka non-package-developers) should see sub-second TTFP, and TTFX for most packages they use that have been updated to use the precompilation goodies in recent versions

I recently looked into making Julia binaries, and it's not at all a good process. They say it's supported, but it's not exactly as easy as "cargo build" to get a Julia binary out. And the build process involves creating this minimal version of Julia you're expected to ship with your binary, so build times were terrible. I don't know if that gets amortized though.

As far as the executable size, it was only 85kb in my test, a bouncing balls simulation. However, it required 300MB of Julia libraries to be shipped with it. About 2/3 of that is in libjulia-codegen.dll, libLLVM-16jl.dll. So you're shipping this chunky runtime and their LLVM backend. If you're willing to pay for that, you can ship a Julia executable. It's a better story than what Python offers, but it's not great if you want small, self-contained executables.

It's a low level language with a high level interface. In theory, GC aside, you should be able to write code as performant as C++ without having to actually write C++. It's also homoiconic and the compiler is part of the language’s API, so you can do neat things with macros that let more or less you temporarily turn it into a different language.

In practice, the Julia package ecosystem is weak and generally correctness is not a high priority. But the language is great, if you're willing to do a lot of the work yourself.

Built-in autodifferentiation and amazing libraries built around it, plus tons of cutting edge applied math libraries that interoperate automatically, thanks to Julia's well conceived approach to the expression problem (multiple dispatch). Aside from that, the language itself is like a refined python so it should be pretty friendly off the bat to ML devs.

What Julia needs though: wayyyy more thorough tooling to support auto generated docs, well integrated with package management tooling and into the web package management ecosystem. Julia attracts really cutting edge research and researchers writing code. They often don't have time to write docs and that shouldn't really matter.

Julia could definitely use some work in the areas discussed in this podcast, not so much the high level interfaces but the low level ones. That's really hard though!

I would use the term "potentially great" rather than plain "great"

but all the normal marketing words: in my opinion it is fast, expressive, and has particularly good APIs for array manipulation

It's very important that readers, writers, maintainers, etc. of code are able to easily understand what that code is doing.

explicit and strict types on arguments to functions is one way, but certainly not the only way, nor probably the best way to effect that

I think Swift is really successful in that there are so many new Apple developers who would use Swift now but wouldn't have used ObjC.

To be fair to Chris, I’ve only seen the message about compiler not being able to type check the expression in swiftui closure hell. I think he left (maybe partly) because of the SwiftUI influence on Swift.

According to wikipedia it was announced in May 2023

True, but aren't we in a better place now? I think the move to free tools was motivated by programmers, and not by their employers. I've read that it became hard to hire people if you used proprietary tools. Even the great Microsoft open-sourced their flagship C# language. And it's ironic but telling that the developers of proprietary software don't trust proprietary tools. And every developer looks at the state of the art in proprietary engineering tooling, such as CAD, and retches a little bit. I've seen many comments on HN along those lines.

And "correlation is not causality," but the occupation with the most vibrant job market until recently was also the one that used free tools. Non-developers like myself looked to that trend and jumped on the bandwagon when we could. I'm doing things with Python that I can't do with Matlab because Python is free.

Interestingly, we may be going back to proprietary tools, if our IDE's become a "terminal" for the AI coding agents, paid for by our employers.

I’d prefer to not touch a hot stove twice. Telling me what processors I can use is Oracle- level rent seeking, and it should be mocked just like Oracle.

And it sucked so hard that GNU and LLVM were born.