Experience converting a mathematical software package to C++20 modules [PDF]
16 comments
·July 1, 2025npalli
Someone
It wouldn’t surprise me if they could do better if they gave up on doing most of the work programmatically.
One part of me agrees with (both from the paper)
> For example, putting a specific piece of code into the right place in each file (or adding necessary header files, as mentioned in Section 5.2) might take 20-30 seconds per file – but doing this for all 1051 files of deal.II then will take approximately a full day of (extremely boring) work. Similarly, individually annotating every class or function we want to export from a module is not feasible for a project of this size, even if from a conceptual perspective it would perhaps be the right thing to do.
and
> Given the size and scope of the library, it is clear that a whole-sale rewrite – or even just substantial modifications to each of its 652 header and 399 implementation files – is not feasible
but another part knows that spending a few days doing such ‘boring’ copy-paste work like that often has unexpected benefits; you get to know the code better and may discover better ways to organize the code.
Maybe, this project is too large for it, as checking that you didn’t mess up things by building the code and running the test suite simply takes too long, but even if it seems to be, isn’t that a good reason to try and get compile times down, so that working on the project becomes more enjoyable?
trostaft
Oh, it’s Wolfgang. In computational math, he has a focus on research software that few others are able to do, he (the deal.ii team more generally) got an award for it last SIAMCSE. Generally a great writer, looking forward to reading this.
Asooka
I would like to see a comparison between modules and precompiled headers. I have a suspicion that using precompiled headers could provide the same build time gains with much less work.
pjmlp
As per Office team, modules are much faster, especially if you also make use of C++ standard library as module, available since C++23.
See VC++ devblogs and CppCon/C++Now talks from the team.
Pre-compiled headers have only worked well on Windows, and OS/2 back in the day.
For whatever reason UNIX compilers never had a great implementation of it.
With exception of clang header maps, which is anyway one of the first approaches to C++ modules.
fpoling
This has been puzzling me for over 3 decades. My first experience with C++ was Borland C++ for DOS. It had precompiled headers and it worked extremely well.
Then around 1995 I got access to HP-UX and native compiler there and GCC. Nobody heard about precompiled headers and people thought the only way to speed up compilation was to get access to computer with more CPUs and rely on make -j.
And then there was no interest to implement precompiled headers from free and proprietary vendors.
The only innovation was unity builds when one includes multiple C++ sources into super-source. But then Google killed support for it in Chromium claiming that with their build farm unity builds made things slower and supporting them in Chromium build system was unbearable burden for Google.
dataflow
Precompiled headers are generally better for system/3rd-party headers. Module are better than PCHs for headers you own, although in some cases you may be better off not using them at all. (I say these because the benefit depends on the frequency with which you need to recompile them, and the relative coupling etc.) Depending on how heavy each one is in your codebase, and how often you modify global build settings, you may have a different experience. And neither is a substitute for keeping headers lightweight and decoupled.
w4rh4wk5
From my experience, compile times ain't an issue if you pay a little attention. Precompiled header, thoughtful forward declarations, and not abusing templates get you a long way.
We are commonly working with games that come with a custom engine and tooling. Compiling everything from scratch (around 1M lines of modern C++ code) takes about 30-40 seconds on my desktop. Rebuilding 1 source file + linking comes in typically under 2 seconds (w/o LTO). We might get this even lower by introducing unity builds, but there's no need for that right now.
ttoinou
40 seconds for 1M lines seems super fast, do you have a fast computer and/or did you spend a lot of time optimizing the compilation pipeline ?
vblanco
The modern cryengine compiles very fast. Their trick is that they have architected everything to go through interfaces that are on very thin headers, and thus their headers end very light and they dont compile the class properties over and over. But its a shame we need to do tricks like this for compile speed as they harm runtime performance.
w4rh4wk5
We didn't create this code base ourselves, we are just working with it. I'd assume the original developers payed attention to compile times during development and introduced forward declarations whenever things got out of hand.
My computer is fast, AMD Ryzen 9 7950X, code is stored on an NVMe SSD. But there certainly are projects with fewer lines of code that take substantially longer to compile.
isatty
The code block styling is less than ideal.
KingLancelot
[dead]
Thanks to author for doing some solid work in providing data points for modules. For those like me looking for the headline metric, here it is in the conclusion
So, alas, underwhelming in this iteration and perhaps speaks to 'module-fication' of existing source code (deal.II, dates from the '90s I believe), rather than doing it from scratch. More work might be needed in structuring the source code into modules as I have known good speedup with just pch, forward decls etc. (more than 10%). Good data point and rich analysis, nevertheless.