16

I'm working on a c++ project built with cmake+ninja with approx 1200 build targets on a 64 thread computer.

There's one translation unit that takes 10min to compile, most others are comparably fast such that a build of all other targets together takes (building on all threads) only about 9 minutes. The slow translation unit is rather independent of the rest, so it doesn't have to be scheduled late, but as it turns out is is scheduled late by default, such that a complete build takes me between 15 and 20 minutes, and at the end there is only one thread working with almost all other targets done. The build would be faster for me, if the slow translation unit would be scheduled first, blocking one thread for about 10 minutes, while all other threads work on the rest of the project and the entire project is built within 10min.

Is there a way in cmake or ninja to shift the priorities for scheduling to point out slow or "please early" targets without messing up dependencies?

pseyfert
  • 3,263
  • 3
  • 21
  • 47

1 Answers1

4

As of February 27, 2021, the answer is no. There are some open issues (#232, #376) and an abandoned PR (#1333) on GitHub requesting this feature in base Ninja. CMake does not provide any way to prioritize a target (at least through 3.20) either.

Messing with dependencies (even order-only) doesn't help here (as you likely know) because that would just force smaller targets to start either completely before or completely after the long target. A priority hint is what's truly needed here.

The only workaround I can think of (and it's not a great one), is to split your long target out into a separate ExternalProject and create a superbuild that builds the large target and the independent portion at the same time. This would require significant restructuring and would be a non-starter for a lot of projects. It might be worth the pain if you're losing a lot of development time to this issue, though.

Alex Reinking
  • 16,724
  • 5
  • 52
  • 86
  • 1
    Well #232 indicates that ninja's build order for independent targets is _likely_ to be based on their order in the build.ninja file, because sorting happens based on the pointer address and the targets are allocated as they are encountered in the file and memory allocators tend to return a series of increasing addresses on each allocation (ignoring complications of intervening frees). This lines up with my empirical experience using cmake. – BeeOnRope Jan 25 '23 at 00:37
  • @BeeOnRope - relying on such brittle behavior would be misguided. [Hyrum's Law](https://www.hyrumslaw.com/) wasn't meant to be a self-fulfilling prophecy. – Alex Reinking Jan 25 '23 at 17:04
  • 1
    Well from my PoV it's OK: if it has been like that already for 11 years, could save me an ~hour of compiling every day _and if it doesn't work my compile is simply slower but no slower than it was before_ (i.e., I would be broken by relying on this behavior: consider it a _hint_ to the compile order) ... seems like a fine tradeoff. – BeeOnRope Jan 25 '23 at 19:21
  • 2
    So I'd argue I'm not at all "relying" on it but taking advantage of it opportunistically with a transparent and automatic fallback. – BeeOnRope Jan 25 '23 at 19:26
  • Well, all I can say is that I disagree with that approach philosophically. I don't build castles on sand, sorry. – Alex Reinking Jan 25 '23 at 20:46