Factors affecting C++ Compilation time – How to reduce them

Well, I never figured out to myself that I will write a C++ article when my main specialization is Java. Anyway during the last three years I’m involved in a cross discipline project involving JAVA and C++.

In this project a JAVA generator generates millions of lines of C++ code which of course have to be compiled, and if you are a C++ guy you certainly already have your hair standing on your head because of the time it will require to compile such huge amount of code. Well you are right, we faced extremely long compilation time (12+ hours in Unix, in windows…), which are major problem in a product that should have a quick time to market.

Worst, the product is used both under Windows and Unix platforms, which means that a solution need to be created for both worlds.

Under Windows – even with the usage of Incredibuild from Xoreax (a great grid compiler platform which allowed to reduce considerably the compilation time), the user still needed to wait 2 hours for compilation, which was not acceptable.

Under Unix – no grid compiler (unless you work only on few platform), we tried to use distcc but the results were still not satisfying and you need additional hardware. We where stuck…

Therefore we begun a research for an alternative that could speed up the builds, and for that we needed to understand the factor that affected compilation time, our main suspicious factor was the number of lines of code to compile, since the code was generated, it was very easy to inflate the output nevertheless we soon understood that we were wrong…

Below are factors impacting the compilation time (according to their impact)

1. In the first place the number of files to compile – this is one of the major factor affecting compilation time, the compiler is not really smart at reusing information it processed between invocation and is not able either to work on a set of files, and it is especially slow (I/O bound) for building dependencies. If you want to really reduce the compilation time reduce the number of file to compile, and it does not mean to write all in a single file, you can use what is called Unity Build.
   A Unity Build group several cpp files in a single one using just the include directive. For example let say you want to compile file1.cpp to file 10.cpp then create a new file group.cpp as follow:

   #include "file1.cpp"
   #include "file2.cpp"
   #include "file3.cpp"
   ..
   ..
   #include "file10.cpp"
   

   Now compile group.cpp and don’t forget to add the file1.cpp…file10.cpp location as include path.
   This method produces miracle (of course you have to balance the number of files you put in a single group/unity). Our compilation reduced from 15 hours to just 2.

2. Include paths – large number of include path directly affect the build time, since the compiler (or pre-compiler) need to scan all the path until it find the requested include. So try to minimize them or at least organize the path list according to the most searched one.
3. NAS (Network Attached Storage) also has a bad impact on the compilation (write is usually fast, but read is slow so library creation is slow).
4. Generate cppdep and compile on the same time – unix compilers support option to create cppdep file and compile at the same time you can save approx 20% of your compilation time.
5. Forward declaration, also know as the “Pimpl idiom” to reduce dependency, greatly help, the problem is that you cannot always refactor the code, to avoid some include that will erase your effort.
6. Usage of template – using C++ template excessively increase compilation time and libraries volume (especially if the template is declared in header).
7. Number of strings constants in a single file. It might should strange, but some compilers (HP and Sun at least) have a performance degradation when the compilation unit contains an large number of string (few thousand).
   Note: Visual Studio compiler is not sensitive to this factor.
   
8. Generic vs inflated code – using call to function or writing the content of function where you need them (like a forced “inline”). Inlining function in this way may produce better performance, but does not affect compilation time, as much as you think it affect. Effectively we reduced millions of lines of code by 75% using call to function instead of inlining their content, but we got no improvement in build time, but at least you gain more maintainable/debugable code.
9. Usage of pre-compiled header might help, but from our test they did not, the compilation time was in fact increased.
10. Usage of header cache folder – similarly to pre-compiled headers, should help (according to vendors) but from our test they most of the time do not.

So if you really want to reduce your compilation time try Unity Build concept, you will gain in:

1. Faster build time
2. Smaller objects size
3. Smaller libraries size
4. Better optimized code

Note: The compilation time is related to the number of cpp files in a single Unity Build (and their dependencies), and this number should be tuned according to the included file content (inline, template usage, headers used…). If you have too much file in a single unity/group file compilation time increase back (still better than when no using group file), nevertheless the library size declines (even if the compilation time increased back).

Mikael.Sitruk