DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> Scientific Ninja

“Master” Header Pitfalls

It’s sometimes surprising how poorly-understood the C++ build process is. The preprocessor and its quirks seems to be particularly opaque to many, leading to the adoption of some harmful preprocessor practices. There’s one issue I’m thinking of in particular right now, mostly because there’s been a rash of users running in to trouble with it over at GDNet: the use of so-called “master” (or “global” or “yes, even the kitchen sink”) headers.

Master headers are those header files that are included by every other header file in the project, and usually include all the “common,” “basic,” or “required,” headers in your project. They are occasionally viewed as beneficial in that they decrease the amount of typing you need to do to build a new component header file (there’s only one header to include), and they increase the clarity of that header file by hiding all those nasty #include directives behind one single, clean directive instead. While this might be true, what you can end up paying for those benefits far outweighs the value of the benefits themselves.

There are many small problems with master headers, but there are three that I consider to be particularly painful.

  • When the master include (or anything included by the master include) is changed, all files including the master are out of date and may need to be recompiled. For a project of sufficient size and sufficient master include proliferation, this can kill your build time, as you must essentially rebuild the entire project every time you change a header, even if that header is only actually relevant to a small subset of source files.
  • They introduce latent, implicit dependancies. That is, since a source file that includes the master is effectively including every file included by the master, it is easy for code to be added to that source file that uses something defined in any of those headers. The source file is thus dependant on that header, but that dependency is not explicit (the header was not included directly), and it may cross subsystems in an unacceptable way.
  • They can render seemingly legal code illegal in subtle ways.

That last point bears some explaination, as its one of the trickier problems with master includes, and usually the one that trips people up enough to finally convince them to revise their include system. Let’s say you’re writing a game, and you have a master include file called GlobalIncludes.h.

1
2
3
4
5
6
7
8
//GlobalIncludes.h
#ifndef GLOBAL_INCLUDES_H_
#define GLOBAL_INCLUDES_H_
 
#include "BaseObject.h"
#include "NiftyPowerUp.h"
 
#endif

GlobalIncludes.h includes all the headers for your game framework, which for our purposes will be only BaseObject.h (the base class of all objects) and NiftyPowerup.h (the class for a powerup that makes the player nifty). We also have corresponding source files for each header.

1
2
3
4
5
6
7
8
9
10
11
12
//BaseObject.h                                   
#ifndef BASE_OBJECT_H_
#define BASE_OBJECT_H_
 
#include "GlobalIncludes.h"
 
class BaseObject
{
	/* ... */
};
 
#endif
1
2
3
4
5
6
7
8
9
10
11
12
//NiftyPowerUp.h
#ifndef NIFTY_POWERUP_H_
#define NIFTY_POWERUP_H_
 
#include "GlobalIncludes.h"
 
class NiftyPowerup : public BaseObject
{
	/* ... */
};
 
#endif
1
2
//BaseObject.cpp
#include "BaseObject.h"
1
2
//NiftyPowerUp.cpp
#include "NiftyPowerUp.h"

Now remember that C++ compilers process source files in isolation — that is, every input source file is preprocessed and compiled independantly of the others. Compilers don’t remember anything about previously compiled files, even if all files are in the same project. This, combined with the fact that the compiler often requires the full definition of the type in order to do anything useful with it, is one reason why the C++ compilation model uses the header file system. With that in mind, let’s consider the process of preprocessing BaseObject.cpp:

  1. BaseObject.cpp will include BaseObject.h; as BASE_OBJECT_H_ has not yet been defined, this will cause the entire text of BaseObject.h to be copied into the translation unit and the preprocessor will define BASE_OBJECT_H_ (likely storing its existence as a defined symbol in an internal table somewhere). The current translation unit will look something like this (I’ve indented the substituted content (and left the #include directive in to symbolize that it hasn’t been processed yet) to try to make it easier to visualize the state of the translation unit):
    1
    2
    3
    4
    5
    6
    7
    8
    
    //BaseObject.cpp
      //BaseObject.h                                   
      #include "GlobalIncludes.h"
     
      class BaseObject
      {
        /* ... */
      };
  2. The #include directive for GlobalIncludes.h will be processed, and since GLOBAL_INCLUDES_H_ isn’t defined yet, the file contents will be copied and the symbol defined. The translation unit now looks like:
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    
    //BaseObject.cpp
      //BaseObject.h                           
        //GlobalIncludes.h
     
        #include "BaseObject.h"
        #include "NiftyPowerUp.h"
     
      class BaseObject
      {
        /* ... */
      };
  3. The #include for BaseObject.h will be attempted, but since BASE_OBJECT_H_ has been defined, not much will happen to the translation unit (the text between the #ifndef and #endif directives will be ignored entirely):
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    
    //BaseObject.cpp
      //BaseObject.h                                   
        //GlobalIncludes.h
     
          //BaseObject.h
        #include "NiftyPowerUp.h"
     
      class BaseObject
      {
        /* ... */
      };
  4. Finally, the preprocessor will process the directive for NiftyPowerUp.h, which has not been seen yet, so we end up with this:
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    
    //BaseObject.cpp
      //BaseObject.h                                   
        //GlobalIncludes.h
     
          //BaseObject.h
          //NiftyPowerUp.h
          #include "GlobalIncludes.h"
     
          class NiftyPowerup : public BaseObject
          {
            /* ... */
          };
     
      class BaseObject
      {
        /* ... */
      };

I could go on, but at this point the problem should be clear: the resulting translation unit defines NiftyPowerUp before its base class, which C++ cannot understand. As a result, the compiler will complain that the base class is undefined. When you look at the error in your IDE, and then look at the code for the offending file (NiftyPowerUp.h), you might scratch your head wondering why this code — which seems perfectly legal, since it looks like the base class definition is included before the derived class, via the master header — doesn’t compile.

This problem can be particularly difficult to track down — especially the first time it ever happens to you. You can fix it temporarily and in small cases by rejiggering the order of some #include directives, or by making sure you also (counter-intuitively perhaps) include the master headers first in all the source files as well. But as a solution, those techniques are woefully inadequate. They don’t scale very well, making your code brittle. The long term, proper solution is to stop using master includes as a container for all other header files in your project, and have file include only what it really needs to function. Those of you unfamiliar with the some of the techniques involved in doing that may want to check out this article written by GDNet’s Ben Sizer (“Kylotan”). It offers some further advice regarding source and header file organization in C++.

So there you have it. Although this little trap is not particularly frequent, it does show up now and again. Combined with the other two reasons I listed earlier, relating to build and code health, you should now have a pretty compelling argument in favor of abandoning master include containers.

Now, this isn’t to say that all types of master includes should be completely expunged from your codebase. Far from it — they have their place, as with most techniques. Master includes can include preprocessor configuration and global-control macros (such as a #define that controls whether or not your code uses a memory tracker for leak debugging, or has logging code compiled in or out, et cetera). For storing those kind of project-wide configuration macros, especially macros with gating logic (which means they cannot be defined on the compiler’s command line), master includes are justified. You just should not abuse them to act as containers for all other header files in your project, or you will suffer for it. Eventually.