Makefile performance: built-in rules

Like any system that has evolved over many years, GNU Make is rife with appendages of questionable utility. One area this is especially noticeable is the collection of built-in rules in gmake. These rules make it possible to do things like compile a C source file to an executable without even having a makefile, or compile and link several source files with a makefile that simply names the executable and each of the objects that go into it.

But this convience comes at a price. Although some of the built-in rules are still relevant in modern environments, many are obsolete or uncommonly used at best. When’s the last time you compiled Pascal code, or used SCCS or RCS as your version control system? And yet every time you run a build, gmake must check every source file against each of these rules, on the off chance that one of them might apply. A simple tweak to your GNU Make command-line is all it takes to get a performance improvement of up to 30% out of your makefiles. Don’t believe me? Read on.

Let’s look at a trivial example:

Touch the file input, then run gmake with the -d option, so you can see as gmake tries each of the built-in rules. GMake will ramble on for hundreds of lines, as you’ll see. Here’s a sample of that output:

What’s going on here? Well, we didn’t provide a rule describing how to build the file input, so gmake is checking to see if any of the built-in rules could be used to generate it. Of course none of them do, so this is all wasted effort. Lucky for us, a single command-line option is all you need to tell gmake not to bother with the default built-in rules: -r. Try that trivial makefile again, this time with -d -r:

All the extra nonsense is gone! And even on this toy example, there is a measurable performance improvement: originally, this makefile runs in about 0.015s (average over three runs); with the built-in rules disabled, it’s just 0.012s. But I can see you won’t be convinced by such a trivial example. So let’s try something a bit bigger:

The directory sub contains 15,000 files named 00001.x through 15000.x. With the built-in rules (and redirecting output to /dev/null), this makefile runs in about 60.2s; without the built-in rules, 42.9s — 28% faster.

Finally, let’s try this optimization on a real build. I built one component of the Accelerator project completely, then ran “no-op” builds (ie, no work to be done, just checking that everything is up-to-date). With built-in rules, this took 6.0s; without, it took 5.2s — 13% faster:

Test results (shorter is better)
Large test, with built-ins:
60.2s
Large test, no built-ins;
42.9s
No-op build, with built-ins:
6.0s
No-op build, no built-ins:
5.2s

Now, if your build actually relies on built-in rules obviously you can’t simply disable them. But you could explicitly define just those rules that you require and disable the rest. For example, if you use the default %.o: %.cpp rule, you could add just that rule to your makefiles:

Once you’ve done that, you can add -r to your command-line and enjoy the benefits. If you go this route, you can see the list of built-in rules by running gmake -p; the built-ins are marked as “built-in” in that output.

Conclusion

Disabling gmake’s array of built-in rules is an easy way to squeeze extra performance out of your makefiles, particularly on large builds and on no-op builds. All you have to do is add -r to your commmand-line. (NB: If you prefer more descriptive command-lines, you can use –no-builtin-rules instead!)


This article is the latest of several looking at different aspects of makefile performance. If you liked this article, you may enjoy the others in the series:

Electric Cloud powers Continuous Delivery. We help organizations developing mobile, embedded systems and enterprise web/IT applications deliver better software faster by automating and accelerating build, test, and deployment processes at scale. Industry leaders like Qualcomm, SpaceX, Cisco, GE, Gap, and E*TRADE use Electric Cloud solutions and services to boost DevOps productivity and Agile throughput.

Follow me

Eric Melski

Eric Melski was part of the team that founded Electric Cloud and is now Chief Architect. Before Electric Cloud, he was a Software Engineer at Scriptics, Inc. and Interwoven. He holds a BS in Computer Science from the University of Wisconsin. Eric also writes about software development at http://blog.melski.net/.
Follow me

Share this:

11 responses to “Makefile performance: built-in rules”

  1. Is there a way to turn on -r from inside the Makefile? Otherwise, it depends on the invoker remembering to use -r.

    Ciao!

  2. Evan M says:

    Put MAKEFLAGS=-r in your Makefile to turn this on by default. (You still pay some extra startup cost relative to passing -r on the command line, but it’s a fraction of a second rather than the huge difference you observe between having -r and not.)

  3. Eric Melski says:

    Adding MAKEFLAGS=-r to your Makefile disables some of the built-in rules, but not all. Try running gmake -d on a Makefile that has set MAKEFLAGS=-r and you’ll still see boatloads of extra rules being checked.

    However, you can explicitly disable each of the built-in rules by adding empty pattern rules. For example, to disable the %.o: %.f built-in, just add the following to your makefile:

    Note that the rule deliberately has no commands! Follow this same pattern for each of the built-ins and you will achieve the same effect as setting -r on the command-line.

  4. Jean says:

    What about .SUFFIXES?

    • Eric Melski says:

      @Jean: .SUFFIXES can be another unexpected source of performance problems for makefiles. I plan to cover that in a future post.

  5. awhan says:

    thanks for this great tip :)

  6. Hi. I read a few of your other posts and i wanted to say thank you for the informative posts.

  7. Vitaly says:

    Hi. If you concerned about GNU make speed you might be interested in Fastmake – continuation of GNU make with speed and syntax improvements

    • Eric Melski says:

      Thanks for the link, Vitaly. Fastmake has some interesting ideas, but unfortunately because of it’s incompatibilities with gmake, I was unable to compare performance on any real build. Also, some of your “syntax improvements” seem to be identical in function to existing features — for example, .FOREACH seems to do exactly what standard gmake $(foreach) does. Anyway, good luck to you, and thanks for stopping by!

  8. John Edwards says:

    Hi. Just want to say thanks for the post, and that we’re seeing an even greater gain than 30%. Our up-to-date builds used to take 8 seconds, and now they take 2. Great tip!

Leave a Reply

Your email address will not be published. Required fields are marked *

Subscribe

Subscribe via RSS
Click here to subscribe to the Electric Cloud Blog via RSS

Subscribe to Blog via Email
Enter your email address to subscribe to this blog and receive notifications of new posts by email.