The last word on SCons performance

My previous look at SCons performance compared SCons and gmake on a variety of build scenarios — full, incremental, and clean. A few people suggested that I try the tips given on the SCons ‘GoFastButton’ wiki page, which are said to significantly improve SCons performance (at the cost of some accuracy, of course). Naturally, I felt that I had to do one last follow-up exploring this avenue. And since that meant I would already be running a bunch of builds, I figured I’d try out SCons’ parallel build features too. My findings follow.

Can SCons “GoFast”?

You can read all about the setup in the previous post, so I’ll just jump straight to the results. According to the “GoFastButton” recommendations, –max-drift=1 –implicit-deps-unchanged will “run your build as fast as possible”, so that’s what I used. In all cases, I did an initial, untimed from-scratch build first, to generate the initial dependency graph, then I ran a second timed build. After that timed run, I ran a clean build and finally ran the full build again, this time with -j 2, to evaluate the impact of SCons parallel build features.

Contrary to my expectations, the GoFast settings had relatively little impact over much of the test range — only about 5-10% faster than without those flags. Only the very largest build showed any significant impact, with a 25% improvement. Unfortunately that impressive result is more likely because SCons uses less memory with GoFast settings enabled. If you recall from the previous tests, with 50,000 source files, SCons’ memory footprint was a hefty 2,023 MB — enough to force my test machine to start swapping. With the GoFast settings, SCons used “only” 1,838 MB — still an awful lot of memory, but just smaller enough to avoid thrashing the system, with the end result being a substantially improved build time.

Building in parallel had a more substantial impact — reducing build times about 30% on the largest build (compared to a serial SCons build with GoFast settings enabled). That’s not as good as I had hoped for (on a large, relatively “flat” build such as this, I expected the build to parallelize very well), but it’s not terrible. Here are the complete results:

SCons full build performance, click for full size

So, GoFast seems to be a bust on full builds. It’s definitely better than vanilla SCons, but still nowhere near as fast as gmake.

Things look a little better on “one-touch” incremental builds though, where GoFast settings cut build times by about 30% across the board:

SCons incremental build performance, click for full size

Ironically, the most impressive results are on clean builds (scons -c). GoFast settings cut build times by about 40% at the low end of the test range, and by more than 50% at the high end of the test range:

SCons clean build performance, click for full size

To my amazement, SCons with GoFast settings actually beats gmake on clean builds. My guess is that this is probably because SCons handles file deletion in-process, while gmake must invoke a separate process (rm).

That’s all folks!

That’s it for my analysis of SCons performance. Thanks to everybody who offered ideas for improving my benchmarks! I hope you found this series of posts interesting.

 


Build Acceleration and Continuous Delivery

Continuous Delivery isn’t continuous if builds and tests take too long to complete. Learn more on how ElectricAccelerator speeds up builds and tests by up to 20X, improving software time to market, infrastructure utilization and developer productivity.

Eric Melski

Eric Melski was part of the team that founded Electric Cloud and is now Chief Architect. Before Electric Cloud, he was a Software Engineer at Scriptics, Inc. and Interwoven. He holds a BS in Computer Science from the University of Wisconsin. Eric also writes about software development at http://blog.melski.net/.

Share this:

11 Responses to “The last word on SCons performance”

  1. Johan Holmberg says:

    In your earlier post about SCons you talked about publishing your generator script for the benchmark. How is that going?

    • Eric Melski says:

      @Johan: yep, just took a little longer to get the OK from management. I put the script up here. The script just generates a build tree in a subdir of your working directory called “sconsbld”. I invoked it as “./genscons.pl -d N -l 2 -f 500″, where N varied from 3 to 99, indicating the number of subdirectories of “sconsbld”.

  2. Parallelize it says:

    Try make -j or make -j 4

    -j [jobs], –jobs[=jobs]
    Specifies the number of jobs (commands) to run simultaneously. If there is more than
    one -j option, the last one is effective. If the -j option is given without an argu‐
    ment, make will not limit the number of jobs that can run simultaneously.

    • Eric Melski says:

      @Parallelize it: I had run the gmake tests with -j 2 also, but I didn’t include that data because I thought it would be a distraction in this article. Anyway, it cut the gmake build times between 35% and 40%, depending on the size of the build. Here’s the graph:

  3. […] This post was mentioned on Twitter by Tech news (BOT) and Proggit Articles, Eric Melski. Eric Melski said: The last word on SCons performance: http://bit.ly/aVFPsb […]

  4. […] The last word on SCons performance: 13% of page views […]

  5. James Hartlow says:

    My guess (experience) is that posting the results with parallel builds would have put SCons in a much better position.

    It’s 2011, everybody compiles with at least a dual or quad core processor. Since SCons generates a perfect dependency tree, he is the perfect tool for parallel compilation.

  6. attosecond says:

    Excellent post, hope to see a follow up soon… Build times impact everyone.

    One comment, do you have figures on how long the latency for starting to compile changed files? In other words, if you make a syntax mistake and hit ‘build’, how long does it take to hit the compile error? That’s a common development scenario.

    Thanks

    • Eric Melski says:

      @attosecond: I’m sorry, I don’t have that information readily available. That would certainly be an interesting test though!

Leave a Reply

Subscribe via RSS
Click here to subscribe to the Electric Cloud Blog via RSS

Subscribe to Blog via Email
Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Upcoming Webinars:


Deploy Better Software, Faster
Dec 3, 11am PST
Register Now »


Continuous Discussions Online Panel

Next Episode: Jenkins in the Enterprise
Dec. 16, 10am PST


Add to your calendar »

Using CMake?
Accelerate Your Builds for Free!

Read the technical document on CMake Build Acceleration
Read Now »
banner_continuous-integration-at-scale

banner_huddle_free-build-acceleration