My previous look at SCons performance compared SCons and gmake on a variety of build scenarios — full, incremental, and clean. A few people suggested that I try the tips given on the SCons ‘GoFastButton’ wiki page, which are said to significantly improve SCons performance (at the cost of some accuracy, of course). Naturally, I felt that I had to do one last follow-up exploring this avenue. And since that meant I would already be running a bunch of builds, I figured I’d try out SCons’ parallel build features too. My findings follow.
Can SCons “GoFast”?
You can read all about the setup in the previous post, so I’ll just jump straight to the results. According to the “GoFastButton” recommendations, –max-drift=1 –implicit-deps-unchanged will “run your build as fast as possible”, so that’s what I used. In all cases, I did an initial, untimed from-scratch build first, to generate the initial dependency graph, then I ran a second timed build. After that timed run, I ran a clean build and finally ran the full build again, this time with -j 2, to evaluate the impact of SCons parallel build features.
Contrary to my expectations, the GoFast settings had relatively little impact over much of the test range — only about 5-10% faster than without those flags. Only the very largest build showed any significant impact, with a 25% improvement. Unfortunately that impressive result is more likely because SCons uses less memory with GoFast settings enabled. If you recall from the previous tests, with 50,000 source files, SCons’ memory footprint was a hefty 2,023 MB — enough to force my test machine to start swapping. With the GoFast settings, SCons used “only” 1,838 MB — still an awful lot of memory, but just smaller enough to avoid thrashing the system, with the end result being a substantially improved build time.
Building in parallel had a more substantial impact — reducing build times about 30% on the largest build (compared to a serial SCons build with GoFast settings enabled). That’s not as good as I had hoped for (on a large, relatively “flat” build such as this, I expected the build to parallelize very well), but it’s not terrible. Here are the complete results:
So, GoFast seems to be a bust on full builds. It’s definitely better than vanilla SCons, but still nowhere near as fast as gmake.
Things look a little better on “one-touch” incremental builds though, where GoFast settings cut build times by about 30% across the board:
Ironically, the most impressive results are on clean builds (scons -c). GoFast settings cut build times by about 40% at the low end of the test range, and by more than 50% at the high end of the test range:
To my amazement, SCons with GoFast settings actually beats gmake on clean builds. My guess is that this is probably because SCons handles file deletion in-process, while gmake must invoke a separate process (rm).
That’s all folks!
That’s it for my analysis of SCons performance. Thanks to everybody who offered ideas for improving my benchmarks! I hope you found this series of posts interesting.
Build Acceleration and Continuous Delivery
Continuous Delivery isn’t continuous if builds and tests take too long to complete. Learn more on how ElectricAccelerator speeds up builds and tests by up to 20X, improving software time to market, infrastructure utilization and developer productivity.
Eric Melski
Latest posts by Eric Melski (see all)
- Why I Love ElectricAccelerator — and You Should Too - February 3, 2014
- Electric Cloud Customer Summit 2012 by the Numbers - October 26, 2012
- The last word on SCons performance - August 11, 2010
In your earlier post about SCons you talked about publishing your generator script for the benchmark. How is that going?
@Johan: yep, just took a little longer to get the OK from management. I put the script up here. The script just generates a build tree in a subdir of your working directory called “sconsbld”. I invoked it as “./genscons.pl -d N -l 2 -f 500″, where N varied from 3 to 99, indicating the number of subdirectories of “sconsbld”.
Try make -j or make -j 4
-j [jobs], –jobs[=jobs]
Specifies the number of jobs (commands) to run simultaneously. If there is more than
one -j option, the last one is effective. If the -j option is given without an argu‐
ment, make will not limit the number of jobs that can run simultaneously.
@Parallelize it: I had run the gmake tests with -j 2 also, but I didn’t include that data because I thought it would be a distraction in this article. Anyway, it cut the gmake build times between 35% and 40%, depending on the size of the build. Here’s the graph:
[…] This post was mentioned on Twitter by Tech news (BOT) and Proggit Articles, Eric Melski. Eric Melski said: The last word on SCons performance: http://bit.ly/aVFPsb […]
Interesting work.
@Matt: Thanks for commenting!
[…] The last word on SCons performance: 13% of page views […]
My guess (experience) is that posting the results with parallel builds would have put SCons in a much better position.
It’s 2011, everybody compiles with at least a dual or quad core processor. Since SCons generates a perfect dependency tree, he is the perfect tool for parallel compilation.
Excellent post, hope to see a follow up soon… Build times impact everyone.
One comment, do you have figures on how long the latency for starting to compile changed files? In other words, if you make a syntax mistake and hit ‘build’, how long does it take to hit the compile error? That’s a common development scenario.
Thanks
@attosecond: I’m sorry, I don’t have that information readily available. That would certainly be an interesting test though!