What is SparkBuild?

At the 2009 Electric Cloud Customer Summit we introduced SparkBuild, a free gmake- and NMAKE-compatible build tool. SparkBuild is now in public beta, and several people have asked us for some more explanation: what is SparkBuild and why should I care? I thought I’d take a crack at answering those questions, hopefully without sounding too “marketingy”. Here goes.

SparkBuild is actually a package containing two components: SparkBuild emake and SparkBuild Insight. As the names imply, these components are derived from the corresponding pieces of ElectricAccelerator and ElectricInsight. That means that they offer some of the same benefits that our commercial product does, but for free, of course. Why did we do this? I’ll be completely honest with you: we’re hoping that people will use SparkBuild, share SparkBuild and talk about SparkBuild, ultimately raising awareness of Electric Cloud and our commerical products. Beyond that though, I’m personally excited about SparkBuild because I want to see these technologies that I’ve worked on for so long get used by as many people as possible.

I think you’ll be interested in SparkBuild because it offers some of the same benefits that our commercial tools do (annotation and build analysis), and even some that aren’t yet part of Accelerator (subbuilds). Read on to learn more about what these features provide and how you can use them.

Annotation

SparkBuild emake can emit XML-annotated build logs, or annotation. The additional information includes data like:

  • The environment and working directory used to run each command.
  • The exact, expanded command-line of each command — even those that were run in “silent” mode with the @ prefix.
  • The runtime of each command.
  • The relationship between submakes in the build.
  • The build dependency graph.

This is the kind of information you’ve been trying to get out of your build tool for years (raise your hand if you’ve ever sprinkled date commands in your makefiles for performance measurement!). Now sophisticated analysis of your build is possible, and many useful reports are trivial one-liners using standard XML processing tools like the XML command-line tools. For example:

  • Pinpoint failed commands: xmlgrep -g -f build.anno 0 “job:type!=remake.failed”
  • Find the prereqs of any target: xmlgrep -g -f build.anno 0 “job.waitingJobs:idList~=Jb4220fbc”
  • Find targets that have a given prereq: xmlgrep -g -f full.xml 0 “job:id=Jb4220fbc” | xmlfmt job.waitingJobs:idList

SparkBuild Insight

Not everybody has the time to code up fancy analysis of the data in annotation, so SparkBuild includes SparkBuild Insight, a derivative of ElectricInsight, our commercial build visualization and analysis tool. Although it doesn’t have all the reports that ElectricInsight does, don’t make the mistake of thinking it’s underpowered. You can:

  • … see the amount of time spent on different types of work, like compiles, links, etc.:
    timebytype
  • … see the specific targets associated with each type of work:
    legend
  • … drill into targets to see the commands executed and the output from those commands:
    details
  • … search for targets using numerous criteria, like runtime, commands, name, and type of work:
    search

Subbuilds

Subbuilds are build avoidance done right: literally skipping entire swaths of the build tree by building only the parts of the tree that are relevant to the work you’re doing now. This is one of those features I wish we had implemented years ago when we first thought of it, because it really is that useful to me on a daily basis. My edit-build-test cycle is faster, and I never have to deal with incremental builds that break because I pulled in another developer’s changes to one of my prerequisite libraries when I resynced to the source repository. You can read more about subbuilds here.

Availability

SparkBuild is in public beta and is available for download at www.sparkbuild.com. You’ll also find documentation and videos explaining how to use SparkBuild, and user forums where you can get help from Electric Cloud employees and what we hope will be a thriving community of users. Try it out and let us know what you think!

Follow me

Eric Melski

Eric Melski was part of the team that founded Electric Cloud and is now Chief Architect. Before Electric Cloud, he was a Software Engineer at Scriptics, Inc. and Interwoven. He holds a BS in Computer Science from the University of Wisconsin. Eric also writes about software development at http://blog.melski.net/.
Follow me

Share this:

8 responses to “What is SparkBuild?”

  1. Art Zaifman says:

    SparkBuild is fantastic and makes it very easy to create vertical IDE’s!

  2. Art Zaifman says:

    We’re experimenting with different ways of how to combine source control systems and runtime environments using your technologies and some in-house developed visualization tools….

  3. Danny says:

    Do SparkBuild support build TI DSP C6000 code?
    Can we use SparkBuild to distribute the load?

    • Eric Melski says:

      @Danny: I’m not personally familiar with the TI DSP C6000 build environment, but from a little bit of googling it appears that it does have a make-based mode. So if you are using makefiles to build your stuff, then yes, SparkBuild should work with it (assuming they are GNU make compatible makefiles, of course).

      As for distributing the load: SparkBuild explicitly does not provide that option. If you are interested in accelerating your build via parallelization and distribution, you should check out ElectricAccelerator, our commercial offering.

      Thanks for the comment!

  4. Anon User says:

    Is this still available? The website seems to redirect to the electric-cloud home page. Assuming it’s not.

Leave a Reply

Your email address will not be published. Required fields are marked *

Subscribe

Subscribe via RSS
Click here to subscribe to the Electric Cloud Blog via RSS

Subscribe to Blog via Email
Enter your email address to subscribe to this blog and receive notifications of new posts by email.