ElectricFlow DSL: Pipeline-As-Code for Orchestrating Releases

For the last decade or so, Infrastructure-as-Code has made its way into IT, allowing you to automate configuration of cloud and infrastructure using declarative code that you can easily test, backup, version and reuse. Here are some of the values of using code to describe configurations:

  • Enforce versioning for better reliability, repeatability, and remediation
  • Make the intent or desired end state of the system more clear
  • Improve team efficiency by making it easier to share and collaborate

Infrastructure-as-code has been key in allowing more complex infrastructure configurations – including on-prem, virtual, and hybrid cloud.

Given the benefits of infrastructure-as-code, it stands to reason that you’d want to do the same for automating pipelines and orchestrating your releases. This can include everything from using code to specify your CI pipeline, applications and environments, defining compliance and gating policies between pipeline stages, and the configuration of all the tools that are integral to your end-to-end DevOps toolchain.

One of the questions I get asked the most by our customers is “How do I save my CI/CD or Release Orchestration projects in ElectricFlow and version them?” Over the years I have written some articles about it (see Versioning Best Practices  and Best Practices for Promoting to Production), and contributed the code for the EC-Admin::projectAsCode to help with this problem.

With the recent release of ElectricFlow 6.0, I’m thrilled to say we now support a robust DSL to bring full pipeline-as-code to ElectricFlow!

ElectricFlow Domain Specific Language (DSL)

ElectricFlow provides a domain specific language (DSL), which is ideal for those times when complex processes can better be represented as code. The DSL serves as another way of interacting and working with the platform, in addition to the GUI. It allows you to define your automation processes as high-level code that is versionable, testable, and reusable.

ElectricFlow’s DSL is based on a general purpose programming language that many already know, Groovy. This gives you a lot of the flexibility due to the nature of the language itself as well as an easy access to the ElectricFlow object library, to describe even the most complex systems.

If we know anything from working with our customers, is that Automation is everywhere, and every pipeline or process is different. You want your scripting language to be flexible and extensible enough to support even your most complex scenarios, yet simple and easily understood so you can get going quickly, and easily reuse your scripts. Let me demonstrate a couple of common DSL scenarios with code samples below, so you get a sense of just how powerful this is.

The Old Way: Scripts

Let’s look at how we may have tried to do something like create a project and pipeline using using code. Previously we’d use Perl (ec-perl) and we would have to write something similar to the code below:

This code works just fine. There’s nothing wrong with scripting, as long as you’re disciplined and well organized. But what happens if you run this code multiple times? It will fail telling you that the project already exists. So you’ll have to add some clean-up code to delete your objects beforehand, or add some error management to check for the projects that exist and then modify or create a new one depending on the result. This is just a small example, but you can see how a traditional script approach requires a lot of overhead to properly maintain your code as it evolves.

The New Way: Pipeline-As-Code with ElectricFlow DSL

Let’s now look at the same example with DSL.

To run the code above, simply save it to a file named mypipeline.groovy and type on your command line:

As you can see, in addition to being slightly shorter, the main advantage is that you can run it multiple times without fear of error. The server will automatically modify the objects that already exists.

This now makes it super easy to save your ElectricFlow objects in simple files that you can check-in to your favorite SCM.

Again – this is a small example, but you can see conceptually how a declarative code configuration framework is more manageable and efficient than traditional scripts.

ElectricFlow Groovy DSL also allows:

  • Easy looping over a list of objects (like projects or procedures) with “.each” syntax
  • Transaction closures that allow you  to execute a procedure created earlier
  • API exposure
  • Parameters
  • Power ElectricFlow substitution mechanism if you run in a step
  • and much much more…

So does it mean that you have now to rewrite all your ElectricFlow procedures and deploy applications in the new DSL?

Fortunately not. This new version comes with an API call to generate DSL code directly from the ElectricFlow server, very similar to the already familiar export mechanism. To accomplish this use:

Voila! This will print the DSL code in the standard output encapsulated by some XML tags, in the usual format for the ElectricFlow API.

Using Pipeline-As-Code for Deployments

Now let’s look at a slightly more complex example of modeling and deploying an application. ElectricFlow’s UI allows you to model your application, environment and deployment processes – so you can version them and reuse them over and over again, for consistent, push-button, deployments.

When we use the DSL to specify our deployment model, the first part is to create a component tied to an artifact:

Then we can create the “retrieve” and “Deploy” steps inside a component process:

Then we need an application process to call the installation of our newly created component:

For simplicity and brevity, this example is incomplete. For instance, it’s missing the Environments and their mapping to the app. But hopefully you get the gist.

Want More?

  • For  complete and more advanced examples, check our DSL GitHub repository, and if you need any assistance, please don’t hesitate to ask us.
  • The full DSL documentation is available here.
  • If you want to learn all about the DSL – including advanced use cases and best practices – be sure to register for the upcoming DevOps Enterprise Summit. During the conference, Electric Cloud customers can take advantage of 2 full days of training – which cover DSL, along with a ton of other goodies to advanced your knowledge.

Laurent Rochette

Laurent Rochette is a Professional Service Engineer with Electric Cloud. He trains customers on our products and help them with deployments and consulting to enable them to use our products effectively. Prior to joining Electric Cloud, Laurent served as an IT Architect at Mentor Graphics. Laurent holds a Master degree in Computer Science from the Grenoble Polytechnic Institute (INPG) in France.

By continuing to browse or by dismissing this alert you agree to the storing of first- and third-party cookies on your device to enhance site navigation, analyze site usage, and assist in our marketing efforts. See privacy policy.