Asset Pipelines With FRP

November 10th, 2015Dan Yoder

Task managers like Gulp and Grunt date back to the Unix make program. Their main job is transforming one file into another with a different format. For example, a common make rule transformed C files into object files. Today, you’ll often hear about asset pipelines. Instead of C files, we’re transforming Stylus into CSS, or CoffeeScript into JavaScript, and so forth.

As it turns out, such transformations are a natural application for functional reactive programming (FRP). Image credit: HISHE

Asset Pipelines For Free

We discovered this more or less by accident. We were developing some examples for Fairmont, our FRP library. We started out using Gulp. Which worked fine, for the most part. But we couldn’t help but notice that Gulp tasks had a strong resemblance to Fairmont’s reactive flows.

Here’s a Gulp task for compiling Jade files:

gulp.src('client/templates/*.jade')
  .pipe(jade())
  .pipe(minify())
  .pipe(gulp.dest('build/minified_templates'))

Here’s a reactive flow that does the same thing:

go(
  glob('*.jade', 'client/templates'),
  map(jade),
  map(minify),
  map(write('build/minified_templates')))

Of course, we’re assuming appropriate definitions for jade and minify in this example. But you get the idea. Fairmont’s FRP support incidentally allows us define asset pipelines. We thought that was pretty cool.

Introducing The Panda-9000

After we ran into some trouble with some Gulp edge-cases, we couldn’t resist taking a stab at a Fairmont-based task runner. After all, that’s just a depth-first traversal of the task tree. That, and a half-dozen helpers, were all we needed to replace Gulp. Sure enough, a few hours later, we had our first version of the Panda-9000.

Static Site Generation

We iterated from there over the next couple of months, using Panda 9000 (or P9K for short) to generate this Web site. That work is packaged up as another open source software package, Haiku9. We’re pretty happy with the results. We have a long ways to go to catch up with the ecosystem around Grunt and Gulp, of course. But we like the foundation provided by Fairmont. There’s a lot of power we get for free with that.

A Powerful Foundation

For example, we often want to provide data to our Jade templates. You can do this in Gulp with the gulp.data and gulp.jade plugins. But you still have to write a function that takes a file and turns it into data. In Panda-9000, this doesn’t require a plug-in at all. You can always add ad-hoc functions to your pipeline. That’s because the pipelines are reactive flows and are basically just composing curried functions to begin with. So we can add some data like this (this is in CoffeeScript).

task "jade", ->
  go [
    glob "**/*.jade", source
    map context source
    tee (context) -> context.data = getDataSomehow context
    tee jade
    tee write target
  ]

We don’t need a special plugin to add processing to our pipeline. All we need is a well-defined protocol, that allows for a data property.

Here’s what our Jade task looks like in real life:

task "survey/jade", "data", ->
  go [
    glob "**/*.jade", source
    reject (path) -> isMatch /(^|\/)_/, path
    map context source
    tee (context) -> Data.augment context
    tee ({target}) -> target.extension = ".html"
    map (context) -> include (Type.create type), context
    tee save
  ]

This version is a bit more involved. This is partly due to a bug and partly because the real-life version is not yet as beautiful as we’d like.

Less Than 200 Lines Of Code

Haiku9 serves as a great example of the power of Panda-9000, which, in turn, demonstrates the power of Fairmont. We’ve effectively built a static site generator on top of Panda-9000 in less than 200 lines of code. And, lest you think we just snuck all the complexity into Haiku9, that weighs in at fewer than 400 lines of code. So the combined total is still fewer than 600 lines. Which is pretty remarkable since (at least for our purposes) we’ve replaced both Gulp and Harp (which is what we were using for static site generation before this).

Learn More

You can learn more about Panda-9000 on the wiki. And, of course, you can also take a look at Haiku9. Both are still alpha releases. We welcome your feedback and contributions.

And if you want to see what’s really powering these libraries, check out Fairmont.