asset_builder

Asset Builder

Simple middleware for compiling, concatenating, minifying, and caching assets with Express.

• Supports javascript, css, less, and handlebar templates.
• Extensible architecture allows support for anything that can be compiled to javascript or css.
• Compiles on demand, with no need to precompile before deploying.
• Caches like a boss.

Installation

npm install asset_builder

or add it ot your package.json file:

{
  "dependencies": {
    "asset_builder":           "0.2"
  },
}

Lets Get To It

Basic example:

app.js

var express   = require("express");
assetbuilder = require("asset_builder");

var app = express.createServer();
app.use(assetbuilder.middleware);
app.use(express.static(path.join(__dirname, "public")));
app.listen(8000);

app/assets/application.js.json

{
  "files": [
    "javascripts/lib/boot.js",
    "javascripts/lib",
    "javascripts/"
  ]
}

Link to assets in the view using the provided stylesheetUrl and javascriptUrl helpers, and it will concatenate (in order):

• The file app/assets/javascripts/lib/boot.js
• All javascript files under app/assets/javascripts/lib/ (except the previously included boot.js)
• All javascript files under app/assets/javascripts/ (except any files already included)

Serve stylesheets too (css or less):

app/assets/application.css.json

{
  "files": [
    "stylesheets/normalize.css",
    "stylesheets/base.less",
    "stylesheets/"
  ]
}

Breaking It Down

Registering The Middleware

var express   = require("express");
assetbuilder = require("asset_builder");

var app = express.createServer();
app.use(assetbuilder.middleware);

Configuring The Middleware

Configuration is only necessary to override the default settings. Below is an example showing the default settings:

assetbuilder.configure({
  src_root: path.join("app", "assets"),
  dst_root: path.join("public", "assets"),
  base_url: "/assets",
  env: "production"
});

Registering View Helpers

View helpers can be registered with express with the following call:

assetbuilder.registerViewHelpers(app);

The views now have access to two url helpers: stylesheetUrl and javscriptUrl An example layout.jade file looks like this:

!!! 5
html
  head
    link(rel="stylesheet", type="text/css", href=stylesheetUrl)
    script(type="text/javascript", src=javascriptUrl)

Serving Assets

In order to serve javascript and css assets, static file middleware needs to be registered after the asset builder.

app.use(assetbuilder.middleware);
app.use(express.static(path.join(__dirname, "public")));

Specify Asset Content

Asset Builder looks for a json file describing the content of the asset to be built. By default, it looks for app/assets/application.js.json and app/assets/application.css.json

Example: application.js.json

{
  "files": [
    "javascripts/lib/boot.js",
    "javascripts/lib",
    "javascripts/"
  ]
}

Paths are relative to the json file. IE: The above looks for app/assets/javascripts/lib/boot.js

Putting It All Together

Here is an example of a typical express middleware stack:

var express      = require("express");
var assetbuilder = require("asset_builder");
var router       = require("./app/router");

var app = express.createServer();
app.configure(function() {
  this.use(express.errorHandler({
    showStack: true,
    dumpExceptions: true
  }));
  this.use(assetbuilder.middleware);
  this.use(express.static(path.join(__dirname, "public")));
  this.use(express.bodyParser());
  this.use(express.methodOverride());
  this.use(express.cookieParser());
  this.use(express.session({ secret: "keyboard cat" }));
  this.use(express.router(router));
  this.set("views", path.join(__dirname, "app", "views"));
  this.set("view engine", "jade");
  this.set("view options", {layout: false });
});

assetbuilder.registerViewHelpers(server);
assetbuilder.configure({
  src_root: path.join(__dirname, "app", "assets"),
  dst_root: path.join(__dirname, "public", "assets"),
  base_url: "/assets",
  env: "production"
});

server.listen(8000);

Options

Don't like the default paths, or url's? No problem.

The options shown below are the defaults:

var assetbuilder = require("asset_builder");
assetbuilder.configure({
  src_root: path.join(__dirname, "app", "assets"),
  dst_root: path.join(__dirname, "public", "assets"),
  base_url: "/assets",
  env: "development"
});

src_root

This is the directory where Asset Builder will look for application.js.json and application.css.json. Filenames within the json manifest files are resolved relative to this root.

dst_root

This is the directory where Asset Builder will write the generated asset files (application.js and application.css). Note that the static file server must be able to serve files from this directory.

base_url

This is the leading portion of the url that will be used by both Asset Builder and the static file server. Ex: /assets/application.js

Note that this setting is tied to the dst_root setting. Files written to dst_root need to be served by the static file server with a request to base_url.

env

The environment Asset Builder is running in.

Defaults to "development", which performs no caching. When set to "production", caching is enabled.

Manifest Files

The asset manifest is a json file named either application.js.json or application.css.json. By default, they are located in the app/assets directory, see the options section to change their location. The manifest files contain an array of filenames and directory names that will be used to build the asset.

Files are included in the order they are specified in the manifest. Directories will recursively include all matching files (js or hbs for javascripts, css or less for stylesheets)

No file will be included twice, even if it matches multiple file or directory listings.

If certain files need to be included before others, specify those files first, ex:

{
  "files": [
    "javascripts/lib/boot.js",
    "javascripts/lib",
    "javascripts/"
  ]
}

Preprocessors

Asset Builder can only serve two types of assets: javascript and css. However, Asset Builder uses preprocessors to convert an input file into one of these types.

Two preprocessors are included (handlebars and less), and custom preprocessors can be registered dynamically.

Handlebars

Handlebar templates will be precompiled by Asset Builder if their path is included in the javascript manifest, and they have a .hbs file extension.

Precompiled templates will be available (in the browser) in a global hash called Handlebars.templates, with the filename as the key.

For example, a template named foo.hbs, would be stored at Handlebars.templates.foo

Example template:

<h1>{{title}}</h1>

For details, see the handlebars website

Less

Less stylesheets will be compiled into css by Asset builder if their path is included in the stylesheet manifest, and they have a .less file extension.

Custom Preprocessors

Custom preprocessors can be registered dynamically using the registerPreprocessor function.

An example of a javascript preprocessor that append a comment onto each file with the .foo extension:

var myPreprocessor = {
  ext: "foo",
  preprocessor: function(data, filepath, hollaback) {
    var error = null;
    var content = data + "// Comment!";
    hollaback(error, content);
  }
};
assetbuilder.registerPreprocessor(myPreprocessor);

See the wiki for more complicated examples, including Ember support.

Caching, Cache-Busting, and Links

There are two kinds of caching performed:

• Server side asset caching
• Client side asset caching

Server Side Caching

Assets that are built can be cached.

• In development mode, assets are built on each request.
• In production mode, assets are built on the first request, and served from cache for all subsequent requests.

Client Side Caching

When an asset request is processed, response headers are set to maximize the chance of client caching.

Each browser uses a set of heuristics to determine if files should be downloaded, or served from cache. Asset builder follows Google's optimize caching recommendations to maximize the chance of client caching.

There are three possible client actions:

1. Makes a normal GET request and downloads the asset.
2. Makes an If-Modified-Since request, and either downloads the asset or uses the cached version depending on response.
3. Makes no requests at all, and uses the cached copy of the asset.

Clearly, option 3 is most desirable. In order to maximize the chances of full caching, Asset Builder sets the following headers:

• Sets the Expires header to be one year from now
• Sets the Last-Modified header to be one month ago
• Sets the Cache-Control header to public to enable caching of assets served over ssl

Example asset response: (Made on January 22, 2012)

HTTP/1.1 200 OK
X-Powered-By: Express
Expires: Mon, 21 Jan 2013 14:50:31 GMT
Cache-Control: public
Last-Modified: Fri, 23 Dec 2011 04:50:31 GMT
Date: Sun, 22 Jan 2012 14:50:31 GMT
ETag: "286234-1327243831000"
Content-Type: application/javascript
Accept-Ranges: bytes
Content-Length: 286234
Connection: keep-alive

Cache Busting

You might be wondering: If the client unconditionally caches the file, how do I prevent stale assets from being used on the client?

Asset Builder's link helpers generate unique links for the assets. For example: assets/application_b1010839e0914dc8cbbfd8d4aecf9a161db917be81a4f7c3317f141dcae700fe.js

When the file name of the asset changes, the client sees it as a new file and will no longer use any previously cached versions.

In development mode, the links generated by the link helpers are unique on each request.

In production mode, the links generated by the link helpers include a SHA256 hash of the asset, so they will only change when the asset changes.

Gzipping

An additional step to maximize performance is to compress (gzip) the contents of the assets.

This can be enabled several ways:

• Use the gzippo npm package
• Configure Nginx to gzip requests matching asset url's

See the Options section on base_url and dst_root to serve assets separately from other static files.

Development vs Production

Development Mode

In development mode, assets are generated for each request. Additionally, the link helpers generate unique links for each request.

Production Mode

In production mode, assets are only built once, and served from cache thereafter. The link helpers generate links based on the SHA256 hash of the asset content, so the links will only change when the content of the assets change.

License

MIT, see the LICENSE file.