Cartero is a client side asset manager, especially suited for organizing, processing, and serving the many JavaScript, stylesheet, and template assets needed in "thick client" web applications built with JavaScript MVC frameworks.
As of the time of this writing Cartero only works with Node.js / Express, but the very small amount of web framework specific logic is easy to port to any environment.
Benefits
- Instead of using separate directories for each type of asset, group your assets into "bundles" of related JavaScript files, stylesheets, templates, and images (e.g. keep
person.coffee
, person.scss
, person.tmpl
together in one directory). - Specify the exact bundles that are required for each page in the page's template.
- Easily manage bundle dependencies.
- All assets that a page requires are automatically injected into the served HTML when the page's template is rendered. No more messing with
<script>
and <link>
tags!
- In development mode, served assets are preprocessed, but not minified or concatenated.
- In production mode, served assets are preprocessed, minified and concatenated.
- All assets that live in the same directory as a page's template are automatically included when that page is rendered.
- Use your preferred JavaScript module system (e.g. RequireJS, Marionette Modules, etc.). If you'd like, even enjoy built in support for client side CommonJS style modules via Browserify!
- Easily run any and all of your favorite preprocessing and minification tasks (scss, coffee, uglify, etc.).
- Works in harmony with package managers like Bower.
Overview
The Asset Library
Get ready for a slight paradigm shift from the traditional js / css / template directory structure. With Cartero, you can keep all your assets, regardless of type, in your application's Asset Library (except for assets that are just used by a particular page, which can be stored with that page's template - see below). Each subdirectory of your Asset Library defines a Bundle that may contain JavaScript files, stylesheets, templates, and images. Additionally, each bundle may contain a bundle.json
file, which contains meta-data about that bundle, such as any dependencies on other bundles. Take the following example library:
assetLibrary/
JQuery/
jquery.js
JQueryUI/
bundle.json = { dependencies : [ "JQuery" ] }
jquery-ui.js
jquery-ui.css
Backbone/
bundle.json = { dependencies : [ "JQuery" ] }
backbone.js
Dialogs/
bundle.json = { dependencies : [ "Backbone", "JQueryUI" ] }
dialogManager.coffee
EditPersonDialog/
bundle.json = { dependencies : [ "Dialogs" ] }
editPersonDialog.coffee
editPersonDialog.scss
editPersonDialog.tmpl
Because of the bundle.json
files (contents inlined), the EditPersonDialog
bundle depends on the Dialogs
bundle, and indirectly depends on the other three bundles. When a page requires a bundle, dependencies are automatically resolved.
It is also possible to implicitly declare dependencies by nesting bundles because, by default, child bundles automatically depend on their parent bundles. For example, we can put the EditPersonDialog
bundle inside the Dialogs
bundle, like so:
assetLibrary/
JQuery/
jquery.js
JQueryUI/
bundle.json = { dependencies : [ "JQuery" ] }
jquery-ui.js
jquery-ui.css
Backbone/
bundle.json = { dependencies : [ "JQuery" ] }
backbone.js
Dialogs/
bundle.json = { dependencies : [ "Backbone", "JQueryUI" ] }
dialogManager.coffee
EditPersonDialog/
editPersonDialog.coffee
editPersonDialog.scss
editPersonDialog.tmpl
Now the bundle named Dialogs/EditPersonDialog
depends on on the Dialogs
bundle (and indirectly depends on the other three bundles) by virtue of the directory structure.
Page specific assets
In addition to the assets in bundles that are required by a page, the assets that live in the same directory as a page's server side template will automatically be included when it is rendered. For example, say your page templates live in a directory named views
, as is typical for most web frameworks.
views/
login/
login.jade
login.coffee
login.scss
peopleList/
peopleList.jade
peopleList.coffee
peopleList.scss
When the login.jade
template is rendered, the login.coffee
and login.scss
assets will automatically be injected into the HTML of the page, as will the peopleList.*
assets when the peopleList.jade
template is rendered.
How it works
The Cartero Grunt Task
The heart of Cartero is an intelligent Grunt.js task that ties together other Grunt.js tasks. You configure and call the Cartero Grunt Task from your application's gruntfile. You specify exactly which preprocessing and minification tasks your application needs, and those tasks are then called by the Cartero task at the appropriate times. After the Cartero task is finished, all of your assets will be preprocessed, and, in production mode, concatenated and minified. Additionally, the Cartero task generates a cartero.json
file that enumerates the assets required for each of your page templates.
The Hook
There is also a very small but important piece of logic for serving up assets and injecting them into rendered HTML, called the Hook. The Hook needs to reside in your web application framework, since it is used at the time your templates are rendered. Currently there is a Hook available only for Node.js / Express, but there is minimal logic involved and it is easy to implement in any environment. Each time you render a template, the Hook is used to look up the template in the cartero.json
file generated by the Cartero Grunt Task, and place raw HTML into three variables that are exposed to the template:
cartero_js
- the raw HTML of the <script>
elements that load all the required JavaScript files.
cartero_css
- the raw HTML of the <link>
elements that load all the required CSS files.
cartero_tmpl
- the raw, concatenated contents of all the required client side template files.
You may then output the contents of those variables in the appropriate places in your template just like any other template variable. For example, if you are using Jade templates, your page structure might look something like this:
// page layout
doctype 5
html(lang="en")
head
title myPage
| !{cartero_js}
| !{cartero_css}
body
| !{cartero_tmpl}
h1 Hello World
Getting started
First, install Cartero via npm:
npm install cartero
Now configure the Cartero Grunt Task in your applcation's gruntfile. (If you haven't used Grunt before, read this first.) Here is the minimal configuration that is required to run the Cartero Grunt Task (all options shown are required):
module.exports = function( grunt ) {
grunt.initConfig( {
cartero : {
options : {
projectDir : __dirname,
library : {
path : "assetLibrary/"
},
views : {
path : "views/",
viewFileExt : ".jade"
}
publicDir : "static/",
tmplExt : ".tmpl",
mode : "dev"
}
dev : {},
prod : {
options : {
mode : "prod"
}
}
}
} );
grunt.loadNpmTasks( "cartero" );
grunt.loadNpmTasks( "grunt-contrib-watch" );
};
The Cartero Grunt Task also takes options that allow you to call arbitrary preprocessing and minification tasks (to compile .scss, uglify JavaScript, etc.), and more. See the reference section for a complete list of options for the Cartero task.
Once you have configured the Cartero Grunt Task, you need to configure the Hook in your web framework. As of this writing there is only a Hook available for Node.js / Express, which is implemented as Express middleware. To install the middleware run:
npm install cartero-express-hook
Then use
it, passing the absolute path of your project directory (i.e. the projectDir
option from the gruntfile configuration).
var app = express();
var carteroMiddleware = require( "cartero-express-hook" );
app.configure( function() {
app.set( "port" , process.env.PORT || 3000 );
app.set( "views" , path.join( __dirname, "views" ) );
app.use( express.static( path.join( __dirname, "static" ) ) );
app.use( carteroMiddleware( __dirname ) );
} );
Now you are ready to go. To let Cartero know which asset bundles are required by which pages, you use Directives. The Cartero Grunt Task scans your page template files for these Directives, which have the form ##cartero_xyz
. The ##cartero_requires
Directive is used to declare dependencies:
// peopleList.jade
// ##cartero_requires "Dialogs/EditPersonDialog"
doctype 5
html(lang="en")
head
title login
| !{cartero_js}
| !{cartero_css}
body
| !{cartero_tmpl}
h1 People List
// ...
When you run either of the following commands from the directory of your gruntfile:
grunt cartero:dev --watch
grunt cartero:prod
The Cartero Grunt Task will fire up, process all of your assets, and put the cartero.json
file used by the Hook in your project folder. The dev
mode --watch
flag tells the Cartero Grunt Task to watch all of your assets for changes and reprocess them as needed. In prod
mode, the task will terminate after minifying and concatenating your assets. In either mode, when you load a page, the three variables cartero_js
, cartero_css
, and cartero_tmpl
with be available to the page's template, and will contain all the raw HTML necessary to load the assets for the page.
Reference
Cartero Grunt Task Options
options : {
"projectDir" : __dirname,
"library" : {
path : "assetLibrary/",
childrenDependOnParents : true,
directoriesToFlatten : /^_.*/,
bundleProperties : grunt.file.readJSON( "bundleProperties.json" ),
namespace : "App"
},
"views" : {
path : "views/",
viewFileExt : ".jade",
filesToIgnore : /^_.*/,
directoriesToIgnore : /^__.*/,
directoriesToFlatten : /^_.*/,
namespace : "Main"
}
"publicDir" : "static/",
"mode" : "dev",
"preprocessingTasks" : [ {
name : "coffee",
inExt : ".coffee",
outExt : ".js",
options : {
sourceMap : true
}
}, {
name : "sass",
inExt : ".scss",
outExt : ".css"
} ],
"minificationTasks" : [ {
name : "htmlmin",
inExt : ".tmpl",
options : {
removeComments : true
}
}, {
name : "uglify",
inExt : ".js",
options : {
mangle : false
}
} ],
browserify : true
}
Properties of bundle.json
Each of your bundles may contain a bundle.json
file that specifies meta-data about the bundle, such as dependencies. (Note: An actual bundle.json file, since it is simple JSON, can not contain JavaScript comments, as does the example.) By using the bundleProperties
grunt taks option, you can alternatively specify this meta-data for all bundles in a central location.
{
"dependencies" : [ "JQuery" ],
"whitelistedFiles" : [ "backbone.js" ],
"filePriority" : [ "backbone.js" ],
"directoriesToFlatten" : [ "mixins" ],
"prioritizeFlattenedDirectories" : false,
"keepSeparate" : true,
"devModeOnlyFiles" : [ "mixins/backbone.subviews-debug.js" ],
"prodModeOnlyFiles" : [ "mixins/backbone.subviews.js" ],
"dynamicallyLoadedFiles" : [ "ie-8.css" ],
"browserify_executeOnLoad" : [ "backbone.js" ]
}
Directives
##cartero_requires bundleName_1, [ bundleName_2, ... ]
This Directive is used in server side templates to specify which bundles they require. Bundles are referred to by their name, which is the full path of their folder, relative to the Asset Library directory in which they reside. If the Asset Library directory has a namespace
property, that namespace should be pre-pended to the bundle name. Generally you will want to enclose the Directive in a "comment" block of whatever template language you are using, as shown here (.erb syntax).
<%# ##cartero_requires "App/Dialogs/EditPersonDialog" %>
<%# All dependencies are automatically resolved and included %>
##cartero_extends parentView
This Directive is used in server side templates to specify that one template should "inherit" all of the assets of another. parentView must be a path relative to the view directory (pre-pended with the view directory's namespace
, if it has one).
<%# ##cartero_extends "layouts/site_layout.twig" %>
##cartero_dir
When your assets are processed, this Directive is replaced with the path of the directory in which it appears. It is similar in concept to the node.js global __dirname
, but the path it evaluates to is relative to your application's publicDir
.
var templateId = "##cartero_dir";
It can be used in any type of asset processed by Cartero, including client side template files.
<script type="text/template" id="##cartero_dir">
...
</script>
##cartero_browserify_executeOnLoad
When the browserify
option in the Cartero Grunk Task is enabled, this directive is used in JavaScript files to specify that they should be automatically executed when they are loaded. You will definitely want to include this directive in your "main" JavaScript files for each page, since otherwise they would never be executed!
FAQ
Q: Does Cartero work with Rails, PHP, etc., or just with Node.js / Express?
The heart of Cartero is an intelligent Grunt.js task, and can be used with any web framework. However, there is a small piece of logic called the Hook which must be called from your web framework, since it is used when each page is rendered. If you are interested in developing a Cartero Hook for your web framework of choice, keep reading - it's not hard.
From a high level perspective, the Hook is responsible for populating the cartero_js
, cartero_css
, and cartero_tmpl
variables and making them available to the template being rendered. The implementation details are somewhat dependent on your web framework, but the general idea will always be similar.
- When the Hook is configured or initialized, it should be passed the absolute path of your
projectDir
. - The Hook needs to be called before your web framework's "render" function to set the value of the three template variables for which it is responsible. It should be passed the template file being rendered.
- The Hook uses the
cartero.json
file that was generated by the Cartero Grunt Task, located in the projectDir
, to lookup the assets needed for the template being rendered. The cartero.json
file has the following format. All paths in the file are relative to projectDir
.
{
publicDir : "static",
parcels : {
"views/peopleList/peopleList.jade" : {
js : [
"static/library-assets/JQuery/jquery.js",
"static/library-assets/JQueryUI/jquery-ui.js",
"static/view-assets/peopleList/peopleList.js"
],
css : [
"static/library-assets/JQueryUI/jquery-ui.css",
"static/view-assets/peopleList/peopleList.css"
],
tmpl : [
"static/library-assets/Dialogs/EditPersonDialog/editPersonDialog.tmpl"
]
},
}
}
- The Hook then generates the raw HTML that will include the assets in the page being rendered and puts it into the
cartero_js
, cartero_css
, and cartero_tmpl
template variables. For the case of js
and css
files, it just needs to transform the paths in the cartero.json
file to be relative to the publicDir
, and then wrap them in <script>
or <link>
tags. For tmpl
assets, the Hook needs to read the files, concatenate their contents, and then put the whole shebang into cartero_tmpl
.
Q: Does Cartero address the issue of cache busting?
Yes. The name of the concatenated asset files generated in prod
mode includes an MD5 digest of their contents. When the contents of one of the files changes, its name will be updated, which will cause browsers to request a new copy of the content. The Rails Asset Pipeline implements the same cache busting technique.
Q: The "watch" task terminates on JS/CSS errors. Can I force it to keep running?
Yes. Use the Grunt --force
flag.
grunt cartero:dev --force --watch
Q: I'm getting the error: EMFILE, too many open files
EMFILE means you've reached the OS limit of concurrently open files. There isn't much we can do about it, however you can increase the limit yourself.
Add ulimit -n [number of files]
to your .bashrc/.zshrc file to increase the soft limit.
If you reach the OS hard limit, you can follow this StackOverflow answer to increase it.
Q: Since Cartero combines files in prod
mode, won't image urls used in my stylesheets break?
Yes and No. They would break, but Cartero automatically scans your .css
files for url()
statements, and fixes their arguments so that they don't break.
Cartero Hook Directory
If you develop a Hook for your web framework, please let us know and we'll add it to the directory.
About
By Oleg Seletsky and David Beck.
Copyright (c) 2013 Rotunda Software, LLC.
Licensed under the MIT License.