express-cdn-wip

express-cdn-wip

A modified version of @niftylettuce's express-cdn package. It includes custom tags, and some minor changes.

## Index

• Features
• Add-On Modules
• How Does It Work?
• Environment Differences
• CDN Setup Instructions
• Quick Start
• Custom Logging
• Lazyweb Requests
• Changelog
• Contributors
• License

Features

• Automatic parsing of background, background-image and content for url({{absoluteUrl}}) in stylesheets and scripts.
• Built-in optimization of images in production mode using binaries from NPM of OptiPNG and JPEGTran.
• Supports Sass, LESS, and Stylus using respective stylesheet compilers.
• JavaScript assets are mangled and minified using UglifyJS.
• Automatic detection of asset changes and will only upload changed assets to S3 in production mode.
• Utilizes cachebusting, which is inspired by express-cachebuster and h5bp.
• All assets are compressed using zlib into a gzip buffer for S3 uploading with Content-Encoding header set to gzip.
• Embed multiple assets as a single <script> or <link> tag using the built-in dynamic view helper.
• Loads and processes assets per view (allowing you to minimize client HTTP requests).
• Combine commonly used assets together using a simple array argument.
• Uploads changed assets automatically and asynchronously to Amazon S3 (only in production mode) using knox.

Add-on Modules

These modules are a not currently a work in progress, see #70.

• express-cdn-cloudfront - Amazon S3 and Amazon CloudFront
• express-cdn-maxcdn - MaxCDN and Amazon S3
• express-cdn-cloudfiles - Rackspace CloudFiles
• express-cdn-cloudflare - CloudFlare and Amazon S3

How does it work?

When the server is first started, the module returns a view helper depending on the server environment (production or development). It also recursively searches through your viewsDir for any views containing instances of the CDN(...) view helper. After parsing each instance and removing duplicates, it will use your S3 credentials to upload a new copy of the production-quality assets. Enjoy :).

Environment Differences

Development Mode:

Assets are untouched, cachebusted, and delivered as typical local files for rapid development.

Production Mode:

Assets are optimized, minified, mangled, gzipped, delivered by Amazon CloudFront CDN, and hosted from Amazon S3.

CDN Setup Instructions

1. Visit https://console.aws.amazon.com/s3/home and click Create Bucket.

• Bucket Name: bucket-name
• Region: US Standard (use options.endpoint with 'bucket.s3-xxx.amazonaws.com' for non US Standard regions)

1. Upload index.html to your new bucket (this will serve as a placeholder in case someone accesses http://cdn.your-site.com/).
2. Select index.html in the Objects and Folders view from your S3 console and click Actions → Make Public.
3. Visit https://console.aws.amazon.com/cloudfront/home and click Create Distribution.

• Choose an origin:
  • Origin Domain Name: bucket-name.s3.amazonaws.com
  • Origin ID: S3-bucket-name
• Create default behavior:
  • Path Pattern: Default (*)
  • Origin: S3-bucket-name
  • Viewer Protocol Policy: HTTP and HTTPS
  • Object Caching: Use Origin Cache Headers
  • Forward Query String: Yes (Improves Caching)
• Distribution details:
  • Alternate Domain Names (CNAMEs): cdn.your-domain.com
  • Default Root Object: index.html
  • Logging: Off
  • Comments: Created with express-cdn by @niftylettuce.
  • Distribution State: Enabled

1. Copy the generated Domain Name (e.g. xyz.cloudfront.net) to your clipboard.
2. Log in to your-domain.com's DNS manager, add a new CNAME "hostname" of cdn, and paste the contents of your clipboard as the the "alias" or "points to" value.
3. After the DNS change propagates, you can test your new CDN by visiting http://cdn.your-domain.com (the index.html file should get displayed).

SSL Configuration

Some additional steps are required to enable SSL access of your assets by cloudfront.

1. Visit https://console.aws.amazon.com/s3/home and open the bucket's properties.
2. On the permissions tab, click the add bucket policy button.

• You can use the Policy Generator to generate the appropiate policy with this settings:
  • Type: S3 Bucket Policy
  • Effect: Allow
  • Principal: AWS
  • AWS Service: Amazon S3
  • Actions: GetObject
  • ARN: arn:aws:s3:::<bucket_name>/* (fill in your bucket name)
• Click on generate policy and paste the output on the add bucket policy window.
• Save your changes.

1. When you configure express-cdn you must reference cloudfront subdomain directly, since CNAMEs are not supported over ssl.

Quick Start

npm install express-cdn

// # express-cdn

var express = require('express')
  , path    = require('path')
  , app     = express.createServer()
  , semver  = require('semver');

var sslEnabled = false

// Set the CDN options
var options = {
    publicDir  : path.join(__dirname, 'public')
  , viewsDir   : path.join(__dirname, 'views')
  , domain     : 'cdn.your-domain.com'
  , bucket     : 'bucket-name'
  , endpoint   : 'bucket-name.s3.amazonaws.com' // optional
  , key        : 'amazon-s3-key'
  , secret     : 'amazon-s3-secret'
  , hostname   : 'localhost'
  , port       : (sslEnabled ? 443 : 1337)
  , ssl        : sslEnabled
  , production : true
};

// Initialize the CDN magic
var CDN = require('express-cdn')(app, options);

app.configure(function() {
  app.set('view engine', 'jade');
  app.set('view options', { layout: false, pretty: true });
  app.enable('view cache');
  app.use(express.bodyParser());
  app.use(express.static(path.join(__dirname, 'public')));
});

// Add the view helper
if (semver.gte(express.version, '3.0.0'))
  app.locals({ CDN: CDN() });
else
  app.dynamicHelpers({ CDN: CDN });

app.get('/', function(req, res, next) {
  res.render('basic');
  return;
});

console.log("Server started: http://localhost:1337");
app.listen(1337);

Views

Jade

// #1 - Load an image
!= CDN('/img/sprite.png')

// #2 - Load an image with a custom tag attribute
!= CDN('/img/sprite.png', { alt: 'Sprite' })

// #3 - Load a script
!= CDN('/js/script.js')

// #4 - Load a script with a custom tag attribute
!= CDN('/js/script.js', { 'data-message': 'Hello' })

// #5 - Load and concat two scripts
!= CDN([ '/js/plugins.js', '/js/script.js' ])

// #6 - Load and concat two scripts with custom tag attributes
!= CDN([ '/js/plugins.js', '/js/script.js' ], { 'data-message': 'Hello' })

// #7 - Load a stylesheet
!= CDN('/css/style.css')

// #8 - Load and concat two stylesheets
!= CDN([ '/css/style.css', '/css/extra.css' ])

// #9 - Load a favicon
!= CDN('/img/favicon.ico')

EJS

<!-- #1 - Load an image -->
<%- CDN('/img/sprite.png') %>

<!-- #2 - Load an image with a custom tag attribute -->
<%- CDN('/img/sprite.png', { alt: 'Sprite' }) %>

<!-- #3 - Load a script -->
<%- CDN('/js/script.js') %>

<!-- #4 - Load a script with a custom tag attribute -->
<%- CDN('/js/script.js', { 'data-message': 'Hello' }) %>

<!-- #5 - Load and concat two scripts -->
<%- CDN([ '/js/plugins.js', '/js/script.js' ]) %>

<!-- #6 - Load and concat two scripts with custom tag attributes -->
<%- CDN([ '/js/plugins.js', '/js/script.js' ], { 'data-message': 'Hello' }) %>

<!-- #7 - Load a stylesheet -->
<%- CDN('/css/style.css') %>

<!-- #8 - Load and concat two stylesheets -->
<%- CDN([ '/css/style.css', '/css/extra.css' ]) %>

<!-- #9 - Load a favicon -->
<%- CDN('/img/favicon.ico') %>

Automatically Rendered HTML

Development Mode

<!-- #1 - Load an image -->
<img src="/img/sprite.png?v=1341214029" />

<!-- #2 - Load an image with a custom tag attribute -->
<img src="/img/sprite.png?v=1341214029" alt="Sprite" />

<!-- #3 - Load a script -->
<script src="/js/script.js?v=1341214029" type="text/javascript"></script>

<!-- #4 - Load a script with a custom tag attribute -->
<script src="/js/script.js?v=1341214029" type="text/javascript" data-message="Hello"></script>

<!-- #5 - Load and concat two scripts -->
<script src="/js/plugins.js?v=1341214029" type="text/javascript"></script>
<script src="/js/script.js?v=1341214029" type="text/javascript"></script>

<!-- #6 - Load and concat two scripts with custom tag attributes -->
<script src="/js/plugins.js?v=1341214029" type="text/javascript" data-message="Hello"></script>
<script src="/js/script.js?v=1341214029" type="text/javascript" data-message="Hello"></script>

<!-- #7 - Load a stylesheet -->
<link href="/css/style.css?v=1341214029" rel="stylesheet" type="text/css" />

<!-- #8 - Load and concat two stylesheets -->
<link href="/css/style.css?v=1341214029" rel="stylesheet" type="text/css" />
<link href="/css/extra.css?v=1341214029" rel="stylesheet" type="text/css" />

<!-- #9 - Load a favicon -->
<link href="/img/favicon.ico?v=1341214029" rel="shortcut icon" />

Production Mode

The protocol will automatically change to "https" or "http" depending on the SSL option.

The module will automatically upload and detect new/modified assets based off timestamp, as it utilizes the timestamp for version control! There is built-in magic to detect if individual assets were changed when concatenating multiple assets together (it adds the timestamps together and checks if the combined asset timestamp on S3 exists!).

<!-- #1 - Load an image -->
<img src="https://cdn.your-site.com/img/sprite.1341382571.png" />

<!-- #2 - Load an image with a custom tag attribute -->
<img src="https://cdn.your-site.com/img/sprite.1341382571.png" alt="Sprite" />

<!-- #3 - Load a script -->
<script src="https://cdn.your-site.com/js/script.1341382571.js" type="text/javascript"></script>

<!-- #4 - Load a script with a custom tag attribute -->
<script src="https://cdn.your-site.com/js/script.1341382571.js" type="text/javascript" data-message="Hello"></script>

<!-- #5 - Load and concat two scripts -->
<script src="https://cdn.your-site.com/plugins%2Bscript.1341382571.js" type="text/javascript"></script>

<!-- #6 - Load and concat two scripts with custom tag attributes -->
<script src="https://cdn.your-site.com/plugins%2Bscript.1341382571.js" type="text/javascript" data-message="Hello"></script>

<!-- #7 - Load a stylesheet -->
<link href="https://cdn.your-site.com/css/style.1341382571.css" rel="stylesheet" type="text/css" />

<!-- #8 - Load and concat two stylesheets -->
<link href="https://cdn.your-site.com/style%2Bextra.1341382571.css" rel="stylesheet" type="text/css" />

<!-- #9 - Load a favicon -->
<link href="https://cdn.your-site.com/img/favicon.1341382571.ico" rel="shortcut icon" />

Custom Logging

By default log messages will be sent to the console. If you would like to use a custom logger function you may pass it in as options.logger

The example below uses the Winston logging library.

var winston = require('winston');
winston.add(winston.transports.File, {filename: 'somefile.log'});

// Set the CDN options
var options = {
    publicDir  : path.join(__dirname, 'public')
  , viewsDir   : path.join(__dirname, 'views')
  , domain     : 'cdn.your-domain.com'
  , bucket     : 'bucket-name'
  , key        : 'amazon-s3-key'
  , secret     : 'amazon-s3-secret'
  , hostname   : 'localhost'
  , port       : 1337
  , ssl        : false
  , production : true
  , logger     : winston.info
};

// Initialize the CDN magic
var CDN = require('express-cdn')(app, options);

app.configure(function() {
  app.set('view engine', 'jade');
  app.set('view options', { layout: false, pretty: true });
  app.enable('view cache');
  app.use(express.bodyParser());
  app.use(express.static(path.join(__dirname, 'public')));
});

// Add the dynamic view helper
app.dynamicHelpers({ CDN: CDN });

Any output from express-cdn is now passed to winston.info() which writes to both console and somefile.log.

Lazyweb Requests

These are feature requests that we would appreciate contributors for:

• Git SHA cachebusting instead of timestamp
• Add support for multiple view directories
• Add cache busting for CSS scraper
• Add font CSS scraper for uploading fonts with proper mimetypes and cachebusting
• Add options to pick CDN network (e.g. MaxCDN vs. Amazon vs. Rackspace)
• Add tests for all asset types.
• Modularization of /lib/main.js please!
• Support Express 3.x.x+ and utilize async with view helper.
• Convert from fs.statSync to fs.stat with callback for image assets modified timestamp hack.
• Investigate why Chrome Tools Audit returns leverage proxy cookieless jargon.

Changelog

• 0.2.3 - Added support for SVG files (by @zhangchiqing)

• 0.2.2 - Fixed uglifyjs license comment regex (by @kudos), fixed wrong mimetypes for minify case in compile method (by @kaskovsky)

• 0.2.1 - Fixed cursor css property also can be url (by @sars)

• 0.2.0 - Support for CSS @media query attribute with parenthesis (by @jfred)

• 0.1.9 - Added cleanCSS support to minify CSS (by @DServy)

• 0.1.8 - Added favicon support (by @mateusz-)

• 0.1.7 - Fixed issue with knox (by @DServy)

• 0.1.6 - Fixed extracting CSS border-image resources and image snot followed by ; in CSS (by @lxe)

• 0.1.5 - Preserved license comments with UglifyJS version 2.0 (by @mateusz-)

• 0.1.4 - Added case insensitive usage of cdn or CDN (by @leostera)

• 0.1.3 - Explicity set x-amz-acl to public-read.

• 0.1.2 - Added protocol relative paths for HTTP/HTTPS via // (by @Nevtep)

• 0.1.1 - Add ability to specify template extension

• 0.1.0 - Fixed endpoint issue, fixed knox issue, added optipng binary, added jpegtran binary, no longer requires optipng or jpegtran server dependencies!

• 0.0.9 - Allowed explicit setting of S3 endpoint (by @eladb)

• 0.0.8 - Enabled string-only output for CDN assets.

  - var href = CDN('/img/full/foo.jpg', { raw : true });
  a(class="fancybox", href="#{href}")
    != CDN('/img/small/foo.jpg', { alt : 'Foo', width : 800, height : 600 })
  

• 0.0.7 - Removed CSS minification due to over-optimization of the clean-css module.

• 0.0.6 - Added temporary support for CSS usage of background-image, background, and contents attributes by absolute image paths.

  /* Valid - Proper way to write CSS with express-cdn */
  #example-valid {
    background: url(/something.png);
  }
  
  /* Invalid - Don't do this! */
  #example-invalid {
    background: url(../something.png);
  }
  

Contributors

• Nick Baugh [e-mail]
• Daniel Santiago
• James Wyse [e-mail]
• Jon Keating [e-mail]
• Andrew de Andrade [e-mail]
• Joshua Gross [e-mail]
• Dominik Lessel [e-mail]
• Elad Ben-Israel [e-mail]
• Aleksey Smolenchuk [e-mail]
• David Staley [e-mail]
• Joshua Frederick [e-mail]
• Rodion [e-mail]
• kaskovsky
• Jonathan Cremin
• Leo Zhang [e-mail]
• Herman-Scheer [e-mail]

License

The MIT License

Copyright (c) 2012- Nick Baugh [e-mail] (http://niftylettuce.com/)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.