bincache

= bincache

BinCache is a system designed to chache compiled binaries on S3. It has resources for bash, ruby and chef.

BinCache tracks a single directory and a series of scripts. After each script is run, the contents of the directory is cached on s3. On sequential runs, the scripts are not re-executed, but instead the cache from S3 is downloaded and placed in the directory. If a script is changed, itself and all following scripts will be re-executed. BinCache does this by hashing the current script together with all previous scripts ran and storing the resulting cache in S3 with that hash as it's name. By ensuring that scripts contain markers to introduce uniquness on different systems, you can make sure that you never have to wait twice for something to compile!

== Install

add rubygems.org to your gem sources list if it is not already there

gem sources -a https://rubygems.org

install bincache

gem install bincache

== Environment

Make sure you set the environment variables with something like:

S3 access key id

export BINCACHE_S3_ACCESS_KEY=1234567890ABCDEFGHIJ

S3 secret access key

export BINCACHE_S3_SECRET_KEY=1234567890ABCDEFHIJKLMNOPQRSTUVWXYZ12345

S3 bucket

export BINCACHE_S3_BUCKET=my_ec2_bucket

prefix inside bucket to store caches

export BINCACHE_S3_PREFIX=my_bincache/

== Bash example

you may have to put rubygem bins in your path by doing something like..

export PATH=$HOME/.gem/ruby/1.8/bin:$PATH

set the directory to cache

dir=/tmp/bincache

create a some sample scripts

cat <<-EOF > script1 echo "I am in script one" > one EOF

cat <<-EOF > script2 echo "I am in script two" > two EOF

run bincache

bincache $dir $script1 $script2

== Ruby example

make sure bincache is loaded

require 'bincache'

set the directory to cache

dir = "/tmp/bincache"

create two sample scripts

script1 = <<EOS echo "I am in script one" > one EOS

script2 = <<EOS echo "I am in script two" > two EOS

create an empty list of scripts and add the sample script to it

scripts = [] scripts << script1 scripts << script2

run bincache

bincache = BinCache.new bincache.run_series(dir,scripts)

== Chef example

require 'bincache'

bincache "script1" do action :run directory "/tmp/bincache" script <<-EOS echo "I am in script one" > one EOS end

bincache "script2" do action :run directory "/tmp/bincache" script <<-EOS echo "I am in script two" > two EOS end

== uniquness example

To tag a script with something unique to ensure that it is distingushible on multiple systems, you need to insert something unique into the comments. For example, you could do:

echo "I am showing you how to uniqueify your script" > example

#{uname -m}

#{lsb_release -ds

When this script is hashed, it will produce different values on different systems. With this technique, you can cache different binaries from different systems with the same code base!

== Copyright

Copyright (c) 2010 Martin Rhoads. See LICENSE for details.