rake_dependencies

RakeDependencies

Rake tasks for managing binary dependencies used within a build.

Installation

Add this line to your application's Gemfile:

gem 'rake_dependencies'

And then execute:

$ bundle

Or install it yourself as:

$ gem install rake_dependencies

Usage

RakeDependencies provides a suite of tasklibs for downloading and extracting a distribution of some dependency. The simplest way to configure all of these tasks is via the RakeDependencies::Tasks::All tasklib. The following provides an example usage with terraform as the target dependency:

RakeDependencies::Tasks::All.new do |t|
  t.namespace = :terraform
  t.dependency = 'terraform'
  t.version = '0.9.0'
  t.path = File.join('vendor', 'terraform')
  t.type = :zip

  t.platform_os_names = { darwin: 'darwin', linux: 'linux' }
  t.platform_cpu_names = { x86_64: 'amd64', arm64: 'arm64' }

  t.uri_template =
    'https://releases.hashicorp.com/terraform/<%= @version %>/' +
      'terraform_<%= @version %>_' +
      '<%= @platform_os_name %>_<%= @platform_cpu_name %><%= @ext %>'
  t.file_name_template =
    'terraform_<%= @version %>_' +
      '<%= @platform_os_name %>_<%= @platform_cpu_name %><%= @ext %>'

  t.needs_fetch = lambda do |parameters|
    terraform_binary = File.join(
      parameters[:path], parameters[:binary_directory], 'terraform')

    !(File.exist?(terraform_binary) &&
      `#{terraform_binary} -version`.lines.first =~ /#{parameters[:version]}/)
  end
end

With this in place, a number of tasks will be defined:

> rake -T
rake terraform:clean     # Clean vendored terraform
rake terraform:download  # Download terraform distribution
rake terraform:ensure    # Ensure terraform present
rake terraform:extract   # Extract terraform archive
rake terraform:fetch     # Fetch terraform

The tasks perform the following:

• <ns>:clean - recursively deletes the directory containing the dependency
• <ns>:download - downloads the distribution from the provided path into the dependency directory
• <ns>:extract - extracts, in the case of a compressed archive, or copies, in the case of an uncompressed distribution, the binaries into the binary directory under the dependency directory
• <ns>:fetch - downloads then extracts
• <ns>:ensure - checks whether the dependency needs to be fetched and cleans and fetches if necessary

With these tasks defined, any task that requires the dependency to be present should depend on <ns>:ensure. Continuing the terraform example:

task :provision_database => ['terraform:ensure'] do
  sh('vendor/terraform/bin/terraform apply infra/database')
end

If the installation_directory attribute is supplied, an additional install task will be defined:

 RakeDependencies::Tasks::All.new do |t|
  t.namespace = :terraform
  t.dependency = 'terraform'

  # ...
  t.installation_directory = "#{ENV['HOME']}/bin"
  # ...
end

Then:

> rake -T
rake terraform:clean     # Clean vendored terraform
rake terraform:download  # Download terraform distribution
rake terraform:ensure    # Ensure terraform present
rake terraform:extract   # Extract terraform archive
rake terraform:fetch     # Fetch terraform
rake terraform:install   # Install terraform

The <ns>:install task copies the binary into the defined installation directory which can be anywhere on the filesystem.

The RakeDependencies::Tasks::All tasklib supports the following configuration parameters:

Name	Description	Default	Required
namespace	The namespace in which to define the tasks	-	no
dependency	The name of the dependency, used in status reporting and as the default binary name	-	yes
version	The version of the dependency to manage, only required if used in templates or needs_fetch	-	no
path	The path in which to install the dependency	-	yes
type	The archive type of the distribution, one of :zip, :tar_gz, :tgz or :uncompressed	:zip	yes
platform_os_names	A map of platform OS identifiers to OS names to use in templates	RakeDependencies::PlatformNames::OS	yes
platform_cpu_names	A map of platform CPU identifiers to CPU names to use in templates	RakeDependencies::PlatformNames::CPU	yes
distribution_directory	The name of the directory under the supplied path into which to download the distribution	'dist'	yes
binary_directory	The name of the directory under the supplied path into which to extract/copy the binaries	'bin'	yes
installation_directory	The name of the directory into which to install the binaries, anywhere on the file system	-	no
uri_template	A template for the URI of the distribution	-	yes
file_name_template	A template for the name of the downloaded file	-	yes
source_binary_name_template	A template for the name of the binary before rename after extraction/copying	-	no
target_binary_name_template	A template for the name to rename the binary to after extraction/copying	-	no
strip_path_template	A template for the path to strip within an archive before extracting	-	no
needs_fetch	A lambda taking a parameter map that should return true if the dependency needs to be fetched, false otherwise	Will always return true	no
clean_task_name	The name of the clean task, required if it should be different from the default	:clean	yes
download_task_name	The name of the download task, required if it should be different from the default	:download	yes
extract_task_name	The name of the extract task, required if it should be different from the default	:extract	yes
install_task_name	The name of the install task, required if it should be different from the default	:install	no
fetch_task_name	The name of the fetch task, required if it should be different from the default	:fetch	yes
ensure_task_name	The name of the ensure task, required if it should be different from the default	:ensure	yes

Notes:

• Each of the templates will have the following instance variables in scope when rendered:
  • @version: the supplied version string
  • @platform: the Gem::Platform on which the task is executing
  • @platform_os_name: the OS name derived from the platform on which the task is executing and the provided platform_os_names map
  • @platform_cpu_name: the CPU name derived from the platform on which the task is executing and the provided platform_cpu_names map
  • @ext: the file extension corresponding to the provided type, one of .zip, .tar.gz, .tgz or empty string for uncompressed files
• The needs_fetch lambda will receive a map with the following entries:
  • path: the supplied path
  • version: the supplied version string
  • binary_directory: the supplied or default binary directory

The RakeDependencies::Tasks::All tasklib uses each of the following tasklibs in its definition:

• RakeDependencies::Tasks::Clean
• RakeDependencies::Tasks::Download
• RakeDependencies::Tasks::Extract
• RakeDependencies::Tasks::Install
• RakeDependencies::Tasks::Fetch
• RakeDependencies::Tasks::Ensure

RakeDependencies::Tasks::Clean

The RakeDependencies::Tasks::Clean tasklib supports the following configuration parameters:

Name	Description	Default	Required
name	The name of the task, required if it should be different from the default	:clean	yes
path	The path in which the dependency is installed	-	yes
dependency	The name of the dependency, used in status reporting	-	yes

RakeDependencies::Tasks::Download

The RakeDependencies::Tasks::Download tasklib supports the following configuration parameters:

Name	Description	Default	Required
name	The name of the task, required if it should be different from the default	:download	yes
dependency	The name of the dependency, used in status reporting	-	yes
version	The version of the dependency to manage, only required if used in templates	-	no
path	The path in which to install the dependency	-	yes
type	The archive type of the distribution, one of :zip, :tar_gz, :tgz or :uncompressed	:zip	yes
platform_os_names	A map of platform OS identifiers to OS names to use in templates	RakeDependencies::PlatformNames::OS	yes
platform_cpu_names	A map of platform CPU identifiers to CPU names to use in templates	RakeDependencies::PlatformNames::CPU	yes
distribution_directory	The name of the directory under the supplied path into which to download the distribution	'dist'	yes
uri_template	A template for the URI of the distribution	-	yes
file_name_template	A template for the name of the downloaded file	-	yes

Notes:

• The templates have the same instance variables in scope when rendered as mentioned above.

RakeDependencies::Tasks::Extract

The RakeDependencies::Tasks::Extract tasklib supports the following configuration parameters:

Name	Description	Default	Required
name	The name of the task, required if it should be different from the default	:extract	yes
dependency	The name of the dependency, used in status reporting	-	yes
version	The version of the dependency to manage, only required if used in templates	-	no
path	The path in which to install the dependency	-	yes
type	The archive type of the distribution, one of :zip, :tar_gz, :tgz or :uncompressed	:zip	yes
platform_os_names	A map of platform OS identifiers to OS names to use in templates	RakeDependencies::PlatformNames::OS	yes
platform_cpu_names	A map of platform CPU identifiers to CPU names to use in templates	RakeDependencies::PlatformNames::CPU	yes
extractors	A map of archive types to extractor classes, see notes for further details	Extractors for all supported types	yes
distribution_directory	The name of the directory under the supplied path into which the distribution was downloaded	'dist'	yes
binary_directory	The name of the directory under the supplied path into which to extract/copy the binaries	'bin'	yes
file_name_template	A template for the name of the downloaded file	-	yes
source_binary_name_template	A template for the name of the binary before rename after extraction/copying	-	no
target_binary_name_template	A template for the name to rename the binary to after extraction/copying	-	no
strip_path_template	A template for the path to strip within an archive before extracting	-	no

Notes:

• The templates have the same instance variables in scope when rendered as mentioned above.
• The extractors map has entries for the following keys:
  • :zip: An extractor class for zip files
  • :tar_gz: An extractor class for tar.gz files
  • :tgz: An alias for :tar_gz using the same extractor class
  • :uncompressed: An extractor class that copies the source to the destination
• The extractor map can be overridden but should include entries for all of the above.

RakeDependencies::Tasks::Install

The RakeDependencies::Tasks::Install tasklib supports the following configuration parameters:

Name	Description	Default	Required
name	The name of the task, required if it should be different from the default	:install	yes
dependency	The name of the dependency, used in status reporting	-	yes
version	The version of the dependency to manage, only required if used in templates	-	no
path	The path in which to install the dependency	-	yes
type	The archive type of the original distribution, one of :zip, :tar_gz, :tgz or :uncompressed	:zip	yes
platform_os_names	A map of platform OS identifiers to OS names to use in templates	RakeDependencies::PlatformNames::OS	yes
platform_cpu_names	A map of platform CPU identifiers to CPU names to use in templates	RakeDependencies::PlatformNames::CPU	yes
binary_directory	The name of the directory under the supplied path into which to extract/copy the binaries	'bin'	yes
installation_directory	The name of the directory into which the binary should be installed	-	yes
binary_name_template	A template for the name of the binary	-	yes

Notes:

• The templates have the same instance variables in scope when rendered as mentioned above.

RakeDependencies::Tasks::Fetch

The RakeDependencies::Tasks::Fetch tasklib supports the following configuration parameters:

Name	Description	Default	Required
name	The name of the task, required if it should be different from the default	:fetch	yes
dependency	The name of the dependency, used in status reporting	-	yes
download_task	The full name including namespaces of the download task	<current-namespace>:download	yes
extract_task	The full name including namespaces of the extract task	<current-namespace>:extract	yes

RakeDependencies::Tasks::Ensure

The RakeDependencies::Tasks::Fetch tasklib supports the following configuration parameters:

Name	Description	Default	Required
name	The name of the task, required if it should be different from the default	:fetch	yes
dependency	The name of the dependency, used in status reporting	-	yes
version	The version of the dependency to manage, only required if used in templates	-	no
path	The path in which to install the dependency	-	yes
binary_directory	The name of the directory under the supplied path into which to extract/copy the binaries	'bin'	yes
needs_fetch	A lambda taking a parameter map that should return true if the dependency needs to be fetched, false otherwise	Will always return true	no
clean_task	The full name including namespaces of the clean task	<current-namespace>:clean	yes
download_task	The full name including namespaces of the download task	<current-namespace>:download	yes
extract_task	The full name including namespaces of the extract task	<current-namespace>:extract	yes
install_task	The full name including namespaces of the install task	<current-namespace>:install	yes

Notes:

• The templates have the same instance variables in scope when rendered as mentioned above.
• The needs_fetch method receives the same parameter map as mentioned above.

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Managing CircleCI keys

To encrypt a GPG key for use by CircleCI:

openssl aes-256-cbc \
  -e \
  -md sha1 \
  -in ./config/secrets/ci/gpg.private \
  -out ./.circleci/gpg.private.enc \
  -k "<passphrase>"

To check decryption is working correctly:

openssl aes-256-cbc \
  -d \
  -md sha1 \
  -in ./.circleci/gpg.private.enc \
  -k "<passphrase>"

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/infrablocks/rake_dependencies. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.

License

The gem is available as open source under the terms of the MIT License.