copy_bot

CopyBot

A gem that provides a rake task for importing a database dump into the local database, as well as for running any additional scripts afterwards.

This can be useful in situations when you want to sync your staging and production data so that you can debug issues in production without having to access the production environment itself. Also, it is useful for populating a database with production data so development and testing can be performed in circumstances that are as close to production as possible. It is advised that the production data is anonymized beforehand.

The gem assumes that the database dump that is to be restored to the local database has already been prepared and made available either on an AWS S3 bucket or on a local path.

Requirements

• aws-sdk-ruby
• rails

Supported databases

• PostgreSQL

Installation

Add this line to your application's Gemfile:

gem 'copy_bot'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install copy_bot

Application configuration

Add config/initializers/copy_bot.rb

CopyBot.setup do |config|
  config.logger = Rails.logger
  config.permitted_environments = ['development', 'staging']
end

Option	Default value	Description
logger	Ruby's built-in Logger class	Logging utility that by default outputs to $stdout
permitted_environments	['development']	Array with environments where the rake task is allowed to be run (since it contains destructive actions)

Step definitions file

It is necessary to define steps that should be run in a .yml file:

steps:
  download_remote_db_dump:
    s3_credentials:
      access_key_id: <%= Rails.application.secrets.aws_access_key_id %>
      access_key: <%= Rails.application.secrets.aws_secret_access_key %>
      region: <%= Rails.application.secrets.aws_region %>
      bucket: <%= Rails.application.secrets.aws_s3_bucket %>
    source_file_path: '/staging_db_dump.sql'
    destination_file_path: './tmp/downloaded_db_dump.sql'
  create_local_db_backup:
    destination_file_path: './tmp/development_backup.sql'
  drop_local_db_tables:
  import_remote_db_to_local_db:
    remote_db_dump_file_path: './tmp/downloaded_db_dump.sql'
  run_migrations_on_local_db:
  delete_remote_db_dump:
    remote_db_dump_file_path: './tmp/downloaded_db_dump.sql'
  execute_custom_command:
    command: 'RAILS_ENV=staging bundle exec rake some_task'

The example above contains all possible steps in the exact order in which they are run. Not all steps are mandatory. If a certain step doesn't need to be run, just omit it from the steps definition file. For example, you may not want to download the database dump from an S3 bucket, but you have acquired it in some other way and you have it available locally. In that case, omit the download_remote_db_dump step and add the relevant remote_db_dump_file_path to the import_remote_db_to_local_db step.

IMPORTANT: These steps include some destructive actions, e.g. drop_local_db_tables will drop all tables in the local database so make sure you don't use this gem in an environment where you do not want to lose data.

Usage

To use the gem, simply run:

$ bundle exec rake copy_bot STEP_DEFINITIONS_FILE_PATH=config/copy_bot/step_definitions.yml

Please note that the step definitions file is passed to the rake task as an environment variable, not an argument.

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and the created tag, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/infinum/copy_bot. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the code of conduct.

License

The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the CopyBot project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.