remont

Remont

A DSL for describing and running the row-level processing of the records in the database (eg. anonymization)

Installation

Add this line to your application's Gemfile:

gem 'remont'

And then execute:

$ bundle

Or install it yourself as:

$ gem install remont

Application configuration

Add config/initializers/remont.rb

Remont.setup do |config|
  config.process_timestamp_attribute = :processed_at
end

Option	Default value	Description
process_timestamp_attribute	nil	processing status attribute identifier

On successful record processing, process_timestamp_attribute on the record will be set to the current processing time (Time.now.getlocal).

Usage

In the following example, the intention is to simulate anonymization of the users and orders table.

• configure the global options

# config/initializers/remont.rb
Remont.setup do |config|
  config.process_timestamp_attribute = :anonymized_at
end

• define the processing script

# db/anonymize.rb
schema model: User  do
  attribute(:email) { '[e-mail]' }
end

schema model: Order do
  attribute(:billing_address) { '23 Wall Street, NY' }
end

• and run remont rake task with the path to the processing script

bundle exec rake "remont[db/anonymize.rb]"

Passing the script path to the rake task is mandatory. Running the rake task would result in updating email (and anonymized_at) column for each row in the users table, and billing_address column for each row in the orders table.

Schema

Defines database table hosting the rows to be processed. The initial dataset is defined through the model option in the schema method. model value must be a subclass of the ActiveRecord::Base.

schema model: Order do
end

Controlling the scope

Scope of the data that will be processed is controlled

• with scope option in the schema method
• by declaring custom scope within the schema block

# skip admin records
schema model: User, scope: { |scope| scope.where.not(role: :admin) } do
end

schema model: Order do
  # process only active records
  scope { |scope| scope.where(status: :active) }
end

Recording the processing end time

Library supports an option to store the processing end time of a record. Enable the behavior by configuring a processing status attribute identifier for the schema. Configure the attribute identifier

• globally (Remont::Config#process_timestamp_attribute, for all schemas)
• or individually per schema (in the schema DSL, overrides the global setting)

schema model: User, process_timestamp_attribute: :anonymized_at do
end

schema model: Order do
  with_process_timestamp_attribute :anonymized_at
end

Set process status attribute to nil to disable the behavior.

Skipping the processed records

In some cases, it's desirable to skip already processed records. You can enable this behavior by declaring without_processed within the schema block. When declared, the processing dataset query will be extended with a condition that excludes already processed records.

schema model: User do
  without_processed
  # ...
end

Configure processing status attribute before declaring the without_processed. An error will be raised otherwise.

Callbacks

Custom pre-processing or post-processing behavior can be declared using before and after callbacks.

schema model: User do
  before { |record| Rails.logger.info("Started processing: #{record.id}") }
  after { |record| Rails.logger.info("Finished processing: #{record.id}") }
end

Attributes

Attributes are processed using processors (an object which responds to the call method). The processor can be defined either as a block or with the :using option passed to the attribute method.

require 'securerandom'

class CachedNick
  def initialize
    @cache = Hash.new { |hash, nick| hash[nick] = SecureRandom.uuid }
  end

  def call(nick, _record)
    @cache[nick]
  end
end

schema model: User do
  attribute(:email) { |email, record| "#{record.id}-#{email}" }
  attribute(:nickname, using: CachedNick.new)
end

Development

After checking out the repo, run bin/setup to install dependencies. Then, run bundle exec rspec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/infinum/remont. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.

License

The gem is available as open source under the terms of the MIT License.

Code of Conduct

Everyone interacting in the Remont project’s codebases, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct.