Parallax
Parallax gem is a quick and simple framework to Ruby IPC and multi-core parallel execution.
Installation
Add this line to your application's Gemfile:
gem 'parallax'
And then execute:
$ bundle
Or install it yourself as:
$ gem install parallax
Usage
You can use this gem in many ways.
To execute parallel code with an array of elements, divide the elements in N groups and execute each chunk in one forked process.
array = [ 'running', 'code', 'in', 'parallel' ]
Parallax.execute array do |worker, array_chunk|
array_chunk.each do |element|
worker.log "[#{worker.index}] #{element}"
end
end
If you need inter-process communication, this can be done by calling worker.send
and passing a list of arguments. The args are serialized and passed via IO pipe to a collector
object, which is by default an instance of Parallax::Collector
class. The collector then parses the args and treats them like a method call where the first arg is the name of the method.
There are a number of predefined methods, built on top of worker.send
that you can call to do a number of tasks:
log
: Used in the example above, the collectors calls the log
method which prints the message to the stdout.
store
: Saves the argument object into a variable called workers_data
in the collector.
The collector object is returned by the Parallax.execute
method, so if you need to access the stored data called with the worker.store
method you can do:
numbers = (0..100).to_a
collector = Parallax.execute numbers, do |worker, numbers_chunk|
numbers_chunk.each do |number|
worker.store number * 2
end
end
puts collector.workers_data.inspect
Other options you can pass to execute are:
processes
: the number of processes in which parallelize the execution. Defaults to Etc.nprocessors
(which is equal to the number of cores of the current running machine).
collector
: a custom collector object that you can implement yourself.
To use a custom collector, you need to include Parallax::Collectable
in your custom collector. Example of a custom collector:
class CustomCollector
include Parallax::Collectable
attr_accessor :name
def initialize(name)
@name = name
end
def store(worker_index, object)
workers_data.push "#{self.name}: worker #{worker_index} stored: #{object}"
end
end
workers_count = 4
numbers = (0..100).to_a
custom_collector = CustomCollector.new('custom_collector')
Parallax.execute numbers, collector: custom_collector, do |worker, numbers_chunk|
numbers_chunk.each do |number|
worker.store number * 2
end
end
puts custom_collector.workers_data.inspect
Development
After checking out the repo, run bin/setup
to install dependencies. Then, run rake spec
to run the tests. You can also run bin/console
for an interactive prompt that will allow you to experiment.
To install this gem onto your local machine, run bundle exec rake install
. To release a new version, update the version number in version.rb
, and then run bundle exec rake release
, which will create a git tag for the version, push git commits and tags, and push the .gem
file to rubygems.org.
Contributing
Bug reports and pull requests are welcome on GitHub at https://github.com/Pluvie/parallax.