forkner

Forkner

Forkner helps you manage multiple child processes and getting data back from them when they finish.

Usage

Consider a situation in which you need to run a bunch of parallel processes, but no more than five at a time. You might do that like this:

require 'forkner'

Forkner.container(5) do |forkner|
    100.times do
        forkner.child do
            # do stuff in the child process
        end
    end
end

# we won't get to this point until all child processes are done

First we load the forkner gem. Then we call Forkner's container method. All child processes within container will completed before the method is done. container yields a Forkner object. Inside the container block we run whatever commands are necessary to set up for running the child processes. In this simple example we simply loop 100 times. Inside the loop, we use the forkner object to fork off into a child process. Everything in child is run within the child process. At the end of that block the child processes exits.

When the loop gets around and calls child again, another child process is only run when there are fewer than five (the number we passed into container) child processes. child pauses until there is a slot available. At the bottom of the container block, Forkner waits until any remaining child processes have finished.

Getting information from the child process

Forkner allows you to communicate information form the child process back to the parent process. Doing so requires two steps. First, set up a block that processes information returned from child processes. Second, in the child processes, finish the block with a value that can be stored as JSON.

Consider this example:

Forkner.container(5) do |forkner|
    forkner.reaper() do |rv|
        puts '-----'
        puts rv['myrand']
        puts rv['timestamp']
    end
    
    10.times do
        forkner.child do
            myrand = rand()
            timestamp = Time.now
            {'myrand'=>myrand, 'timestamp'=>timestamp}
        end
    end
end

In this example, inside the container block we call the Forkner object's reaper method with a block. In that block we get a single parameter which contains information from the child process. In this case we know it's a hash and we output two of its values.

Inside the child block, the child generates a random value and a timestamp. The last line in the block is a hash of those values.

Behind the scenes, that hash is converted to JSON and stored in a temporary file. When the reaper method is called, the hash is reconstituted from the JSON file and returned to the reaper block. That's why it's important that the child block end with a value that can be stored as JSON.

Using Forkner without the container block

If you prefer to get a little closer to the metal, you can directly create a Forkner object and call its child method. Just be sure to call the wait_all method after all the child processes have been called. The following code does exactly the same thing as the previous example.

forkner = Forkner.new(5)

forkner.reaper() do |rv|
    puts '-----'
    puts rv['myrand']
    puts rv['timestamp']
end

10.times do
    forkner.child do
        myrand = rand()
        timestamp = Time.now
        {'myrand'=>myrand, 'timestamp'=>timestamp}
    end
end

forkner.wait_all

Install

gem install forkner

Author

Mike O'Sullivan [e-mail]

History

version	date	notes
1.0	Jan 7, 2020	Initial upload.
1.1	Jan 7, 2020	Fixed some typos. No changes to functionality.
1.2	Jan 19, 2020	Fixed other documentation problems. No changes to functionality.