Job Queues

Resque is a Redis-backed Ruby library for creating background jobs, placing those jobs on multiple queues, and processing them later. Background jobs can be any Ruby class or module that responds to perform. Your existing classes can easily be converted to background jobs or you can create new classes specifically to do work. Or, you can do both. Resque is heavily inspired by DelayedJob (which rocks) and is comprised of three parts: * A Ruby library for creating, querying, and processing jobs * A Rake task for starting a worker which processes jobs * A Sinatra app for monitoring queues, jobs, and workers.

Light weight job scheduling on top of Resque. Adds methods enqueue_at/enqueue_in to schedule jobs in the future. Also supports queueing jobs on a fixed, cron-like schedule.

Light weight job scheduling extension for Sidekiq that adds support for queueing jobs in a recurring way.

A job queue that uses PostgreSQL's advisory locks for speed and reliability.

resque-status is an extension to the resque queue system that provides simple trackable jobs. It provides a Resque::Plugins::Status::Hash class which can set/get the statuses of jobs and a Resque::Plugins::Status class that when included provides easily trackable/killable jobs.

queue_classic is a queueing library for Ruby apps. (Rails, Sinatra, Etc...) queue_classic features asynchronous job polling, database maintained locks and no ridiculous dependencies. As a matter of fact, queue_classic only requires pg.

Offload CarrierWave's image processing and storage to a background process using Delayed Job, Resque, Sidekiq, Qu, Queue Classic or Girl Friday

A Rails-based frontend to the Resque job queue system.

Makes sure that for special jobs, there can be only one job with the same workload in one queue. Example: class CacheSweeper include Resque::Plugins::UniqueJob @queue = :cache_sweeps def self.perform(article_id) # Cache Me If You Can... end end

`qless` is meant to be a performant alternative to other queueing systems, with statistics collection, a browser interface, and strong guarantees about job losses. It's written as a collection of Lua scipts that are loaded into the Redis instance to be used, and then executed by the client library. As such, it's intended to be extremely easy to port to other languages, without sacrificing performance and not requiring a lot of logic replication between clients. Keep the Lua scripts updated, and your language-specific extension will also remain up to date.

Test::Unit support for resque job queueing

Adds Redis::Queue class which can be used as Distributed-Queue based on Redis. Redis is often used as a messaging server to implement processing of background jobs or other kinds of messaging tasks. It implements Reliable-queue pattern decribed here: http://redis.io/commands/rpoplpush

A job queueing and background workers system using Beanstalkd. Inspired by the Minion gem.

A Rails engine that provides a simple web interface for exposing the Delayed::Job queue.

A simple Resque plugin that times and saves some simple metrics for Resque jobs back into redis. Based on this system you could build some simple auto-scaling mechanism based on the speed and ETA of queues. Also includes a hook/callback mechanism for recording/sending the metrics to your favorite tool (AKA statsd/graphite).

A Resque plugin to add functionality to pause resque jobs through the web interface. Using a `pause` allows you to stop the job for a slice of time. The job finish the process it are doing and don't get a new task to do, until the queue is released. You can use this functionality to do some maintenance whithout kill workers, for example.

Super simple job queue over AMQP

Ensures that for a given queue, only one worker is working on a job at any given time. Example: require 'resque/plugins/lonely_job' class StrictlySerialJob extend Resque::Plugins::LonelyJob @queue = :serial_work def self.perform # only one at a time in this block, no parallelism allowed for this # particular queue end end

resque-restriction is an extension to resque queue system that restricts the execution number of certain jobs in a period time, the exceeded jobs will be executed at the next period.

Simple job queue system based on mongodb

A Resque plugin. Two or more jobs with the same lock cannot be processed simultaneously by multiple workers. When this situation occurs the second job gets pushed back to the queue.

Declare job classes that can be run by a variety of queueing backends.

This gem makes Rails console sessions less dangerous in specified environments by warning, color-coding, auto-sandboxing, and allowing read-only external connections (disables job queueing, non-GET requests, etc.)

Simple Scheduler adds the ability to enhance Heroku Scheduler by using Sidekiq to queue jobs in the future. This allows for defining specific run times (Ex: Every Sunday at 4 AM) and running tasks more often than Heroku Scheduler's 10 minute limit.

resque-throttle is an extension to the resque queue system that restricts the frequency in which certain jobs are run. Add more description here.

Normally resque processes queues in a fixed order. This can lead to jobs in queues at the end of the list not getting process for very long periods. resque-fairly provides a mechanism where by workers are distributed across the set of queues with pending jobs fairly. This results in a much more predictable mean time to handling for jobs in queues that are not the first in the list.

Compatible with Resque 1.x. Use Resque.push if you are using >= 2.x. Resque is great. So is job processing with redis. Our biggest drawback has been that resque requires the class that will be processing a job to be loaded when the job is enqueued. But what happens when the implementing job is defined in a separate application and isn't currently loaded into memory? Enter Resque Remote. Resque Remote's simple goal is to allow you to add a job to a queue with a string identifier for the class rather than the class constant. It is assumed that the worker-side of the equation _will_ have the class in memory and hence will be able to run it no problem. Feedback, comments and questions are welcome at bj [dot] neilsen [at] gmail [dot] com.

HireFire automatically "hires" and "fires" (aka "scales") Delayed Job and Resque workers on Heroku. When there are no queue jobs, HireFire will fire (shut down) all workers. If there are queued jobs, then it'll hire (spin up) workers. The amount of workers that get hired depends on the amount of queued jobs (the ratio can be configured by you). HireFire is great for both high, mid and low traffic applications. It can save you a lot of money by only hiring workers when there are pending jobs, and then firing them again once all the jobs have been processed. It's also capable to dramatically reducing processing time by automatically hiring more workers when the queue size increases.

job_boss allows you to queue jobs which are unqueued by a "Job Boss" daemon and handed off to workers to process

Resque-mongo is a fork of resque that uses mongo as a queue backend. Resque is a Redis-backed Ruby library for creating background jobs, placing those jobs on multiple queues, and processing them later. Background jobs can be any Ruby class or module that responds to perform. Your existing classes can easily be converted to background jobs or you can create new classes specifically to do work. Or, you can do both. Resque is heavily inspired by DelayedJob (which rocks) and is comprised of three parts: * A Ruby library for creating, querying, and processing jobs * A Rake task for starting a worker which processes jobs * A Sinatra app for monitoring queues, jobs, and workers.

Manage your organization's workflows entirely through Google Docs and irb. Mobilize schedules jobs, queues workers, sends failure notifications, and integrates mobilize-hadoop, -http, -mysql, and -mongodb packages to allow seamless transport of TSV and JSON data between any two endpoints.

Rate limit how many times a job can be run from a queue

Light weight job scheduling on top of Resque. Adds methods enqueue_at/enqueue_in to schedule jobs in the future. Also supports queueing jobs on a fixed, cron-like schedule.

Prioritize Sidekiq jobs within queues

resque plugin for automatically scaling workers based on the amount of time it takes a job to go through the queue and/or the length of the queue

= Ungulate According to Wikipedia, this can mean "hoofed animal". Camels have hooves. This is a gem for uploading and processing images using an Amazon Web Services stack. It comes with a few goodies: * ungulate_server.rb - simple queue runner that expects a YAML-encoded job description for RMagick * Ungulate::FileUpload - a model for e.g. Rails that does some cryptography stuff - example to follow * A view helper for Rails: "ungulate_upload_form_for" == Installation gem install ungulate == Documentation http://wiki.github.com/camelpunch/ungulate/ == Note on Patches/Pull Requests * Fork the project. * Make your feature addition or bug fix. * Add tests for it. This is important so I don't break it in a future version unintentionally. * Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull) * Send me a pull request. Bonus points for topic branches. == Copyright Copyright (c) 2011 Camel Punch Limited. See LICENSE for details.

Cloudist is a simple, highly scalable job queue for Ruby applications, it can run within Rails, DaemonKit or your own custom application. Refer to github page for examples

Resque is a Redis-backed Ruby library for creating background jobs, placing those jobs on multiple queues, and processing them later. Resque-igo is the same thing, but for mongo. It would not exist without the work of defunkt and ctrochalakis on github. Background jobs can be any Ruby class or module that responds to perform. Your existing classes can easily be converted to background jobs or you can create new classes specifically to do work. Or, you can do both. Resque is heavily inspired by DelayedJob (which rocks) and is comprised of three parts: * A Ruby library for creating, querying, and processing jobs * A Rake task for starting a worker which processes jobs * A Sinatra app for monitoring queues, jobs, and workers.

Quebert is a worker queue framework built around beanstalkd. Use it in your Rails apps for job processing.

Sidekiq::Undertaker is a plugin for Sidekiq. It allows exploring, reviving (retrying) or burying (deleting) dead jobs. For easy exploring the dead-jobs queue is broken down into time windows (buckets) of hours, days and weeks.

Rails 4 compatible queue with support for executing jobs later.

A Resque plugin. If you want only one instance of your job queued at a time, extend it with this module. For example: class ExampleJob extend Resque::Jobs::Queue::Lock def self.perform(repo_id) heavy_lifting end end

resque-status is an extension to the resque queue system that provides simple trackable jobs. It provides a Resque::Plugins::Status::Hash class which can set/get the statuses of jobs and a Resque::Plugins::Status class that when included provides easily trackable/killable jobs.

Resque is a Redis-backed Ruby library for creating background jobs, placing those jobs on multiple queues, and processing them later. Mongo-Resque is the same thing, but for mongo. It would not exist without the work of defunkt and ctrochalakis on github.

Timberline is a simple and extensible queuing system built in Ruby and backed by Redis. It makes as few assumptions as possible about how you want to interact with your queues while also allowing for some functionality that should be universally useful, like allowing for automatic retries of jobs and queue statistics.

Job queue system to run job with Docker

Queues and Background Job Runners are becoming as ubiquitous to Rails applications as Databases. Why not treat each of them as the same generic component, conforming to an interface and convention that is well understood.

Edamame combines the Beanstalk priority queue with a Tokyo Tyrant database and God monitoring to produce a persistent distributed priority job queue system. Like beanstalk, it is fast, lightweight, distributed, priority queuing, reliable scheduling; it adds persistence, named jobs and job querying/enumeration.

This gem makes it possible to send mails asynchronously using Resque by simply rewriting `SomeMailer.some_mail(ar_resource, 1234).deliver` to `SomeMailer.async_deliver.some_mail(ar_resource, 1234)`. Using ActiveRecord objects as arguments to mailers is still possible. This is achieved by storing the class name and the record id as arguments in the Resque queue which will be transformed back to records by the mailer job and passed along to the mailer.

The Cloudq Client gem is an awesome client interface to the cloudq a remote job queue engine that allows you to publish jobs and subscribe from jobs anywhere in the cloud....