qamasu

==================================================== Qamasu . Job Queue Application written in Python

Qamasu is JobQueue system that respects TheSchwartz.

Suited to load leveling.

Implemented using optimistic lock.

Requirements

• Python>=2.6
• Django>=1.0

Python3 needs Django1.5

Usage

Set Qamasu up! ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Qamasu is a Django application.

You need add qamasu to your or new django project's INSTALLED_APPS.

And //manage.py syncdb//.

Write your worker. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Define GRAB_FOR in seconds that is max time worker grabbed for a work.

Define def work_safely(manager, job): that is a work you need.

See sample worker_ in workers directory for detail.

.. _sample worker: http://bitbucket.org/tsuyukimakoto/qamasu/src/tip/workers/random_wait.py

Registration ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ You need add worker to abilities.

register_func insert fuction record into database table if not exist.

::

>>> from qamasu import Qamasu
>>> qamasu = Qamasu([])
>>> qamasu.register_func('workers.random_wait')

Queue! ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Once fuction is registered to qamasu, you can enqueue jobs.

Add hundreds Queues.::

>>> from qamasu import Qamasu
>>> from random import uniform
>>> qamasu = Qamasu(['workers.random_wait',])
>>> for x in xrange(1,500):
      arg = dict(random_number=uniform(1,5))
      qamasu.enqueue('workers.random_wait', arg)

Add a highest-priority queue.::

>>> qamasu.enqueue('workers.random_wait', dict(random_number=uniform(1,5)), priority=1)

Work! Work! Work! ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Process enqueued job.

1. Instantiate Qamasu with availabilities.
2. call work method. infinite loop inside this method.

work method process queue as FIFO.

::

>>> from qamasu import Qamasu
>>> qamasu = Qamasu(['workers.random_wait',])
>>> qamasu.work()

Use work_prioritizing method if you tend to process job respects to priority. ::

>>> from qamasu import Qamasu
>>> qamasu = Qamasu(['workers.random_wait',])
>>> qamasu.work_prioritizing()

Caution!

For MySQL backend ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

You must set worker's transaction isolation level to read commited before working qamasu when you use InnoDB. ::

>>> from django.db import connection
>>> from qamasu import Qamasu
>>> connection.cursor().execute('set session transaction isolation level read committed')
>>> qamasu = Qamasu(['workers.random_wait',])
>>> qamasu.work()

Or you have to set transaction isolation level read committed. It's global settings and dangerous. ::

[mysqld] transaction-isolation=Read-Committed