Security News
Highlights from the 2024 Rails Community Survey
A record 2,709 developers participated in the 2024 Ruby on Rails Community Survey, revealing key tools, practices, and trends shaping the Rails ecosystem.
oxen-queue
Advanced tools
CAUTION: This project isn't yet ready for use by humans.
A no-frills, high-throughput worker queue backed by MySQL.
Oxen is designed to help you chew through a very high number of jobs by leveraging significant concurrency. It is resilient to misbehaving jobs, dropped database connections, and other ills. At Opteo, we mostly use it to work though scheduled batch tasks that aren't reasonable to run in a fat Promise.all().
There are already several great job queue libraries out there, but in the context of our use-cases, they either struggled with a high number of jobs, handled unexpected disconnections poorly, or had issues with race conditions.
You'll be happy with Oxen if you:
Oxen isn't for you if:
Infrastructure Requirements:
NPM
To install via npm, run:
npm install oxen-queue
Here's how you initialise the queue.
const oxen_queue = require('oxen-queue')
const ox = new oxen_queue({
mysql_config: {
// this object gets passed onto the mysql2 library
user: 'mysql_user',
password: 'mysql_password',
// anything else you need to pass to the mysql lib
},
db_table: 'oxen_queue', // (optional) name the table that oxen will use
job_type: 'avatar_renders', // give this queue a job type. Other instances of oxen with the same job type will be the same queue.
})
/* If this is your first time running oxen, run this line to automatically create the database table. You should only need to run this once. */
await ox.createTable()
Each job is saved as a single row in your table. The actual job body is JSON.stringify'ed and put into a VARCHAR(1000) field, so anything that will survive that process will fit into a job. If 1000 characters isn't enough for you, feel free to alter your table to use a TEXT field.
const oxen_queue = require('oxen-queue')
const ox = new oxen_queue({ /* Initialisation args here */ }}
// adding a job with a string body
ox.addJob({
body : 'job_body_here'
})
// adding a job with an object body
ox.addJob({
body : { oh : 'hello', arr : [1, 2]}
})
// shorthand for adding a job with no additional parameters
ox.addJob('job_body_here')
TODO: define all available args for addJob() and describe job consumer.
FAQs
A resilient worker queue backed by MySQL.
We found that oxen-queue demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 9 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
A record 2,709 developers participated in the 2024 Ruby on Rails Community Survey, revealing key tools, practices, and trends shaping the Rails ecosystem.
Security News
In 2023, data breaches surged 78% from zero-day and supply chain attacks, but developers are still buried under alerts that are unable to prevent these threats.
Security News
Solo open source maintainers face burnout and security challenges, with 60% unpaid and 60% considering quitting.