mongodb-queue
Advanced tools
A really light-weight way to create queue with a nice API if you're already using MongoDB.
Create a connection to your MongoDB database, and use it to create a queue object:
var mongodb = require('mongodb')
var mongoDbQueue = require('mongodb-queue')
var con = 'mongodb://localhost:27017/test'
mongodb.MongoClient.connect(con, function(err, db) {
var myQueue = mongoDbQueue(db, 'my-queue')
})
Add a message to a queue:
queue.add('Hello, World!', function(err, id) {
// Message with payload 'Hello, World!' added.
// 'id' is returned, useful for logging.
})
Get a message from the queue:
queue.get(function(err, msg) {
console.log('msg.id=' + msg.id)
console.log('msg.ack=' + msg.ack)
console.log('msg.payload=' + msg.payload) // 'Hello, World!'
console.log('msg.tries=' + msg.tries)
})
Ping a message to keep it's visibility open for long-running tasks
queue.ping(msg.ack, function(err, id) {
// Visibility window now increased for this message id.
// 'id' is returned, useful for logging.
})
Ack a message (and remove it from the queue):
queue.ack(msg.ack, function(err) {
// This msg removed from queue for this message id.
// 'id' is returned, useful for logging.
})
And if you haven't already, you should call this to make sure indexes have been added in MongoDB. Of course, if you've called this once (in some kind one-off script) you don't need to call it in your program. Of course, check the changelock to see if you need to update them with new releases:
queue.ensureIndexes(function(err) {
// The indexes needed have been added to MongoDB.
})
To create a queue, call the exported function with the MongoClient, the name
and a set of opts. The MongoDB collection used is the same name as the name
passed in:
var mongoDbQueue = require('mongodb-queue')
// an instance of a queue
var queue1 = mongoDbQueue(db, 'a-queue')
// another queue which uses the same collection as above
var queue2 = mongoDbQueue(db, 'a-queue')
Note: but don't use the same queue name twice with different options, otherwise things might get confusing.
To pass options, try this:
var resizeQueue = mongoDbQueue(db, 'resize-queue', { visibility : 30, delay : 15 })
This is the name of the MongoDB Collection you wish to use to store the messages. Each queue you create will be it's own collection.
e.g.
var resizeQueue = mongoDbQueue(db, 'resize-queue')
var notifyQueue = mongoDbQueue(db, 'notify-queue')
This will create two collections in MongoDB called resize-image and notify-owner.
Default: 30
By default, if you don't ack a message within the first 30s after receiving it, it is placed back in the queue so it can be fetched again. This is called the visibility window.
You may set this visibility window on a per queue basis. For example, to set the visibility to 15 seconds:
var queue = mongoDbQueue(db, 'queue', { visibility : 15 })
All messages in this queue now have a visibility window of 15s, instead of the default 30s.
Default: 0
When a message is added to a queue, it is immediately available for retrieval.
However, there are times when you might like to delay messages coming off a queue.
ie. if you set delay to be 10, then every message will only be available for
retrieval 10s after being added.
To delay all messages by 10 seconds, try this:
var queue = mongoDbQueue(db, 'queue', { delay : 10 })
This is now the default for every message added to the queue.
Whilst using MongoDB recently and having a need for lightweight queues, I realised that the atomic operations that MongoDB provides are ideal for this kind of job.
Since everything it atomic, it is impossible to lose messages in or around your application. I guess MongoDB could lose them but it's a safer bet it won't compared to your own application.
As an example of the atomic nature being used, messages stay in the same collection
and are never moved around or deleted, just a couple of fields are set, incremented
or deleted. We always use MongoDB's excellent collection.findAndModify() so that
each message is updated atomically inside MongoDB and we never have to fetch something,
change it and store it back.
delaydelay optionqueueNameWritten by Andrew Chilton - Twitter.
MIT - http://chilts.mit-license.org/2014/
(Ends)
FAQs
Message queues which uses MongoDB.
The npm package mongodb-queue receives a total of 855 weekly downloads. As such, mongodb-queue popularity was classified as not popular.
We found that mongodb-queue demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Socket is tracking a new trend where malicious actors are now exploiting the popularity of LLM research to spread malware through seemingly useful open source packages.
Security News
Research
Noxia, a new dark web bulletproof host, offers dirt cheap servers for Python, Node.js, Go, and Rust, enabling cybercriminals to distribute malware and execute supply chain attacks.
Company News
Socket safeguards companies from software supply chain attacks by detecting and preventing threats in open source code and empowering developers to secure their applications and critical services against malware and other security risks.