bottleneck

What is bottleneck?

The bottleneck npm package is a rate limiter that allows you to control the rate at which tasks are executed. It is useful for managing resource usage and ensuring that your application does not exceed the rate limits imposed by APIs or other systems. It can be used to throttle function calls, manage task execution concurrency, and queue tasks.

What are bottleneck's main functionalities?

Task Scheduling

This feature allows you to schedule tasks to be executed with a minimum time interval between them and a maximum number of concurrent tasks. The code sample demonstrates how to create a limiter that only allows one task to run at a time with a minimum of 1 second between task starts.

const Bottleneck = require('bottleneck');
const limiter = new Bottleneck({ maxConcurrent: 1, minTime: 1000 });

async function myFunction(id) {
  console.log(`Working on task ${id}`);
}

for (let i = 0; i < 5; i++) {
  limiter.schedule(() => myFunction(i));
}

Priority Queueing

This feature allows you to assign priorities to tasks. Tasks with higher priority (lower priority number) will be executed before those with lower priority. The code sample shows how to schedule tasks with different priorities.

const Bottleneck = require('bottleneck');
const limiter = new Bottleneck({ maxConcurrent: 2 });

async function myFunction(priority, id) {
  console.log(`Working on task ${id} with priority ${priority}`);
}

limiter.schedule({ priority: 1 }, myFunction, 1, 'A');
limiter.schedule({ priority: 5 }, myFunction, 5, 'B');
limiter.schedule({ priority: 3 }, myFunction, 3, 'C');

Rate Limiting

This feature allows you to limit the rate at which functions are executed. It can be used to avoid hitting rate limits of external APIs. The code sample demonstrates how to set up a limiter with a reservoir that refreshes over time, and how to handle the 'depleted' event when the rate limit is exceeded.

const Bottleneck = require('bottleneck');
const limiter = new Bottleneck({ reservoir: 100, // initial value
  reservoirRefreshAmount: 100,
  reservoirRefreshInterval: 60 * 1000 // must be divisible by 250
});

limiter.on('depleted', () => console.log('Rate limit exceeded'));

// Function to be rate-limited
async function myFunction() {
  console.log('Doing something.');
}

setInterval(() => {
  limiter.submit(myFunction);
}, 10);

Other packages similar to bottleneck

p-throttle

p-throttle is a package that allows you to throttle function calls. It is similar to bottleneck in that it helps to control the rate of function execution, but it is built on top of Promises and does not provide the same level of detailed configuration as bottleneck, such as priority queueing or reservoir-based limiting.

async-ratelimiter

async-ratelimiter is a rate limiting library that uses Redis for storage. It is designed for more distributed use cases where you need to maintain rate limits across multiple processes or servers. Unlike bottleneck, which is more focused on in-process rate limiting, async-ratelimiter is suitable for scenarios where you need a shared rate limit state.

leaky-bucket

leaky-bucket is another rate limiting library that implements the leaky bucket algorithm. It is similar to bottleneck in providing rate limiting capabilities, but it has a different algorithmic approach. While bottleneck uses a more general approach with configurable reservoirs, leaky-bucket strictly adheres to the leaky bucket algorithm, which may be more suitable for certain use cases.

README

bottleneck

Bottleneck is a simple and efficient Asynchronous Rate Limiter for Node.JS and the browser. When dealing with services with limited resources, it's important to ensure that they don't become overloaded. Bottleneck is the easiest solution as it doesn't add any complexity to the code.

#Install

Node

npm install bottleneck

Browser

<script type="text/javascript" src="bottleneck.min.js"></script>

#Example

Most APIs have a rate limit. For example, the Reddit.com API limits programs to 1 request every 2 seconds.

var Bottleneck = require("bottleneck"); //Node.JS only

// Never more than 1 request running at a time.
// Wait at least 2000ms between each request.
var limiter = new Bottleneck(1, 2000);

Instead of doing

someAsyncCall(arg1, arg2, argN, callback);

You do

limiter.submit(someAsyncCall, arg1, arg2, argN, callback);

And now you can be assured that someAsyncCall will abide by your rate guidelines!

All the submitted requests will be executed in order.

#Docs

###Constructor

var limiter = new Bottleneck(maxConcurrent, minTime, highWater, strategy);

• maxConcurrent : How many requests can be running at the same time. Default: 0 (unlimited)
• minTime : How long to wait after launching a request before launching another one. Default: 0ms
• highWater : How long can the queue get? Default: 0 (unlimited)
• strategy : Which strategy use if the queue gets longer than the high water mark. Default: Bottleneck.strategy.LEAK.

###submit()

This adds a request to the queue, see the example above.

It returns true if the strategy was executed. Therefore it will always return false if highWater is set to 0.

Note: If a callback isn't necessary, you must pass null or an empty function instead.

Make sure that all the requests will eventually complete! This is very important if you are using a maxConcurrent value that isn't 0 (unlimited), otherwise those uncompleted requests will be clogging up the limiter and no new requests will be getting through. A way to do this is to use a timer that will always call the callback. It's safe to call the callback more than once, subsequent calls are ignored.

###strategies

A strategy is a simple algorithm that is executed every time submit would cause the queue to exceed highWater.

####Bottleneck.strategy.LEAK When submitting a new request, if the queue length reaches highWater, drop the oldest request in the queue. This is useful when requests that have been waiting for too long are not important anymore.

####Bottleneck.strategy.OVERFLOW When submitting a new request, if the queue length reaches highWater, do not add the new request.

####Bottleneck.strategy.BLOCK When submitting a new request, if the queue length reaches highWater, the limiter falls into "blocked mode". No new requests will be accepted until it unblocks. It will unblock after penalty milliseconds have passed without receiving a new request. penalty is equal to 15 * minTime (or 5000 if minTime is 0) by default and can be changed by calling changePenalty(). This strategy is ideal when bruteforce attacks are to be expected.

###check()

limiter.check();

If a task was submitted right now, would it be run immediately? Returns a boolean.

###stopAll()

limiter.stopAll(interrupt);

Cancels all queued up requests and prevents additonal requests from being submitted.

• interrupt : If true, prevent the tasks currently running from calling their callback when they're done. Default: false

###changeSettings()

limiter.changeSettings(maxConcurrent, minTime, highWater, strategy);

Same parameters as the constructor, pass null to skip a parameter and keep it to its current value.

###changePenalty()

limiter.changePenalty(penalty);

This changes the penalty value used by the BLOCK strategy.

Thoughts

The main design goal for Bottleneck is to be extremely small and transparent to use. It's meant to add the least possible complexity to the code.

Let's take a DNS server as an example of how Bottleneck can be used. It's a service that sees a lot of abuse. Bottleneck is tiny, so it's not unreasonable to create one instance of it for each origin IP. The BLOCK strategy will then easily lock out abusers and prevent the server from being used for a DNS amplification attack.

Other times, the application acts as a client and Bottleneck is used to not overload the server. In those cases, it's often better to not set any highWater mark so that no request is ever lost.

As long as highWater is 0, all requests are assured to be executed at some point. Once again, when using a maxConcurrent value greater than 0, make sure that all requests will call the callback eventually.

---

Most of the time, using Bottleneck is as simple as the first example above. However, when Bottleneck is used on a synchronous call, it (obviously) becomes asynchronous, so the returned value of that call can't be used directly. The following example should make it clear why.

This is the original code that we want to rate-limit:

var req = http.request(options, function(res){
	//do stuff with res
});
req.write("some string", "utf8");
req.end();

The following code snippet will NOT work, because http.request is not executed synchronously therefore req doesn't contain the expected request object.

// DOES NOT WORK
var req = limiter.submit(http.request, options, function(res){
	//do stuff with res
});
req.write("some string", "utf8");
req.end();

This is the right way to do it:

limiter.submit(function(){
	var req = http.request(options, function(res){
		//do stuff with res
	});
	req.write("some string", "utf8");
	req.end();
}, null);

---

Pull requests and suggestions are welcome.