New Case Study:See how Anthropic automated 95% of dependency reviews with Socket.Learn More →
rolling-rate-limiter
Advanced tools
rolling-rate-limiter - npm Package Compare versions
Comparing version 0.1.11 to 0.2.0
| module.exports = { | ||
| extends: "classdojo/node", | ||
| rules: { | ||
| "one-var": 0, | ||
| "max-len": 0, | ||
| } | ||
| } | ||
| extends: 'peterkhayes', | ||
| env: { | ||
| node: true, | ||
| }, | ||
| parserOptions: { | ||
| project: './tsconfig.json', | ||
| }, | ||
| overrides: [ | ||
| { | ||
| files: ['*.test.ts'], | ||
| env: { | ||
| jest: true, | ||
| }, | ||
| }, | ||
| ], | ||
| }; |
| { | ||
| "name": "rolling-rate-limiter", | ||
| "version": "0.1.11", | ||
| "description": "Rate limiter that supports a rolling window, either in-memory or backed by redis", | ||
| "version": "0.2.0", | ||
| "description": "Rate limiter that supports a rolling window, either in-memory or backed by Redis", | ||
| "main": "index.js", | ||
| "scripts": { | ||
| "test": "mocha" | ||
| "lint": "eslint . --fix", | ||
| "test": "jest" | ||
| }, | ||
@@ -13,2 +14,5 @@ "repository": { | ||
| }, | ||
| "engines": { | ||
| "node": ">= 4.0.0" | ||
| }, | ||
| "keywords": [ | ||
@@ -23,3 +27,3 @@ "rate", | ||
| "author": "Peter Hayes", | ||
| "license": "ISC", | ||
| "license": "MIT", | ||
| "bugs": { | ||
@@ -31,13 +35,20 @@ "url": "https://github.com/peterkhayes/rolling-rate-limiter/issues" | ||
| "microtime-nodejs": "~1.0.0", | ||
| "uuid": "^3.0.1" | ||
| "uuid": "^8.3.0" | ||
| }, | ||
| "devDependencies": { | ||
| "async": "~0.9.0", | ||
| "chai": "~1.10.0", | ||
| "eslint": "^3.19.0", | ||
| "eslint-config-peterkhayes": "^1.2.6", | ||
| "fakeredis": "~0.3.0", | ||
| "lodash": "^4.17.4", | ||
| "mocha": "~2.1.0" | ||
| "@types/ioredis": "^4.17.3", | ||
| "@types/jest": "^26.0.12", | ||
| "@types/node": "^14.6.2", | ||
| "@types/redis": "^2.8.26", | ||
| "@types/uuid": "^8.3.0", | ||
| "async": "~3.2.0", | ||
| "eslint": "^7.8.0", | ||
| "eslint-config-peterkhayes": "^4.0.0", | ||
| "ioredis": "^4.17.3", | ||
| "jest": "^26.4.2", | ||
| "lodash": "^4.17.20", | ||
| "redis": "^3.0.2", | ||
| "ts-jest": "^26.3.0", | ||
| "typescript": "^4.0.2" | ||
| } | ||
| } | ||
| } |
165
README.md
| # Rolling Rate Limiter | ||
| [](https://travis-ci.org/classdojo/rolling-rate-limiter) | ||
| ## Description | ||
| This is an implementation of a rate limiter in node.js that allows for rate limiting with a rolling window. | ||
| This is an implementation of a rate limiter in node.js that allows for rate limiting with a rolling window. It can use either in-memory storage or Redis as a backend. If Redis is used, multiple rate limiters can share one instance with different namespaces, and multiple processes can share rate limiter state safely. | ||
| This means that if a user is allowed 5 actions per 60 seconds, any action will be blocked if 5 actions have already occured in the preceeding 60 seconds, without any set points at which this interval resets. This contrasts with many existing implementations, in which a user could make 5 requests at 0:59 and another 5 requests at 1:01. | ||
| This means that if a user is allowed 5 actions per 60 seconds, any action will be blocked if 5 actions have already occured in the preceeding 60 seconds, without any set points at which this interval resets. This contrasts with some other rate limiter implementations, in which a user could make 5 requests at 0:59 and another 5 requests at 1:01. | ||
| It can use either in-memory storage or Redis as a backend. If Redis is used, multiple rate limiters can share one instance with different namespaces, and multiple processes can share rate limiter state safely without race conditions. The implementation uses what I believe to be a novel algorithm, with sorted sets. | ||
| **Important Note**: | ||
| As a consequence of the way the Redis algorithm works, if an action is blocked, it is still "counted". This means that if a user is continually attempting actions more quickly than the allowed rate, __all__ of their actions will be blocked until they pause or slow their requests. | ||
| ## Examples | ||
| This behavior is somewhat counterintuitive, but it's the only way that I have found that uses an atomic `MULTI` set of commands for Redis. Without this, race conditions would be possible. [See more below.](#method-of-operation). | ||
| ### In-memory | ||
| ## Quick start | ||
| Basic use in an Express application. | ||
| ```javascript | ||
| /* | ||
| Setup: | ||
| */ | ||
| const { RedisRateLimiter } = require('rolling-rate-limiter'); | ||
| var RateLimiter = require("rolling-rate-limiter"); | ||
| const limiter = new RedisRateLimiter({ | ||
| client: redisClient, // client instance from `redis` or `ioredis` | ||
| namespace: 'rate-limiter', // prefix for redis keys | ||
| interval: 60000, // milliseconds | ||
| maxInInterval: 10, | ||
| }); | ||
| var limiter = RateLimiter({ | ||
| interval: 1000 // in miliseconds | ||
| maxInInterval: 10, | ||
| minDifference: 100 // optional: the minimum time (in miliseconds) between any two actions | ||
| }); | ||
| /* | ||
| Action: | ||
| */ | ||
| function attemptAction(userId) { | ||
| // Argument should be a unique identifier for a user if one exists. | ||
| // If none is provided, the limiter will not differentiate between users. | ||
| var timeLeft = limiter(userId) | ||
| if (timeLeft > 0) { | ||
| // limit was exceeded, action should not be allowed | ||
| // timeLeft is the number of ms until the next action will be allowed | ||
| // note that this can be treated as a boolean, since 0 is falsy | ||
| app.use(function(req, res, next) { | ||
| limiter.limit(req.ipAddress).then((wasBlocked) => { | ||
| if (wasBlocked) { | ||
| return res.status(429).send("Too many requests"); | ||
| } else { | ||
| // limit was not exceeded, action should be allowed | ||
| return next(); | ||
| } | ||
| } | ||
| /* | ||
| Note that the in-memory version can also operate asynchronously. | ||
| The syntax is identical to the redis implementation below. | ||
| */ | ||
| }) | ||
| }); | ||
| ``` | ||
| ### With a redis backend | ||
| This allows multiple processes (e.g. multiple instances of a server application) to use a single redis to share rate limiter state. Make sure that the limiters have identical configurations in each instance. | ||
| ```javascript | ||
| /* | ||
| Setup: | ||
| */ | ||
| ## Available limiters | ||
| * `RedisRateLimiter` - Stores state in Redis. Can use `redis` or `ioredis` clients. | ||
| * `InMemoryRateLimiter` - Stores state in memory. Useful in testing or outside of web servers. | ||
| var RateLimiter = require("rolling-rate-limiter"); | ||
| var Redis = require("redis"); | ||
| var client = Redis.createClient(config); | ||
| ## Configuration options | ||
| * `interval: number` - The length of the rate limiter's interval, in milliseconds. For example, if you want a user to be able to perform 5 actions per minute, this should be `60000`. | ||
| * `maxInInterval: number` - The number of actions allowed in each interval. For example, in the scenario above, this would be `5` | ||
| * `minDifference?: number` - Optional. The minimum time allowed between consecutive actions, in milliseconds. | ||
| * `client: Client` (Redis only) - The Redis client to use. | ||
| * `namespace: string` (Redis only) - A string to prepend to all keys to prevent conflicts with other code using Redis. | ||
| var limiter = RateLimiter({ | ||
| redis: client, | ||
| namespace: "UserLoginLimiter", // optional: allows one redis instance to handle multiple types of rate limiters. defaults to "rate-limiter-{string of 8 random characters}" | ||
| interval: 1000, | ||
| maxInInterval: 10, | ||
| minDifference: 100 | ||
| }); | ||
| ## Instance Methods | ||
| All methods take an `Id`, which should be of type `number | string`. Commonly, this will be a user's id. | ||
| /* | ||
| Action: | ||
| */ | ||
| function attemptAction(userId, cb) { | ||
| limiter(userId, function(err, timeLeft, actionsLeft) { | ||
| if (err) { | ||
| // redis failed or similar. | ||
| } else if (timeLeft) { | ||
| // limit was exceeded, action should not be allowed | ||
| } else { | ||
| // limit was not exceeded, action should be allowed | ||
| } | ||
| }); | ||
| } | ||
| * `limit(id: Id): Promise<boolean>` - Attempt to perform an action. Returns `false` if the action should be allowed, and `true` if the action should be blocked. | ||
| * `wouldLimit(id: Id): Promise<boolean>` - Return what would happen if an action were attempted. Returns `false` if an action would not have been blocked, and `true` if an action would have been blocked. Does not "count" as an action. | ||
| * `limitWithInfo(id: Id): Promise<RateLimitInfo>` - Attempt to perform an action. Returns whether the action should be blocked, as well as additional information about why it was blocked and how long the user must wait. | ||
| * `wouldLimitWithInfo(id: Id): Promise<RateLimitInfo>` - Returns info about what would happened if an action were attempted and why. Does not "count" as an action. | ||
| ``` | ||
| `RateLimitInfo` contains the following properties: | ||
| * `blocked: boolean` - Whether the action was blocked (or would have been blocked). | ||
| * `blockedDueToCount: boolean` - Whether the action was blocked (or would have been blocked) because of the `interval` and `maxInInterval` properties. | ||
| * `blockedDueToMinDifference: boolean` - Whether the action was blocked (or would have been blocked) because of the `minDistance` property. | ||
| * `millisecondsUntilAllowed: number` - The number of milliseconds the user must wait until they can make another action. If another action would immediately be permitted, this is `0`. | ||
| * `actionsRemaining: number` - The number of actions a user has left within the interval. Does not account for `minDifference`. | ||
| ### As a middleware | ||
| You can easily use this module to set up a request rate limiter middleware in Express. | ||
| ```javascript | ||
| var limiter = RateLimiter({ | ||
| redis: redisClient, | ||
| namespace: "requestRateLimiter", | ||
| interval: 60000, | ||
| maxInInterval: 100, | ||
| minDifference: 100 | ||
| }); | ||
| app.use(function(req, res, next) { | ||
| // "req.ipAddress" could be replaced with any unique user identifier | ||
| // Note that the limiter returns the number of miliseconds until an action | ||
| // will be allowed. Since 0 is falsey, this can be treated as a boolean. | ||
| limiter(req.ipAddress, function(err, timeLeft) { | ||
| if (err) { | ||
| return res.status(500).send(); | ||
| } else if (timeLeft) { | ||
| return res.status(429).send("You must wait " + timeLeft + " ms before you can make requests."); | ||
| } else { | ||
| return next(); | ||
| } | ||
| }); | ||
| }); | ||
| ``` | ||
| ## Method of operation | ||
| * Each identifier/user corresponds to a __sorted set__ data structure. The keys and values are both equal to the (microsecond) times at which actions were attempted, allowing easy manipulation of this list. | ||
| * When a new action comes in for a user, all elements in the set that occurred earlier than (current time - interval) are dropped from the set. | ||
| * If the number of elements in the set is still greater than the maximum, the current action is blocked. | ||
| * If a minimum difference has been set and the most recent previous element is too close to the current time, the current action is blocked. | ||
| * The current action is then added to the set. | ||
| * __Note__: if an action is blocked, it is still added to the set. This means that if a user is continually attempting actions more quickly than the allowed rate, __all__ of their actions will be blocked until they pause or slow their requests. | ||
| * If the limiter uses a redis instance, the keys are prefixed with namespace, allowing a single redis instance to support separate rate limiters. | ||
| * All redis operations for a single rate-limit check/update are performed as an atomic transaction, allowing rate limiters running on separate processes or machines to share state safely. | ||
| * Each identifier/user corresponds to a _sorted set_ data structure. The keys and values are both equal to the (microsecond) times at which actions were attempted, allowing easy manipulation of this list. | ||
| * When a new action comes in for a user, all elements in the set that occurred earlier than (current time - interval) are dropped from the set. | ||
| * If the number of elements in the set is still greater than the maximum, the current action is blocked. | ||
| * If a minimum difference has been set and the most recent previous element is too close to the current time, the current action is blocked. | ||
| * The current action is then added to the set. | ||
| * _Note_: if an action is blocked, it is still added to the set. This means that if a user is continually attempting actions more quickly than the allowed rate, _all_ of their actions will be blocked until they pause or slow their requests. | ||
| * If the limiter uses a redis instance, the keys are prefixed with namespace, allowing a single redis instance to support separate rate limiters. | ||
| * All redis operations for a single rate-limit check/update are performed as an atomic transaction, allowing rate limiters running on separate processes or machines to share state safely. |
New alerts
Major refactor
Supply chain riskPackage has recently undergone a major refactor. It may be unstable or indicate significant internal changes. Use caution when updating to versions that include significant changes.
Found 1 instance in 1 package
Fixed alerts
Mixed license
License(Experimental) Package contains multiple licenses.
Found 1 instance in 1 package
Improved metrics
- Total package byte prevSize
- increased by1.48%
26601
- Number of package files
- increased by12.5%
9
- Number of low license alerts
- decreased by-100%
0
Worsened metrics
- Dev dependency count
- increased by100%
14
- Lines of code
- decreased by-3.94%
536
- Number of lines in readme file
- decreased by-47.01%
71
- Number of medium supply chain risk alerts
- increased byInfinity%
1
Dependency changes
+ Addeduuid@8.3.2(transitive)
- Removeduuid@3.4.0(transitive)
Updateduuid@^8.3.0