Socket
Socket
Sign inDemoInstall

cacheable-request

Package Overview
Dependencies
15
Maintainers
1
Versions
61
Alerts
File Explorer

Advanced tools

npm Scripts

Install Socket

Detect and block malicious and high-risk dependencies

Install

Comparing version 0.4.0 to 0.5.0

test/cacheable-request-class.js
test/cacheable-request-instance.js

6

package.json
{
"name": "cacheable-request",
"version": "0.4.0",
"version": "0.5.0",
"description": "Wrap native HTTP requests with RFC compliant cache support",

@@ -51,5 +51,9 @@ "main": "src/index.js",

"get-stream": "^3.0.0",
"keyv-sqlite": "^1.2.4",
"nyc": "^11.0.2",
"pify": "^3.0.0",
"sqlite3": "^3.1.9",
"this": "^1.0.2",
"xo": "^0.19.0"
}
}

115

README.md

@@ -9,3 +9,3 @@ # cacheable-request

[RFC 7234](http://httpwg.org/specs/rfc7234.html) compliant HTTP caching for native Node.js HTTP/HTTPS requests. Caching works out of the box with `new Map()` or is easily pluggable with a wide range of cache adapters.
[RFC 7234](http://httpwg.org/specs/rfc7234.html) compliant HTTP caching for native Node.js HTTP/HTTPS requests. Caching works out of the box in memory or is easily pluggable with a wide range of storage adapters.

@@ -22,34 +22,59 @@ ## Install

const http = require('http');
const cacheableRequest = require('cacheable-request');
const CacheableRequest = require('cacheable-request');
// Then instead of
const opts = {
host: 'example.com'
};
const req = http.request(opts, cb);
const req = http.request('http://example.com', cb);
req.end();
// You can do
const cache = new Map();
const opts = {
host: 'example.com',
cache: cache
};
const cacheReq = cacheableRequest(http.request, opts, cb);
const cacheableRequest = new CacheableRequest(http.request);
const cacheReq = cacheableRequest('http://example.com', cb);
cacheReq.on('request', req => req.end());
// Future requests to 'example.com' will be returned from cache if still valid
// Or pass in any other http.request API compatible method:
cacheableRequest(https.request, opts, cb);
cacheableRequest(electron.net, opts, cb);
// You pass in any other http.request API compatible method to be wrapped with cache support:
const cacheableRequest = new CacheableRequest(https.request);
const cacheableRequest = new CacheableRequest(electron.net);
```
## Cache Adapters
## Storage Adapters
> TODO
`cacheable-request` uses [Keyv](https://github.com/lukechilds/keyv) to support a wide range of storage adapters.
For example, to use Redis as a cache backend, you just need to install the official Redis Keyv storage adapter:
```
npm install --save @keyv/redis
```
And then you can pass `CacheableRequest` your connection string:
```js
const cacheableRequest = new CacheableRequest(http.request, 'redis://user:pass@localhost:6379');
```
[View all official Keyv storage adapters.](https://github.com/lukechilds/keyv#official-storage-adapters)
Keyv also supports anything that follows the Map API so it's easy to write your own storage adapter or use a third-party solution.
e.g The following are all valid storage adapters
```js
const storageAdapter = new Map();
// or
const storageAdapter = require('./my-storage-adapter');
// or
const QuickLRU = require('quick-lru');
const storageAdapter = new QuickLRU({ maxSize: 1000 });
const cacheableRequest = new CacheableRequest(http.request, storageAdapter);
```
View the [Keyv docs](https://github.com/lukechilds/keyv) for more information on how to use storage adapters.
## API
### cacheableRequest(request, opts, [cb])
### new cacheableRequest(request, [storageAdapter])
Returns an event emitter.
Returns the provided request function wrapped with cache support.

@@ -62,33 +87,47 @@ #### request

#### opts
#### storageAdapter
Type: `object`
Type: `Keyv storage adapter`<br>
Default: `new Map()`
A [Keyv](https://github.com/lukechilds/keyv) storage adapter instance, or connection string if using with an official Keyv storage adapter.
### Instance
#### cacheableRequest(opts, [cb])
Returns an event emitter.
##### opts
Type: `object`, `string`
Any of the default request functions options plus:
##### opts.cache
###### opts.cache
Type `cache adapter instance`
Type: `boolean`<br>
Default: `true`
The cache adapter should follow the [Map](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map) API. You can pass `new Map()` to cache items in memory, or a [Keyv storage adapter](https://github.com/lukechilds/keyv#official-storage-adapters) if you want a shared persistent store.
If the cache should be used. Setting this to false will completely bypass the cache for the current request.
The `cache` option can be omitted and the request will be passed directly through to the request function with no caching.
###### opts.strictTtl
##### opts.strictTtl
Type: `boolean`<br>
Default: `false`
Type: `object`<br>
Default `false`
If set to `false`, after a cached resource's TTL expires it is kept in the cache and will be revalidated on the next request with `If-None-Match`/`If-Modified-Since` headers.
If set to `false` expired resources are still kept in the cache and will be revalidated on the next request with `If-None-Match`/`If-Modified-Since` headers.
If set to `true` once a cached resource has expired it is deleted and will have to be re-requested.
#### cb
##### cb
Type: `function`
The callback function which will receive the response as an argument. The response can be either a [Node.js HTTP response stream](https://nodejs.org/api/http.html#http_class_http_incomingmessage) or a [responselike object](https://github.com/lukechilds/responselike).
The callback function which will receive the response as an argument.
#### .on('request', request)
The response can be either a [Node.js HTTP response stream](https://nodejs.org/api/http.html#http_class_http_incomingmessage) or a [responselike object](https://github.com/lukechilds/responselike). The response will also have a `fromCache` property set with a boolean value.
##### .on('request', request)
`request` event to get the request object of the request.

@@ -98,14 +137,14 @@

#### .on('response', response)
##### .on('response', response)
`response` event to get the response object from the HTTP request or cache.
#### .on('error', error)
##### .on('error', error)
`error` event emitted in case of an error with the cache logic.
`error` event emitted in case of an error with the cache.
**Note:** You still need to handle requst errors on `request`. e.g:
**Note:** You should still handle request errors in the `request` event. e.g:
```js
cacheableRequest(http.request, opts, cb)
cacheableRequest('example.com', cb)
.on('error', handleCacheError)

@@ -112,0 +151,0 @@ .on('request', req => {

189

src/index.js

@@ -13,107 +13,112 @@ 'use strict';

const cacheKey = opts => {
const url = normalizeUrl(urlLib.format(opts));
return `${opts.method}:${url}`;
};
class CacheableRequest {
constructor(request, cacheAdapter) {
if (typeof request !== 'function') {
throw new TypeError('Parameter `request` must be a function');
}
const cacheableRequest = (request, opts, cb) => {
if (typeof opts === 'string') {
opts = urlLib.parse(opts);
}
opts = Object.assign({
headers: {},
method: 'GET',
cache: undefined,
strictTtl: false
}, opts);
opts.headers = lowercaseKeys(opts.headers);
this.cache = new Keyv({
uri: typeof cacheAdapter === 'string' && cacheAdapter,
store: typeof cacheAdapter !== 'string' && cacheAdapter,
namespace: 'cacheable-request'
});
if (typeof request !== 'function') {
throw new TypeError('Parameter `request` must be a function');
return this.createCacheableRequest(request);
}
const cache = new Keyv({
uri: typeof opts.cache === 'string' && opts.cache,
store: typeof opts.cache !== 'string' && opts.cache,
namespace: 'got'
});
const ee = new EventEmitter();
const key = cacheKey(opts);
let revalidate = false;
const makeRequest = opts => {
const req = request(opts, response => {
if (revalidate) {
const revalidatedPolicy = CachePolicy.fromObject(revalidate.cachePolicy).revalidatedPolicy(opts, response);
if (!revalidatedPolicy.modified) {
const headers = revalidatedPolicy.policy.responseHeaders();
response = new Response(response.statusCode, headers, revalidate.body, revalidate.url);
response.cachePolicy = revalidatedPolicy.policy;
response.fromCache = true;
}
createCacheableRequest(request) {
return (opts, cb) => {
if (typeof opts === 'string') {
opts = urlLib.parse(opts);
}
opts = Object.assign({
headers: {},
method: 'GET',
cache: true,
strictTtl: false
}, opts);
opts.headers = lowercaseKeys(opts.headers);
if (!response.fromCache) {
response.cachePolicy = new CachePolicy(opts, response);
response.fromCache = false;
}
const ee = new EventEmitter();
const url = normalizeUrl(urlLib.format(opts));
const key = `${opts.method}:${url}`;
let revalidate = false;
let clonedResponse;
if (opts.cache && response.cachePolicy.storable()) {
clonedResponse = cloneResponse(response);
getStream.buffer(response)
.then(body => {
const value = {
cachePolicy: response.cachePolicy.toObject(),
url: response.url,
statusCode: response.fromCache ? revalidate.statusCode : response.statusCode,
body
};
const ttl = opts.strictTtl ? response.cachePolicy.timeToLive() : undefined;
return cache.set(key, value, ttl);
})
.catch(err => ee.emit('error', err));
} else if (opts.cache && revalidate) {
cache.delete(key)
.catch(err => ee.emit('error', err));
}
const makeRequest = opts => {
const req = request(opts, response => {
if (revalidate) {
const revalidatedPolicy = CachePolicy.fromObject(revalidate.cachePolicy).revalidatedPolicy(opts, response);
if (!revalidatedPolicy.modified) {
const headers = revalidatedPolicy.policy.responseHeaders();
response = new Response(response.statusCode, headers, revalidate.body, revalidate.url);
response.cachePolicy = revalidatedPolicy.policy;
response.fromCache = true;
}
}
ee.emit('response', clonedResponse || response);
if (typeof cb === 'function') {
cb(clonedResponse || response);
}
});
ee.emit('request', req);
};
if (!response.fromCache) {
response.cachePolicy = new CachePolicy(opts, response);
response.fromCache = false;
}
const get = opts => Promise.resolve()
.then(() => opts.cache ? cache.get(key) : undefined)
.then(cacheEntry => {
if (typeof cacheEntry === 'undefined') {
return makeRequest(opts);
}
let clonedResponse;
if (opts.cache && response.cachePolicy.storable()) {
clonedResponse = cloneResponse(response);
getStream.buffer(response)
.then(body => {
const value = {
cachePolicy: response.cachePolicy.toObject(),
url: response.url,
statusCode: response.fromCache ? revalidate.statusCode : response.statusCode,
body
};
const ttl = opts.strictTtl ? response.cachePolicy.timeToLive() : undefined;
return this.cache.set(key, value, ttl);
})
.catch(err => ee.emit('error', err));
} else if (opts.cache && revalidate) {
this.cache.delete(key)
.catch(err => ee.emit('error', err));
}
const policy = CachePolicy.fromObject(cacheEntry.cachePolicy);
if (policy.satisfiesWithoutRevalidation(opts)) {
const headers = policy.responseHeaders();
const response = new Response(cacheEntry.statusCode, headers, cacheEntry.body, cacheEntry.url);
response.cachePolicy = policy;
response.fromCache = true;
ee.emit('response', clonedResponse || response);
if (typeof cb === 'function') {
cb(clonedResponse || response);
}
});
ee.emit('request', req);
};
ee.emit('response', response);
if (typeof cb === 'function') {
cb(response);
}
} else {
revalidate = cacheEntry;
opts.headers = policy.revalidationHeaders(opts);
makeRequest(opts);
}
});
const get = opts => Promise.resolve()
.then(() => opts.cache ? this.cache.get(key) : undefined)
.then(cacheEntry => {
if (typeof cacheEntry === 'undefined') {
return makeRequest(opts);
}
get(opts).catch(err => ee.emit('error', err));
const policy = CachePolicy.fromObject(cacheEntry.cachePolicy);
if (policy.satisfiesWithoutRevalidation(opts)) {
const headers = policy.responseHeaders();
const response = new Response(cacheEntry.statusCode, headers, cacheEntry.body, cacheEntry.url);
response.cachePolicy = policy;
response.fromCache = true;
return ee;
};
ee.emit('response', response);
if (typeof cb === 'function') {
cb(response);
}
} else {
revalidate = cacheEntry;
opts.headers = policy.revalidationHeaders(opts);
makeRequest(opts);
}
});
module.exports = cacheableRequest;
get(opts).catch(err => ee.emit('error', err));
return ee;
};
}
}
module.exports = CacheableRequest;

137

test/cache.js

@@ -7,10 +7,11 @@ import { request } from 'http';

import delay from 'delay';
import cacheableRequest from '../';
import sqlite3 from 'sqlite3';
import pify from 'pify';
import CacheableRequest from 'this';
let s;
// Simple wrapper that returns a promise, reads stream and sets up some options
const cacheableRequestHelper = (path, o) => new Promise(resolve => {
const opts = Object.assign({}, url.parse(s.url + path), o);
cacheableRequest(request, opts, response => {
// Promisify cacheableRequest
const promisify = cacheableRequest => opts => new Promise(resolve => {
cacheableRequest(opts, response => {
getStream(response).then(body => {

@@ -106,5 +107,7 @@ response.body = body;

const cache = new Map();
const cacheableRequest = new CacheableRequest(request, cache);
const cacheableRequestHelper = promisify(cacheableRequest);
const firstResponseInt = Number((await cacheableRequestHelper(endpoint, { cache })).body);
const secondResponseInt = Number((await cacheableRequestHelper(endpoint, { cache })).body);
const firstResponseInt = Number((await cacheableRequestHelper(s.url + endpoint)).body);
const secondResponseInt = Number((await cacheableRequestHelper(s.url + endpoint)).body);

@@ -118,5 +121,7 @@ t.is(cache.size, 0);

const cache = new Map();
const cacheableRequest = new CacheableRequest(request, cache);
const cacheableRequestHelper = promisify(cacheableRequest);
const firstResponse = await cacheableRequestHelper(endpoint, { cache });
const secondResponse = await cacheableRequestHelper(endpoint, { cache });
const firstResponse = await cacheableRequestHelper(s.url + endpoint);
const secondResponse = await cacheableRequestHelper(s.url + endpoint);

@@ -130,5 +135,7 @@ t.is(cache.size, 1);

const cache = new Map();
const cacheableRequest = new CacheableRequest(request, cache);
const cacheableRequestHelper = promisify(cacheableRequest);
const firstResponse = await cacheableRequestHelper(endpoint + '?foo', { cache });
const secondResponse = await cacheableRequestHelper(endpoint + '?bar', { cache });
const firstResponse = await cacheableRequestHelper(s.url + endpoint + '?foo');
const secondResponse = await cacheableRequestHelper(s.url + endpoint + '?bar');

@@ -139,2 +146,21 @@ t.is(cache.size, 2);

test('Setting opts.cache to false bypasses cache for a single request', async t => {
const endpoint = '/cache';
const cache = new Map();
const cacheableRequest = new CacheableRequest(request, cache);
const cacheableRequestHelper = promisify(cacheableRequest);
const opts = url.parse(s.url + endpoint);
const optsNoCache = Object.assign({ cache: false }, opts);
const firstResponse = await cacheableRequestHelper(opts);
const secondResponse = await cacheableRequestHelper(opts);
const thirdResponse = await cacheableRequestHelper(optsNoCache);
const fourthResponse = await cacheableRequestHelper(opts);
t.false(firstResponse.fromCache);
t.true(secondResponse.fromCache);
t.false(thirdResponse.fromCache);
t.true(fourthResponse.fromCache);
});
test('TTL is passed to cache', async t => {

@@ -152,14 +178,39 @@ const endpoint = '/cache';

};
const cacheableRequest = new CacheableRequest(request, cache);
const cacheableRequestHelper = promisify(cacheableRequest);
const opts = Object.assign({ strictTtl: true }, url.parse(s.url + endpoint));
t.plan(2);
await cacheableRequestHelper(endpoint, { cache, strictTtl: true });
await cacheableRequestHelper(opts);
});
test('TTL is not passed to cache if strictTtl is false', async t => {
const endpoint = '/cache';
const store = new Map();
const cache = {
get: store.get.bind(store),
set: (key, val, ttl) => {
t.true(typeof ttl === 'undefined');
return store.set(key, val, ttl);
},
delete: store.delete.bind(store)
};
const cacheableRequest = new CacheableRequest(request, cache);
const cacheableRequestHelper = promisify(cacheableRequest);
const opts = Object.assign({ strictTtl: false }, url.parse(s.url + endpoint));
t.plan(1);
await cacheableRequestHelper(opts);
});
test('Stale cache entries with Last-Modified headers are revalidated', async t => {
const endpoint = '/last-modified';
const cache = new Map();
const cacheableRequest = new CacheableRequest(request, cache);
const cacheableRequestHelper = promisify(cacheableRequest);
const firstResponse = await cacheableRequestHelper(endpoint, { cache });
const secondResponse = await cacheableRequestHelper(endpoint, { cache });
const firstResponse = await cacheableRequestHelper(s.url + endpoint);
const secondResponse = await cacheableRequestHelper(s.url + endpoint);

@@ -176,5 +227,7 @@ t.is(cache.size, 1);

const cache = new Map();
const cacheableRequest = new CacheableRequest(request, cache);
const cacheableRequestHelper = promisify(cacheableRequest);
const firstResponse = await cacheableRequestHelper(endpoint, { cache });
const secondResponse = await cacheableRequestHelper(endpoint, { cache });
const firstResponse = await cacheableRequestHelper(s.url + endpoint);
const secondResponse = await cacheableRequestHelper(s.url + endpoint);

@@ -191,6 +244,8 @@ t.is(cache.size, 1);

const cache = new Map();
const cacheableRequest = new CacheableRequest(request, cache);
const cacheableRequestHelper = promisify(cacheableRequest);
const firstResponse = await cacheableRequestHelper(endpoint, { cache });
const firstResponse = await cacheableRequestHelper(s.url + endpoint);
t.is(cache.size, 1);
const secondResponse = await cacheableRequestHelper(endpoint, { cache });
const secondResponse = await cacheableRequestHelper(s.url + endpoint);

@@ -207,5 +262,7 @@ t.is(cache.size, 0);

const cache = new Map();
const cacheableRequest = new CacheableRequest(request, cache);
const cacheableRequestHelper = promisify(cacheableRequest);
const response = await cacheableRequestHelper(endpoint, { cache });
const cachedResponse = await cacheableRequestHelper(endpoint, { cache });
const response = await cacheableRequestHelper(s.url + endpoint);
const cachedResponse = await cacheableRequestHelper(s.url + endpoint);

@@ -219,7 +276,9 @@ t.false(response.fromCache);

const cache = new Map();
const cacheableRequest = new CacheableRequest(request, cache);
const cacheableRequestHelper = promisify(cacheableRequest);
const firstResponse = await cacheableRequestHelper(endpoint, { cache });
const firstResponse = await cacheableRequestHelper(s.url + endpoint);
await delay(1100);
const secondResponse = await cacheableRequestHelper(endpoint, { cache });
const thirdResponse = await cacheableRequestHelper(endpoint, { cache });
const secondResponse = await cacheableRequestHelper(s.url + endpoint);
const thirdResponse = await cacheableRequestHelper(s.url + endpoint);

@@ -237,5 +296,7 @@ t.is(firstResponse.statusCode, 200);

const cache = new Map();
const cacheableRequest = new CacheableRequest(request, cache);
const cacheableRequestHelper = promisify(cacheableRequest);
const firstResponse = await cacheableRequestHelper(endpoint, { cache });
const secondResponse = await cacheableRequestHelper(endpoint, { cache });
const firstResponse = await cacheableRequestHelper(s.url + endpoint);
const secondResponse = await cacheableRequestHelper(s.url + endpoint);

@@ -251,6 +312,7 @@ t.is(firstResponse.statusCode, 200);

const cache = new Map();
const opts = Object.assign({}, url.parse(s.url + endpoint), { cache });
const cacheableRequest = new CacheableRequest(request, cache);
const cacheableRequestHelper = promisify(cacheableRequest);
cacheableRequestHelper(endpoint, { cache }).then(() => {
cacheableRequest(request, opts);
cacheableRequestHelper(s.url + endpoint).then(() => {
cacheableRequest(s.url + endpoint);
setTimeout(() => {

@@ -262,4 +324,23 @@ t.end();

test('Keyv cache adapters load via connection uri', async t => {
const endpoint = '/cache';
const cacheableRequest = new CacheableRequest(request, 'sqlite://test/testdb.sqlite');
const cacheableRequestHelper = promisify(cacheableRequest);
const db = new sqlite3.Database('test/testdb.sqlite');
const query = pify(db.all).bind(db);
const firstResponse = await cacheableRequestHelper(s.url + endpoint);
await delay(1000);
const secondResponse = await cacheableRequestHelper(s.url + endpoint);
const cacheResult = await query(`SELECT * FROM keyv WHERE "key" = "cacheable-request:GET:${s.url + endpoint}"`);
t.false(firstResponse.fromCache);
t.true(secondResponse.fromCache);
t.is(cacheResult.length, 1);
await query(`DELETE FROM keyv`);
});
test.after('cleanup', async () => {
await s.close();
});
test/cacheable-request.js
.npmignore

Sorry, the diff of this file is not supported yet

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap

About

Packages

Stay in touch

Get open source security insights delivered straight into your inbox.

  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc