Security News
Research
Data Theft Repackaged: A Case Study in Malicious Wrapper Packages on npm
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
fetch-dedupe
Advanced tools
A (very) thin wrapper around
fetch()
that prevents duplicate requests.
A common feature of libraries or frameworks that build abstractions around HTTP requests is that they deduplicate requests that are exactly the same. This library extracts that functionality.
Install using npm:
npm install fetch-dedupe
or yarn:
yarn add fetch-dedupe
This example demonstrates using Fetch Dedupe with the ES2015 module syntax.
import { fetchDedupe } from 'fetch-dedupe';
const fetchOptions = {
method: 'PATCH',
body: JSON.stringify({ a: 12 })
};
// The API of `fetchDedupe` is the same as fetch, except that it
// has an additional argument. Pass the `requestKey` in that
// third argument
fetchDedupe('/test/2', fetchOptions).then(res => {
console.log('Got some data', res.data);
});
// Additional requests are deduped. Nifty.
fetchDedupe('/test/2', fetchOptions).then(res => {
console.log('Got some data', res.data);
});
When using fetch
, you typically read the body yourself by calling, say, .json()
on the
response. Fetch Dedupe reads the body for you, so you cannot do it, or else an error
will be thrown.
// Normal usage of `fetch`:
fetch(url, init)
.then(res => res.json())
.then(data => console.log('got some cool data', data));
// The same code using `fetchDedupe`:
fetchDedupe(url, init)
.then(res =>
console.log('got some cool data', res.data)
);
// Don't do this! It will throw an error.
fetchDedupe(url, init)
.then(res => res.json())
.then(data => console.log('got some cool data', data));
This library exports the following methods:
fetchDedupe()
getRequestKey()
isRequestInFlight()
clearRequestCache()
fetchDedupe( input [, init] [, dedupeOptions] )
A wrapper around global.fetch()
. The first two arguments are the same ones that you're used to.
Refer to
the fetch() documentation on MDN
for more.
Note that init
is optional, as with global.fetch()
.
The third option is dedupeOptions
, and it is also optional. This is an object with three attributes:
responseType
: Any of the methods from the Body mixin.
The default is "json"
, unless the response status code is "204"
, in which case "text"
will be used to prevent
an error.requestKey
: A string that is used to determine if two requests are identical. You may pass this
to configure how the request key is generated. A default key will be generated for you if this is
omitted.dedupe
: Whether or not to dedupe the request. Pass false
and it will be as if this library
was not even being used. Defaults to true
.Given the two possible value types of input
, optional second argument, there are a way few ways that you can
call fetchDedupe
. Let's run through valid calls to fetchDedupe
:
import { fetchDedupe } from 'fetch-dedupe';
// Omitting everything except for the URL
fetchDedupe('/test/2');
// Just a URL and some init option
fetchDedupe('/test/2', {
method: 'DELETE'
});
// Omitting `init` and using a URL string as `input`
fetchDedupe('/test/2', {responseType: 'json'});
// Using a URL string as `input`, with numerous `init` configurations
// and specifying several `dedupeOptions`
fetchDedupe('/test/2', {
method: 'PATCH',
body: JSON.stringify({value: true}),
credentials: 'include'
}, {
responseType: 'json',
requestKey: generateCustomKey(opts),
dedupe: false
})
// Omitting `init` and using a Request as `input`
const req = new Request('/test/2');
fetchDedupe(req, {responseType: 'json'});
// Request as `input` with an `init` object. Note that the `init`
// object takes precedence over the Request values.
fetchDedupe(req, {method: 'PATCH'}, {responseType: 'json'});
getRequestKey({ url, method, responseType, body })
Returns a unique request key based on the passed-in values. All of the values,
including body
, must be strings.
Every value is optional, but the deduplication logic is improved by adding the most information that you can.
Note: The
method
option is case-insensitive.
Note: You do not need to use this method to generate a request key. You can generate the key in whatever way that you want. This should work for most use cases, though.
import { getRequestKey } from 'fetch-dedupe';
const keyOne = getRequestKey({
url: '/books/2',
method: 'get'
});
const keyTwo = getRequestKey({
url: '/books/2',
method: 'patch',
body: JSON.stringify({
title: 'My Name is Red'
})
});
keyOne === keyTwo;
// => false
isRequestInFlight( requestKey )
Pass in a requestKey
to see if there's already a request in flight for it. This
can be used to determine if a call to fetchDedupe()
will actually hit the network
or not.
import { isRequestInFlight, getRequestKey } from 'fetch-dedupe';
const key = getRequestKey({
url: '/books/2',
method: 'get'
});
// Is there already a request in flight for this?
const readingBooksAlready = isRequestInFlight(key);
Now: We strongly recommend that you manually pass in
requestKey
tofetchDedupe
if you intend to use this method. In other words, do not rely on being able to reliably reproduce the request key that is created when arequestKey
is not passed in.
clearRequestCache()
Wipe the cache of in-flight requests.
Warning: this is not safe to use in application code. It is mostly useful for testing.
Empty text strings are not valid JSON.
JSON.parse('');
// > Uncaught SyntaxError: Unexpected end of JSON input
Consequently, using json
as the responseType
when a response's body is empty will cause an
Error to be thrown. To avoid this, we recommend using text
in these situations instead.
APIs generally use empty bodies in conjunction with a 204 status code for responses
of "write" requests (deletes, updates, and less commonly creates). For this reason, the default
behavior of responseType
is "json"
except in situations when a 204 code is returned, in which
case "text"
will be used instead.
If your API returns empty bodies with other codes, then you will need to manage that in your
application. As a "worst-case scenario," you can always pass "text"
and then try/catch a
JSON.parse
within a .then()
.
responseType
even an option?The argument that is returned to you in the .then
callback of a call to fetch()
is a
Response object. The body of a Response
object can only be read a single time, because it is a
ReadableStream.
For Fetch Dedupe to work, it must pass the result of a single request to many "consumers." The only way for this to work is if the library reads it for you, which requires that the library know what its content type is.
Just strings for now, which should work for the majority of APIs. Support for other body types is in the works.
These are projects that build abstractions around HTTP requests using Fetch Dedupe under the hood.
Are you using it on a project? Add it to this list by opening a Pull Request
Apollo inspired me to write this library.
v2.0.0 (2018/2/4)
Breaking
dedupeOptions
is now optional. The responseType
is "json"
by default, unless the
status code is 204, in which case it will be "text"
.FAQs
A thin wrapper around fetch that prevents duplicate requests.
The npm package fetch-dedupe receives a total of 5,834 weekly downloads. As such, fetch-dedupe popularity was classified as popular.
We found that fetch-dedupe demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.