Security News
ESLint is Now Language-Agnostic: Linting JSON, Markdown, and Beyond
ESLint has added JSON and Markdown linting support with new officially-supported plugins, expanding its versatility beyond JavaScript.
make-fetch-happen
Advanced tools
The make-fetch-happen npm package is a wrapper around the node-fetch package which provides additional features such as caching, retries, proxy support, and more. It is designed to make HTTP requests in Node.js environments more robust and feature-rich.
Caching
This feature allows make-fetch-happen to cache responses locally, which can be reused for future requests to the same resource, saving bandwidth and time.
const fetch = require('make-fetch-happen').defaults({
cacheManager: './my-cache' // path where cache will be stored
});
fetch('https://example.com').then(response => response.json()).then(data => console.log(data));
Retries
This feature enables automatic retries of failed requests, with customizable settings for the number of retries, delay strategy, and more.
const fetch = require('make-fetch-happen').defaults({
retry: {
retries: 3, // maximum amount of retries
factor: 2, // the exponential factor for delay between retries
minTimeout: 1000 // the number of milliseconds before starting the first retry
}
});
fetch('https://example.com').then(response => response.json()).then(data => console.log(data));
Proxy Support
This feature allows requests to be made through a specified HTTP or HTTPS proxy.
const fetch = require('make-fetch-happen').defaults({
proxy: 'http://myproxy.com:8080'
});
fetch('https://example.com').then(response => response.json()).then(data => console.log(data));
Timeouts
This feature allows setting a maximum time to wait for a response before aborting the request.
const fetch = require('make-fetch-happen').defaults({
timeout: 5000 // time in milliseconds
});
fetch('https://example.com').then(response => response.json()).then(data => console.log(data));
Axios is a popular HTTP client for the browser and Node.js. It supports promise-based API, interceptors, request and response transformations, and automatic transforms for JSON data. Compared to make-fetch-happen, Axios has a larger community and more extensive documentation but does not have built-in caching or offline support.
Got is a human-friendly and powerful HTTP request library for Node.js. It features stream support, promise-based API, and advanced retrying, among other things. Got is comparable to make-fetch-happen in terms of retrying and stream support but differs in its API design and plugin system.
node-fetch is a light-weight module that brings the Fetch API to Node.js. It is a minimalistic and straightforward implementation of the standard without additional features like caching or retries. make-fetch-happen is built on top of node-fetch, adding more advanced features on top of the basic functionality provided by node-fetch.
Superagent is a small progressive client-side HTTP request library, and Node.js module with the same API, sporting many high-level HTTP client features. It is known for its fluent API and chaining capabilities. While it offers features like retries and plugins, it does not have built-in caching like make-fetch-happen.
make-fetch-happen
is a Node.js
library that wraps minipass-fetch
with additional
features minipass-fetch
doesn't intend to include, including HTTP Cache support, request
pooling, proxies, retries, and more!
$ npm install --save make-fetch-happen
const fetch = require('make-fetch-happen').defaults({
cachePath: './my-cache' // path where cache will be written (and read)
})
fetch('https://registry.npmjs.org/make-fetch-happen').then(res => {
return res.json() // download the body as JSON
}).then(body => {
console.log(`got ${body.name} from web`)
return fetch('https://registry.npmjs.org/make-fetch-happen', {
cache: 'no-cache' // forces a conditional request
})
}).then(res => {
console.log(res.status) // 304! cache validated!
return res.json().then(body => {
console.log(`got ${body.name} from cache`)
})
})
minipass-fetch
for the core fetch
API implementationCache-Control
, ETag
, 304
s, cache fallback on error, etc).@npmcli/agent
)@npmcli/agent
)> fetch(uriOrRequest, [opts]) -> Promise<Response>
This function implements most of the fetch
API: given a uri
string or a Request
instance, it will fire off an http request and return a Promise containing the relevant response.
If opts
is provided, the minipass-fetch
-specific options will be passed to that library. There are also additional options specific to make-fetch-happen that add various features, such as HTTP caching, integrity verification, proxy support, and more.
fetch('https://google.com').then(res => res.buffer())
> fetch.defaults([defaultUrl], [defaultOpts])
Returns a new fetch
function that will call make-fetch-happen
using defaultUrl
and defaultOpts
as default values to any calls.
A defaulted fetch
will also have a .defaults()
method, so they can be chained.
const fetch = require('make-fetch-happen').defaults({
cachePath: './my-local-cache'
})
fetch('https://registry.npmjs.org/make-fetch-happen') // will always use the cache
> minipass-fetch options
The following options for minipass-fetch
are used as-is:
These other options are modified or augmented by make-fetch-happen:
User-Agent
set to make-fetch happen. Connection
is set to keep-alive
or close
automatically depending on opts.agent
.For more details, see the documentation for minipass-fetch
itself.
> make-fetch-happen options
make-fetch-happen augments the minipass-fetch
API with additional features available through extra options. The following extra options are available:
opts.cachePath
- Cache target to read/writeopts.cache
- fetch
cache mode. Controls cache behavior.opts.cacheAdditionalHeaders
- Store additional headers in the cacheopts.proxy
- Proxy agentopts.noProxy
- Domain segments to disable proxying for.opts.ca, opts.cert, opts.key, opts.strictSSL
opts.localAddress
opts.maxSockets
opts.retry
- Request retry settingsopts.onRetry
- a function called whenever a retry is attemptedopts.integrity
- Subresource Integrity metadata.opts.dns
- DNS cache optionsopts.agent
- http/https/proxy/socks agent options. See @npmcli/agent
for more info.> opts.cachePath
A string Path
to be used as the cache root for cacache
.
NOTE: Requests will not be cached unless their response bodies are consumed. You will need to use one of the res.json()
, res.buffer()
, etc methods on the response, or drain the res.body
stream, in order for it to be written.
The default cache manager also adds the following headers to cached responses:
X-Local-Cache
: Path to the cache the content was found inX-Local-Cache-Key
: Unique cache entry key for this responseX-Local-Cache-Mode
: Always stream
to indicate how the response was read from cacacheX-Local-Cache-Hash
: Specific integrity hash for the cached entryX-Local-Cache-Status
: One of miss
, hit
, stale
, revalidated
, updated
, or skip
to signal how the response was createdX-Local-Cache-Time
: UTCString of the cache insertion time for the entryUsing cacache
, a call like this may be used to
manually fetch the cached entry:
const h = response.headers
cacache.get(h.get('x-local-cache'), h.get('x-local-cache-key'))
// grab content only, directly:
cacache.get.byDigest(h.get('x-local-cache'), h.get('x-local-cache-hash'))
fetch('https://registry.npmjs.org/make-fetch-happen', {
cachePath: './my-local-cache'
}) // -> 200-level response will be written to disk
> opts.cache
This option follows the standard fetch
API cache option. This option will do nothing if opts.cachePath
is null. The following values are accepted (as strings):
default
- Fetch will inspect the HTTP cache on the way to the network. If there is a fresh response it will be used. If there is a stale response a conditional request will be created, and a normal request otherwise. It then updates the HTTP cache with the response. If the revalidation request fails (for example, on a 500 or if you're offline), the stale response will be returned.no-store
- Fetch behaves as if there is no HTTP cache at all.reload
- Fetch behaves as if there is no HTTP cache on the way to the network. Ergo, it creates a normal request and updates the HTTP cache with the response.no-cache
- Fetch creates a conditional request if there is a response in the HTTP cache and a normal request otherwise. It then updates the HTTP cache with the response.force-cache
- Fetch uses any response in the HTTP cache matching the request, not paying attention to staleness. If there was no response, it creates a normal request and updates the HTTP cache with the response.only-if-cached
- Fetch uses any response in the HTTP cache matching the request, not paying attention to staleness. If there was no response, it returns a network error. (Can only be used when request’s mode is "same-origin". Any cached redirects will be followed assuming request’s redirect mode is "follow" and the redirects do not violate request’s mode.)(Note: option descriptions are taken from https://fetch.spec.whatwg.org/#http-network-or-cache-fetch)
const fetch = require('make-fetch-happen').defaults({
cachePath: './my-cache'
})
// Will error with ENOTCACHED if we haven't already cached this url
fetch('https://registry.npmjs.org/make-fetch-happen', {
cache: 'only-if-cached'
})
// Will refresh any local content and cache the new response
fetch('https://registry.npmjs.org/make-fetch-happen', {
cache: 'reload'
})
// Will use any local data, even if stale. Otherwise, will hit network.
fetch('https://registry.npmjs.org/make-fetch-happen', {
cache: 'force-cache'
})
> opts.cacheAdditionalHeaders
The following headers are always stored in the cache when present:
cache-control
content-encoding
content-language
content-type
date
etag
expires
last-modified
link
location
pragma
vary
This option allows a user to store additional custom headers in the cache.
fetch('https://registry.npmjs.org/make-fetch-happen', {
cacheAdditionalHeaders: ['my-custom-header'],
})
> opts.proxy
A string or new url.URL()
-d URI to proxy through. Different Proxy handlers will be
used depending on the proxy's protocol.
Additionally, process.env.HTTP_PROXY
, process.env.HTTPS_PROXY
, and
process.env.PROXY
are used if present and no opts.proxy
value is provided.
(Pending) process.env.NO_PROXY
may also be configured to skip proxying requests for all, or specific domains.
fetch('https://registry.npmjs.org/make-fetch-happen', {
proxy: 'https://corporate.yourcompany.proxy:4445'
})
fetch('https://registry.npmjs.org/make-fetch-happen', {
proxy: {
protocol: 'https:',
hostname: 'corporate.yourcompany.proxy',
port: 4445
}
})
> opts.noProxy
If present, should be a comma-separated string or an array of domain extensions that a proxy should not be used for.
This option may also be provided through process.env.NO_PROXY
.
> opts.ca, opts.cert, opts.key, opts.strictSSL
These values are passed in directly to the HTTPS agent and will be used for both
proxied and unproxied outgoing HTTPS requests. They mostly correspond to the
same options the https
module accepts, which will be themselves passed to
tls.connect()
. opts.strictSSL
corresponds to rejectUnauthorized
.
> opts.localAddress
Passed directly to http
and https
request calls. Determines the local
address to bind to.
> opts.maxSockets
Default: 15
Maximum number of active concurrent sockets to use for the underlying Http/Https/Proxy agents. This setting applies once per spawned agent.
15 is probably a pretty good value for most use-cases, and balances speed with, uh, not knocking out people's routers. 🤓
> opts.retry
An object that can be used to tune request retry settings. Retries will only be attempted on the following conditions:
POST
AND408
, 420
, 429
, or any status in the 500-range. ORECONNRESET
, ECONNREFUSED
, EADDRINUSE
, ETIMEDOUT
, or the fetch
error request-timeout
.The following are worth noting as explicitly not retried:
getaddrinfo ENOTFOUND
and will be assumed to be either an unreachable domain or the user will be assumed offline. If a response is cached, it will be returned immediately.If opts.retry
is false
, it is equivalent to {retries: 0}
If opts.retry
is a number, it is equivalent to {retries: num}
The following retry options are available if you want more control over it:
For details on what each of these do, refer to the retry
documentation.
fetch('https://flaky.site.com', {
retry: {
retries: 10,
randomize: true
}
})
fetch('http://reliable.site.com', {
retry: false
})
fetch('http://one-more.site.com', {
retry: 3
})
> opts.onRetry
A function called with the response or error which caused the retry whenever one is attempted.
fetch('https://flaky.site.com', {
onRetry(cause) {
console.log('we will retry because of', cause)
}
})
> opts.integrity
Matches the response body against the given Subresource Integrity metadata. If verification fails, the request will fail with an EINTEGRITY
error.
integrity
may either be a string or an ssri
Integrity
-like.
fetch('https://registry.npmjs.org/make-fetch-happen/-/make-fetch-happen-1.0.0.tgz', {
integrity: 'sha1-o47j7zAYnedYFn1dF/fR9OV3z8Q='
}) // -> ok
fetch('https://malicious-registry.org/make-fetch-happen/-/make-fetch-happen-1.0.0.tgz', {
integrity: 'sha1-o47j7zAYnedYFn1dF/fR9OV3z8Q='
}) // Error: EINTEGRITY
FAQs
Opinionated, caching, retrying fetch client
The npm package make-fetch-happen receives a total of 15,438,530 weekly downloads. As such, make-fetch-happen popularity was classified as popular.
We found that make-fetch-happen demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 6 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
ESLint has added JSON and Markdown linting support with new officially-supported plugins, expanding its versatility beyond JavaScript.
Security News
Members Hub is conducting large-scale campaigns to artificially boost Discord server metrics, undermining community trust and platform integrity.
Security News
NIST has failed to meet its self-imposed deadline of clearing the NVD's backlog by the end of the fiscal year. Meanwhile, CVE's awaiting analysis have increased by 33% since June.