Socket
Socket
Sign inDemoInstall

make-fetch-happen

Package Overview
Dependencies
64
Maintainers
5
Versions
104
Alerts
File Explorer

Advanced tools

Install Socket

Detect and block malicious and high-risk dependencies

Install

make-fetch-happen

Opinionated, caching, retrying fetch client


Version published
Maintainers
5
Weekly downloads
17,521,230
decreased by-8.93%

Weekly downloads

Package description

What is make-fetch-happen?

The make-fetch-happen npm package is a wrapper around the node-fetch package which provides additional features such as caching, retries, proxy support, and more. It is designed to make HTTP requests in Node.js environments more robust and feature-rich.

What are make-fetch-happen's main functionalities?

Caching

This feature allows make-fetch-happen to cache responses locally, which can be reused for future requests to the same resource, saving bandwidth and time.

const fetch = require('make-fetch-happen').defaults({
  cacheManager: './my-cache' // path where cache will be stored
});

fetch('https://example.com').then(response => response.json()).then(data => console.log(data));

Retries

This feature enables automatic retries of failed requests, with customizable settings for the number of retries, delay strategy, and more.

const fetch = require('make-fetch-happen').defaults({
  retry: {
    retries: 3, // maximum amount of retries
    factor: 2, // the exponential factor for delay between retries
    minTimeout: 1000 // the number of milliseconds before starting the first retry
  }
});

fetch('https://example.com').then(response => response.json()).then(data => console.log(data));

Proxy Support

This feature allows requests to be made through a specified HTTP or HTTPS proxy.

const fetch = require('make-fetch-happen').defaults({
  proxy: 'http://myproxy.com:8080'
});

fetch('https://example.com').then(response => response.json()).then(data => console.log(data));

Timeouts

This feature allows setting a maximum time to wait for a response before aborting the request.

const fetch = require('make-fetch-happen').defaults({
  timeout: 5000 // time in milliseconds
});

fetch('https://example.com').then(response => response.json()).then(data => console.log(data));

Other packages similar to make-fetch-happen

Readme

Source

make-fetch-happen

npm version license Travis Coverage Status

make-fetch-happen is a Node.js library that wraps minipass-fetch with additional features minipass-fetch doesn't intend to include, including HTTP Cache support, request pooling, proxies, retries, and more!

Install

$ npm install --save make-fetch-happen

Table of Contents

Example

const fetch = require('make-fetch-happen').defaults({
  cachePath: './my-cache' // path where cache will be written (and read)
})

fetch('https://registry.npmjs.org/make-fetch-happen').then(res => {
  return res.json() // download the body as JSON
}).then(body => {
  console.log(`got ${body.name} from web`)
  return fetch('https://registry.npmjs.org/make-fetch-happen', {
    cache: 'no-cache' // forces a conditional request
  })
}).then(res => {
  console.log(res.status) // 304! cache validated!
  return res.json().then(body => {
    console.log(`got ${body.name} from cache`)
  })
})

Features

  • Builds around minipass-fetch for the core fetch API implementation
  • Request pooling out of the box
  • Quite fast, really
  • Automatic HTTP-semantics-aware request retries
  • Cache-fallback automatic "offline mode"
  • Proxy support (http, https, socks, socks4, socks5)
  • Built-in request caching following full HTTP caching rules (Cache-Control, ETag, 304s, cache fallback on error, etc).
  • Node.js Stream support
  • Transparent gzip and deflate support
  • Subresource Integrity support
  • Literally punches nazis
  • Built in DNS cache
  • (PENDING) Range request caching and resuming

Contributing

The make-fetch-happen team enthusiastically welcomes contributions and project participation! There's a bunch of things you can do if you want to contribute! The Contributor Guide outlines the process for community interaction and contribution. Please don't hesitate to jump in if you'd like to, or even ask us questions if something isn't clear.

All participants and maintainers in this project are expected to follow the npm Code of Conduct, and just generally be excellent to each other.

Please refer to the Changelog for project history details, too.

Happy hacking!

API

> fetch(uriOrRequest, [opts]) -> Promise<Response>

This function implements most of the fetch API: given a uri string or a Request instance, it will fire off an http request and return a Promise containing the relevant response.

If opts is provided, the minipass-fetch-specific options will be passed to that library. There are also additional options specific to make-fetch-happen that add various features, such as HTTP caching, integrity verification, proxy support, and more.

Example
fetch('https://google.com').then(res => res.buffer())
> fetch.defaults([defaultUrl], [defaultOpts])

Returns a new fetch function that will call make-fetch-happen using defaultUrl and defaultOpts as default values to any calls.

A defaulted fetch will also have a .defaults() method, so they can be chained.

Example
const fetch = require('make-fetch-happen').defaults({
  cachePath: './my-local-cache'
})

fetch('https://registry.npmjs.org/make-fetch-happen') // will always use the cache
> minipass-fetch options

The following options for minipass-fetch are used as-is:

  • method
  • body
  • redirect
  • follow
  • timeout
  • compress
  • size

These other options are modified or augmented by make-fetch-happen:

  • headers - Default User-Agent set to make-fetch happen. Connection is set to keep-alive or close automatically depending on opts.agent.
  • agent
    • If agent is null, an http or https Agent will be automatically used. By default, these will be http.globalAgent and https.globalAgent.
    • If opts.proxy is provided and opts.agent is null, the agent will be set to an appropriate proxy-handling agent.
    • If opts.agent is an object, it will be used as the request-pooling agent argument for this request.
    • If opts.agent is false, it will be passed as-is to the underlying request library. This causes a new Agent to be spawned for every request.

For more details, see the documentation for minipass-fetch itself.

> make-fetch-happen options

make-fetch-happen augments the minipass-fetch API with additional features available through extra options. The following extra options are available:

> opts.cachePath

A string Path to be used as the cache root for cacache.

NOTE: Requests will not be cached unless their response bodies are consumed. You will need to use one of the res.json(), res.buffer(), etc methods on the response, or drain the res.body stream, in order for it to be written.

The default cache manager also adds the following headers to cached responses:

  • X-Local-Cache: Path to the cache the content was found in
  • X-Local-Cache-Key: Unique cache entry key for this response
  • X-Local-Cache-Mode: Always stream to indicate how the response was read from cacache
  • X-Local-Cache-Hash: Specific integrity hash for the cached entry
  • X-Local-Cache-Status: One of miss, hit, stale, revalidated, updated, or skip to signal how the response was created
  • X-Local-Cache-Time: UTCString of the cache insertion time for the entry

Using cacache, a call like this may be used to manually fetch the cached entry:

const h = response.headers
cacache.get(h.get('x-local-cache'), h.get('x-local-cache-key'))

// grab content only, directly:
cacache.get.byDigest(h.get('x-local-cache'), h.get('x-local-cache-hash'))
Example
fetch('https://registry.npmjs.org/make-fetch-happen', {
  cachePath: './my-local-cache'
}) // -> 200-level response will be written to disk
> opts.cache

This option follows the standard fetch API cache option. This option will do nothing if opts.cachePath is null. The following values are accepted (as strings):

  • default - Fetch will inspect the HTTP cache on the way to the network. If there is a fresh response it will be used. If there is a stale response a conditional request will be created, and a normal request otherwise. It then updates the HTTP cache with the response. If the revalidation request fails (for example, on a 500 or if you're offline), the stale response will be returned.
  • no-store - Fetch behaves as if there is no HTTP cache at all.
  • reload - Fetch behaves as if there is no HTTP cache on the way to the network. Ergo, it creates a normal request and updates the HTTP cache with the response.
  • no-cache - Fetch creates a conditional request if there is a response in the HTTP cache and a normal request otherwise. It then updates the HTTP cache with the response.
  • force-cache - Fetch uses any response in the HTTP cache matching the request, not paying attention to staleness. If there was no response, it creates a normal request and updates the HTTP cache with the response.
  • only-if-cached - Fetch uses any response in the HTTP cache matching the request, not paying attention to staleness. If there was no response, it returns a network error. (Can only be used when request’s mode is "same-origin". Any cached redirects will be followed assuming request’s redirect mode is "follow" and the redirects do not violate request’s mode.)

(Note: option descriptions are taken from https://fetch.spec.whatwg.org/#http-network-or-cache-fetch)

Example
const fetch = require('make-fetch-happen').defaults({
  cachePath: './my-cache'
})

// Will error with ENOTCACHED if we haven't already cached this url
fetch('https://registry.npmjs.org/make-fetch-happen', {
  cache: 'only-if-cached'
})

// Will refresh any local content and cache the new response
fetch('https://registry.npmjs.org/make-fetch-happen', {
  cache: 'reload'
})

// Will use any local data, even if stale. Otherwise, will hit network.
fetch('https://registry.npmjs.org/make-fetch-happen', {
  cache: 'force-cache'
})
> opts.proxy

A string or new url.URL()-d URI to proxy through. Different Proxy handlers will be used depending on the proxy's protocol.

Additionally, process.env.HTTP_PROXY, process.env.HTTPS_PROXY, and process.env.PROXY are used if present and no opts.proxy value is provided.

(Pending) process.env.NO_PROXY may also be configured to skip proxying requests for all, or specific domains.

Example
fetch('https://registry.npmjs.org/make-fetch-happen', {
  proxy: 'https://corporate.yourcompany.proxy:4445'
})

fetch('https://registry.npmjs.org/make-fetch-happen', {
  proxy: {
    protocol: 'https:',
    hostname: 'corporate.yourcompany.proxy',
    port: 4445
  }
})
> opts.noProxy

If present, should be a comma-separated string or an array of domain extensions that a proxy should not be used for.

This option may also be provided through process.env.NO_PROXY.

> opts.ca, opts.cert, opts.key, opts.strictSSL

These values are passed in directly to the HTTPS agent and will be used for both proxied and unproxied outgoing HTTPS requests. They mostly correspond to the same options the https module accepts, which will be themselves passed to tls.connect(). opts.strictSSL corresponds to rejectUnauthorized.

> opts.localAddress

Passed directly to http and https request calls. Determines the local address to bind to.

> opts.maxSockets

Default: 15

Maximum number of active concurrent sockets to use for the underlying Http/Https/Proxy agents. This setting applies once per spawned agent.

15 is probably a pretty good value for most use-cases, and balances speed with, uh, not knocking out people's routers. 🤓

> opts.retry

An object that can be used to tune request retry settings. Retries will only be attempted on the following conditions:

  • Request method is NOT POST AND
  • Request status is one of: 408, 420, 429, or any status in the 500-range. OR
  • Request errored with ECONNRESET, ECONNREFUSED, EADDRINUSE, ETIMEDOUT, or the fetch error request-timeout.

The following are worth noting as explicitly not retried:

  • getaddrinfo ENOTFOUND and will be assumed to be either an unreachable domain or the user will be assumed offline. If a response is cached, it will be returned immediately.

If opts.retry is false, it is equivalent to {retries: 0}

If opts.retry is a number, it is equivalent to {retries: num}

The following retry options are available if you want more control over it:

  • retries
  • factor
  • minTimeout
  • maxTimeout
  • randomize

For details on what each of these do, refer to the retry documentation.

Example
fetch('https://flaky.site.com', {
  retry: {
    retries: 10,
    randomize: true
  }
})

fetch('http://reliable.site.com', {
  retry: false
})

fetch('http://one-more.site.com', {
  retry: 3
})
> opts.onRetry

A function called with the response or error which caused the retry whenever one is attempted.

Example
fetch('https://flaky.site.com', {
  onRetry(cause) {
    console.log('we will retry because of', cause)
  }
})
> opts.integrity

Matches the response body against the given Subresource Integrity metadata. If verification fails, the request will fail with an EINTEGRITY error.

integrity may either be a string or an ssri Integrity-like.

Example
fetch('https://registry.npmjs.org/make-fetch-happen/-/make-fetch-happen-1.0.0.tgz', {
  integrity: 'sha1-o47j7zAYnedYFn1dF/fR9OV3z8Q='
}) // -> ok

fetch('https://malicious-registry.org/make-fetch-happen/-/make-fetch-happen-1.0.0.tgz', {
  integrity: 'sha1-o47j7zAYnedYFn1dF/fR9OV3z8Q='
}) // Error: EINTEGRITY
> opts.dns

An object that provides options for the built-in DNS cache. The following options are available:

Note: Due to limitations in the current proxy agent implementation, users of proxies will not benefit from the DNS cache.

  • ttl: Milliseconds to keep cached DNS responses for. Defaults to 5 * 60 * 1000 (5 minutes)
  • lookup: A custom lookup function, see dns.lookup() for implementation details. Defaults to require('dns').lookup.

Keywords

FAQs

Last updated on 17 Oct 2022

Did you know?

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc