Issue detection

We automatically detect npm package issues for you.

Integrations

We integrate with all of the tools you use.

Blog

Keep up to date with all the news.

Love

See why developers love Socket.

@adobe/helix-fetch

Advanced tools

npm
@adobe/helix-fetch

@adobe/helix-fetch

Light-weight Fetch implementation transparently supporting both HTTP/1(.1) and HTTP/2

3.1.2

latest

GitHub

npm

Version published: 8 months ago

Maintainers: 21

Weekly downloads: 976; increased by34.25%

Weekly downloads

Changelog

Source

3.1.2 (2022-08-07)

Bug Fixes

deps: update dependency lru-cache to v7.13.2 (#298) (47e0290)

Readme

Source

Light-weight Fetch API implementation transparently supporting both HTTP/1(.1) and HTTP/2

About
Features
Installation
Upgrading
API
- Context
Common Usage Examples
Advanced Usage Examples
More examples
Development
Acknowledgement
License

About

helix-fetch in general adheres to the Fetch API Specification, implementing a subset of the API. However, there are some notable deviations:

Response.body returns a Node.js Readable stream.
Response.blob() is not implemented. Use Response.buffer() instead.
Response.formData() is not implemented.
Cookies are not stored by default. However, cookies can be extracted and passed by manipulating request and response headers.
The following values of the fetch() option cache are supported: 'default' (the implicit default) and 'no-store'. All other values are currently ignored.
The following fetch() options are ignored due to the nature of Node.js and since helix-fetch doesn't have the concept of web pages: mode, referrer, referrerPolicy, integrity and credentials.
The fetch() option keepalive is not supported. But you can use the h1.keepAlive context option, as demonstrated here.

helix-fetch also supports the following extensions:

Response.buffer() returns a Node.js Buffer.
Response.url contains the final url when following redirects.
The body that can be sent in a Request can also be a Readable Node.js stream, a Buffer, a string or a plain object.
There are no forbidden header names.
The Response object has an extra property httpVersion which is one of '1.0', '1.1' or '2.0', depending on what was negotiated with the server.
The Response object has an extra property fromCache which determines whether the response was retrieved from cache.
The Response object has an extra property decoded which determines whether the response body was automatically decoded (see Fetch option decode below).
Response.headers.plain() returns the headers as a plain object.
The Fetch option follow allows to limit the number of redirects to follow (default: 20).
The Fetch option compress enables transparent gzip/deflate/br content encoding (default: true).
The Fetch option decode enables transparent gzip/deflate/br content decoding (default: true).

Note that non-standard Fetch options have been aligned with node-fetch where appropriate.

Features

supports reasonable subset of the standard Fetch specification
Transparent handling of HTTP/1(.1) and HTTP/2 connections
RFC 7234 compliant cache
Support gzip/deflate/br content encoding
HTTP/2 request and response multiplexing support
HTTP/2 Server Push support (transparent caching and explicit listener support)
overridable User-Agent
low-level HTTP/1.* agent/connect options support (e.g. keepAlive, rejectUnauthorized)

Installation

Note:

As of v2 Node version >= 12 is required.

$ npm install @adobe/helix-fetch

Upgrading

Upgrading from an old version of helix-fetch? Check out the following files:

1.x to 2.x Upgrade Guide
Changelog

API

Apart from the standard Fetch API

fetch()
Request
Response
Headers
Body

helix-fetch exposes the following extensions:

context() - creates a new customized API context
reset() - resets the current API context, i.e. closes pending sessions/sockets, clears internal caches, etc ...
onPush() - registers an HTTP/2 Server Push listener
offPush()- deregisters a listener previously registered with onPush()
clearCache() - clears the HTTP cache (cached responses)
cacheStats() - returns cache statistics
noCache() - creates a customized API context with disabled caching (convenience)
h1() - creates a customized API context with enforced HTTP/1.1 protocol (convenience)
keepAlive() - creates a customized API context with enforced HTTP/1.1 protocol and persistent connections (convenience)
h1NoCache() - creates a customized API context with disabled caching and enforced HTTP/1.1 protocol (convenience)
keepAliveNoCache() - creates a customized API context with disabled caching and enforced HTTP/1.1 protocol with persistent connections (convenience)
createUrl() - creates a URL with query parameters (convenience)
timeoutSignal() - ceates a timeout signal (convenience)

Context

An API context allows to customize certain aspects of the implementation and provides isolation of internal structures (session caches, HTTP cache, etc.) per API context.

The following options are supported:

interface ContextOptions {
  /**
   * Value of `user-agent` request header
   * @default 'helix-fetch/<version>'
   */
  userAgent?: string;
  /**
   * The maximum total size of the cached entries (in bytes). 0 disables caching.
   * @default 100 * 1024 * 1024
   */
  maxCacheSize?: number;
  /**
   * The protocols to be negotiated, in order of preference
   * @default [ALPN_HTTP2, ALPN_HTTP1_1, ALPN_HTTP1_0]
   */
  alpnProtocols?: ReadonlyArray< ALPNProtocol >;
  /**
   * How long (in milliseconds) should ALPN information be cached for a given host?
   * @default 60 * 60 * 1000
   */
  alpnCacheTTL?: number;
  /**
   * (HTTPS only, applies to HTTP/1.x and HTTP/2)
   * If not false, the server certificate is verified against the list of supplied CAs. An 'error' event is emitted if verification fails; err.code contains the OpenSSL error code.
   * @default true
   */
  rejectUnauthorized?: boolean;
  /**
   * Maximum number of ALPN cache entries
   * @default 100
   */
  alpnCacheSize?: number;
  h1?: Http1Options;
  h2?: Http2Options;
};

interface Http1Options {
  /**
   * Keep sockets around in a pool to be used by other requests in the future.
   * @default false
   */
  keepAlive?: boolean;
  /**
   * When using HTTP KeepAlive, how often to send TCP KeepAlive packets over sockets being kept alive.
   * Only relevant if keepAlive is set to true.
   * @default 1000
   */
  keepAliveMsecs?: number;
  /**
   * (HTTPS only)
   * If not false, the server certificate is verified against the list of supplied CAs. An 'error' event is emitted if verification fails; err.code contains the OpenSSL error code.
   * @default true
   */
  rejectUnauthorized?: boolean;
  /**
   * (HTTPS only)
   * Maximum number of TLS cached sessions. Use 0 to disable TLS session caching.
   * @default 100
   */
  maxCachedSessions?: number;
}

interface Http2Options {
  /**
   * Max idle time in milliseconds after which a session will be automatically closed. 
   * @default 5 * 60 * 1000
   */
  idleSessionTimeout?: number;
  /**
   * Enable HTTP/2 Server Push?
   * @default true
   */
  enablePush?: boolean;
  /**
   * Max idle time in milliseconds after which a pushed stream will be automatically closed. 
   * @default 5000
   */
  pushedStreamIdleTimeout?: number;
  /**
   * (HTTPS only)
   * If not false, the server certificate is verified against the list of supplied CAs. An 'error' event is emitted if verification fails; err.code contains the OpenSSL error code.
   * @default true
   */
  rejectUnauthorized?: boolean;
};

Common Usage Examples

Access Response Headers and other Meta data

  const { fetch } = require('@adobe/helix-fetch');

  const resp = await fetch('https://httpbin.org/get');
  console.log(resp.ok);
  console.log(resp.status);
  console.log(resp.statusText);
  console.log(resp.httpVersion);
  console.log(resp.headers.plain());
  console.log(resp.headers.get('content-type'));

Fetch JSON

  const { fetch } = require('@adobe/helix-fetch');

  const resp = await fetch('https://httpbin.org/json');
  const jsonData = await resp.json();

Fetch text data

  const { fetch } = require('@adobe/helix-fetch');

  const resp = await fetch('https://httpbin.org/');
  const textData = await resp.text();

Fetch binary data

  const { fetch } = require('@adobe/helix-fetch');

  const resp = await fetch('https://httpbin.org//stream-bytes/65535');
  const imageData = await resp.buffer();

Specify a timeout for a `fetch` operation

Using timeoutSignal(ms) extension:

  const { fetch, timeoutSignal, AbortError } = require('@adobe/helix-fetch');

  const signal = timeoutSignal(1000);
  try {
    const resp = await fetch('https://httpbin.org/json', { signal });
    const jsonData = await resp.json();
  } catch (err) {
    if (err instanceof AbortError) {
      console.log('fetch timed out after 1s');
    }
  } finally {
    // avoid pending timers which prevent node process from exiting
    signal.clear();
  }

Using AbortController:

  const { fetch, AbortController, AbortError } = require('@adobe/helix-fetch');

  const controller = new AbortController();
  const timerId = setTimeout(() => controller.abort(), 1000);
  const { signal } = controller;

  try {
    const resp = await fetch('https://httpbin.org/json', { signal });
    const jsonData = await resp.json();
  } catch (err) {
    if (err instanceof AbortError) {
      console.log('fetch timed out after 1s');
    }
  } finally {
    // avoid pending timers which prevent node process from exiting
    clearTimeout(timerId);
  }

Stream an image

  const fs = require('fs');
  const { fetch } = require('@adobe/helix-fetch');

  const resp = await fetch('https://httpbin.org/image/jpeg');
  resp.body.pipe(fs.createWriteStream('saved-image.jpg'));

Post JSON

  const { fetch } = require('@adobe/helix-fetch');

  const method = 'POST';
  const body = { foo: 'bar' };
  const resp = await fetch('https://httpbin.org/post', { method, body });

Post JPEG image

  const fs = require('fs');
  const { fetch } = require('@adobe/helix-fetch');

  const method = 'POST';
  const body = fs.createReadStream('some-image.jpg');
  const headers = { 'content-type': 'image/jpeg' };
  const resp = await fetch('https://httpbin.org/post', { method, body, headers });

Post form data

  const { FormData, Blob, File } = require('formdata-node'); // spec-compliant implementations
  const { fileFromPath } = require('formdata-node/file-from-path'); // helper for creating File instance from disk file

  const { fetch } = require('@adobe/helix-fetch');

  const method = 'POST';
  const fd = new FormData();
  fd.set('field1', 'foo');
  fd.set('field2', 'bar');
  fd.set('blob', new Blob([0x68, 0x65, 0x6c, 0x69, 0x78, 0x2d, 0x66, 0x65, 0x74, 0x63, 0x68]));
  fd.set('file', new File(['File content goes here'], 'file.txt'));
  fd.set('other_file', await fileFromPath('/foo/bar.jpg', 'bar.jpg', { type: 'image/jpeg' }));
  const resp = await fetch('https://httpbin.org/post', { method, body: fd });

GET with query parameters object

const { createUrl, fetch } = require('@adobe/helix-fetch');

const qs = {
  helix: 'dummy',
  foo: 'bar',
  rumple: "stiltskin",
};

const resp = await fetch(createUrl('https://httpbin.org/json', qs));

or using URLSearchParams:

const { fetch } = require('@adobe/helix-fetch');

const body = new URLSearchParams({
  helix: 'dummy',
  foo: 'bar',
  rumple: "stiltskin",
});

const resp = await fetch('https://httpbin.org/json', { body });

Cache

Responses of GET and HEAD requests are by default cached, according to the rules of RFC 7234:

const { fetch } = require('@adobe/helix-fetch');

const url = 'https://httpbin.org/cache/60'; // -> max-age=60 (seconds)
// send initial request, priming cache
let resp = await fetch(url);
assert(resp.ok);
assert(!resp.fromCache);

// re-send request and verify it's served from cache
resp = await fetch(url);
assert(resp.ok);
assert(resp.fromCache);

You can disable caching per request with the cache: 'no-store' option:

const { fetch } = require('@adobe/helix-fetch');

const resp = await fetch('https://httbin.org/', { cache: 'no-store' });
assert(resp.ok);
assert(!resp.fromCache);

You can disable caching entirely:

const { fetch } = require('@adobe/helix-fetch').noCache();

Advanced Usage Examples

HTTP/2 Server Push

Note that pushed resources will be automatically and transparently added to the cache. You can however add a listener which will be notified on every pushed (and cached) resource.

  const { fetch, onPush } = require('@adobe/helix-fetch');

  onPush((url, response) => console.log(`received server push: ${url} status ${response.status}`));

  const resp = await fetch('https://nghttp2.org');
  console.log(`Http version: ${resp.httpVersion}`);

Force HTTP/1(.1) protocol

  const { fetch } = require('@adobe/helix-fetch').h1();

  const resp = await fetch('https://nghttp2.org');
  console.log(`Http version: ${resp.httpVersion}`);

HTTP/1.1 Keep-Alive

const { fetch } = require('@adobe/helix-fetch').keepAlive();

const resp = await fetch('https://httpbin.org/status/200');
console.log(`Connection: ${resp.headers.get('connection')}`); // -> keep-alive

Self-signed Certificates

const { fetch } = require('@adobe/helix-fetch').context({ rejectUnauthorized: false });

const resp = await fetch('https://localhost:8443/');  // a server using a self-signed certificate

Set cache size limit

  const { fetch, cacheStats } = require('@adobe/helix-fetch').context({
    maxCacheSize: 100 * 1024, // 100kb (Default: 100mb)
  });

  let resp = await fetch('https://httpbin.org/bytes/60000'); // ~60kb response
  resp = await fetch('https://httpbin.org/bytes/50000'); // ~50kb response
  console.log(cacheStats());

Disable caching

  const { fetch } = require('@adobe/helix-fetch').noCache();

  let resp = await fetch('https://httpbin.org/cache/60'); // -> max-age=60 (seconds)
  // re-fetch
  resp = await fetch('https://httpbin.org/cache/60');
  assert(!resp.fromCache);

Set a custom user agent

  const { fetch } = require('@adobe/helix-fetch').context({
    userAgent: 'custom-fetch'
  });

  const resp = await fetch('https://httpbin.org//user-agent');
  const json = await resp.json();
  console.log(json['user-agent']);

More examples

More example code can be found in the test source files.

Development

Build

$ npm install

Test

$ npm test

Lint

$ npm run lint

Troubleshooting

You can enable helix-fetch low-level debug console output by setting the DEBUG environment variable to helix-fetch*, e.g.:

$ DEBUG=helix-fetch* node test.js

This will produce console outout similar to:

  ...
  helix-fetch:core established TLS connection: #48 (www.nghttp2.org) +2s
  helix-fetch:core www.nghttp2.org -> h2 +0ms
  helix-fetch:h2 reusing socket #48 (www.nghttp2.org) +2s
  helix-fetch:h2 GET www.nghttp2.org/httpbin/user-agent +0ms
  helix-fetch:h2 session https://www.nghttp2.org established +1ms
  helix-fetch:h2 caching session https://www.nghttp2.org +0ms
  helix-fetch:h2 session https://www.nghttp2.org remoteSettings: {"headerTableSize":8192,"enablePush":true,"initialWindowSize":1048576,"maxFrameSize":16384,"maxConcurrentStreams":100,"maxHeaderListSize":4294967295,"maxHeaderSize":4294967295,"enableConnectProtocol":true} +263ms
  helix-fetch:h2 session https://www.nghttp2.org localSettings: {"headerTableSize":4096,"enablePush":true,"initialWindowSize":65535,"maxFrameSize":16384,"maxConcurrentStreams":4294967295,"maxHeaderListSize":4294967295,"maxHeaderSize":4294967295,"enableConnectProtocol":false} +0ms
  helix-fetch:h2 session https://www.nghttp2.org closed +6ms
  helix-fetch:h2 discarding cached session https://www.nghttp2.org +0ms
  ...

Additionally, you can enable Node.js low-level debug console output by setting the NODE_DEBUG environment variable appropriately, e.g.

$ export NODE_DEBUG=http*,stream*
$ export DEBUG=helix-fetch*

$ node test.js

Note: this will flood the console with highly verbose debug output.

Acknowledgement

Thanks to node-fetch and github/fetch for providing a solid implementation reference.

License

Apache 2.0

Keywords

fetch
whatwg
Fetch API
http
https
http2

FAQs

What is @adobe/helix-fetch?

Light-weight Fetch implementation transparently supporting both HTTP/1(.1) and HTTP/2

Is @adobe/helix-fetch popular?

The npm package @adobe/helix-fetch receives a total of 870 weekly downloads. As such, @adobe/helix-fetch popularity was classified as not popular.

Is @adobe/helix-fetch well maintained?

We found that @adobe/helix-fetch demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 21 open source maintainers collaborating on the project.

Last updated on 07 Aug 2022

Did you know?

Socket installs a Github app to automatically flag issues on every pull request and report the health of your dependencies. Find out what is inside your node modules and prevent malicious activity before you update the dependencies.

Install Socket

support@socket.dev

Product

Package Issues
Integrations
Docs
Pricing
FAQ
Roadmap

About

About
Love
Blog
Discord Community
JobsHiring
Send Feedback
Contact Us
System Status

Packages

npm Package Directory
Explore npm Packages
Maven Package Directory
Explore Maven Packages
PyPi Package Directory
Explore PyPi Packages
Most Popular JS Packages
Top JavaScript Maintainers
Removed npm Packages
Random npm Package

Stay in touch

Get open source security insights delivered straight into your inbox.

Enter your email

Terms

Privacy

Security

Made with ⚡️ by Socket Inc

@adobe/helix-fetch

Bug Fixes

About

Features

Installation

Upgrading

API

Context

Common Usage Examples

Access Response Headers and other Meta data

Fetch JSON

Fetch text data

Fetch binary data

Specify a timeout for a fetch operation

Stream an image

Post JSON

Post JPEG image

Post form data

GET with query parameters object

Cache

Advanced Usage Examples

HTTP/2 Server Push

Force HTTP/1(.1) protocol

HTTP/1.1 Keep-Alive

Self-signed Certificates

Set cache size limit

Disable caching

Set a custom user agent

More examples

Development

Build

Test

Lint

Troubleshooting

Acknowledgement

License

Keywords

Specify a timeout for a `fetch` operation