@adobe/fetch
Advanced tools
Light-weight Fetch implementation transparently supporting both HTTP/1(.1) and HTTP/2
@adobe/fetch in general adheres to the Fetch API Specification, implementing a subset of the API. However, there are some notable deviations:
Response.body returns a Node.js Readable stream.Response.blob() is not implemented. Use Response.buffer() instead.Response.formData() is not implemented.fetch() option cache are supported: 'default' (the implicit default) and 'no-store'. All other values are currently ignored.fetch() options are ignored due to the nature of Node.js and since @adobe/fetch doesn't have the concept of web pages: mode, referrer, referrerPolicy, integrity and credentials.fetch() option keepalive is not supported. But you can use the h1.keepAlive context option, as demonstrated here.@adobe/fetch also supports the following extensions:
Response.buffer() returns a Node.js Buffer.Response.url contains the final url when following redirects.body that can be sent in a Request can also be a Readable Node.js stream, a Buffer, a string or a plain object.Response object has an extra property httpVersion which is one of '1.0', '1.1' or '2.0', depending on what was negotiated with the server.Response object has an extra property fromCache which determines whether the response was retrieved from cache.Response object has an extra property decoded which determines whether the response body was automatically decoded (see Fetch option decode below).Response.headers.plain() returns the headers as a plain object.follow allows to limit the number of redirects to follow (default: 20).compress enables transparent gzip/deflate/br content encoding (default: true).decode enables transparent gzip/deflate/br content decoding (default: true).Note that non-standard Fetch options have been aligned with node-fetch where appropriate.
gzip/deflate/br content encodingkeepAlive, rejectUnauthorized)Note:
As of v2 Node version >= 12 is required.
$ npm install @adobe/fetch
Apart from the standard Fetch API
fetch()RequestResponseHeadersBody@adobe/fetch exposes the following extensions:
context() - creates a new customized API contextreset() - resets the current API context, i.e. closes pending sessions/sockets, clears internal caches, etc ...onPush() - registers an HTTP/2 Server Push listeneroffPush()- deregisters a listener previously registered with onPush()clearCache() - clears the HTTP cache (cached responses)cacheStats() - returns cache statisticsnoCache() - creates a customized API context with disabled caching (convenience)h1() - creates a customized API context with enforced HTTP/1.1 protocol (convenience)keepAlive() - creates a customized API context with enforced HTTP/1.1 protocol and persistent connections (convenience)h1NoCache() - creates a customized API context with disabled caching and enforced HTTP/1.1 protocol (convenience)keepAliveNoCache() - creates a customized API context with disabled caching and enforced HTTP/1.1 protocol with persistent connections (convenience)createUrl() - creates a URL with query parameters (convenience)timeoutSignal() - ceates a timeout signal (convenience)An API context allows to customize certain aspects of the implementation and provides isolation of internal structures (session caches, HTTP cache, etc.) per API context.
The following options are supported:
interface ContextOptions {
/**
* Value of `user-agent` request header
* @default 'adobe-fetch/<version>'
*/
userAgent?: string;
/**
* The maximum total size of the cached entries (in bytes). 0 disables caching.
* @default 100 * 1024 * 1024
*/
maxCacheSize?: number;
/**
* The protocols to be negotiated, in order of preference
* @default [ALPN_HTTP2, ALPN_HTTP1_1, ALPN_HTTP1_0]
*/
alpnProtocols?: ReadonlyArray< ALPNProtocol >;
/**
* How long (in milliseconds) should ALPN information be cached for a given host?
* @default 60 * 60 * 1000
*/
alpnCacheTTL?: number;
/**
* (HTTPS only, applies to HTTP/1.x and HTTP/2)
* If not false, the server certificate is verified against the list of supplied CAs. An 'error' event is emitted if verification fails; err.code contains the OpenSSL error code.
* @default true
*/
rejectUnauthorized?: boolean;
/**
* Maximum number of ALPN cache entries
* @default 100
*/
alpnCacheSize?: number;
h1?: Http1Options;
h2?: Http2Options;
};
interface Http1Options {
/**
* Keep sockets around in a pool to be used by other requests in the future.
* @default false
*/
keepAlive?: boolean;
/**
* When using HTTP KeepAlive, how often to send TCP KeepAlive packets over sockets being kept alive.
* Only relevant if keepAlive is set to true.
* @default 1000
*/
keepAliveMsecs?: number;
/**
* (HTTPS only)
* If not false, the server certificate is verified against the list of supplied CAs. An 'error' event is emitted if verification fails; err.code contains the OpenSSL error code.
* @default true
*/
rejectUnauthorized?: boolean;
/**
* (HTTPS only)
* Maximum number of TLS cached sessions. Use 0 to disable TLS session caching.
* @default 100
*/
maxCachedSessions?: number;
}
interface Http2Options {
/**
* Max idle time in milliseconds after which a session will be automatically closed.
* @default 5 * 60 * 1000
*/
idleSessionTimeout?: number;
/**
* Enable HTTP/2 Server Push?
* @default true
*/
enablePush?: boolean;
/**
* Max idle time in milliseconds after which a pushed stream will be automatically closed.
* @default 5000
*/
pushedStreamIdleTimeout?: number;
/**
* (HTTPS only)
* If not false, the server certificate is verified against the list of supplied CAs. An 'error' event is emitted if verification fails; err.code contains the OpenSSL error code.
* @default true
*/
rejectUnauthorized?: boolean;
};
const { fetch } = require('@adobe/fetch');
const resp = await fetch('https://httpbin.org/get');
console.log(resp.ok);
console.log(resp.status);
console.log(resp.statusText);
console.log(resp.httpVersion);
console.log(resp.headers.plain());
console.log(resp.headers.get('content-type'));
const { fetch } = require('@adobe/fetch');
const resp = await fetch('https://httpbin.org/json');
const jsonData = await resp.json();
const { fetch } = require('@adobe/fetch');
const resp = await fetch('https://httpbin.org/');
const textData = await resp.text();
const { fetch } = require('@adobe/fetch');
const resp = await fetch('https://httpbin.org//stream-bytes/65535');
const imageData = await resp.buffer();
fetch operationUsing timeoutSignal(ms) extension:
const { fetch, timeoutSignal, AbortError } = require('@adobe/fetch');
const signal = timeoutSignal(1000);
try {
const resp = await fetch('https://httpbin.org/json', { signal });
const jsonData = await resp.json();
} catch (err) {
if (err instanceof AbortError) {
console.log('fetch timed out after 1s');
}
} finally {
// avoid pending timers which prevent node process from exiting
signal.clear();
}
Using AbortController:
const { fetch, AbortController, AbortError } = require('@adobe/fetch');
const controller = new AbortController();
const timerId = setTimeout(() => controller.abort(), 1000);
const { signal } = controller;
try {
const resp = await fetch('https://httpbin.org/json', { signal });
const jsonData = await resp.json();
} catch (err) {
if (err instanceof AbortError) {
console.log('fetch timed out after 1s');
}
} finally {
// avoid pending timers which prevent node process from exiting
clearTimeout(timerId);
}
const fs = require('fs');
const { fetch } = require('@adobe/fetch');
const resp = await fetch('https://httpbin.org/image/jpeg');
resp.body.pipe(fs.createWriteStream('saved-image.jpg'));
const { fetch } = require('@adobe/fetch');
const method = 'POST';
const body = { foo: 'bar' };
const resp = await fetch('https://httpbin.org/post', { method, body });
const fs = require('fs');
const { fetch } = require('@adobe/fetch');
const method = 'POST';
const body = fs.createReadStream('some-image.jpg');
const headers = { 'content-type': 'image/jpeg' };
const resp = await fetch('https://httpbin.org/post', { method, body, headers });
const { FormData, Blob, File } = require('formdata-node'); // spec-compliant implementations
const { fileFromPath } = require('formdata-node/file-from-path'); // helper for creating File instance from disk file
const { fetch } = require('@adobe/fetch');
const method = 'POST';
const fd = new FormData();
fd.set('field1', 'foo');
fd.set('field2', 'bar');
fd.set('blob', new Blob([0x68, 0x65, 0x6c, 0x69, 0x78, 0x2d, 0x66, 0x65, 0x74, 0x63, 0x68]));
fd.set('file', new File(['File content goes here'], 'file.txt'));
fd.set('other_file', await fileFromPath('/foo/bar.jpg', 'bar.jpg', { type: 'image/jpeg' }));
const resp = await fetch('https://httpbin.org/post', { method, body: fd });
const { createUrl, fetch } = require('@adobe/fetch');
const qs = {
fake: 'dummy',
foo: 'bar',
rumple: "stiltskin",
};
const resp = await fetch(createUrl('https://httpbin.org/json', qs));
or using URLSearchParams:
const { fetch } = require('@adobe/fetch');
const body = new URLSearchParams({
fake: 'dummy',
foo: 'bar',
rumple: "stiltskin",
});
const resp = await fetch('https://httpbin.org/json', { body });
Responses of GET and HEAD requests are by default cached, according to the rules of RFC 7234:
const { fetch } = require('@adobe/fetch');
const url = 'https://httpbin.org/cache/60'; // -> max-age=60 (seconds)
// send initial request, priming cache
let resp = await fetch(url);
assert(resp.ok);
assert(!resp.fromCache);
// re-send request and verify it's served from cache
resp = await fetch(url);
assert(resp.ok);
assert(resp.fromCache);
You can disable caching per request with the cache: 'no-store' option:
const { fetch } = require('@adobe/fetch');
const resp = await fetch('https://httbin.org/', { cache: 'no-store' });
assert(resp.ok);
assert(!resp.fromCache);
You can disable caching entirely:
const { fetch } = require('@adobe/fetch').noCache();
Note that pushed resources will be automatically and transparently added to the cache. You can however add a listener which will be notified on every pushed (and cached) resource.
const { fetch, onPush } = require('@adobe/fetch');
onPush((url, response) => console.log(`received server push: ${url} status ${response.status}`));
const resp = await fetch('https://nghttp2.org');
console.log(`Http version: ${resp.httpVersion}`);
const { fetch } = require('@adobe/fetch').h1();
const resp = await fetch('https://nghttp2.org');
console.log(`Http version: ${resp.httpVersion}`);
const { fetch } = require('@adobe/fetch').keepAlive();
const resp = await fetch('https://httpbin.org/status/200');
console.log(`Connection: ${resp.headers.get('connection')}`); // -> keep-alive
const { fetch } = require('@adobe/fetch').context({ rejectUnauthorized: false });
const resp = await fetch('https://localhost:8443/'); // a server using a self-signed certificate
const { fetch, cacheStats } = require('@adobe/fetch').context({
maxCacheSize: 100 * 1024, // 100kb (Default: 100mb)
});
let resp = await fetch('https://httpbin.org/bytes/60000'); // ~60kb response
resp = await fetch('https://httpbin.org/bytes/50000'); // ~50kb response
console.log(cacheStats());
const { fetch } = require('@adobe/fetch').noCache();
let resp = await fetch('https://httpbin.org/cache/60'); // -> max-age=60 (seconds)
// re-fetch
resp = await fetch('https://httpbin.org/cache/60');
assert(!resp.fromCache);
const { fetch } = require('@adobe/fetch').context({
userAgent: 'custom-fetch'
});
const resp = await fetch('https://httpbin.org//user-agent');
const json = await resp.json();
console.log(json['user-agent']);
More example code can be found in the test source files.
$ npm install
$ npm test
$ npm run lint
You can enable @adobe/fetch low-level debug console output by setting the DEBUG environment variable to adobe/fetch*, e.g.:
$ DEBUG=adobe/fetch* node test.js
This will produce console outout similar to:
...
adobe/fetch:core established TLS connection: #48 (www.nghttp2.org) +2s
adobe/fetch:core www.nghttp2.org -> h2 +0ms
adobe/fetch:h2 reusing socket #48 (www.nghttp2.org) +2s
adobe/fetch:h2 GET www.nghttp2.org/httpbin/user-agent +0ms
adobe/fetch:h2 session https://www.nghttp2.org established +1ms
adobe/fetch:h2 caching session https://www.nghttp2.org +0ms
adobe/fetch:h2 session https://www.nghttp2.org remoteSettings: {"headerTableSize":8192,"enablePush":true,"initialWindowSize":1048576,"maxFrameSize":16384,"maxConcurrentStreams":100,"maxHeaderListSize":4294967295,"maxHeaderSize":4294967295,"enableConnectProtocol":true} +263ms
adobe/fetch:h2 session https://www.nghttp2.org localSettings: {"headerTableSize":4096,"enablePush":true,"initialWindowSize":65535,"maxFrameSize":16384,"maxConcurrentStreams":4294967295,"maxHeaderListSize":4294967295,"maxHeaderSize":4294967295,"enableConnectProtocol":false} +0ms
adobe/fetch:h2 session https://www.nghttp2.org closed +6ms
adobe/fetch:h2 discarding cached session https://www.nghttp2.org +0ms
...
Additionally, you can enable Node.js low-level debug console output by setting the NODE_DEBUG environment variable appropriately, e.g.
$ export NODE_DEBUG=http*,stream*
$ export DEBUG=adobe/fetch*
$ node test.js
Note: this will flood the console with highly verbose debug output.
Thanks to node-fetch and github/fetch for providing a solid implementation reference.
FAQs
Light-weight Fetch implementation transparently supporting both HTTP/1(.1) and HTTP/2
The npm package @adobe/fetch receives a total of 8,103 weekly downloads. As such, @adobe/fetch popularity was classified as popular.
We found that @adobe/fetch demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 0 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
GitHub removed 27 malicious pull requests attempting to inject harmful code across multiple open source repositories, in another round of low-effort attacks.
Security News
RubyGems.org has added a new "maintainer" role that allows for publishing new versions of gems. This new permission type is aimed at improving security for gem owners and the service overall.
Security News
Node.js will be enforcing stricter semver-major PR policies a month before major releases to enhance stability and ensure reliable release candidates.