Research
Security News
Threat Actor Exposes Playbook for Exploiting npm to Build Blockchain-Powered Botnets
A threat actor's playbook for exploiting the npm ecosystem was exposed on the dark web, detailing how to build a blockchain-powered botnet.
The 'got' npm package is a human-friendly and powerful HTTP request library for Node.js. It provides an easy-to-use API for making HTTP requests and supports many features like streams, pagination, JSON parsing, and more.
Simplified HTTP requests
This feature allows you to perform HTTP GET requests with a promise-based API. The example shows how to fetch a webpage and log the HTML content.
const got = require('got');
got('https://sindresorhus.com').then(response => {
console.log(response.body);
}).catch(error => {
console.log(error.response.body);
});
JSON support
This feature automatically parses JSON responses. The example demonstrates fetching JSON data from an API and logging the parsed object.
const got = require('got');
got('https://api.example.com/data', { responseType: 'json' }).then(response => {
console.log(response.body);
}).catch(error => {
console.log(error.response.body);
});
POST requests
This feature allows you to send POST requests with JSON bodies. The example shows how to send a POST request with a JSON payload and receive a JSON response.
const got = require('got');
got.post('https://api.example.com/submit', {
json: {
key: 'value'
},
responseType: 'json'
}).then(response => {
console.log(response.body);
}).catch(error => {
console.log(error.response.body);
});
Error handling
This feature provides comprehensive error handling for various types of request failures. The example demonstrates how to handle different error scenarios when a request fails.
const got = require('got');
got('https://api.example.com/wrong-endpoint').then(response => {
console.log(response.body);
}).catch(error => {
if (error.response) {
console.log('The server responded with a non-2xx status code.');
} else if (error.request) {
console.log('The request was made but no response was received');
} else {
console.log('An error occurred when trying to perform the request.');
}
});
Stream support
This feature allows you to use got as a stream. The example shows how to stream a webpage's content and write it to a file.
const got = require('got');
const fs = require('fs');
const stream = got.stream('https://sindresorhus.com');
stream.pipe(fs.createWriteStream('index.html'));
Axios is a promise-based HTTP client for the browser and Node.js. It provides an API similar to got but also works in the browser. Axios has interceptors that allow you to transform requests and responses before they are handled by then or catch.
Request is a simplified HTTP request client that was very popular but is now deprecated. It had a callback-based API but also supported promises. Got is considered a modern alternative to Request with promise support by default.
Node-fetch is a light-weight module that brings the Fetch API to Node.js. It is a minimalistic and straightforward API that resembles the Fetch API provided by modern browsers, making it familiar to front-end developers.
Superagent is a small progressive client-side HTTP request library. It has a fluent API that allows chaining methods to configure requests, and it can be used on both server and client side. Compared to got, it has a more object-oriented style.
Simplified HTTP requests
A nicer interface to the built-in http
module.
Created because request
is bloated (several megabytes!).
$ npm install got
const got = require('got');
(async () => {
try {
const response = await got('sindresorhus.com');
console.log(response.body);
//=> '<!doctype html> ...'
} catch (error) {
console.log(error.response.body);
//=> 'Internal server error ...'
}
})();
const fs = require('fs');
const got = require('got');
got.stream('sindresorhus.com').pipe(fs.createWriteStream('index.html'));
// For POST, PUT, and PATCH methods `got.stream` returns a `stream.Writable`
fs.createReadStream('index.html').pipe(got.stream.post('sindresorhus.com'));
It's a GET
request by default, but can be changed by using different methods or in the options
.
Returns a Promise for a response
object with a body
property, a url
property with the request URL or the final URL after redirects, and a requestUrl
property with the original request URL.
The response object will normally be a Node.js HTTP response stream, however if returned from the cache it will be a responselike object which behaves in the same way.
The response will also have a fromCache
property set with a boolean value.
Type: string
Object
The URL to request as simple string, a http.request
options, or a WHATWG URL
.
Properties from options
will override properties in the parsed url
.
If no protocol is specified, it will default to https
.
Type: Object
Any of the http.request
options.
Type: boolean
Default: false
Returns a Stream
instead of a Promise
. This is equivalent to calling got.stream(url, [options])
.
Type: string
Buffer
stream.Readable
This is mutually exclusive with stream mode.
Body that will be sent with a POST
request.
If present in options
and options.method
is not set, options.method
will be set to POST
.
If content-length
or transfer-encoding
is not set in options.headers
and body
is a string or buffer, content-length
will be set to the body length.
Type: string
null
Default: 'utf8'
Encoding to be used on setEncoding
of the response data. If null
, the body is returned as a Buffer.
Type: boolean
Default: false
This is mutually exclusive with stream mode.
If set to true
and Content-Type
header is not set, it will be set to application/x-www-form-urlencoded
.
body
must be a plain object or array and will be stringified.
Type: boolean
Default: false
This is mutually exclusive with stream mode.
If set to true
and Content-Type
header is not set, it will be set to application/json
.
Parse response body with JSON.parse
and set accept
header to application/json
. If used in conjunction with the form
option, the body
will the stringified as querystring and the response parsed as JSON.
body
must be a plain object or array and will be stringified.
Type: string
Object
Query string object that will be added to the request URL. This will override the query string in url
.
Type: number
Object
Milliseconds to wait for the server to end the response before aborting request with ETIMEDOUT
error.
This also accepts an object with separate connect
, socket
, and request
fields for connection, socket, and entire request timeouts.
Type: number
Function
Default: 2
Number of request retries when network errors happens. Delays between retries counts with function 1000 * Math.pow(2, retry) + Math.random() * 100
, where retry
is attempt number (starts from 0).
Option accepts function
with retry
and error
arguments. Function must return delay in milliseconds (0
return value cancels retry).
Note: if retries
is number
, ENOTFOUND
and ENETUNREACH
error will not be retried (see full list in is-retry-allowed
module).
Type: boolean
Default: true
Defines if redirect responses should be followed automatically.
Note that if a 303
is sent by the server in response to any request type (POST
, DELETE
, etc.), got will automatically
request the resource pointed to in the location header via GET
. This is in accordance with the spec.
Type: boolean
Default: true
Decompress the response automatically.
If this is disabled, a compressed response is returned as a Buffer
. This may be useful if you want to handle decompression yourself or stream the raw compressed data.
Type: Object
Default: false
Cache adapter instance for storing cached data.
Type: boolean
Default: false
When used in Electron, Got will use electron.net
instead of the Node.js http
module. According to the Electron docs, it should be fully compatible, but it's not entirely. See #315.
Type: boolean
Default: true
Determines if a got.HTTPError
is thrown for error responses (non-2xx status codes).
If this is disabled, requests that encounter an error status code will be resolved with the response
instead of throwing. This may be useful if you are checking for resource availability and are expecting error responses.
stream
method will return Duplex stream with additional events:
request
event to get the request object of the request.
Tip: You can use request
event to abort request:
got.stream('github.com')
.on('request', req => setTimeout(() => req.abort(), 50));
response
event to get the response object of the final request.
redirect
event to get the response object of a redirect. The second argument is options for the next request to the redirect location.
Progress events for uploading (sending request) and downloading (receiving response). The progress
argument is an object like:
{
percent: 0.1,
transferred: 1024,
total: 10240
}
If it's not possible to retrieve the body size (can happen when streaming), total
will be null
.
Note: Progress events can also be used with promises.
(async () => {
const response = await got('sindresorhus.com')
.on('downloadProgress', progress => {
// Report download progress
})
.on('uploadProgress', progress => {
// Report upload progress
});
console.log(response);
})();
error
event emitted in case of protocol error (like ENOTFOUND
etc.) or status error (4xx or 5xx). The second argument is the body of the server response in case of status error. The third argument is response object.
Sets options.method
to the method name and makes a request.
Each error contains (if available) statusCode
, statusMessage
, host
, hostname
, method
, path
, protocol
and url
properties to make debugging easier.
In Promise mode, the response
is attached to the error.
When a cache method fails, for example if the database goes down, or there's a filesystem error.
When a request fails. Contains a code
property with error class code, like ECONNREFUSED
.
When reading from response stream fails.
When json
option is enabled, server response code is 2xx, and JSON.parse
fails.
When server response code is not 2xx. Includes statusCode
, statusMessage
, and redirectUrls
properties.
When server redirects you more than 10 times. Includes a redirectUrls
property, which is an array of the URLs Got was redirected to before giving up.
When given an unsupported protocol.
When the request is aborted with .cancel()
.
The promise returned by Got has a .cancel()
method which, when called, aborts the request.
(async () => {
const request = got(url, options);
…
// In another part of the code
if (something) {
request.cancel();
}
…
try {
await request;
} catch (error) {
if (request.canceled) { // Or `error instanceof got.CancelError`
// Handle cancelation
}
// Handle other errors
}
})();
Got implements RFC 7234 compliant HTTP caching which works out of the box in memory or is easily pluggable with a wide range of storage adapters. Fresh cache entries are served directly from cache and stale cache entries are revalidated with If-None-Match
/If-Modified-Since
headers. You can read more about the underlying cache behaviour in the cacheable-request
documentation.
You can use the JavaScript Map
type as an in memory cache:
const got = require('got');
const map = new Map();
(async () => {
let response = await got('sindresorhus.com', {cache: map});
console.log(response.fromCache);
//=> false
response = await got('sindresorhus.com', {cache: map});
console.log(response.fromCache);
//=> true
})();
Got uses Keyv internally to support a wide range of storage adapters. For something more scalable you could use an official Keyv storage adapter:
$ npm install @keyv/redis
const got = require('got');
const KeyvRedis = require('@keyv/redis');
const redis = new KeyvRedis('redis://user:pass@localhost:6379');
got('sindresorhus.com', {cache: redis});
Got supports anything that follows the Map API, so it's easy to write your own storage adapter or use a third-party solution.
For example, the following are all valid storage adapters:
const storageAdapter = new Map();
// or
const storageAdapter = require('./my-storage-adapter');
// or
const QuickLRU = require('quick-lru');
const storageAdapter = new QuickLRU({maxSize: 1000});
got('sindresorhus.com', {cache: storageAdapter});
View the Keyv docs for more information on how to use storage adapters.
You can use the tunnel
module with the agent
option to work with proxies:
const got = require('got');
const tunnel = require('tunnel');
got('sindresorhus.com', {
agent: tunnel.httpOverHttp({
proxy: {
host: 'localhost'
}
})
});
If you require different agents for different protocols, you can pass a map of agents to the agent
option. This is necessary because a request to one protocol might redirect to another. In such a scenario, got
will switch over to the right protocol agent for you.
const got = require('got');
const HttpAgent = require('agentkeepalive');
const HttpsAgent = HttpAgent.HttpsAgent;
got('sindresorhus.com', {
agent: {
http: new HttpAgent(),
https: new HttpsAgent()
}
});
You can use the cookie
module to include cookies in a request:
const got = require('got');
const cookie = require('cookie');
got('google.com', {
headers: {
cookie: cookie.serialize('foo', 'bar')
}
});
You can use the form-data
module to create POST request with form data:
const fs = require('fs');
const got = require('got');
const FormData = require('form-data');
const form = new FormData();
form.append('my_file', fs.createReadStream('/foo/bar.jpg'));
got.post('google.com', {
body: form
});
You can use the oauth-1.0a
module to create a signed OAuth request:
const got = require('got');
const crypto = require('crypto');
const OAuth = require('oauth-1.0a');
const oauth = OAuth({
consumer: {
key: process.env.CONSUMER_KEY,
secret: process.env.CONSUMER_SECRET
},
signature_method: 'HMAC-SHA1',
hash_function: (baseString, key) => crypto.createHmac('sha1', key).update(baseString).digest('base64')
});
const token = {
key: process.env.ACCESS_TOKEN,
secret: process.env.ACCESS_TOKEN_SECRET
};
const url = 'https://api.twitter.com/1.1/statuses/home_timeline.json';
got(url, {
headers: oauth.toHeader(oauth.authorize({url, method: 'GET'}, token)),
json: true
});
Requests can also be sent via unix domain sockets. Use the following URL scheme: PROTOCOL://unix:SOCKET:PATH
.
PROTOCOL
- http
or https
(optional)SOCKET
- absolute path to a unix domain socket, e.g. /var/run/docker.sock
PATH
- request path, e.g. /v2/keys
got('http://unix:/var/run/docker.sock:/containers/json');
// or without protocol (http by default)
got('unix:/var/run/docker.sock:/containers/json');
Requests to AWS services need to have their headers signed. This can be accomplished by using the aws4
package. This is an example for querying an "Elasticsearch Service" host with a signed request.
const url = require('url');
const AWS = require('aws-sdk');
const aws4 = require('aws4');
const got = require('got');
const config = require('./config');
// Reads keys from the environment or `~/.aws/credentials`. Could be a plain object.
const awsConfig = new AWS.Config({ region: config.region });
function request(uri, options) {
const awsOpts = {
region: awsConfig.region,
headers: {
accept: 'application/json',
'content-type': 'application/json'
},
method: 'GET',
json: true
};
// We need to parse the URL before passing it to `got` so `aws4` can sign the request
const opts = Object.assign(url.parse(uri), awsOpts, options);
aws4.sign(opts, awsConfig.credentials);
return got(opts);
}
request(`https://${config.host}/production/users/1`);
request(`https://${config.host}/production/`, {
// All usual `got` options
});
It's a good idea to set the 'user-agent'
header so the provider can more easily see how their resource is used. By default, it's the URL to this repo.
const got = require('got');
const pkg = require('./package.json');
got('sindresorhus.com', {
headers: {
'user-agent': `my-module/${pkg.version} (https://github.com/username/my-module)`
}
});
Bear in mind, if you send an if-modified-since
header and receive a 304 Not Modified
response, the body will be empty. It's your responsibility to cache and retrieve the body contents.
Sindre Sorhus | Vsevolod Strukchinsky | Alexander Tesfamichael | Luke Childs |
MIT
FAQs
Human-friendly and powerful HTTP request library for Node.js
The npm package got receives a total of 18,760,920 weekly downloads. As such, got popularity was classified as popular.
We found that got demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
A threat actor's playbook for exploiting the npm ecosystem was exposed on the dark web, detailing how to build a blockchain-powered botnet.
Security News
NVD’s backlog surpasses 20,000 CVEs as analysis slows and NIST announces new system updates to address ongoing delays.
Security News
Research
A malicious npm package disguised as a WhatsApp client is exploiting authentication flows with a remote kill switch to exfiltrate data and destroy files.