Security News
tea.xyz Spam Plagues npm and RubyGems Package Registries
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
fetch
Advanced tools
Readme
Fetch url contents. Supports gzipped content for quicker download, redirects (with automatic cookie handling, so no eternal redirect loops), streaming and piping etc.
npm install fetch
See examples folder for a complete example
fetch.fetchUrl(url [, options], callback)
Where
callback(error, meta, body)
Example
var fetchUrl = require("fetch").fetchUrl;
// source file is iso-8859-15 but it is converted to utf-8 automatically
fetchUrl("http://kreata.ee/iso-8859-15.php", function(error, meta, body){
console.log(body.toString());
});
NB If the file has been marked with charset other than utf-8, it is converted automatically.
By default iconv-lite
is used for charset conversion. If you want to use node-iconv
module instead,
add "iconv": "*"
to your package.json file, it will be picked up by fetch
automatically.
fetch.FetchStream(url [, options]) -> Stream
Where
With events:
function(chunk){}
function(meta){}
Example
var FetchStream = require("fetch").FetchStream;
var fetch = new FetchStream("http://google.com");
fetch.on("data", function(chunk){
console.log(chunk);
});
Possible option values
{'Header-Field':'value'}
Infinity
['name=val']
fetchUrl
fetchUrl
, set to true to disable automatic charset decoding to utf-8fetchUrl
, set input encodingtrue
, default behavior), or ignore and allow them (false
)Meta object contains following fields:
Request headers can be set with options.headers
options = {
headers:{
"X-My-Header": "This is a custom header field"
}
}
User-Agent value can be set with options.headers['User-Agent']
value. Defaults to "FetchStream"
options = {
headers: {
"User-Agent": "MyUseragent/1.0"
}
}
Cookies can be set with options.cookies
which takes an array with cookie definitions
options = {
cookie: ["name=value", "key=value; path=/; secure"]
}
Paths, domain, expire and other cookie settings are honored, so try not to set cookies with expire dates in the past. If domain is not set, any domain will pass, same for paths.
NB Do not set cookie field directly in request header as it will be overwritten.
Cookies can be shared between different requests, this can be achieved with CookieJar
var fetch = require("fetch");
var cookies = new fetch.CookieJar();
// add one cookie for testing
cookies.setCookie('alfa=beta; path=/;');
// create a FetchStream with custom CookieJar
var f = fetch.FetchStream("http://www.example.com/page1",{cookieJar: cookies});
f.on("end", function(){
// if cookies were set with the previos request, the data is
// saved in 'cookieJar' and passed to the next request
fetch.FetchStream("http://www.example.com/page1",{cookieJar: cookies});
});
Redirects are on by default, use options.disableRedirects
to disable. Maximum redirect count can be set with options.maxRedirects
(defaults to 10)
options = {
disableRedirects: true
}
options = {
maxRedirects: 100
}
Gzip and Deflate support is automatically on. This is problematic in Node v0.5.9 and below since Zlib support on these versions is buggy with unpacking and tends to yield in error.
options = {
disableGzip: true
}
FetchStream
is a readable Stream object and thus can be piped. For example stream URL contents directly to a file:
var FetchStream = require("fetch").FetchStream,
fs = require("fs"),
out;
out = fs.createWriteStream('file.html');
new FetchStream("http://www.example.com/index.php").pipe(out);
BSD
FAQs
Fetch URL contents
The npm package fetch receives a total of 77,164 weekly downloads. As such, fetch popularity was classified as popular.
We found that fetch demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
Security News
As cyber threats become more autonomous, AI-powered defenses are crucial for businesses to stay ahead of attackers who can exploit software vulnerabilities at scale.
Security News
UnitedHealth Group disclosed that the ransomware attack on Change Healthcare compromised protected health information for millions in the U.S., with estimated costs to the company expected to reach $1 billion.