Security News
tea.xyz Spam Plagues npm and RubyGems Package Registries
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
@scrapingant/scrapingant-client
Advanced tools
Readme
@scrapingant/scrapingant-client
is the official library to access ScrapingAnt API from your
JavaScript applications. It runs both in Node.js and browser and provides useful features like
automatic retries and parameters encoding to improve the ScrapingAnt usage experience.
const ScrapingAntClient = require('@scrapingant/scrapingant-client');
const client = new ScrapingAntClient({ apiKey: '<YOUR-SCRAPINGANT-API-KEY>' });
// Scrape the example.com site.
client.scrape('https://example.com')
.then(res => console.log(res))
.catch(err => console.error(err.message));
In order to get API key you'll need to register at ScrapingAnt Service
Network communication sometimes fails, that's a given. The client will automatically retry requests that
failed due to a network error, an internal error of the ScrapingAnt API (HTTP 500+).
By default, it will retry up to 8 times. First retry will be attempted after ~500ms, second after ~1000ms
and so on. You can configure those parameters using the maxRetries
and minDelayBetweenRetriesMillis
options of the ScrapingAntClient
constructor.
All public classes, methods and their parameters can be inspected in this API reference.
ScrapingAntClient is the official library to access ScrapingAnt API from your JavaScript applications. It runs both in Node.js and browser.
new ScrapingAntClient(options)
Param | Type | Default |
---|---|---|
[options] | object | |
[options.maxRetries] | number | 8 |
[options.minDelayBetweenRetriesMillis] | number | 500 |
[options.timeoutSecs] | number | 60 |
[options.apiKey] | string |
scrapingAntClient.scrape(url, [parameters])
⇒ ScrapingAnt API response
https://docs.scrapingant.com/request-response-format#available-parameters
Param | Type |
---|---|
url | string |
[parameters] | object |
[parameters.browser] | boolean |
[parameters.cookies] | string |
[parameters.headers] | object |
[parameters.js_snippet] | string |
[parameters.proxy_type] | string |
[parameters.proxy_country] | string |
[parameters.wait_for_selector] | string |
[parameters.return_text] | boolean |
IMPORTANT NOTE: parameters.js_snippet
will be encoded to Base64 automatically by the ScrapingAnt JS client library.
An ScrapingAntApiError
is thrown for successful HTTP requests that reach the API,
but the API responds with an error response. Typically, those are internal errors,
which are automatically retried, or validation errors, which are thrown immediately,
because a correction by the user is needed.
Properties
Name | Type | Description |
---|---|---|
message | string | Error message returned by the API. |
statusCode | number | HTTP status code of the error. |
httpMethod | string | HTTP method of the API call. |
const ScrapingAntClient = require('@scrapingant/scrapingant-client');
const client = new ScrapingAntClient({ apiKey: '<YOUR-SCRAPINGANT-API-KEY>' });
// Get the residential IP info using httpbin.org
client.scrape('https://httpbin.org/ip', { proxy_type: 'residential' })
.then(res => console.log(res))
.catch(err => console.error(err.message));
const ScrapingAntClient = require('@scrapingant/scrapingant-client');
const client = new ScrapingAntClient({ apiKey: '<YOUR-SCRAPINGANT-API-KEY>' });
// Scrape the httpbin.org site and get all the cookies sent before
client.scrape('https://httpbin.org/cookies', { cookies: 'cookieName1=cookieVal1;cookieName2=cookieVal2' })
.then(res => console.log(res))
.catch(err => console.error(err.message));
const ScrapingAntClient = require('@scrapingant/scrapingant-client');
const client = new ScrapingAntClient({ apiKey: '<YOUR-SCRAPINGANT-API-KEY>' });
// Scrape the httpbin.org site and get all the headers that would be sent before
client.scrape('https://httpbin.org/headers', { headers: { scraping: "is cool!" } })
.then(res => console.log(res))
.catch(err => console.error(err.message));
const ScrapingAntClient = require('@scrapingant/scrapingant-client');
const client = new ScrapingAntClient({ apiKey: '<YOUR-SCRAPINGANT-API-KEY>' });
// Scrape the httpbin.org site and replace all the content with "Hello, world"
const customJsSnippet = "var str = 'Hello, world!';\n" +
"var htmlElement = document.getElementsByTagName('html')[0];\n" +
"htmlElement.innerHTML = str;"
client.scrape('https://httpbin.org/cookies', { js_snippet: customJsSnippet })
.then(res => console.log(res))
.catch(err => console.error(err.message));
FAQs
ScrapingAnt API client for JavaScript
The npm package @scrapingant/scrapingant-client receives a total of 85 weekly downloads. As such, @scrapingant/scrapingant-client popularity was classified as not popular.
We found that @scrapingant/scrapingant-client demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 2 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
Security News
As cyber threats become more autonomous, AI-powered defenses are crucial for businesses to stay ahead of attackers who can exploit software vulnerabilities at scale.
Security News
UnitedHealth Group disclosed that the ransomware attack on Change Healthcare compromised protected health information for millions in the U.S., with estimated costs to the company expected to reach $1 billion.