secret-agent

SecretAgent

SecretAgent is a web browser that's built for scraping.

• Built for scraping - it's the first modern headless browsers designed specifically for scraping instead of just automated testing.
• Designed for web developers - We've recreated a fully compliant DOM directly in NodeJS allowing you bypass the headaches of previous scraper tools.
• Powered by Chrome - The powerful Chrome engine sits under the hood, allowing for lightning fast rendering.
• Emulates any modern browser - BrowserEmulators make it easy to disguise your script as practically any browser.
• Avoids detection along the entire stack - Don't be blocked because of TLS fingerprints in your networking stack.

Check out our website for more details.

Installation

npm i --save secret-agent

or

yarn add secret-agent

Usage

SecretAgent provides access to the W3C DOM specification without the need for Puppeteer's complicated evaluate callbacks and multi-context switching:

const agent = require('secret-agent');

(async () => {
  await agent.goto('https://example.org');
  const title = await agent.document.title;
  const intro = await agent.document.querySelector('p').textContent;
  await agent.close();
})();

Browse the full API docs.

Contributing

We'd love your help in making SecretAgent a better tool. Please don't hesitate to send a pull request.

License

MIT

Changelog

1.6.5 (2022-05-25)

Bug Fixes

• cookies: sub domain cookies not saving (712808d)
• core: file picker not working after profile (4f7c737)
• core: speed for many user profile sites (54a4061)
• dependencies (65cae71)
• github no longer allowing git prefix (9693c89)
• lint issues (db3f353)