puppeteer-extra
Advanced tools
puppeteer-extra is a modular plugin framework for the popular headless browser automation library Puppeteer. It allows users to easily extend Puppeteer's functionality with plugins, making it more versatile and powerful for various web scraping, automation, and testing tasks.
Stealth Plugin
The Stealth Plugin helps to avoid detection by various anti-bot measures on websites. It modifies the browser's behavior to make it look more like a regular user.
const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
puppeteer.use(StealthPlugin());
(async () => {
const browser = await puppeteer.launch({ headless: true });
const page = await browser.newPage();
await page.goto('https://example.com');
await page.screenshot({ path: 'example.png' });
await browser.close();
})();Adblocker Plugin
The Adblocker Plugin blocks ads and trackers, making page loads faster and reducing the amount of data processed.
const puppeteer = require('puppeteer-extra');
const AdblockerPlugin = require('puppeteer-extra-plugin-adblocker');
puppeteer.use(AdblockerPlugin());
(async () => {
const browser = await puppeteer.launch({ headless: true });
const page = await browser.newPage();
await page.goto('https://example.com');
await page.screenshot({ path: 'example.png' });
await browser.close();
})();Recaptcha Plugin
The Recaptcha Plugin helps to solve reCAPTCHAs automatically using third-party services like 2Captcha, making it easier to automate interactions with websites that use CAPTCHA challenges.
const puppeteer = require('puppeteer-extra');
const RecaptchaPlugin = require('puppeteer-extra-plugin-recaptcha');
puppeteer.use(RecaptchaPlugin({ provider: { id: '2captcha', token: 'YOUR_2CAPTCHA_API_KEY' } }));
(async () => {
const browser = await puppeteer.launch({ headless: true });
const page = await browser.newPage();
await page.goto('https://example.com');
const { captchas } = await page.solveRecaptchas();
await page.screenshot({ path: 'example.png' });
await browser.close();
})();Puppeteer is the core library for headless browser automation. It provides a high-level API to control Chrome or Chromium over the DevTools Protocol. While it is powerful, it lacks the modular plugin system of puppeteer-extra, making it less flexible for certain tasks.
Playwright is a newer library from Microsoft that provides similar functionality to Puppeteer but supports multiple browsers (Chromium, Firefox, and WebKit). It offers a more robust and versatile API but does not have a plugin system like puppeteer-extra.
Selenium WebDriver is a widely-used tool for browser automation that supports multiple browsers and programming languages. It is more mature and has a larger community but is generally considered more complex and slower compared to Puppeteer and puppeteer-extra.
A light-weight wrapper around
puppeteerto enable cool plugins through a clean interface.
yarn add puppeteer puppeteer-extra
# - or -
npm install puppeteer puppeteer-extra
You can also use a specific puppeteer version:
# puppeteer-extra works with any version, e.g. the prerelease:
yarn add puppeteer@next puppeteer-extra
// puppeteer-extra is a drop-in replacement for puppeteer,
// it augments the installed puppeteer with plugin functionality.
// Any number of plugins can be added through `puppeteer.use()`
const puppeteer = require('puppeteer-extra')
// Add stealth plugin and use defaults (all tricks to hide puppeteer usage)
const StealthPlugin = require('puppeteer-extra-plugin-stealth')
puppeteer.use(StealthPlugin())
// Add plugin to anonymize the User-Agent and signal Windows as platform
const UserAgentPlugin = require('puppeteer-extra-plugin-anonymize-ua')
puppeteer.use(UserAgentPlugin({ makeWindows: true }))
// That's it, the rest is puppeteer usage as normal 😊
puppeteer.launch({ headless: true }).then(async browser => {
const page = await browser.newPage()
await page.setViewport({ width: 800, height: 600 })
console.log(`Testing the user agent plugin..`)
await page.goto('https://httpbin.org/headers')
await page.waitFor(1000)
await page.screenshot({ path: 'headers.png', fullPage: true })
console.log(`Testing the stealth plugin..`)
await page.goto('https://bot.sannysoft.com')
await page.waitFor(5000)
await page.screenshot({ path: 'stealth.png', fullPage: true })
console.log(`All done, check the screenshots. ✨`)
await browser.close()
})
The above example uses the stealth and anonymize-ua plugin, which need to be installed as well:
yarn add puppeteer-extra-plugin-stealth puppeteer-extra-plugin-anonymize-ua
# - or -
npm install puppeteer-extra-plugin-stealth puppeteer-extra-plugin-anonymize-ua
If you'd like to see debug output just run your script like so:
DEBUG=puppeteer-extra,puppeteer-extra-plugin:* node myscript.js
puppeteer-extra-plugin-recaptchapage.solveRecaptchas().puppeteer-extra-plugin-stealthpuppeteer-extra-plugin-devtoolspuppeteer-extra-plugin-replpuppeteer-extra-plugin-block-resourcespuppeteer-extra-plugin-flashpuppeteer-extra-plugin-anonymize-uapuppeteer-extra-plugin-user-preferencespuppeteer-extra-plugin-font-size.Check out the packages folder for more plugins.
PRs and new plugins are welcome! :tada: The plugin API for puppeteer-extra is clean and fun to use. Have a look the PuppeteerExtraPlugin base class documentation to get going and check out the existing plugins (minimal example is the anonymize-ua plugin) for reference.
We use a monorepo powered by Lerna (and yarn workspaces), ava for testing, the standard style for linting and JSDoc heavily to auto-generate markdown documentation based on code. :-)
puppeteer-extra and all plugins are tested continously against all relevant NodeJS (v8-v13) and puppeteer versions.
We never broke compatibility and still support puppeteer down to version 1.6.2 (Released Aug 1, 2018).
A few plugins won't work in headless mode (it's noted if that's the case) due to Chrome limitations (e.g. the user-preferences plugin), look into xvfb-run if you still require a headless experience in these circumstances.
Modular plugin framework to teach puppeteer new tricks.
This module acts a drop-in replacement for puppeteer.
Allows PuppeteerExtraPlugin's to register themselves and to extend puppeteer with additional functionality.
Type: function ()
Example:
const puppeteer = require('puppeteer-extra')
puppeteer.use(require('puppeteer-extra-plugin-anonymize-ua')())
puppeteer.use(
require('puppeteer-extra-plugin-font-size')({ defaultFontSize: 18 })
)
;(async () => {
const browser = await puppeteer.launch({ headless: false })
const page = await browser.newPage()
await page.goto('http://example.com', { waitUntil: 'domcontentloaded' })
await browser.close()
})()
Outside interface to register plugins.
Type: function (plugin): this
plugin PuppeteerExtraPluginExample:
const puppeteer = require('puppeteer-extra')
puppeteer.use(require('puppeteer-extra-plugin-anonymize-ua')())
puppeteer.use(require('puppeteer-extra-plugin-user-preferences')())
const browser = await puppeteer.launch(...)
Launch a new browser instance with given arguments.
Augments the original puppeteer.launch method with plugin lifecycle methods.
All registered plugins that have a beforeLaunch method will be called
in sequence to potentially update the options Object before launching the browser.
Type: function (options): Puppeteer.Browser
options Object? Regular Puppeteer launch options (optional, default {})Attach Puppeteer to an existing Chromium instance.
Augments the original puppeteer.connect method with plugin lifecycle methods.
All registered plugins that have a beforeConnect method will be called
in sequence to potentially update the options Object before launching the browser.
Type: function (options)
Get all registered plugins.
Type: Array<PuppeteerExtraPlugin>
Collects the exposed data property of all registered plugins.
Will be reduced/flattened to a single array.
Can be accessed by plugins that listed the dataFromPlugins requirement.
Implemented mainly for plugins that need data from other plugins (e.g. user-preferences).
Type: function (name)
name string? Filter data by name property (optional, default null)Regular Puppeteer method that is being passed through.
Type: function (): string
Regular Puppeteer method that is being passed through.
Type: function ()
Regular Puppeteer method that is being passed through.
Type: function (options): PuppeteerBrowserFetcher
options Object?FAQs
Teach puppeteer new tricks through plugins.
The npm package puppeteer-extra receives a total of 37,462 weekly downloads. As such, puppeteer-extra popularity was classified as popular.
We found that puppeteer-extra demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
A malicious npm package disguised as a WhatsApp client is exploiting authentication flows with a remote kill switch to exfiltrate data and destroy files.
Security News
PyPI now supports digital attestations, enhancing security and trust by allowing package maintainers to verify the authenticity of Python packages.
Security News
GitHub removed 27 malicious pull requests attempting to inject harmful code across multiple open source repositories, in another round of low-effort attacks.