puppeteer-extra-plugin-stealth
Advanced tools
Stealth mode: Applies various techniques to make detection of headless puppeteer harder.
The puppeteer-extra-plugin-stealth package is designed to make headless puppeteer browsers less detectable by web servers. It achieves this by applying various techniques to mask the fact that a browser is being controlled by automation scripts.
Bypass WebDriver Detection
This feature allows you to bypass WebDriver detection by websites. The plugin modifies the navigator.webdriver property and other related properties to make the browser appear as if it is not being controlled by automation scripts.
const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
puppeteer.use(StealthPlugin());
(async () => {
const browser = await puppeteer.launch({ headless: true });
const page = await browser.newPage();
await page.goto('https://example.com');
// Perform actions on the page
await browser.close();
})();Masking Browser Fingerprints
This feature helps in masking various browser fingerprints such as user-agent, languages, and other properties that can be used to detect automated browsing.
const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
puppeteer.use(StealthPlugin());
(async () => {
const browser = await puppeteer.launch({ headless: true });
const page = await browser.newPage();
await page.goto('https://example.com');
// Perform actions on the page
await browser.close();
})();Evading Chrome Headless Detection
This feature evades detection mechanisms that check for headless Chrome. It modifies various properties and behaviors to make the headless browser appear as a regular browser.
const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth');
puppeteer.use(StealthPlugin());
(async () => {
const browser = await puppeteer.launch({ headless: true });
const page = await browser.newPage();
await page.goto('https://example.com');
// Perform actions on the page
await browser.close();
})();puppeteer-extra is a modular plugin framework for puppeteer. It allows you to add various plugins to enhance puppeteer's functionality. While it includes the stealth plugin, it also offers other plugins for ad-blocking, recaptcha solving, and more.
puppeteer-cluster is a library that allows you to create a cluster of puppeteer workers to perform parallel tasks. While it does not focus on stealth, it is useful for scaling up web scraping tasks efficiently.
playwright is a Node.js library to automate Chromium, Firefox, and WebKit with a single API. It offers built-in features to handle headless detection and provides more control over browser contexts and sessions compared to puppeteer.
A plugin for puppeteer-extra to prevent detection.
yarn add puppeteer-extra-plugin-stealth
# - or -
npm install puppeteer-extra-plugin-stealth
If this is your first puppeteer-extra plugin here's everything you need:
yarn add puppeteer puppeteer-extra puppeteer-extra-plugin-stealth
# - or -
npm install puppeteer puppeteer-extra puppeteer-extra-plugin-stealth
// puppeteer-extra is a drop-in replacement for puppeteer,
// it augments the installed puppeteer with plugin functionality
const puppeteer = require('puppeteer-extra')
// add stealth plugin and use defaults (all evasion techniques)
const StealthPlugin = require('puppeteer-extra-plugin-stealth')
puppeteer.use(StealthPlugin())
// puppeteer usage as normal
puppeteer.launch({ headless: true }).then(async browser => {
console.log('Running tests..')
const page = await browser.newPage()
await page.goto('https://bot.sannysoft.com')
await page.waitFor(5000)
await page.screenshot({ path: 'testresult.png', fullPage: true })
await browser.close()
console.log(`All done, check the screenshot. ✨`)
})
puppeteer-extraand most plugins are written in TS, so you get perfect type support out of the box. :)
import puppeteer from 'puppeteer-extra'
import StealthPlugin from 'puppeteer-extra-plugin-stealth'
puppeteer
.use(StealthPlugin())
.launch({ headless: true })
.then(async browser => {
const page = await browser.newPage()
await page.goto('https://bot.sannysoft.com')
await page.waitFor(5000)
await page.screenshot({ path: 'stealth.png', fullPage: true })
await browser.close()
})
Please check this wiki entry in case you have TypeScript related import issues.
Please check out the main documentation to learn more about puppeteer-extra (Firefox usage, other Plugins, etc).
January 2020
puppeeteer-extra with stealth passes all public bot tests.Please note: I consider this a friendly competition in a rather interesting cat and mouse game. If the other team (👋) wants to detect headless chromium there are still ways to do that (at least I noticed a few, which I'll tackle in future updates).
It's probably impossible to prevent all ways to detect headless chromium, but it should be possible to make it so difficult that it becomes cost-prohibitive or triggers too many false-positives to be feasible.
If something new comes up or you experience a problem, please do your homework and create a PR in a respectful way (this is Github, not reddit) or I might not be motivated to help. :)
v2.4.7user-agent-override - Used to set a stealthy UA string, language & platform. This also fixes issues with the prior method of setting the Accept-Language header through request interception (#104, kudos to @Niek)navigator.vendor - Makes it possible to optionally override navigator.vendor (#110, thanks @Niek)navigator.webdriver: Now uses ES6 Proxies to pass instanceof tests (#117, thanks @aabbccsmith)user-agent, accept-language (now obsolete)v2.4.2 / v2.4.1iframe.contentWindow - We now proxy the original window object and smartly redirect calls that might reveal it's true identity, as opposed to mocking it like peasants :)accept-language - More robust and it's now possible to set a custom locale if needed.v2.4.0Let's ring the bell for round 2 in this cat and mouse fight 😄
media.codecs - we spoof the presence of proprietary codecs in Chromium nowiframe.contentWindow - Found a way to fix srcdoc frame based detection without breaking recaptcha inline popup & other iframes (please report any issues)accept-language - Adds a missing Accept-Language header in headless (capitalized correctly, page.setExtraHTTPHeaders is all lowercase which can be detected)chrome.runtime - More extensive mocking of the chrome objectareyouheadless testv2.1.2navigator.plugins - we fully emulate plugins/mimetypes in headless now 🎉webgl.vendor - is otherwise set to "Google" in headlesswindow.outerdimensions - fix missing window.outerWidth/outerHeight and viewportnavigator.webdriver now returns undefined instead of false Chromium + headless |  Chromium + headful |  Chrome + headless |  Chrome + headful |
 Chromium + headless |  Chromium + headful |  Chrome + headless |  Chrome + headful |
Note: The
MQ_SCREENtest is broken on their page (will fail in regular Chrome as well).
Tests have been done using this test site and these scripts.
Using stealth also seems to help with maintaining a normal reCAPTCHA v3 score.
Regular Puppeteer | Stealth Puppeteer |
Note: The official test is to be taken with a grain of salt, as the score is calculated individually per site and multiple other factors (past behaviour, IP address, etc). Based on anecdotal observations it still seems to work as a rough indicator.
Tip: Have a look at the recaptcha plugin if you have issues with reCAPTCHAs.
opts Object? Options (optional, default {})
Extends: PuppeteerExtraPlugin
Stealth mode: Applies various techniques to make detection of headless puppeteer harder. 💯
There are a couple of ways the use of puppeteer can easily be detected by a target website.
The addition of HeadlessChrome to the user-agent being only the most obvious one.
The goal of this plugin is to be the definite companion to puppeteer to avoid detection, applying new techniques as they surface.
As this cat & mouse game is in it's infancy and fast-paced the plugin is kept as flexibile as possible, to support quick testing and iterations.
This plugin uses puppeteer-extra's dependency system to only require
code mods for evasions that have been enabled, to keep things modular and efficient.
The stealth plugin is a convenience wrapper that requires multiple evasion techniques
automatically and comes with defaults. You could also bypass the main module and require
specific evasion plugins yourself, if you whish to do so (as they're standalone puppeteer-extra plugins):
// bypass main module and require a specific stealth plugin directly:
puppeteer.use(
require('puppeteer-extra-plugin-stealth/evasions/console.debug')()
)
PRs are welcome, if you want to add a new evasion technique I suggest you look at the template to kickstart things.
Thanks to Evan Sangaline and Paul Irish for kickstarting the discussion!
Example:
const puppeteer = require('puppeteer-extra')
// Enable stealth plugin with all evasions
puppeteer.use(require('puppeteer-extra-plugin-stealth')())
;(async () => {
// Launch the browser in headless mode and set up a page.
const browser = await puppeteer.launch({
args: ['--no-sandbox'],
headless: true
})
const page = await browser.newPage()
// Navigate to the page that will perform the tests.
const testUrl =
'https://intoli.com/blog/' +
'not-possible-to-block-chrome-headless/chrome-headless-test.html'
await page.goto(testUrl)
// Save a screenshot of the results.
const screenshotPath = '/tmp/headless-test-result.png'
await page.screenshot({ path: screenshotPath })
console.log('have a look at the screenshot:', screenshotPath)
await browser.close()
})()
Get all available evasions.
Please look into the evasions directory for an up to date list.
Example:
const pluginStealth = require('puppeteer-extra-plugin-stealth')()
console.log(pluginStealth.availableEvasions) // => Set { 'user-agent', 'console.debug' }
puppeteer.use(pluginStealth)
Get all enabled evasions.
Enabled evasions can be configured either through opts or by modifying this property.
Example:
// Remove specific evasion from enabled ones dynamically
const pluginStealth = require('puppeteer-extra-plugin-stealth')()
pluginStealth.enabledEvasions.delete('console.debug')
puppeteer.use(pluginStealth)
opts Object? Options
Default export, PuppeteerExtraStealthPlugin
Copyright © 2020, berstend̡̲̫̹̠̖͚͓̔̄̓̐̄͛̀͘. Released under the MIT License.
FAQs
Stealth mode: Applies various techniques to make detection of headless puppeteer harder.
The npm package puppeteer-extra-plugin-stealth receives a total of 219,451 weekly downloads. As such, puppeteer-extra-plugin-stealth popularity was classified as popular.
We found that puppeteer-extra-plugin-stealth demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.
Security News
Research
Socket researchers found a malicious Maven package impersonating the legitimate ‘XZ for Java’ library, introducing a backdoor for remote code execution.