header-generator
Advanced tools
header-generator - npm Package Compare versions
Comparing version 1.1.3 to 2.1.39
| { | ||
| "name": "header-generator", | ||
| "version": "1.1.3", | ||
| "description": "NodeJs package for generating browser-like headers.", | ||
| "author": { | ||
| "name": "Apify", | ||
| "email": "support@apify.com", | ||
| "url": "https://apify.com" | ||
| }, | ||
| "homepage": "https://github.com/apify/header-generator#readme", | ||
| "license": "Apache-2.0", | ||
| "main": "src/main.js", | ||
| "files": [ | ||
| "src" | ||
| ], | ||
| "dependencies": { | ||
| "generative-bayesian-network": "0.1.0-beta.1", | ||
| "ow": "^0.23.0" | ||
| }, | ||
| "devDependencies": { | ||
| "@apify/eslint-config": "^0.1.3", | ||
| "eslint": "^7.19.0", | ||
| "fs-extra": "^9.1.0", | ||
| "jest": "^26.6.3", | ||
| "jsdoc-to-markdown": "^7.0.0", | ||
| "markdown-toc": "^1.2.0" | ||
| }, | ||
| "scripts": { | ||
| "build-docs": "npm run build-toc && node docs/build-docs.js", | ||
| "build-toc": "markdown-toc docs/README.md -i", | ||
| "lint": "eslint ./src --ext .js,.jsx", | ||
| "lint:fix": "eslint ./src --ext .js,.jsx --fix", | ||
| "test": "jest --maxWorkers=3 --forceExit" | ||
| }, | ||
| "bugs": { | ||
| "url": "https://github.com/apify/header-generator/issues" | ||
| }, | ||
| "repository": { | ||
| "type": "git", | ||
| "url": "git+https://github.com/apify/header-generator.git" | ||
| } | ||
| "name": "header-generator", | ||
| "version": "2.1.39", | ||
| "description": "NodeJS package for generating realistic browser-like HTTP headers.", | ||
| "author": { | ||
| "name": "Apify", | ||
| "email": "support@apify.com", | ||
| "url": "https://apify.com" | ||
| }, | ||
| "homepage": "https://github.com/apify/fingerprint-suite", | ||
| "license": "Apache-2.0", | ||
| "engines": { | ||
| "node": ">=16.0.0" | ||
| }, | ||
| "main": "index.js", | ||
| "module": "index.mjs", | ||
| "types": "index.d.ts", | ||
| "exports": { | ||
| ".": { | ||
| "import": "./index.mjs", | ||
| "require": "./index.js", | ||
| "types": "./index.d.ts" | ||
| } | ||
| }, | ||
| "dependencies": { | ||
| "browserslist": "^4.21.1", | ||
| "generative-bayesian-network": "^2.1.39", | ||
| "ow": "^0.28.1", | ||
| "tslib": "^2.4.0" | ||
| }, | ||
| "scripts": { | ||
| "build": "npm run clean && npm run compile", | ||
| "postbuild": "cp -r src/data_files/ && cp ../../README.md ", | ||
| "clean": "rimraf ./dist", | ||
| "compile": "tsc -p tsconfig.build.json && gen-esm-wrapper ./index.js ./index.mjs", | ||
| "copy": "ts-node -T ../../scripts/copy.ts" | ||
| }, | ||
| "bugs": { | ||
| "url": "https://github.com/apify/fingerprint-suite/issues" | ||
| }, | ||
| "repository": { | ||
| "type": "git", | ||
| "url": "git+https://github.com/apify/fingerprint-suite.git" | ||
| } | ||
| } |
207
README.md
@@ -1,144 +0,109 @@ | ||
| # Header generator | ||
| NodeJs package for generating browser-like headers. | ||
| <h1 align="center"> | ||
| <a href="https://github.com/apify/fingerprint-suite/"> | ||
| <picture> | ||
| <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/apify/fingerprint-suite/master/static/logo_big_light.svg"> | ||
| <img alt="Fingerprinting suite" src="https://raw.githubusercontent.com/apify/fingerprint-suite/master/static/logo_big_dark.svg" width="500"> | ||
| </picture> | ||
| </a> | ||
| <br> | ||
| </h1> | ||
| <!-- toc --> | ||
| <p align=center> | ||
| <a href="https://www.npmjs.com/package/fingerprint-injector" rel="nofollow"><img src="https://img.shields.io/npm/v/fingerprint-injector/latest.svg" alt="NPM dev version" data-canonical-src="https://img.shields.io/npm/v/fingerprint-injector/next.svg" style="max-width: 100%;"></a> | ||
| <a href="https://discord.gg/jyEM2PRvMU" rel="nofollow"><img src="https://img.shields.io/discord/801163717915574323?label=discord" alt="Chat on discord" data-canonical-src="https://img.shields.io/discord/801163717915574323?label=discord" style="max-width: 100%;"></a> | ||
| </p> | ||
| - [Installation](#installation) | ||
| - [Usage](#usage) | ||
| - [Result example](#result-example) | ||
| - [API Reference](#api-reference) | ||
| `fingerprint-suite` is a handcrafted assembly of tools for browser fingerprint generation and injection. | ||
| Today's websites are increasingly using fingerprinting to track users and identify them. | ||
| With the help of `fingerprint-suite` you can generate and inject browser fingerprints into your browser, allowing you to fly your scrapers under the radar. | ||
| <!-- tocstop --> | ||
| > Would you like to work with us on our fingerprinting tools or similar projects? [We are hiring!](https://apify.com/jobs#senior-node.js-engineer) | ||
| ## Installation | ||
| Run the `npm install header-generator` command. No further setup is needed afterwards. | ||
| ## Usage | ||
| To use the generator, you need to create an instance of the `HeaderGenerator` class which is exported from this package. Constructor of this class accepts a `HeaderGeneratorOptions` object, which can be used to globally specify what kind of headers you are looking for: | ||
| ```js | ||
| const HeaderGenerator = require('header-generator'); | ||
| let headerGenerator = new HeaderGenerator({ | ||
| browsers: [ | ||
| {name: "firefox", minVersion: 80}, | ||
| {name: "chrome", minVersion: 87}, | ||
| "safari" | ||
| ], | ||
| devices: [ | ||
| "desktop" | ||
| ], | ||
| operatingSystems: [ | ||
| "windows" | ||
| ] | ||
| }); | ||
| ``` | ||
| You can then get the headers using the `getHeaders` method, either with no argument, or with another `HeaderGeneratorOptions` object, this time specifying the options only for this call (overwriting the global options when in conflict) and using the global options specified beforehands for the unspecified options: | ||
| ```js | ||
| let headers = headersGenerator.getHeaders({ | ||
| operatingSystems: [ | ||
| "linux" | ||
| ], | ||
| locales: ["en-US", "en"] | ||
| }); | ||
| ``` | ||
| This method always generates a random realistic set of headers, excluding the request dependant headers, which need to be filled in afterwards. Since the generation is randomized, multiple calls to this method with the same parameters can generate multiple different outputs. | ||
| ## Result example | ||
| A result that can be generated for the usage example above: | ||
| ```json | ||
| { | ||
| "sec-ch-ua-mobile": "?0", | ||
| "user-agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.72 Safari/537.36", | ||
| "accept-encoding": "gzip, deflate, br", | ||
| "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9", | ||
| "upgrade-insecure-requests": "1", | ||
| "accept-language": "en-US,en;0.9", | ||
| "sec-fetch-site": "same-site", | ||
| "sec-fetch-mode": "navigate", | ||
| "sec-fetch-user": "?1", | ||
| "sec-fetch-dest": "document" | ||
| } | ||
| ``` | ||
| ## API Reference | ||
| All public classes, methods and their parameters can be inspected in this API reference. | ||
| ## Overview | ||
| <a name="HeaderGenerator"></a> | ||
| `fingerprint-suite` is a modular toolkit for browser fingerprint generation and injection. It consists of the following `npm` packages, which you can use separately, or together: | ||
| ### HeaderGenerator | ||
| HeaderGenerator randomly generates realistic browser headers based on specified options. | ||
| - [`header-generator`](https://www.npmjs.com/package/header-generator): generates configurable, realistic HTTP headers | ||
| - [`fingerprint-generator`](https://www.npmjs.com/package/fingerprint-generator): generates realistic browser fingerprints, affecting the HTTP headers and browser JS APIs | ||
| - [`fingerprint-injector`](https://www.npmjs.com/package/fingerprint-injector): injects browser fingerprints into your Playwright or Puppeteer managed browser instance | ||
| - [`generative-bayesian-network`](https://www.npmjs.com/package/generative-bayesian-network): our fast implementation of a Bayesian generative network used to generate realistic browser fingerprints | ||
| ## Quick start | ||
| * [HeaderGenerator](#HeaderGenerator) | ||
| * [`new HeaderGenerator(options)`](#new_HeaderGenerator_new) | ||
| * [`.getHeaders(options, requestDependentHeaders)`](#HeaderGenerator+getHeaders) | ||
| * [`.orderHeaders(headers, order)`](#HeaderGenerator+orderHeaders) | ||
| The following example shows how to use the fingerprinting tools to camouflage your Playwright-managed Chromium instance. | ||
| ```typescript | ||
| import { chromium } from 'playwright'; | ||
| import { newInjectedContext } from 'fingerprint-injector'; | ||
| * * * | ||
| (async () => { | ||
| const browser = await chromium.launch({ headless: false }); | ||
| const context = await newInjectedContext( | ||
| browser, | ||
| { | ||
| // Constraints for the generated fingerprint (optional) | ||
| fingerprintOptions: { | ||
| devices: ['mobile'], | ||
| operatingSystems: ['ios'], | ||
| }, | ||
| // Playwright's newContext() options (optional, random example for illustration) | ||
| newContextOptions: { | ||
| geolocation: { | ||
| latitude: 51.50853, | ||
| longitude: -0.12574, | ||
| } | ||
| } | ||
| }, | ||
| ); | ||
| <a name="new_HeaderGenerator_new"></a> | ||
| const page = await context.newPage(); | ||
| // ... your code using `page` here | ||
| })(); | ||
| ``` | ||
| #### `new HeaderGenerator(options)` | ||
| Here is the same example using Puppeteer: | ||
| | Param | Type | Description | | ||
| | --- | --- | --- | | ||
| | options | [<code>HeaderGeneratorOptions</code>](#HeaderGeneratorOptions) | default header generation options used unless overridden | | ||
| ```typescript | ||
| import puppeteer from 'puppeteer'; | ||
| import { newInjectedPage } from 'fingerprint-injector'; | ||
| (async () => { | ||
| const browser = await puppeteer.launch({ headless: false }); | ||
| const page = await newInjectedPage( | ||
| browser, | ||
| { | ||
| // constraints for the generated fingerprint | ||
| fingerprintOptions: { | ||
| devices: ['mobile'], | ||
| operatingSystems: ['ios'], | ||
| }, | ||
| }, | ||
| ); | ||
| * * * | ||
| // ... your code using `page` here | ||
| await page.goto('https://example.com'); | ||
| })(); | ||
| ``` | ||
| <a name="HeaderGenerator+getHeaders"></a> | ||
| ## Performance | ||
| With ever-improving performance of antibot fingerprinting services, we use some of the industry-leading services to benchmark our performance. | ||
| The following table shows how is the latest build of `fingerprint-suite` doing in comparison to other popular fingerprinting tools. | ||
| #### `headerGenerator.getHeaders(options, requestDependentHeaders)` | ||
| Generates a single set of ordered headers using a combination of the default options specified in the constructor | ||
| and their possible overrides provided here. | ||
|  | ||
| ## Support | ||
| | Param | Type | Description | | ||
| | --- | --- | --- | | ||
| | options | [<code>HeaderGeneratorOptions</code>](#HeaderGeneratorOptions) | specifies options that should be overridden for this one call | | ||
| | requestDependentHeaders | <code>Object</code> | specifies known values of headers dependent on the particular request | | ||
| If you find any bug or issue with any of the fingerprinting tools, please [submit an issue on GitHub](https://github.com/apify/fingerprint-suite/issues). | ||
| For questions, you can ask on [Stack Overflow](https://stackoverflow.com/questions/tagged/apify) or contact support@apify.com | ||
| ## Contributing | ||
| * * * | ||
| Your code contributions are welcome and you'll be praised for eternity! | ||
| If you have any ideas for improvements, either submit an issue or create a pull request. | ||
| For contribution guidelines and the code of conduct, | ||
| see [CONTRIBUTING.md](https://github.com/apify/fingerprint-suite/blob/master/CONTRIBUTING.md). | ||
| <a name="HeaderGenerator+orderHeaders"></a> | ||
| ## License | ||
| #### `headerGenerator.orderHeaders(headers, order)` | ||
| Returns a new object that contains ordered headers. | ||
| | Param | Type | Description | | ||
| | --- | --- | --- | | ||
| | headers | <code>object</code> | specifies known values of headers dependent on the particular request | | ||
| | order | <code>Array.<string></code> | an array of ordered header names, optional (will be deducted from `user-agent`) | | ||
| * * * | ||
| <a name="BrowserSpecification"></a> | ||
| ### `BrowserSpecification` | ||
| | Param | Type | Description | | ||
| | --- | --- | --- | | ||
| | name | <code>string</code> | One of `chrome`, `firefox` and `safari`. | | ||
| | minVersion | <code>number</code> | Minimal version of browser used. | | ||
| | maxVersion | <code>number</code> | Maximal version of browser used. | | ||
| | httpVersion | <code>string</code> | Http version to be used to generate headers (the headers differ depending on the version). Either 1 or 2. If none specified the httpVersion specified in `HeaderGeneratorOptions` is used. | | ||
| * * * | ||
| <a name="HeaderGeneratorOptions"></a> | ||
| ### `HeaderGeneratorOptions` | ||
| | Param | Type | Description | | ||
| | --- | --- | --- | | ||
| | browsers | <code>Array.<(BrowserSpecification\|string)></code> | List of BrowserSpecifications to generate the headers for, or one of `chrome`, `firefox` and `safari`. | | ||
| | operatingSystems | <code>Array.<string></code> | List of operating systems to generate the headers for. The options are `windows`, `macos`, `linux`, `android` and `ios`. | | ||
| | devices | <code>Array.<string></code> | List of devices to generate the headers for. Options are `desktop` and `mobile`. | | ||
| | locales | <code>Array.<string></code> | List of at most 10 languages to include in the [Accept-Language](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Language) request header in the language format accepted by that header, for example `en`, `en-US` or `de`. | | ||
| | httpVersion | <code>string</code> | Http version to be used to generate headers (the headers differ depending on the version). Can be either 1 or 2. Default value is 2. | | ||
| * * * | ||
| This project is licensed under the Apache License 2.0 - | ||
| see the [LICENSE.md](https://github.com/apify/fingerprint-suite/blob/master/LICENSE.md) file for details. |
New alerts
License Policy Violation
LicenseThis package is not allowed per your license policy. Review the package's license to ensure compliance.
Found 1 instance in 1 package
Filesystem access
Supply chain riskAccesses the file system, and could potentially read sensitive data.
Found 1 instance in 1 package
Fixed alerts
License Policy Violation
LicenseThis package is not allowed per your license policy. Review the package's license to ensure compliance.
Found 1 instance in 1 package
Improved metrics
- Dev dependency count
- decreased by-100%
0
- Number of package files
- increased by190%
29
Worsened metrics
- Total package byte prevSize
- decreased by-96.27%
201135
- Dependency count
- increased by100%
4
- Lines of code
- decreased by-41.71%
823
- Number of lines in readme file
- decreased by-24.14%
110
- Number of low supply chain risk alerts
- increased byInfinity%
1
Dependency changes
+ Addedbrowserslist@^4.21.1
+ Addedtslib@^2.4.0
+ Addedadm-zip@0.5.16(transitive)
+ Addedbrowserslist@4.24.0(transitive)
+ Addedcaniuse-lite@1.0.30001664(transitive)
+ Addedelectron-to-chromium@1.5.29(transitive)
+ Addedescalade@3.2.0(transitive)
+ Addedgenerative-bayesian-network@2.1.55(transitive)
+ Addednode-releases@2.0.18(transitive)
+ Addedow@0.28.2(transitive)
+ Addedpicocolors@1.1.0(transitive)
+ Addedtslib@2.7.0(transitive)
+ Addedupdate-browserslist-db@1.1.1(transitive)
- Removedgenerative-bayesian-network@0.1.0-beta.1(transitive)
- Removedow@0.23.1(transitive)
- Removedtype-fest@1.4.0(transitive)
Updatedow@^0.28.1