Pappet is a command-line tool to crawl websites using puppeteer. It allows you to crawl multiple pages in parallel and recursively. Take screenshots or create PDFs in seconds.
npm i -g pappet
Usage: pappet [OPTION]... [URL]...
Options:
--help Show help
--version Show version number
-q, --quit Be quit
-t, --tabs Set number of pages
-s, --screenshot Take a screenshot
-p, --pdf Take a PDF
-m, --mhtml Save as mhtml
-r, --recursive Recursively visit links
-l, --level Set recursion depth
-w, --width Set page width
-h, --height Set page height
-f, --full-page Take a screenshot of the full scrollable page
-L, --relative Follow relative links only
--device-scale-factor Specify device scale factor
--is-mobile Take meta viewport into account
--has-touch Support touch events
--is-landscape Set viewport in landscape mode
--https-only Follow HTTPS links only
--same-origin Only visit pages with same origin
--disable-js Disable javascript
--user-agent Set user agent
--pattern Only follow links that match the supplied regular expression
Pappet allows you to write most of the options in a long and short form. I will use the shorter syntax here for convenience.
pappet -s https://example.com
You can also specify the width and height of the viewport.
pappet -s -w 800 -h 600 https://example.com
Take a screenshot of the full scrollable page. By specifying the option -f or --full-page.
pappet -sf https://example.com
pappet -p https://example.com
pappet -m https://example.com
pappet -sr https://example.com
When crawling websites recursively you should specify a maximum depth (default 1) for not crawling to long.
You can do that by using the -l or --level option.
pappet -srl2 https://example.com
You can specify a infinite number of URLs to crawl.
pappet -rs https://example.com https://example.com
When doing this it's recommended to use the option -t or --tabs.
This will set the maximum number of pages used by puppeteer (default 2). The more pages you have the faster it will be.
Be careful by specifying this option. Some websites may block you, for sending to many requests at once.
pappet -rst4 https://example.com
pappet -sf --disable-js https://example.com
This will only follow links of the same origin (example.com).
pappet -rs --same-origin https://example.com
pappet -rsL https://example.com
pappet -rp --pattern "/articles/.*" https://example.com
FAQs
A command-line tool to crawl websites using puppeteer.
The npm package pappet receives a total of 0 weekly downloads. As such, pappet popularity was classified as not popular.
We found that pappet demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
/Research
Malicious npm package impersonates Nodemailer and drains wallets by hijacking crypto transactions across multiple blockchains.
Security News
This episode explores the hard problem of reachability analysis, from static analysis limits to handling dynamic languages and massive dependency trees.
Security News
/Research
Malicious Nx npm versions stole secrets and wallet info using AI CLI tools; Socket’s AI scanner detected the supply chain attack and flagged the malware.