Node.js/TypeScript HTTP client with browser TLS fingerprint impersonation (JA3/JA4). Bypass Cloudflare and anti-bot detection. Rust-powered, fetch()-compatible.
Command line for Crawler Free: read web pages as clean Markdown, links or facts through the free hosted service.
A literal web crawler: drop a document, watch a spider read it, get the gist.
MCP server connecting AI agents to Scavio: web search, page extraction, and structured e-commerce, social, travel, jobs, real-estate, app-store, ad-library and company-filing data. 665 tools across 90 platforms plus Extract, Company Enrich, site map, craw
Lightning-fast, enterprise-grade HTTP client for modern JavaScript. Full HTTP/2 support, intelligent cookie management, multiple adapters (HTTP, Fetch, cURL, XHR), streaming, proxy support (HTTP/HTTPS/SOCKS), and cross-environment compatibility.
Html Metadata scraper and parser for Node.js
A port of n0madic/twitter-scraper to Node.js.
A CLI tool to scrape and analyze Hacker News entries
Device detection cloud services for the 51Degrees Pipeline API
Device detection on-premise services for the 51Degrees Pipeline API
A command-line tool acting as an MCP (ModelContextProtocol) server, using Playwright to crawl web content for AI models.
Parse HTTP headers to detect the device type, model, operating system, browser, and crawler information
Lightweight crawler written in TypeScript using ES6 generators.
MDAA DataOps Crawler L3 Construct
The SEO command for AI agents: a free Google Search Console MCP server and CLI. Reads your real Search Console, Google Analytics and Bing data for traffic drops, quick wins, content decay, cannibalization and indexing, plus DataForSEO and Ahrefs keyword r
Pure javascript cross-platform module to extract text from PDFs.
MDAA dataops-crawler module
SEO and AEO analysis for coding agents: a CLI and MCP server that audit URLs, static build output or raw HTML for search engines and AI answer engines, and return machine-actionable findings.
This module helps detect if a user agent derives from known robots/crawlers/spiders
n8n community node for Scavio — real-time structured JSON from Google, ChatGPT, Gemini, Amazon, Walmart, eBay, Target, Home Depot, Costco, Publix, IKEA, Wayfair, Ashley Furniture, Maisons du Monde, Best Buy, TCGplayer, Nintendo eShop, H&M, Zara, Uniqlo, A
> 当前支持的模式
Nest TypeScript starter repository
The official MCP Server for the Crawler Dev API
Log crawler progress and RAM usage
A simple command line app for controlling a GitHub Crawler
爬虫节点
Easily create XML sitemaps for your website.
Shared utilities and base functionality for implementing device detection engines for the 51Degrees Pipeline API in Node.js.
Always up-to-date list of the top 100 most common browser user-agents for HTTP clients.
crawls a npm package and it's dependencies for their licenses
a module to interact with the crawler tables stored in hbase
wordpress数据导入插件
Node.js/TypeScript HTTP client with browser TLS fingerprint impersonation (JA3/JA4). Bypass Cloudflare and anti-bot detection. Rust-powered, fetch()-compatible.
MCP server for web scraping, structured data extraction, and screenshots — give Claude, Cursor, Windsurf, and AI agents the ability to scrape any website with automatic anti-bot bypass
Web crawler engine with headless browser rendering (writes .nitpicker archives via @nitpicker/archive)
Real-time Spine animation profiler and performance analyzer for PixiJS