Crawler is a web spider written with Nodejs. It gives you the full power of jQuery on the server to parse a big number of pages as they are downloaded, asynchronously. Scraping should be simple and fun!
爬虫调度程序
Crawler4nodejs is an open source web crawler for Node.js which provides a simple interface for crawling the Web.
HTTP request module customized for crawlers.
## services
小说爬虫npm包
Expired domains finder for crawler.ninja
Crawler scans the ARK network to get information about the peers in the network.
Clubhouse social graph crawler with Neo4j support.
An utility to crawl generic objects paths
a simplified directed web crawler, easy to use for scraping pages and downloading resources of page.
Super simple website crawler
Parse HTTP headers to detect the device type, model, operating system, browser, and crawler information
Search videos on YouTube without API key
An extremely simple web crawler, based on puppeteer.
![Nightcrawler](docs/logo.png) ============
## install
Forked from https://github.com/github/lightcrawler
[![Build Status](https://travis-ci.org/montacasa/crawler-xml-filters.svg?branch=dev)](https://travis-ci.org/montacasa/crawler-xml-filters)
Plugin for bauer-crawler to make http requests.
Yet another node torrent scraper based on x-ray. (Support iptorrents, torrentleech, torrent9, Yyggtorrent, ThePiratebay, torrentz2, 1337x, KickassTorrent, Rarbg, TorrentProject, Yts, Limetorrents, Eztv)
**[★ Online documentation ★](https://apiel.github.io/test-crawler/)**
Analyzes license information for multiple node.js modules (package.json files) as part of your software project.
jsoncrawler.js lets you search complex json data
React component to protect email addresses from crawlers.
A web scraper for NodeJs
Automatically extracts structured information from webpages
Bright CLI is a CLI tool that can initialize, stop, polling and maintain scans in Bright solutions.
Easily create XML sitemaps for your website with url-parse 1.5.10 based on sitemap generator
Pure javascript cross-platform module to extract text from PDFs.
A web crawler module designed to scarp data from Ptt.
Wordpress posts crawler for node.js