Comparing version 3.2.0 to 3.2.1
{ | ||
"name": "x-crawl", | ||
"version": "3.2.0", | ||
"version": "3.2.1", | ||
"author": "coderHXL", | ||
@@ -5,0 +5,0 @@ "description": "x-crawl is a flexible nodejs crawler library.", |
@@ -5,15 +5,15 @@ # x-crawl | ||
x-crawl is a flexible nodejs crawler library. | ||
x-crawl is a flexible nodejs crawler library. It is used to batch crawl data, network requests and download file resources. Support crawling data asynchronously or synchronously. Since it runs on nodejs, it is friendly to JS/TS developers. | ||
If it helps you, you can give [x-crawl repository](https://github.com/coder-hxl/x-crawl) a Star support. | ||
If you feel good, you can support [x-crawl repository](https://github.com/coder-hxl/x-crawl) with a Star. | ||
## Features | ||
- Crawl pages, JSON, file resources, etc. with simple configuration. | ||
- The built-in puppeteer crawls the page, and uses the jsdom library to parse the page. | ||
- Support asynchronous/synchronous way to crawl data. | ||
- Support Promise/Callback method to get the result. | ||
- Anthropomorphic request interval. | ||
- Crawl pages, JSON, file resources, etc. with simple configuration. | ||
- Polling function, timing crawling. | ||
- Anthropomorphic request interval. | ||
- Written in TypeScript, has type hints, provides generics. | ||
- The built-in puppeteer crawls the page and uses the jsdom library to parse the page. | ||
- Written in TypeScript, has type hints, and provides generics. | ||
@@ -310,4 +310,7 @@ ## Relationship with puppeteer | ||
The count attribute of the callback function records the current number of polling operations, and stopPolling is a callback function that can be called to terminate subsequent polling operations. | ||
Callback function parameters: | ||
- The count attribute records the current number of polling operations. | ||
- stopPolling is a callback function, calling it can terminate subsequent polling operations. | ||
### Request interval time | ||
@@ -314,0 +317,0 @@ |
109161
810