@attestate/crawler - npm Package Compare versions

Comparing version 0.0.1 to 0.2.0

changelog.md

 # changelog
 
+## 0.2.0
+
+- Merge crawl path and configuration file into configuration file
+- Move `EXTRACTION_WORKER_CONCURRENCY` into configuration file
+
 ## 0.1.0
@@ -4,0 +9,0 @@

package.json

 {
   "name": "@attestate/crawler",
-  "version": "0.0.1",
+  "version": "0.2.0",
   "description": "@attestate/crawler is a tool chain to retrieve on-chain data from Ethereum.",
@@ -58,3 +58,2 @@ "main": "./src/boot.mjs",
       "DATA_DIR": "data",
-      "EXTRACTION_WORKER_CONCURRENCY": "12",
       "IPFS_HTTPS_GATEWAY": "https://cloudflare-ipfs.com/ipfs/",
@@ -61,0 +60,0 @@ "ARWEAVE_HTTPS_GATEWAY": "https://arweave.net"

readme.md

@@ -6,18 +6,19 @@ # @attestate/crawler
 
-The `@attestate/crawler` is a Node.js framework for scalably querying
-Ethereum's JSON-RPC endpoints and to slice out portions of the Ethereum
-network's data for local usage.
+A JavaScript library for retrieving on-chain storage and generating data
+derivatives.
 
-To modularize the crawler's data sources, developers can implement custom
-strategies or re-use existing ones to compose crawl paths. Conceptually, the
-crawler implements the Extract, Transform and Load separation, to minimize
-downtime, dependencies on external interfaces and to increase reproducability.
+- **Modular:** Extract, Transform and Load stage separation makes composing
+  simple components into complex workflows possible. Stages are reunnable to
+  recover errors.
+- **Embedded:** Unlike Graph Protocol, attestate's crawler does not have a
+  separate server process. It can directly write to self-contained databases
+  like LevelDB or sqlite.
+- **Batteries-included:** Support for Ethereum, IPFS, Arweave, GraphQL and
+  rate-limit pooling for Infura and Alchemy.
 
-The entire project is licensed licensed under the GPL3, to make it truly owned
-by the community instead of a single entity. Branded as the "Neume Network",
-the Attestate Crawler codebase has successfully been used to download the
-metadata of the music NFT ecosystem.
+* **[Getting Started](https://attestate.com/crawler/main/getting-started.html)**
+* Documentation: [attestate.com/crawler/main](https://attestate.com/crawler/main)
 
-- **[Getting Started](https://attestate.com/crawler/getting-started.html)**
-- Documentation: [attestate.com/crawler](https://attestate.com/crawler)
-- License: [SPDX-License-Identifier: GPL-3.0-only](https://spdx.org/licenses/GPL-3.0-only.html)
+### License
+
+[SPDX-License-Identifier: GPL-3.0-only](https://spdx.org/licenses/GPL-3.0-only.html)

No alert changes

Improved metrics

Total package byte prevSize

122711

increased by9.42%

Number of package files

49

increased by19.51%

Number of lines in readme file

24

increased by4.35%

Worsened metrics

Lines of code

1609

decreased by-2.37%

No dependency changes