event-log-harvester

Ethereum Event Logs Harvester

Description

The Ethereum Event Logs Harvester is a TypeScript utility that simplifies the process of fetching, decoding, and processing event logs from the Ethereum blockchain. It is designed to handle multiple event filters, decode logs according to user-defined schemas, and process the results using customizable callbacks. The utility supports both synchronous and asynchronous decoding, ensuring flexibility and performance in handling large datasets.

Summary of Key Features

1. Event Log Harvester with Ethers.js:

• The tool is designed to fetch and decode Ethereum event logs based on user-defined filters.
• It supports batching of getLogs requests to efficiently handle large ranges of blocks.

2. EventFilter Interface:

• addresses: An array of contract addresses to listen to.
• topics: The topics to filter events by (e.g., event signatures).
• abi: The ABI (Application Binary Interface) required to decode the events.
• fromBlock & toBlock: The block range in which to look for logs, inclusive.
• callback: An optional function that is called with the logs and batch metadata after processing each batch for that specific filter.

3. Batched Fetching with Error Handling:

• Batching: The tool processes blocks in batches, fetching logs for multiple filters in a single getLogs request.
• Error Handling and Resilience: If the getLogs request fails, the tool retries up to a specified number of times (maxRetries), with an exponential backoff (1s * retry count).

4. Callbacks for Custom Post-Processing:

• Filter-Specific Callbacks: Each EventFilter can have its own callback function, which is invoked after each batch, allowing custom post-processing like database writes, logging, etc.
• Generic and Flexible: The callbacks make the tool highly adaptable to various use cases without needing to modify the core logic.

5. Inclusive Block Range:

• The tool includes logs from the fromBlock and toBlock (i.e., logs within [fromBlock, toBlock] are processed).

6. Implementation Flow:

• Batch Processing: The tool iterates over the entire block range, processing blocks in batches.
• Filter Activation: Only filters whose block ranges overlap with the current batch are active during that batch.
• Decoding Logs: Logs are decoded using the provided ABI, and only logs within the defined block range for each filter are included.
• Callback Invocation: After processing each batch, the respective callback for each filter is invoked with the relevant logs.

7. Example Usage:

• The tool can be configured with different filters, each with its own block range, topics, addresses, and optional callback for custom post-processing.

Install

To install the package, run the following command:

npm install event-log-harvester