Chain Syncer
ChainSyncer is a module that helps synchronize a backend with a blockchain by catching on-chain events and writing them immediately to the database. This is more flexible than storing all data on-chain or off-chain because it allows for a hybrid approach that simplifies development of decentralized applications (dApps).
Updates
-
3.0.0 - Project codebase moved to Typescript. Also some naming changes.
-
4.0.0 - Reworked reliability and speed of the module. Now it is much faster and more reliable. If some of the RPCs are down - it will automatically switch to another one. Also ChainSyncer now tries to batch load events from the blockchain. This allows to reduce the number of RPC calls and speed up the process. And the last one - Chain Syncer moved to Ethers V6
Install
Works only with ethers v6 package, so don't forget to install it:
$ npm i chain-syncer ethers@^6.0.0
Example usage
Using Ethers package:
const { ChainSyncer, InMemoryAdapter } = require('chain-syncer');
const default_adapter = new InMemoryAdapter();
const contracts = {
'Items': {
abi: [ ],
address: '0xe9e7CEA3DedcA5984780Bafc599bD69ADd087D56',
start_block: 27118825
}
}
const syncer = new ChainSyncer(default_adapter, {
tick_interval: 3000,
query_block_limit: 1000,
verbose: true,
block_time: 3500,
rpc_url: 'https://data-seed-prebsc-1-s1.binance.org:8545',
network_id: 97,
async contractsResolver(contract_name) {
return contracts[contract_name];
},
});
syncer.on('Items.Transfer', async (
{ global_index, from_address, block_number, block_timestamp, transaction_hash },
from,
to,
token_id,
) => {
const item = await Item.findOne({ _id: token_id });
if(!item) {
return false;
}
item.owner = to;
item.updatedAt = new Date(block_timestamp * 1000);
await item.save();
}));
await syncer.start();
API
constructor(adapter, options)
Name | Description | Required | Example |
---|
adapter | An adapter interface for storing event data. As a test you can use built-in InMemoryAdapter , but better build your own adapter to any DB. | required | new InMemoryAdapter() |
options | An options object | required | - |
options.tick_interval | determinates how often will process unprocessed events | optional (default: 2000 ) | 3000 (every 3 seconds) |
options.query_block_limit | Maximum amount of blocks that can be scanned per tick. For example official BSC RPC allows up to 2000 blocks per request. | optional (default: 100 ) | 2000 |
options.verbose | A flag which enables debug mode and logging | optional (default: false ) | true |
options.mode | Module mode. Possible: 'mono' or 'scanner' . 'mono' mode is made for monolith applications - processing and scanning happens on the same app. 'scanner' mode is made for microservices arch - app is only scanning for new events, processing happens on the client nodes (currently we are building client package, but dont be shy to contribute) | optional (default: 'mono' ) | 'scanner' |
options.block_time | Block time of a network you are working with. For example 3500 for BSC. | required | 3500 (BSC network) |
options.rpc_url | RPC url that is used to get blockchain data. If array of urls is passed - it will work as fallback urls. | required | 'https://data-seed-prebsc-1-s1.binance.org:8545' or [ 'https://bscrpc.com', 'https://bscrpc2.com', 'https://bscrpc3.com' ] |
options.contractsResolver | An async function that returns object with ethers.js contract instance and tx hash of its deploy | required | async () => ({ contract_abi: contracts[contract_name].abi, start_block: 20304005, address: contracts[contract_name].network.address }) |
options.safe_rescan_every_n_block | Because of unreliability of most of the RPCs, the syncer may miss some events from the latest blocks, so the syncer will rescan previous options.safe_rescan_every_n_block * 2 blocks every options.safe_rescan_every_n_block block. | optional : (default: 100 ) | 50 |
on(event, listener)
⚠️ MUST BE CALLED ONLY BEFORE SYNCER STARTS
Name | Description | Required | Example |
---|
event | Event is a string which contains contract and event | required | 'Items.Transfer' |
listener | Listener function (handler), last argument is always object of event parameters. If false returned from listener - event will be postponed till next processing tick | required | async ({ global_index, from_address, block_number, block_timestamp, transaction_hash }, ...event_args) => { ... } |
start()
Starts scanner and processor
stop()
Stops scanner and processor
Available storage adapters
MongoDB - @chainsyncer/mongodb-adapter
Please feel free to help with all possible adapter (for Postgres, MySQL etc...)
Contact with me if you want @chainsyncer/* namespace in NPM.
FAQ
How to use Chain Syncer for multiple networks?
Chain Syncer is not designed to work with multiple networks at the same time. You can use it for multiple networks, but you have to create separate instance of Chain Syncer for each network.
How to make sure that I have the latest state of the field?
Use global_index to make sure that you have the latest state of the field.
Why sometimes I receive errors while syncing?
Because of unreliability of most of the RPCs, the syncer may miss some events from the latest blocks, so the syncer will rescan previous options.safe_rescan_every_n_block * 2
blocks every options.safe_rescan_every_n_block
block. Also use options.rpc_url
as array of urls so if the first one fails - it will try to use the second one and so on.
Contributing
Feel free to contribute to this project.
Testing
First of all let's start docker container with local Ganache node:
$ sh test.sh
Then enter the container:
$ docker exec -it chainsyncer_tester sh
And finally run tests:
$ npm run test
OR just run in one line:
$ sh test.sh full
Building
$ npm run build
Please make sure you built dist
folder before committing.
License
Chain Syncer is released under the MIT license. © 2022 Miroslaw Shpak