Destination Tools
Prototype tools for dev-ops to use to identify and diagnose issues with Destinations:
Pre-Requisites
This application requires node.js
version 11 or greater, and yarn
if you want to install from the repository.
Installation
You can install from the repository and use locally.
git clone git@github.com:bitbrew/destination-tools
cd desination-tools
yarn
./bin/dtools ls
You can also install an application from npm:
npm i -g bbdtools
dtools ls
Usage
Configuration
You can change the following parameters:
- Redis Hostname and Port
- Number of Kafka partitions in destination topics
dtools config 127.0.0.1 6379 50
Example output:
Successfully changed configuration to:
Number of partitons: 50
Redis host: 127.0.0.1
Redis port: 6379
List all destinations
dtools ls
Example output:
destinationId closed open retrying clearing cancelled
------------------------------------ ------ ---- -------- -------- ---------
02838e3e-e1c5-434b-b91a-d03abffbbfce 50 0 0 0 0
3eeb7885-ee2a-46de-b6e1-98b202e1a486 0 50 0 0 0
86a6c111-7de6-4421-934d-81ea4fce43ee 0 1 1 49 0
Get destination status
dtools status 3eeb7885-ee2a-46de-b6e1-98b202e1a486
Example output:
Total sent records: 0
Current queued batches (queued): 125
Current queued records (rqueued): 125
Details
ptn state sent retry queued rqueued offset
--- ----- ---- ----- ------ ------- ------
0 Open ✔ 2 2 2
1 Open ✔ 2 2 2
2 Open ✔ 2 2 2
3 Open ✔ 2 2 2
4 Open ✔ 2 2 2
5 Open ✔ 2 2 2
Get a retry batch from a partition
dtools retry 3eeb7885-ee2a-46de-b6e1-98b202e1a486 44
Example output:
{
"retryBatch": {
"dipId": {
"destinationId": "3eeb7885-ee2a-46de-b6e1-98b202e1a486",
"partition": 44
},
"tenantId": "kafka-fill",
"records": [
{
"eventId": "304dd66c-30e5-4e52-af8c-86c97f08f867",
"messageId": "985c52fc-9154-4b13-a41b-d44c97e6e25a",
"deviceId": "0000001000",
"ruleId": "rule-id",
"ingestionTimestamp": "2019-03-19T14:09:54.000Z",
"payload": "{\"payload\":\"KknPvQvQV3yDklUaoztwbspMtYC3EGTzgvvkKJWJVxZc0lSWsWxSapwkTayfn0P488WX301FZEikA4OZ8FkTdiiSj7COqO9qNDi2\"}"
}
],
"retryCount": 32,
"liveSendAt": "2019-03-19T14:12:52.000Z",
"retryAfter": "2019-03-19T19:28:21.000Z"
}
}
List pod assigned to partition
dtools pods
Example output
sender ptn pod
------ --- --------------------------------------------------
Dynamo 0 dynamo-db-sender-v1-18-0-snapshot-5d98c55d84-dpznk
Dynamo 1 dynamo-db-sender-v1-18-0-snapshot-5d98c55d84-dpznk
Dynamo 2 dynamo-db-sender-v1-18-0-snapshot-5d98c55d84-dpznk
Dynamo 3 dynamo-db-sender-v1-18-0-snapshot-5d98c55d84-dpznk
Dynamo 4 dynamo-db-sender-v1-18-0-snapshot-5d98c55d84-dpznk
...