dyno

Dyno

The aws-sdk dynamo client is very close to the API, dyno tries to help reduce the amount of repetitive code needed to interact with dynamodb.

First it guesses types. Dynamo is very specific about its types:

{key:{S: 'value'}}

in dyno can be written like:

{key:'value'}

When dyno doesn't do anything to improve a command, it simply passes it through to aws-sdk dynamodb client.

This is the case right now with commands like scan

Installing

 npm install dyno -S

Usage

CLI

Dyno includes a cli for working with DynamoDB tables.

Setup

Dyno assumes that credentials for AWS will be provided in the ENVIRONMENT as described in the aws-sdk docs

Common flags:

dyno <region> <sub command> -e <Endpoint URL>

dyno us-east-1 tables

dyno local tables
// setting the region to local, will set the endpoint to http://localhost:4567

List tables:

dyno local tables

{"TableNames":['my-table', 'my-other-table']}

Describe a table:

dyno local table my-table

{"Table":{"AttributeDefinitions":[{"AttributeName":"collection","AttributeType":"S"},....]}}

Scan a table:

Outputs line delimited JSON in wire-format for every item in the table.

dyno local scan my-table

{"id":{"N":"0.9410678697749972"},"collection":{"S":"somethign:0"},"attr":{"S":"moredata 64"}}
{"id":{"N":"0.9417226337827742"},"collection":{"S":"somethign:0"},"attr":{"S":"moredata 24"}}
{"id":{"N":"0.9447696127463132"},"collection":{"S":"somethign:0"},"attr":{"S":"moredata 48"}}
{"id":{"N":"0.9472108569461852"},"collection":{"S":"somethign:0"},"attr":{"S":"moredata 84"}}
....

Export a table:

Outputs the table schema then does a scan (like above)

dyno local export my-table

{"AttributeDefinitions":[{"AttributeName":"collection","AttributeType":"S"},...]}
{"id":{"N":"0.9410678697749972"},"collection":{"S":"somethign:0"},"attr":{"S":"moredata 64"}}
{"id":{"N":"0.9417226337827742"},"collection":{"S":"somethign:0"},"attr":{"S":"moredata 24"}}
{"id":{"N":"0.9447696127463132"},"collection":{"S":"somethign:0"},"attr":{"S":"moredata 48"}}
{"id":{"N":"0.9472108569461852"},"collection":{"S":"somethign:0"},"attr":{"S":"moredata 84"}}
....

Import a table:

Receives an exported table on stdin. Expects the first line to be the table schema, and the rest of the lines to be items.

dyno us-west-1 export my-table | dyno local import my-table-copy

Put items into a table:

Receives an items as new line delimited json on stdin.

pro tip: That is the same as the output of scan.

dyno us-west-1 scan my-table | dyno local put my-table-copy

JavaScript API

See complete API Documentation.

Setup

var dyno = require('dyno')({
    accessKeyId: 'XXX',
    secretAccessKey: 'XXX',
    region: 'us-east-1',
    table: 'test'
});

getItem

var item = {id: 'yo', range: 5};
dyno.getItem(item, function(err, resp){});

// multiple items
var items = [
        {id: 'yo', range: 5},
        {id: 'guten tag', range: 5},
        {id: 'nihao', range: 5}
    ];
dyno.getItems(items, function(err, resp){})

Set the table name per command:

var item = {id: 'yo', range: 5};
dyno.getItem(item, {table:'myothertablename'}, function(err, resp){});

deleteItems

var keys = [
        {id: 'yo', range: 5},
        {id: 'guten tag', range: 5},
        {id: 'nihao', range: 5}
    ];
dyno.deleteItems(keys, function(err, resp){})

query

var query = {id: {'EQ':'yo'}, {range:{'BETWEEN':[4,6]}};

dyno.query(query, function(err, resp){
    assert.deepEqual(resp, {count : 1, items : [{id : 'yo', range : 5 }]});
});

dyno.query(query, {attributes:['range']}, function(err, resp){
    assert.deepEqual(resp, {count : 1, items : [{range : 5 }]});
});

The last key evaluated by dynamodb can be found in the query callback's third argument.

dyno.query(query, {pages: 1}, function(err, resp, metas) {
    next = metas.pop().last;
    ...
});

This key can be passed back in to another query to get the next page of results.

dyno.query(query, {start: next, pages: 1}, function(err, resp, metas) {
    ...
});

Type Utilties

Dyno.createSet(array, type)

Turn a native JavaScript array into an explicit set. The type is preserved when converting to the DynamoDB wire format.

Dyno.toDynamoTypes(nativeObj)

Convert a native JavaScript object into the DynamoDB wire format.

Dyno.typesFromDynamo(writeObj)

Convert an object in DynamoDB wire format to native JavaScript objects.

multi + kinesisConfig

Dyno includes a special multi-table client that let's you provide configuration details for both a read (replica) table and a write (primary) table. All read operations are performed on the read table and all write operations on the write table.

You can supply a kinesisConfig object to the write client. The idea is to provide a stopgap until DynamoDB Streams are out of preview. Adding kinesisConfig to a dyno config object will result in write operations being "logged" to a Kinesis stream. The Kinesis stream can then be utilized elsewhere to keep a follower database up-to-speed.

multi and kinesisConfig are is useful when used together for setting up basic cross-region replication.

var readConfig = {
  region: 'eu-west-1',
  table: 'follower'
};

var writeConfig = {
  region: 'us-east-1',
  table: 'leader',
  kinesisConfig: {
    stream: 'identifier-for-kinesis-stream',
    region: 'us-east-1',
    key: ['id', 'range']
  }
};

var dyno = Dyno.multi(readConfig, writeConfig);

dyno.getItem('...'); // reads from the follower database in eu-west-1
dyno.putItem('...'); // writes to the leader database + kinesis stream

Changelog

v0.16.0

• Fixed a bug that did not allow the CLI tool to function using EC2 IAM Role-based credentials
• Adds a function to estimate the number of bytes required to store an item
• Allows caller to set concurrency on batch requests