Socket
Socket
Sign inDemoInstall

dynamodb-toolkit

Package Overview
Dependencies
0
Maintainers
1
Versions
52
Alerts
File Explorer

Advanced tools

Install Socket

Detect and block malicious and high-risk dependencies

Install

    dynamodb-toolkit

Helpers for AWS DynamoDB to build efficient web applications


Version published
Weekly downloads
125
increased by468.18%
Maintainers
1
Install size
92.9 kB
Created
Weekly downloads
 

Readme

Source

dynamodb-toolkit NPM version

No-dependencies opinionated micro-library for AWS DynamoDB to build small efficient RESTful APIs and high-performance command-line utilities with a simple intuitive API.

  • Designed to work with existing code. Your legacy code can be gradually improved. No special requirements for database tables or indices!
  • Battle-proven. Used in mission-critical applications.
  • Provides a flexible way preparing your data objects for storing and reviving them back:
    • Supports complex indexing: design your own queries!
    • Validate objects before storing.
    • Check the consistency of a database before storing objects.
    • Rich set of efficient read/write/delete/clone/move operations.
    • Various low-level and high-level utilities for DynamoDB.
  • Automatically encoding/decoding your beautiful JSON data to and fro DynamoDB internal format.
  • Supports vagaries of mass read/write/delete operations correctly handling throughput-related errors using the recommended exponential back-off algorithm.
    • Supports transactions and batch requests.
  • Implements efficiently on the server:
    • Paging (in offset/limit terms) and sorting on an index (ascending and descending) to interface with list/table visual components.
    • Subsetting AKA projection: read operations can return a subset required of fields instead of the whole shebang which can be huge especially for mass read operations — think lists and tables.
    • Searching AKA filtering: filters results looking for a substring in a predefined set of fields.
    • Patching: only necessary fields are transferred to be updated/deleted.
    • Cloning: making updated copies in a database.
    • Moving: effectively renaming objects.
  • Thoroughly asynchronous. No callbacks.
  • Working with multiple databases at the same time potentially using different credentials.

Extensively documented: see the wiki.

Example

const Adapter = require('dynamodb-toolkit');

const AWS = require('aws-sdk');
AWS.config.update({region: 'us-east-2'});

const adapter = new Adapter({
  client: new AWS.DynamoDB(),
  table: 'test',
  keyFields: ['name']
});

// ...

const planet = await adapter.get({name: 'Alderaan'});
await adapter.patch({name: planet.name, diameter: planet.diameter * 2});
// ...
await adapter.clone({name: planet.name}, planet => {
  const newPlanet = {...planet, name: planet.name + ' Two'};
  return newPlanet;
});
await adapter.delete({name: planet.name});

Adapter

The full documentation: Adapter. Here is a cheatsheet:

What you have to define yourself

  • Data properties:
    • keyFields — a required list of keys starting with the partition key.
  • Methods:

What you may want to define yourself

What you immediately get

Stand-alone utilities

Included utility functions:

  • Helping to create clients with all necessary settings.
  • Various operations on parameters (params) including adding projections, filtering.
  • Batching operations for efficiency.
  • Transactions.
  • Preparing patches.
  • Mass operations: reading by keys, list with pagination, copying, deleting, iterating over, getting totals, writing.
  • See the wiki for more details.

Koa

The library provides a helper for Koa to write HTTP REST servers: KoaAdapter. It takes care of query parameters, extracts POST/PUT JSON bodies, sends responses encoded as JSON with proper HTTP status codes, and prepares parameters for mass operations.

Example

Define a Koa adapter:

const koaAdapter = new KoaAdapter(adapter, {
  sortableIndices: {name: '-t-name-index'},

  augmentItemFromContext(item, ctx) {
    if (ctx.params.planet) {
      item.name = ctx.params.planet;
    }
    return item;
  },

  augmentCloneFromContext(ctx) {
    // how to transform an object when cloning/moving (the default)
    return item => ({...item, name: item.name + ' COPY'});
  },

  async getAll(ctx) {
    // custom getAll(), which supports sorting
    let index, descending;
    if (ctx.query.sort) {
      let sortName = ctx.query.sort;
      descending = ctx.query.sort[0] == '-';
      if (descending) {
        sortName = sortName.substr(1);
      }
      index = this.sortableIndices[sortName];
    }
    const options = this.makeOptions(ctx);
    options.descending = !!descending;
    ctx.body = await this.adapter.getAll(options, null, index);
  },

  async load(ctx) {
    // custom operation
    const data = JSON.parse(zlib.gunzipSync(fs.readFileSync(path.join(__dirname, 'data.json.gz'))));
    await this.adapter.putAll(data);
    ctx.body = {processed: data.length};
  }
});

Most operations were trivial. Some operations take more than a couple of lines for flexibility's sake.

Define the routing table:

const router = new Router();

router
  // mass operations
  .get('/', async ctx => koaAdapter.getAll(ctx))
  .delete('/', async ctx => koaAdapter.deleteAll(ctx))
  .put('/-load', async ctx => koaAdapter.load(ctx))
  .put('/-clone', async ctx => koaAdapter.cloneAll(ctx))
  .put('/-clone-by-names', async ctx => koaAdapter.cloneByNames(ctx))
  .put('/-move', async ctx => koaAdapter.moveAll(ctx))
  .put('/-move-by-names', async ctx => koaAdapter.moveByNames(ctx))
  .get('/-by-names', async ctx => koaAdapter.getByNames(ctx))
  .delete('/-by-names', async ctx => koaAdapter.deleteByNames(ctx))

  // item operations
  .post('/', async ctx => koaAdapter.post(ctx))
  .get('/:planet', async ctx => koaAdapter.get(ctx))
  .put('/:planet', async ctx => koaAdapter.put(ctx))
  .patch('/:planet', async ctx => koaAdapter.patch(ctx))
  .delete('/:planet', async ctx => koaAdapter.delete(ctx))
  .put('/:planet/-clone', async ctx => koaAdapter.clone(ctx))
  .put('/:planet/-move', async ctx => koaAdapter.move(ctx));

module.exports = router;

It cannot be simpler than that!

Documentation

See wiki for the full documentation.

Versions

  • 2.3.0 Added __separator to a patch object.
  • 2.2.0 Added readOrderedListByKeys() and indirection to Adapter's GET-like methods.
  • 2.1.1 Bugfix in addProjection() to avoid duplicates.
  • 2.1.0 Added moveXXX() operations, some minor implementation improvements.
  • 2.0.1 Bugfix in the default implementation of revive().
  • 2.0.0 Minor API change, better support for paths, support for AWS.DynamoDB.DocumentClient, and so on.
  • 1.16.0 Switched conversion to AWS.DynamoDB.Convert, added getPath() and setPath() utilities.
  • 1.15.1 Added seq() for sequential asynchronous operations.
  • 1.15.0 Updated API to work with "db-raw" objects from a database.
  • 1.14.0 Updated API to work with "raw" objects from a database.
  • 1.13.9 Added an overridable clone function.
  • 1.13.8 Serialized generic mass operations. Slower but less resource consumption.
  • 1.13.7 Better work with boolean results in mass operations.
  • 1.13.6 Bugfix to return a number of processed records from generic mass operations.
  • 1.13.5 Bugfix to unbreak generic ops.
  • 1.13.4 Bugfix in generic operations.
  • 1.13.3 Added addKeyFields(), simplified getAllByParams(), fixed genericDeleteAllByParams().
  • 1.13.2 Bugfix in pagination with filtering.
  • 1.13.1 Added an alternative way to go over database records: iterateList().
  • 1.13.0 Refactored xxxList() functions to use getBatch() and applyBatch().
  • 1.12.0 Updated getTotal() API and added a no-limit version of paginateList().
  • 1.11.2 Typo fix in Adapter.makeCheck().
  • 1.11.1 Bugfix in Adapter related to transactions.
  • 1.11.0 New style of batches, support for transactional consistency checks.
  • 1.10.6 Bugfix in applyTransaction().
  • 1.10.5 Added generic implementations of compound operations.
  • 1.10.4 Added checkExistence() helper.
  • 1.10.3 Minor improvements, added batch makers working on raw keys and items.
  • 1.10.2 Restructured names to keys methods of KoaAdapter.
  • 1.10.1 Added xxxByNames methods to KoaAdapter.
  • 1.10.0 Added support for multi-table batches and transactions. Minor overall improvements.
  • 1.9.2 Added profile-related utilities.
  • 1.9.1 Bugfix: added missing normalization for scanAllByParams(), updated dev deps.
  • 1.9.0 Added: projecting sub-objects, new way to clone objects, and unit tests.
  • 1.8.2 Bugfix in scanAllByParams().
  • 1.8.1 Added makeListParams() and scanAllByParams().
  • 1.8.0 New signature for readList(), added streamable reading.
  • 1.7.2 Technical release.
  • 1.7.1 Minor performance tweaks, bugfixes in the test case.
  • 1.7.0 Added deleteByNames() and cloneByNames(), renamed getAllByKeys() to getByKeys().
  • 1.6.3 Added updateParams() to add writing conditions.
  • 1.6.2 Bugfix in KoaAdapter's mass methods: augmenting instead of copying.
  • 1.6.1 Return of clone() and cloneAll() as default methods.
  • 1.6.0 Added doClone().
  • 1.5.0 Refactored, simplified, added more canned list-based operations.
  • 1.4.1 Added ability to return undefined for mapping functions.
  • 1.4.0 Added helper methods: toDynamo(), toDynamoKey(), fromDynamo().
  • 1.3.0 clone() added to KoaAdapter. A related bug was fixed.
  • 1.2.0 Major refactoring of internals. Some API changes.
  • 1.1.1 Bugfix: added the missing index file.
  • 1.1.0 Made a search prefix a parameter.
  • 1.0.0 The initial public release

License

The 3-Clause BSD License

Acknowledgements

The test JSON file containing planets of the Star Wars universe is extracted from SWAPI and used under BSD license.

Keywords

FAQs

Last updated on 22 Jan 2021

Did you know?

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc