Huge News!Announcing our $40M Series B led by Abstract Ventures.Learn More
Socket
Sign inDemoInstall
Socket

db-streamer

Package Overview
Dependencies
Maintainers
1
Versions
16
Alerts
File Explorer

Advanced tools

Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

db-streamer

A library to stream data into a SQL database.

  • 0.3.0
  • Source
  • npm
  • Socket score

Version published
Weekly downloads
1
decreased by-50%
Maintainers
1
Weekly downloads
 
Created
Source

db-streamer

npm version

A library to stream data into a SQL database.

Current Status

Project is in alpha version. Currently supports streaming data into PostgreSQL table.

Usage

var dbStreamer = require('db-streamer');

// create inserter
var inserter = dbStreamer.getInserter({
  dbConnString: 'postgres://streamer:streamer@localhost:5432/streamer-test',
  tableName: 'test_table',
  columns: ['a', 'b', 'c']
});

// establish connection
inserter.connect(function(err, client) {

  // push some rows
  inserter.push({a: 1, b: 'one', c: new Date() });
  inserter.push({a: 2, b: 'two', c: new Date() });
  inserter.push({a: 3, b: 'three', c: new Date() });

  // create child table inserter using deferring strategy
  // this is useful to avoid missing foreign key conflicts as a result of race conditions
  var childInserter = dbStreamer.getInserter({
    tableName: 'child_table',
    columns: ['a', 'd', 'e'],
    client: client,
    deferUntilEnd: true
  });

  childInserter.push({a: 2, d: 'asdf', e: new Date() });
  childInserter.push({a: 3, d: 'ghjk', e: new Date() });

  childInserter.setEndHandler(callback);

  // set end callback
  inserter.setEndHandler(function() {
    childInserter.end();
  });

  // announce end
  inserter.end();

});

Inserter Config

KeyDescription
dbConnStringA database connection string.
tableNameThe tablename to insert into.
columnsArray of column names.
clientOptional. A database client. Provide this in place of dbConnString.
deferUntilEndBoolean (default=false). Stream output to temporary file which is then streamed in all at once into table upon calling end.

Inserter Config (Sequelize Bulk Insert alternative)

KeyDescription
useSequelizeBulkInsertBoolean. Perform the insert using a combination of async.cargo and sequelize bulkInsert. Must provide sequelizeModel parameter too.
sequelizeModelThe sequelize model to perform a bulk insert with.
deferUntilEndBoolean (default=false). Pause all cargo iterations until calling end.

PostgreSQL

Additional Dependencies

In order to use this library, you must also install the following libraries:

npm install pg --save
npm install pg-copy-streams --save
npm install pg-hstore --save

Keywords

FAQs

Package last updated on 24 Sep 2015

Did you know?

Socket

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

Packages

npm

Stay in touch

Get open source security insights delivered straight into your inbox.


  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc