elasticsearch-bulk-index-stream
A writable stream for bulk indexing records in Elasticsearch
Usage
Format
The records written to the stream has to have the following format:
{
index: 'name-of-index',
type: 'recordType',
id: 'recordId',
body: {
name: 'Foo Bar'
}
}
Buffering
The highWaterMark
option set on the stream defines how many items
will be buffered before doing a bulk indexing operation. The stream
will also write all buffered items if its is closed, before emitting
the finish
event.
Logging
A bunyan,
winston or similar logger
instance that have methods like debug
, error
and info
may be
sent in as options.logger
to the constructor.
Example
var ElasticsearchBulkIndexStream = require('elasticsearch-bulk-index-stream');
var stream = new ElasticsearchBulkIndexStream(elasticsearchClient, { highWaterMark: 256 });
someInputStream.pipe(stream);
See
Elasticsearch readable and writable streams. The main difference
between the bulk writer in elasticsearch-streams
and this library is
that this library requires the index
and type
of the data being
written to exist in the record instead of being set in a callback when
the records written.
License
MIT