lifion-kinesis
Advanced tools
Lifion's Node.js client for Amazon Kinesis Data Streams.
To install the module:
npm install lifion-kinesis --save
The main module export is a Kinesis class that instantiates as a readable stream.
const Kinesis = require('lifion-kinesis');
const client = new Kinesis({
streamName: 'sample-stream',
consumerName: 'sample-consumer'
/* other options from AWS.Kinesis */
});
client.on('data', chunk => {
console.log('Incoming data:', chunk);
});
To take advantage of back-pressure, the client can be piped to a writable stream:
const Kinesis = require('lifion-kinesis');
const { pipeline } = require('stream');
pipeline([
new Kinesis(/* options */),
new Writable({
objectMode: true,
write(data, encoding, callback) {
console.log(data);
callback();
}
})
]);
Incoming Features:
GetRecords API, with automatic checkpointing.Readable ⏏
PromiseReadable ⏏A specialization of the readable stream class implementing a
consumer of Kinesis Data Streams using the
enhanced fan-out feature. Upon connection, instances of this
class will subscribe to receive data from all the shards of the given stream. Incoming data can
be retrieved through either the data event or by piping the instance to a writable stream.
Kind: Exported class
Extends: Readable
Initializes a new instance of the Kinesis client.
| Param | Type | Default | Description |
|---|---|---|---|
| options | Object | The initialization options. In addition to the below options, this object can also contain the AWS.Kinesis options. | |
| [options.compression] | string | The kind of data compression to use with records. The currently available compression options are either "LZ-UTF8" or none. | |
| options.consumerName | string | The unique name of the consumer for the given stream. This option is required. | |
| [options.createStreamIfNeeded] | boolean | false | Whether if the Kinesis stream should be created if it doesn't exist upon connection. |
| [options.compression] | Object | The kind of compression to enforce in the stream. | |
| [options.compression.type] | string | The encryption type to use. | |
| [options.compression.keyId] | string | The GUID for the customer-managed AWS KMS key to use for encryption. This value can be a globally unique identifier, a fully specified ARN to either an alias or a key, or an alias name prefixed by "alias/". | |
| [options.logger] | Object | An object with the warn, debug, and error functions that will be used for logging purposes. If not provided, logging will be omitted. | |
| [options.shardCount] | number | 1 | The number of shards that the newly-created stream will use (if the createStreamIfNeeded option is set). |
| options.streamName | string | The name of the stream to consume data from. This option is required. | |
| [options.tags] | Object | If provided, the client will ensure that the stream is tagged with these hash of tags upon connection. If the stream is already tagged same tag keys, they won't be overriden. If the stream is already tagged with different tag keys, they won't be removed. |
PromiseInitializes the Kinesis client, then it proceeds to:
Kind: instance method of Kinesis
Fulfil: Once the stream is active, encrypted, tagged, the enhanced fan-out consumer is active,
and the client is subscribed to the data in all the stream shards.
Reject: Error - If at least one of the above steps fails to succeed.
v1.0.0 (2018-11-29)
#36: Update dependency aws-sdk to ^2.366.0#33: Update dependency lint-staged to ^8.1.0#32: Update dependency husky to ^1.2.0#34: Update dependency npm-watch to ^0.5.0#35: Update dependency aws-sdk to ^2.363.0#1: Enhanced Fan-Out Support#31: Update dependency eslint-config-lifion to ^1.0.3#29: Update dependency lint-staged to ^8.0.5#30: Update dependency auto-changelog to ^1.10.2#23: Update dependency prettier to ^1.15.2#28: Update dependency auto-changelog to ^1.10.1#25: Update dependency eslint to ^5.9.0#26: Update dependency auto-changelog to ^1.10.0#27: Update dependency husky to ^1.1.4#24: Update dependency auto-changelog to ^1.9.0#20: Update dependency eslint to ^5.8.0#21: Update dependency lint-staged to v8#22: Update dependency husky to ^1.1.3#19: Update dependency eslint to ^5.7.0#18: Update dependency husky to ^1.1.2#17: Update dependency semver to ^5.6.0#16: Update dependency eslint-config-lifion to ^1.0.2#15: Feature/enable npm publish ci#13: Update dependency auto-changelog to ^1.8.1#12: Update dependency eslint-config-lifion to ^1.0.1#11: Update dependency husky to ^1.1.1#10: Update dependency eslint to ^5.6.1#9: Update dependency husky to ^1.0.1#7: Update dependency eslint-config-lifion to v1#6: Update dependency eslint to v5#8: Update dependency husky to v1#5: Update dependency npm-watch to ^0.4.0#2: Configure Renovate1be561c: Initial commit9894deb: Add basic CircleCI integration27a7d4f: Re-create package-lock.json5823729: Add integration with CircleCIb5b9ce1: Set NVM to use the current LTS3ce84f9: Add management of stream consumers, refactor stream management562bea1: Add test coveragee5588a6: Add enhanced-fanout shard subscriber and initial parsera9765f1: Upgrade aws-sdk, eslint-config-lifion, and eslint38b435f: Update documentation4faac4d: Ensure streams are created, encrypted, and taggedc695f98: Add a Kinesis records decoder0dd328a: Recreate the shard subscription stream when expiring972c7f6: Adopt lifion-aws-event-stream40f156f: Remove the check-dependencies script in favor of Renovate1bb389b: Include the shard ID in the piped recordscd5d904: Connect the parsed records into the clientc591875: Fix the utils tests4f9fad8: Add test coverage for lib/utils197f2c4: Add deploy and publish to circleci30c0a3b: Replace toBeCalledWith calls with its canonical version5ad427a: Fix directory for .npmrc and add reports39f098d: Make the tag test cases more specific0ffdc6a: Rename project to lifion-kinesisce26fc3: Change the Renovate configuration07fd3aa: Delete renovate.jsonc8d578e: Add renovate.jsona641c0d: Make sure the CI stops if the tests fail23ac7b7: Correct the package-lock.json fileFAQs
Lifion client for Amazon Kinesis Data streams
The npm package lifion-kinesis receives a total of 4,127 weekly downloads. As such, lifion-kinesis popularity was classified as popular.
We found that lifion-kinesis demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 3 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Bun 1.2 enhances its JavaScript runtime with 90% Node.js compatibility, built-in S3 and Postgres support, HTML Imports, and faster, cloud-first performance.
Security News
Biden's executive order pushes for AI-driven cybersecurity, software supply chain transparency, and stronger protections for federal and open source systems.
Security News
Fluent Assertions is facing backlash after dropping the Apache license for a commercial model, leaving users blindsided and questioning contributor rights.