Security News
Combatting Alert Fatigue by Prioritizing Malicious Intent
In 2023, data breaches surged 78% from zero-day and supply chain attacks, but developers are still buried under alerts that are unable to prevent these threats.
A Node.js stream implementation of Amazon's Kinesis.
Allows the consumer to pump data directly into (and out of) a Kinesis stream.
This makes it trivial to setup Kinesis as a logging sink with Bunyan, or any other logging library.
For setting up a local Kinesis instance (eg for testing), check out Kinesalite.
var fs = require('fs'),
Transform = require('stream').Transform,
kinesis = require('kinesis'),
KinesisStream = kinesis.KinesisStream
// Uses credentials from process.env by default
kinesis.listStreams({region: 'us-west-1'}, function(err, streams) {
if (err) throw err
console.log(streams)
// ["http-logs", "click-logs"]
})
var kinesisSink = kinesis.stream('http-logs')
// OR new KinesisStream('http-logs')
fs.createReadStream('http.log').pipe(kinesisSink)
var kinesisSource = kinesis.stream({name: 'click-logs', oldest: true})
// Data is retrieved as Record objects, so let's transform into Buffers
var bufferify = new Transform({objectMode: true})
bufferify._transform = function(record, encoding, cb) {
cb(null, record.Data)
}
kinesisSource.pipe(bufferify).pipe(fs.createWriteStream('click.log'))
// Create a new Kinesis stream using the raw API
kinesis.request('CreateStream', {StreamName: 'test', ShardCount: 2}, function(err) {
if (err) throw err
kinesis.request('DescribeStream', {StreamName: 'test'}, function(err, data) {
if (err) throw err
console.dir(data)
})
})
Returns a readable and writable Node.js stream for the given Kinesis stream
options
include:
region
: a string, or (deprecated) object with AWS credentials, host, port, etc (resolved from env or file by default)credentials
: an object with accessKeyId
/secretAccessKey
properties (resolved from env, file or IAM by default)shards
: an array of shard IDs, or shard objects. If not provided, these will be fetched and cached.oldest
: if truthy, then will start at the oldest records (using TRIM_HORIZON
) instead of the latestwriteConcurrency
: how many parallel writes to allow (1
by default)cacheSize
: number of PartitionKey-to-SequenceNumber mappings to cache (1000
by default)agent
: HTTP agent used (uses Node.js defaults otherwise)timeout
: HTTP request timeout (uses Node.js defaults otherwise)initialRetryMs
: first pause before retrying under the default policy (50
by default)maxRetries
: max number of retries under the default policy (10
by default)errorCodes
: array of Node.js error codes to retry on (['EADDRINFO', 'ETIMEDOUT', 'ECONNRESET', 'ESOCKETTIMEDOUT', 'ENOTFOUND', 'EMFILE']
by default)errorNames
: array of Kinesis exceptions to retry on
(['ProvisionedThroughputExceededException', 'ThrottlingException']
by default)retryPolicy
: a function to implement a retry policy different from the default oneCalls the callback with an array of all stream names for the AWS account
Makes a generic Kinesis request with the given action (eg, ListStreams
) and data as the body.
options
include:
region
: a string, or (deprecated) object with AWS credentials, host, port, etc (resolved from env or file by default)credentials
: an object with accessKeyId
/secretAccessKey
properties (resolved from env, file or IAM by default)agent
: HTTP agent used (uses Node.js defaults otherwise)timeout
: HTTP request timeout (uses Node.js defaults otherwise)initialRetryMs
: first pause before retrying under the default policy (50
by default)maxRetries
: max number of retries under the default policy (10
by default)errorCodes
: array of Node.js error codes to retry on (['EADDRINFO', 'ETIMEDOUT', 'ECONNRESET', 'ESOCKETTIMEDOUT', 'ENOTFOUND', 'EMFILE']
by default)errorNames
: array of Kinesis exceptions to retry on
(['ProvisionedThroughputExceededException', 'ThrottlingException']
by default)retryPolicy
: a function to implement a retry policy different from the default oneFAQs
A stream implementation of Amazon's Kinesis
The npm package kinesis receives a total of 165 weekly downloads. As such, kinesis popularity was classified as not popular.
We found that kinesis demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
In 2023, data breaches surged 78% from zero-day and supply chain attacks, but developers are still buried under alerts that are unable to prevent these threats.
Security News
Solo open source maintainers face burnout and security challenges, with 60% unpaid and 60% considering quitting.
Security News
License exceptions modify the terms of open source licenses, impacting how software can be used, modified, and distributed. Developers should be aware of the legal implications of these exceptions.