kafka-streams
Advanced tools
Changelog
2020-02-24, Version 5.0.0
node-rdkafka anymore. In case you are using the
native client with kafka-streams or kafka-connect you have to install it manually (see below).If you are using the native mode (config: { noptions: {} }).
You will have to manually install node-rdkafka alongside kafka-streams.
(This requires a Node.js version between 9 and 12 and will not work with Node.js >= 13, last tested with 12.16.1)
On Mac OS High Sierra / Mojave:
CPPFLAGS=-I/usr/local/opt/openssl/include LDFLAGS=-L/usr/local/opt/openssl/lib yarn add --frozen-lockfile node-rdkafka@2.7.4
Otherwise:
yarn add --frozen-lockfile node-rdkafka@2.7.4
(Please also note: Doing this with npm does not work, it will remove your deps, npm i -g yarn)
Readme
// suggested Node.js version: v12.16.1
npm install --save kafka-streams
const {KafkaStreams} = require("kafka-streams");
const config = require("./config.json");
const factory = new KafkaStreams(config);
const kstream = factory.getKStream("input-topic");
const ktable = factory.getKTable(/* .. */);
kstream.merge(ktable).filter(/* .. */).map(/* .. */).reduce(/* .. */).to("output-topic");
CHANGES: The latest version brings a lot of changes, please check here before updating.
>= 0.11.x>= 8.x.xIf you are using the native mode (config: { noptions: {} }).
You will have to manually install node-rdkafka alongside kafka-streams.
(This requires a Node.js version between 9 and 12 and will not work with Node.js >= 13, last tested with 12.16.1)
On Mac OS High Sierra / Mojave:
CPPFLAGS=-I/usr/local/opt/openssl/include LDFLAGS=-L/usr/local/opt/openssl/lib yarn add --frozen-lockfile node-rdkafka@2.7.4
Otherwise:
yarn add --frozen-lockfile node-rdkafka@2.7.4
(Please also note: Doing this with npm does not work, it will remove your deps, npm i -g yarn)
kafka-streams :octopus: equivalent for nodejs :sparkles::turtle::rocket::sparkles: build on super fast :fire: observables using most.js :metal:
ships with sinek :pray: for backpressure
comes with js and native Kafka client, for more performance and SSL, SASL and Kerberos features
the lib also comes with a few window operations that are more similar to Apache Flink,
yet they still feel natural in this api :squirrel:
overwriteable local-storage solution allows for any kind of datastore e.g. RocksDB, Redis, Postgres..
async (Promises) and sync stream operators e.g. stream$.map() or stream$.asyncMap()
super easy API :goberserk:
the lib is based on sinek, which is based on kafka-node's ConsumerGroups
librdkafka for more performanceYes.
Probably, yes. :smile:
Forks or Stars give motivation :bowtie:
FAQs
kafka-streams for Node.js
The npm package kafka-streams receives a total of 339 weekly downloads. As such, kafka-streams popularity was classified as not popular.
We found that kafka-streams demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
As cyber threats become more autonomous, AI-powered defenses are crucial for businesses to stay ahead of attackers who can exploit software vulnerabilities at scale.
Security News
UnitedHealth Group disclosed that the ransomware attack on Change Healthcare compromised protected health information for millions in the U.S., with estimated costs to the company expected to reach $1 billion.
Security News
GitHub is susceptible to a CDN flaw that allows attackers to host malware on any public repository.