Node Red for working with apache kafka, a streaming product.
First initial release using kafka-node .
- Kafka Broker
- Kafka Admin
- Kafka Consumer
- Kafka ConsumerGroup
- Kafka Offset
- Kafka Producer
Has a test GUI which allows topics to be added.
Note: all nodes run in debug mode for 100 messages then turns off.
Kafka Broker
Defines the client interface to kafka.
Kafka Admin
Provide the ability to process administration tasks such as create and list topic.
Kafka Consumer
Consumer of topic messages in kafka which are generated into node-red message.
Provides types of base and high level.
Kafka Consumer Group
Consumer of topic messages in kafka which are generated into node-red message.
Kafka Offset
Get various offsets from Kafka. Which type are set via msg.action or msg.topic. msg.payload states the types of options.
Kafka Producer
Converts a node-red message into a kafka messages.
Provides types of base and high level.
Simple Web Admin Panel
Simple Web page monitor and admin panel
Producing flow can be found in test flow
Install
Run the following command in the root directory of your Node-RED install or via GUI install
npm install node-red-contrib-kafka-manager
Tests
Test/example flow in test/generalTest.json
Includes sample script for start kafka in windows using node-red
Version
0.2.9 Change debugging mechanism and add kafka-node to dependencies
0.2.8 Added all admin api's per Kafka 2.3 but dependent on kafka-node update.
Remove refresh metadata, automated if problem. Fix consumer group errors. Add tests for admin calls.
0.2.7 If offsetOutOfRange pause consumer. Added in deleteTopics but dependant on kafka-node update.
0.2.6 More fixes for error processing on invalid topic
0.2.4 Fix for error processing
0.2.3 Fix for multi nodes on broker
0.2.2 Stopped bug where producer on connection initiates a null message. Fix bug with restart logic on fail and order of messages on failed retry
0.2.1 When messages being queued as producer is waiting on connection or reconnection show producer in problem state.
0.2.0 Add nodes consumer group and offset.
Another fix to issue with initial no kafka.
describe groups implemented
0.1.0 Add in High level producer/consumer.
Further fixes to make connection more robust on kafka up/down
Multi host per broker
Multi Topic for consumer
0.0.2 Add in High level producer.
Make connection more robust on kafka up/down due to bugs and problem points in in kafka-node
0.0.1 base
Author
Peter Prib