hive-io-domain-example

hive-io-domain-example

An example CQRS/ES domain module to help describe implementation details when leveraging the Hive^io framework.

Contents

• Overview
  • Endpoints
  • Source Code
• Getting Started
  • Prerequisites
  • Installing
  • Environment Variables

Overview

This example evolves the previous hive-io-rest-example into a highly distributed architecture in order to handle different magnitudes of network traffic for viewed metrics and content management. It is a contrived but robust example to illustrate different ways to use Actors in the Hive^io framework.

Endpoints

Once you get the app running using the setup instructions below, you can use the application from the following endpoint(s):

• http://localhost/posts/<postId> (GET)
• http://localhost/posts (GET, POST)
  • CreateContent API JSON Schema
    • PostId API JSON Schema ref
    • Content API JSON Schema ref
• http://localhost/posts/<postId>/content (PATCH, DELETE)
  • PATCH EditContent API JSON Schema
    • PostId API JSON Schema ref
    • Content API JSON Schema ref
  • PATCH EnableContent Payload = {}
  • DELETE DisableContent Payload = {}
• http://localhost/posts/<postId>/viewed (POST)
  • ViewContent Payload = {}

NOTE: Network data payloads follow the Flux Standard Action specification for network transport.

Source Code

Getting Started

This is a straight forward CQRS/ES example of a Post Entity that contains text, a couple boolean flags, and a count of how many views it has. It is a highly distributed application with the expectation that viewed traffic will be much larger than content management traffic. It stores these Posts in MongoDB. It implements an Actor System to handle logging to Fluentd. Here's how to use it.

Prerequisites

To use, you'll need:

• Required
  • Docker
  • Docker Compose

Installing

To start using:

1. Create the following files:
   • Producer.dockerfile

     FROM fnalabs/hive-producer-js:latest
     RUN npm install hive-io-domain-example
     

   • Stream-Processor.dockerfile

     FROM fnalabs/hive-stream-processor-js:latest
     RUN npm install hive-io-domain-example
     

   • Consumer.dockerfile

     FROM fnalabs/hive-consumer-js:latest
     RUN npm install hive-io-domain-example
     

   • Rest.dockerfile

     FROM fnalabs/hive-rest-js:latest
     RUN npm install hive-io-domain-example
     

   • docker-compose.yml

     version: '3.5'
     services:
       # proxy for layer 7 routing
       hive-io-proxy:
         image: fnalabs/hive-io-proxy:latest
         depends_on:
           - hive-producer-js
           - hive-rest-js
           - hive-stream-processor-js
         ports:
           - 80:80
         networks:
           - hive-io
         restart: on-failure
       fluentd:
         image: fluent/fluentd:v1.2.1
         networks:
           - hive-io
         restart: on-failure
     
       # producers
       hive-producer-js:
         build:
           context: .
           dockerfile: Producer.dockerfile
         image: hive-producer-js
         environment:
           CLUSTER_SIZE: 1
           EVENT_STORE_URL: "kafka:9092"
           EVENT_STORE_ID: "producer-client"
           FLUENTD_HOST: fluentd
           FLUENTD_PORT: 24224
           FLUENTD_TIMEOUT: 3.0
           FLUENTD_RECONNECT: 600000
         depends_on:
           - kafka
           - fluentd
         networks:
           - hive-io
     
       # stream processors
       hive-stream-processor-js:
         build:
           context: .
           dockerfile: Stream-Processor.dockerfile
         image: hive-stream-processor-js
         environment:
           CLUSTER_SIZE: 1
           CACHE_URL: "redis://redis:6379"
           EVENT_STORE_URL: "kafka:9092"
           EVENT_STORE_ID: stream-processor-client
           FLUENTD_HOST: fluentd
           FLUENTD_PORT: 24224
           FLUENTD_TIMEOUT: 3.0
           FLUENTD_RECONNECT: 600000
         depends_on:
           - redis
           - kafka
           - fluentd
         networks:
           - hive-io
       redis:
         image: redis:4.0.9-alpine
         networks:
           - hive-io
         restart: on-failure
     
       # log stream containers
       kafka:
         image: confluentinc/cp-kafka:4.1.1-2
         depends_on:
           - zookeeper
         environment:
           KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
           KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
           KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
         networks:
           - hive-io
         restart: on-failure
       zookeeper:
         image: confluentinc/cp-zookeeper:4.1.1-2
         environment:
           ZOOKEEPER_CLIENT_PORT: 2181
         networks:
           - hive-io
         restart: on-failure
     
       # consumers
       hive-consumer-js:
         build:
           context: .
           dockerfile: Consumer.dockerfile
         image: hive-consumer-js
         environment:
           CLUSTER_SIZE: 1
           MONGO_URL: "mongodb://mongo:27017/post"
           EVENT_STORE_URL: "kafka:9092"
           EVENT_STORE_ID: consumer-client
           EVENT_STORE_OFFSET: earliest
           FLUENTD_HOST: fluentd
           FLUENTD_PORT: 24224
           FLUENTD_TIMEOUT: 3.0
           FLUENTD_RECONNECT: 600000
         depends_on:
           - mongo
           - kafka
           - fluentd
         networks:
           - hive-io
       mongo:
         image: mongo:3.6.5
         networks:
           - hive-io
         restart: on-failure
     
       # rest services
       hive-rest-js:
         build:
           context: .
           dockerfile: Rest.dockerfile
         image: hive-rest-js
         environment:
           ACTOR: PostQueryActor
           ACTOR_LIB: hive-io-domain-example
           CLUSTER_SIZE: 1
           MONGO_URL: "mongodb://mongo:27017/post"
           FLUENTD_HOST: fluentd
           FLUENTD_PORT: 24224
           FLUENTD_TIMEOUT: 3.0
           FLUENTD_RECONNECT: 600000
         depends_on:
           - mongo
           - fluentd
         networks:
           - hive-io
     
     # networking specifics
     networks:
       hive-io:
         driver: bridge
     

2. Run the following commands:

   $ docker-compose up
   

Environment Variables

The table below contains a reference to the custom environment variables used in the example. Standard environment variables are documented for the following service containers:

• hive-producer-js
• hive-stream-processor-js
• hive-consumer-js
• hive-rest-js

Name	Type	Default	Description
MONGO_URL	String	'mongodb://mongo:27017/post'	url to connect to MongoDB instance
FLUENTD_HOST	String	'fluentd'	Hostname of Fluentd instance
FLUENTD_PORT	Number	24224	Port of Fluentd instance
FLUENTD_TIMEOUT	Number	3.0	Timeout (in sec) for Fluentd client
FLUENTD_RECONNECT	Number	600000	Reconnect Interval (in sec) for Fluentd client