micro-kit-aws-queue

module-aws-queue

This module implements a generic queue using Amazon SNS/SQS to allow micro-services to communicate without having to use AWS directly.

Endpoints

Note that all endpoints are asynchronous, returning a promise that gets resolved once the endpoint has completed.

ensureTopicExists(topic)

Check to see if the given topic exists in SNS. If not, the topic is created.

The returned promise will be resolved with the ARN for the desired topic.

publishToTopic(message, topic)

Publish the given message to the given topic (which is automatically assured)

message must be either a string or a JSON-compatible data structure.

ensureQueueExists(queue)

Ensure the given queue exists in SQS. If not, the queue is created.

The returned promise will be resolved with the ARN for the queue.

subscribeQueueToTopic(queue, topic)

Given a queue and topic name, subscribe the given queue to the given topic.

The returned promise will be resolved with the ARN for the subscription.

listenToQueue(queue, frequency, callback)

Regularly poll the given queue to see if any new messages have arrived. The parameters are as follows:

• queue: the name of the queue to listen to.
• frequency: the desired polling frequency, in seconds.
• callback: an asynchronous function which takes a single parameter, message, which is a string containing the message received from the queue.

Note that the callback function is asynchronous; when the returned promise is resolved, the message will be removed from the queue.

stopListeningToQueue(queue)

Stop polling for new messages on the given queue.

sendMessageToQueue(queue, message)

Sends message to the queue, queue must be name of the queue and message must be either a string or a JSON-compatible data structure.

The returned promise will be resolved with the MessageId.

Configuration

This module uses dotenv to store configuration details in an environment file. It looks for a file named .env in the current directory, and loads the following configuration settings from this file:

• AWS_REGION: The desired AWS region string.
• AWS_ACCESS_KEY_ID: The access key to use for authentication.
• AWS_SECRET_ACCESS_KEY: The secret to use for authentication.
• SNS_ENDPOINT: An optional URL to use for accessing the SNS server. Leave undefined to use the default AWS server.
• SQS_ENDPOINT: An optional URL to use for accessing the SQS server. Leave undefined to use the default AWS server.

Compiling

The queuing module uses flow for type checking. To check that the code is all using the correct data types, type:

npm run flow

To strip the flow data type definitions from the source, type:

npm run build

This creates a plain-vanilla JS version of the module in the flow-typed directory.

Testing

Note that you must type npm run build before running any of these tests.

To run the unit tests:

npm run unit

To run the integration tests:

npm run integration

You must have the AWS credentials set in the .env file for this to work.

To run Istanbul coverage:

npm run coverage

To check that the source matches the Javascript Standard Style rules:

npm run standard

How to set up local AWS SQS/SNS for development

Use this project, docker version or go https://github.com/p4tin/goaws

You can find more about docker here: https://docs.docker.com/engine/installation/

Docker version: All you need to do is run this commands Command to download the image:

docker pull pafortin/goaws

Command to run it on port 4100:

docker run -d --name goaws -p 4100:4100 pafortin/goaws

You can also run it with docker-compose. Create a file docker-compose.yml and insert this code:

awsQueue:
  restart: always
  container_name: awsQueue
  image: pafortin/goaws
  ports:
  - "4100:4100"

Then run the file with command

docker-compose up

Now you have your own AWS SQS/SNS systems on http://localhost:4100

To-Do

Still to do:

• A gitlab-ci file for automated testing.

• Docker configuration to run the tests on Docker.

• Documentation needs update