@casechek/aws-cdk-elasticsearch-index

Elasticsearch Index CDK Construct

This construct allows you to deploy an elasticsearch index and all its settings and mappings within a stack.

Table of Contents

• Elasticsearch Index CDK Construct
  • Table of Contents
  • Usage
    • Installation
    • Create an Elasticsearch Domain (Optional)
    • Create an Index
  • Tip for testing in an existing project
  • Contributing
    • Required Software
    • Obtaining the Source
    • Docker
      • Running the docker-compose Stack
    • Testing
      • Running jest tests
      • Running cucumber tests
  • Useful commands

Usage

Installation

Install the package in your CDK application.

npm install @casechek/aws-cdk-elasticsearch-index

Create an Elasticsearch Domain (Optional)

This is currently in the experimental stage over at AWS; however, it is the easiest way to setup a basic Elasticsearch cluster.

// in your stack constructor
import {CfnDomain} from '@aws-cdk/aws-cdk-elasticsearch-index';

const elasticsearchDomain = new CfnDomain(
  this,
  'ElasticsearchDomain',
  {
    accessPolicies: {
      Version: '2012-10-17',
      Statement: [
        {
          Effect: 'Allow',
          Principal: {
            AWS: '*',
          },
          Action: 'es:*',
          Resource: `arn:aws:es:${this.region}:${this.account}:domain/${this.stackName}/*`,
        },
      ],
    },
    ebsOptions: {
      ebsEnabled: true,
      volumeSize: 20,
      volumeType: 'standard',
    },
    elasticsearchClusterConfig: {
      instanceCount: 1,
      instanceType: 't2.medium.elasticsearch',
    },
    domainName: this.stackName,
    encryptionAtRestOptions: {
      enabled: false,
    },
    nodeToNodeEncryptionOptions: {
      enabled: true,
    },
    vpcOptions: {
      securityGroupIds: [securityGroupIdFromSomewhere],
      subnetIds: [privateSubnetIdFromSomewhere],
    },
    elasticsearchVersion: '7.1',
  }
);

Create an Index

The construct

import {ElasticsearchIndex} from 'aws-cdk-elasticsearch-index'
const es = new ElasticsearchIndex(this, 'ElasticsearchIndex', {
  mappingJSONPath: PATH_TO_A_JSON_FILE_CONTAINING_FULL_INDEX_CONFIGURATION,
  elasticSearchIndex: NAME_OF_THE_INDEX_TO_CREATE,
  elasticSearchEndpoint: FULL_URL_TO_THE_ELASTICSEARCH_ENDPOINT_INCLUDING_SCHEME,
  vpc: VPC_OF_YOUR_ELASTIC_SEARCH_DOMAIN, // If you want to host the lambda functions responsible for resource creation in your vpc
  policyArn: elasticsearchDomain.attrArn, // where elasticSearch domain is your elasticsearch CDK construct instance, only required if you are using AWS Elasticsearch
});

Tip for testing in an existing project

It's helpful to be able to test a change you're working on in your own project's stack. To do that with a working branch of this repo, here are some tips:

Note: This is assuming you have aws-cdk-elasticsearch-index and some my_project project directory in 2 folders in ~/dev/.

• In my_project's package.json, set "@casechek/aws-cdk-elasticsearch-index": "../aws-cdk-elasticsearch-index", for the dependency version
• Run npm install in my_project (maybe npm ci is needed for a clean install)
• Run npm install in aws-cdk-elasticsearch-index then run npm run build
• Then wipe the node_modules directory (run rm -rf node_modules) in aws-cdk-elasticsearch-index
• Then link your project's node_modules with the one in custom construct's directory:
  • run ln -s ../my_project/node_modules in aws-cdk-elasticsearch-index

With this, you should be able to deploy your project's stack, and it will be using your modified custom construct.

Note: Don't forget to remove the symlink if you're going through various npm installs, or you'll run into a stack memory issue. In order to do that, you can delete ./node_modules.

Contributing

Required Software

• git
• docker
• docker-compose
• nodejs
• aws-cdk

Obtaining the Source

git clone [e-mail]:incompass/aws-cdk-elasticsearch-domain

Docker

To help contributing to this project, a docker-compose configuration is provided that includes:

• lambci/docker-lambda for the on-event lambda function
• elasticsearch
• kibana
• mockserver

Running the docker-compose Stack

docker-compose up

Testing

This package includes both jest and cucumber-js tests. Jest is used for unit testing the lambda functions and for testing the cdk stack using the AWS supplied helpers. Cucumber is used for e2e testing the lambda functions. All contributions must include relevant tests.

Running jest tests

npm run test

Running cucumber tests

You need to start the docker-compose stack before running the end to end tests. In one terminal, run:

docker-compose up

In another window, set the appropriate environment variables and run cucumber:

Bash:

AWS_ENDPOINT=http://localhost \
AWS_REGION=us-east-1 \
S3_ENDPOINT=http://localhost:1080 \
ON_EVENT_PORT=9001 \
ON_EVENT_S3_BUCKET_NAME=test-bucket \
ON_EVENT_S3_OBJECT_KEY=test-object-key \
ON_EVENT_INDEX=test-index \
ELASTICSEARCH_ENDPOINT=http://localhost:9200 \
ELASTICSEARCH_INDEX=test-index \
npm run cucumber -- --tags=@on-event

Powershell:

$env:AWS_ENDPOINT='http://localhost';
$env:AWS_REGION='us-east-1';
$env:S3_ENDPOINT='http://localhost:1080';
$env:ON_EVENT_PORT='9001';
$env:ON_EVENT_S3_BUCkET_NAME='test-bucket';
$env:ON_EVENT_S3_OBJECT_KEY='test-object-key';
$env:ON_EVENT_INDEX='test-index';
$env:ELASTICSEARCH_ENDPOINT='http://localhost:9200';
$env:ELASTICSEARCH_INDEX='test-index';
npm run cucumber -- --tags=@on-event

Commit Messages

This package uses the conventional commit message format. All PRs will be squashed and merged.

Useful commands

• npm run build compile typescript to js
• npm run watch watch for changes and compile
• npm run test perform the jest unit tests
• npm run cucumber perform the cucumber e2e tests