github.com/elastic/elasticsearch-clients-benchmarks

Elasticsearch Clients Benchmarks

Benchmarking framework for Elasticsearch clients.

Introduction

This repository contains all neccessary tooling for benchmarking the Elasticsearch clients.

It uses Terraform to create a fully configured infrastructure for running the benchmarks, includes the JSON data for benchmarks, and provides scripts to setup the reporting Elasticsearch cluster and Kibana dashboards.

Usage

You need to install Terraform and the Google Cloud Platform SDK to create and interact with the infrastructure; please follow the respective guidelines.

On Mac OS X, you can install them via Homebrew:

brew install terraform
brew cask install google-cloud-sdk

Clone this repository:

git clone https://github.com/elastic/elasticsearch-clients-benchmarks.git
cd elasticsearch-clients-benchmarks

To create the infrastructure in Google Cloud Platform (GCP), you need to export an environment variable pointing to a file with a service account:

export GOOGLE_CLOUD_KEYFILE_JSON=/.../foo-123def456.json

You need to export environment variables with the reporting Elasticsearch cluster URL and credentials, which are consumed by Terraform:

export TF_VAR_reporting_url='http://localhost:9200'
export TF_VAR_reporting_username='username'
export TF_VAR_reporting_password='password'

You need to export an environment variable with the fully-qualified client Docker image:

export CLIENT_IMAGE="docker.elastic.co/clients/go-elasticsearch:d3adb33f"

NOTE: The value is illustrative; see https://container-library.elastic.co/r/clients/go-elasticsearch for a list of images.

To create the infrastructure with the default settings, run the Terraform commands:

cd terraform/gcp
terraform init
terraform apply --var client_image="$CLIENT_IMAGE"

After the apply command finishes, verify that the runner instance is able to communicate with the target instance:

gcloud compute --project 'elastic-clients' ssh $(terraform output runner_instance_name) \
  --zone 'europe-west1-b' \
  --command="curl -s '$(terraform output --json cluster_urls | jq -r '.[0]')/_cat/nodes?v&h=name,ip,master'"
# > name   ip       master
# > es-001 10.0.0.2 *

In order to run the client benchmarks, execute them over an SSH connection:

gcloud compute --project 'elastic-clients' ssh $(terraform output runner_instance_name) \
  --zone='europe-west1-b' \
  --ssh-flag='-t' \
  --command="\
  CLIENT_BRANCH=master \
  CLIENT_BENCHMARK_ENVIRONMENT=development \
  /home/runner/runner.sh \
  'source /environment.sh && cd _benchmarks/benchmarks && go run cmd/main.go'"
# > Running benchmarks for go-elasticsearch@8.0.0-SNAPSHOT; linux/go1.14
# >  [ping]          1000×     mean=0s runner=success report=success
# >  ...

When you're finished running the benchmarks, destroy the infrastructure:

terraform destroy --var client_image="$CLIENT_IMAGE"

License

(c) 2020 Elasticsearch B.V. Licensed under the Apache License, Version 2.0.