polkadot-secure-validator - npm Package Compare versions

Comparing version 2.0.0 to 2.1.0

package.json

 {
   "name": "polkadot-secure-validator",
-  "version": "2.0.0",
+  "version": "2.1.0",
   "main": "src/index.js",
@@ -5,0 +5,0 @@ "repository": "https://github.com/w3f/polkadot-secure-validator",

README.md

@@ -5,108 +5,45 @@ [![CircleCI](https://circleci.com/gh/w3f/polkadot-secure-validator.svg?style=svg)](https://circleci.com/gh/w3f/polkadot-secure-validator)
 
-This repo describes a potential setup for a Polkadot validator that aims to prevent
-some types of potential attacks.
+This repo describes a potential setup for a Polkadot validator that aims to
+prevent some types of potential attacks, as described in the
+[Polkadot Secure Validator approach](https://hackmd.io/QSJlqjZpQBihEU_ojmtR8g).
+The [Workflow](#workflow) section describes the [Platform Layer](#platform-layer)
+and the [Application Layer](#application-layer) in more detail.
 
-## How to use
+![Polkadot Secure Network Chart](secure_network_chart.svg)
 
-This repo has code for creating a complete implementation of the approach
-described [here](https://hackmd.io/QSJlqjZpQBihEU_ojmtR8g) from scratch, including
-both layers described in [Workflow](#workflow). This can be done on a host with
-NodeJS, Yarn and Git installed with:
+## Usage
 
-### Prerequisites
+There are two ways of using this repository:
 
-Before using polkadot-secure-validator you need to have installed:
+* **Platform & Application Layer**
 
-* NodeJS (we recommend using [nvm](https://github.com/nvm-sh/nvm))
+  Configure credentials for infrastructure providers such as AWS, Azure, GCP
+  and/or Packet, then execute the Terraform process to automatically deploy the
+  required machines ([Platform Layer](#platform-layer)) and setup the
+  [Application Layer](#application-layer).
 
-* [Yarn](https://yarnpkg.com/lang/en/docs/install)
+  See the [Complete Guide](GUIDE_COMPLETE.md) for more.
 
-* [Terraform](https://www.terraform.io/downloads.html) (the snap package available via your package manager will not work)
+* **Application Layer**
 
-* [Ansible](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html) (v2.8+, available through pip)
+  Setup Debian-based machines yourself, which only need basic SSH access and
+  configure those in an inventory. The Ansible scripts will setup the entire
+  [Application Layer](#application-layer).
 
-You will need credentials as environment variables for all the infrastructure providers
-used in the platform creation phase. The tool now supports AWS, Azure, GCP and packet,
-these are the required variables:
+  See the [Ansible Guide](GUIDE_ANSIBLE.md) for more.
 
-* AWS: `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` of an IAM account with EC2
-and VPC write access.
-* Azure: `ARM_CLIENT_ID`, `ARM_CLIENT_SECRET`, `ARM_SUBSCRIPTION_ID`,
-`ARM_TENANT_ID`, `TF_VAR_client_id` (same as `ARM_CLIENT_ID`),
-`TF_VAR_client_secret` (same as `ARM_CLIENT_SECRET`). All these credentials
-should correspond to a service principal with at least a `Contributor` role,
-see [here](https://docs.microsoft.com/en-us/azure/role-based-access-control/role-assignments-portal)
-for details or [create an issue](https://github.com/w3f/polkadot-secure-validator/issues/new) for
-finer grained access control.
-* GCP: `GOOGLE_APPLICATION_CREDENTIALS` (path to json file with credentials of
-the service account you want to use; this service account needs to have write
-access to compute and network resources).
-* PACKET: `TF_VAR_auth_token`.
-* DigitalOcean: `TF_VAR_do_token`.
-
-The tool allows you to specify which providers to use, so you don't need to have
-accounts in all of them, see [here](https://github.com/w3f/polkadot-secure-validator/blob/master/config/main.sample.json)
-for an example of how to define the providers. You could use, for instance,
-packet for the validators and GCP for the public nodes. Keep in mind that, the
-more distributed your public nodes, the fewer opportunities to be affected by
-potential incidents in the respective cloud providers.
-
-You need two additional environment variables to allow ansible to connect to the
-created machines:
-
-* `SSH_ID_RSA_PUBLIC`: path to private SSH key you want to use for the public
-nodes.
-
-* `SSH_ID_RSA_VALIDATOR`: path to private SSH key you want to use for the
-validators.
-
-You can easily create and add them to your ssh-agent as follows:
-
-```bash
-$ ssh-keygen -f <path>
-$ ssh-add <path>
-```
-
-### Synchronization
-
-```
-$ git clone https://github.com/w3f/secure-validator
-$ cd secure-validator
-$ yarn
-$ cp config/main.template.json config/main.json
-# now you should complete and customize config/main.json, using main.sample.json as a reference
-$ yarn sync -c config/main.json
-```
-
-You can also just provision a set of previously created machines with the ansible code
-[here](./ansible). We have provided an [example inventory](./ansible/inventory.sample)
-that you can customize.
-
-The `sync` command is idempotent, unless there are errors it will always have
-the same results. You can execute it as much as you want, it will only make
-changes when the actual infrastructure state doesn't match the desired state.
-
-### Cleaning up
-
-You can remove all the created infrastructure with:
-
-```
-$ yarn clean -c config/main.json
-```
-
 ## Structure
 
-The secure validator setup is composed of a bare-metal machine that runs the
-actual validator and a set of cloud nodes connected to it. The validator is
-isolated from the internet and only has access to the Polkadot network through
-the cloud nodes, which are accessible from the internet and are connected to
-the rest of the Polkadot network.
+The secure validator setup is composed of one or more validators and a set of 
+public nodes nodes connected to it. The validators are isolated from the internet 
+and only have access to the Polkadot network through
+the public nodes.
 
-The connection between the validator node and the cloud nodes is performed
+The connection between the validator nodes and the public nodes is performed
 defining a VPN to which all these nodes belong. The Polkadot instance running in
-the validator node is configured to only listen on the VPN-attached interface,
-and uses the cloud node's VPN address in the `--reserved-nodes` parameter. It is
+the validator nodes are configured to only listen on the VPN-attached interface,
+and uses the public node's VPN address in the `--reserved-nodes` parameter. It is
 also protected by a firewall that only allows connections on the VPN port.
 
-This way, the only nodes allowed to connect to the validator are the public nodes
+This way, the only nodes allowed to connect to the validators are the public nodes
 through the VPN. Messages sent by other validators can still reach it through
@@ -133,16 +70,13 @@ gossiping, and these validators can know the IP address of the secure validator
 
-### Platform creation
+### Platform Layer
 
-Because of the different nature of the validator and the cloud nodes, the
-platform is hybrid, consisting of a bare-metal machine and cloud instances.
-However, we use terraform for creating both. The code for setting up the
-bare-metal machine is in the [terraform](/terraform) dir
-of this repository.
+Both validator and public nodes are created in a similar way using the terraform
+modules located at [terraform](/terraform) directory. We have created code for 
+several providers but it is possible to add new ones, please reach out if you 
+are interested in any provider currently not available.
 
-The cloud instances are created on 3 different cloud providers for increased
-resiliency, and the bare-metal machine on packet.com. As part of the creation
-process of the cloud instances we define a hardware firewall to only allow access
-on the VPN and p2p ports.
+Besides the actual machines the terraform modules create the minimum required networking
+infrastructure for adding firewall rules to protect the nodes.
 
-### Application creation
+### Application Layer
 
@@ -209,3 +143,3 @@ This is done through the ansible playbook and roles located at [ansible](/ansible), the
 
-* Private (validator) node:
+* Private (validator) nodes:
 
@@ -212,0 +146,0 @@ * Start Polkadot service: the private (validator) node is started with the node's VPN address as part

src/index.js

@@ -11,2 +11,3 @@ #!/usr/bin/env node
 const sync = require('./lib/actions/sync');
+const plan = require('./lib/actions/plan');
 const version = require('./lib/version');
@@ -30,3 +31,9 @@
 
+program
+  .command('plan')
+  .description('Shows changes in the infrastructure layer that would be performed by sync.')
+  .option('-c, --config [path]', 'Path to config file.', './config/main.json')
+  .action(plan.do);
 
+
 program.allowUnknownOption(false);
@@ -33,0 +40,0 @@

src/lib/clients/terraform.js

@@ -23,3 +23,3 @@ const fs = require('fs-extra');
 
-  async sync() {
+  async sync(method='apply') {
     this._initializeTerraform();
@@ -36,3 +36,3 @@ try {
     try {
-      validatorSyncPromises = await this._create('validator', sshKeys.validatorPublicKey, this.config.validators.nodes);
+      validatorSyncPromises = await this._create('validator', sshKeys.validatorPublicKey, this.config.validators.nodes, method);
     } catch(e) {
@@ -44,3 +44,3 @@ console.log(`Could not get validator sync promises: ${e.message}`);
     try {
-      publicNodeSyncPromises = await this._create('publicNode', sshKeys.publicNodePublicKey, this.config.publicNodes.nodes);
+      publicNodeSyncPromises = await this._create('publicNode', sshKeys.publicNodePublicKey, this.config.publicNodes.nodes, method);
     } catch(e) {
@@ -82,3 +82,3 @@ console.log(`Could not get publicNodes sync promises: ${e.message}`);
 
-  async _create(type, sshKey, nodes) {
+  async _create(type, sshKey, nodes, method='apply') {
     const createPromises = [];
@@ -96,4 +96,9 @@
 
-        await this._cmd(`apply -auto-approve`, options);
+        let cmd = method;
+        if (method === 'apply'){
+          cmd += ' -auto-approve';
+        }
 
+        await this._cmd(cmd, options);
+
         resolve(true);
@@ -100,0 +105,0 @@ }));

src/lib/platform.js

@@ -13,3 +13,3 @@ const asyncUtils = require('./async.js');
   async sync() {
-    await this.tf.sync();
+    await this.tf.sync('apply');
 
@@ -22,2 +22,6 @@ const validatorIpAddresses = await this._extractOutput('validator', this.config.validators.nodes);
 
+  async platform() {
+    return this.tf.sync('plan');
+  }
+
   async clean() {
@@ -24,0 +28,0 @@ return this.tf.clean();

No alert changes

Improved metrics

Total package byte prevSize

138121

increased by0.74%

Number of package files

163

increased by3.82%

Lines of code

772

increased by4.18%

Worsened metrics

Number of lines in readme file

453

decreased by-12.72%

No dependency changes