nc-vault-env
This package provides a convenient way to launch a subprocess with environment variables populated from Vault.
How it works?
This tool fetches specified secrets then run your app with environment variables that contain secrets.
Also, propagate received signals to subprocess.
Getting started.
For Windows users only ;)
Generally you can also try to execute steps below w/o use of Docker
but do it on your own risk.
- Install Docker
docker run --rm -it -v ${PWD}:/codebase node:8 bash
cd codebase/
Add secrets to Vault
-
Vault install hashi-corp-vault
-
Configure Vault CLI
We need to setup following environment variable:
export VAULT_ADDR=https://vault.devops.namecheap.net
-
Run command to sign in into Vault server
vault auth -method=ldap username=<your_AD_username>
Input your AD password then
Expected response looks like this
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.
Key Value
--- -----
token <token>
token_accessor <token>
token_duration 768h
token_renewable true
token_policies [default <roles>]
token_meta_policies default,<team>
token_meta_username <your_username>
-
Add secret to the vault.
$ vault write secret/data password=pass**123 login=jack_sparrow
Success! Data written to: secret/data
Install & configure nc-vault-env
-
Install npm package
NPM Package: nc-vault-env
nc-vault-env
written in nodejs, so you need to install suitable versions.
It currently has been tested with 6.x
and 8.x
.
npm install -g nc-vault-env
-
Create config.json
In working directory you create config.json file
{
"vault": {
"address": "<%= env('VAULT_ADDR') %>",
"auth": {
"type": "token",
"config": {
"token": "<%= env('VAULT_TOKEN') %>"
}
}
},
"secrets": [
{
"path": "secret/data",
"format": "MY_ENV_<%= key %>",
"upcase": true
}
]
}
-
Export Vault auth token to env variable
$ export VAULT_TOKEN=$(cat ~/.vault-token)
-
Run command
$ nc-vault-env -c ./config.json -- printenv
MY_ENV_PASSWORD=pass**123
MY_ENV_LOGIN=jack_sparrow
WARNING: This command working on linux, ubuntu and macOS only.
Integrate with your application
-
For correct work in dockerfile you need add following lines:
RUN apt-get update \
&& apt-get install -y build-essential curl \
&& curl -sL https://deb.nodesource.com/setup_8.x | bash - \
&& apt-get install -y nodejs \
&& npm install -g nc-vault-env \
&& rm -rf /var/lib/apt/lists/*
COPY vault-env.conf.json .
CMD ["nc-vault-env", "-c", "./vault-env.conf.json", "--", "./<your_start_script>.sh"]
CLI
Options:
option | description |
---|
-c, --config | path to configuration file. |
-v, --verbosity | verbosity level. Supported "error", "warn", "info", "debug", "trace". Default is "info". |
-f, --log-format | logging format. Supported "json" and "text". Default is "json". |
Dummy mode
When you just want to skip secrets fetching and just run app/script without them we may use dummy mode.
Just pass env variable VAULTENV_DUMMY=true
, bash example:
VAULTENV_DUMMY=true nc-vault-env -c ./vault-env.conf.json -- ./<your_start_script>.sh
Configuration File API
Configuration files are written in json.
{
"vault": {
"address": "<%= env('VAULT_ADDR') %>",
"auth": {
"type": "token",
"mount": "<%= env('VAULT_AWS_AUTH_MOUNT') %>",
"config": {
}
}
},
"secrets": [
{
"path": "secret/my_awesome_team_namespace/<%= env('ENVIRONMENT') %>/rmq",
"format": "RMQ_<%= key %>",
"upcase": true
},
{
"path": "secret/my_awesome_team_namespace/<%= env('ENVIRONMENT') %>/mssql",
"format": "user id=<%= username %>;password=<%= password %>",
"key": "ConnectionString"
},
{
"path": "secret/my_awesome_team_namespace/<%= env('ENVIRONMENT') %>/shared",
"format": "<%= folder %>:<%= key %>",
"upcase": true,
"folder": true
},
{
"path": "secret/my_awesome_team_namespace/<%= env('ENVIRONMENT') %>/local",
"format": "<%= folder %>:<%= key %>",
"upcase": true,
"folder": true
}
]
}
Templating
Templating based on Lodash template function.
Predefined functions:
fn | description | usage |
---|
env | provides access to environment variables. | <%= env('VAULT_ADDR') %> |
| | |
Troubleshooting
For debugging purpose you can run this locally using you vault token (token auth backend).
This way assumes that you have access to all of your app's secrets.
Please be aware that debug or trace log level prints secret to stdout, so be careful with enable this level on real environment.
cat config.json
export VAULT_ADDR=https://vault.devops.namecheap.net
export VAULT_TOKEN=$(cat ~/.vault-token)
nc-vault-env -c config.json -f text -v trace -- run_my_app.sh
Recipes.
Add multiple items in value
- For adding multiple values in vault
$ vault write secret/data foo=world excited=yes count=1
Success! Data written to: secret/data
- Secret was added to /secret/data in format:
{
"foo": "world",
"excited": "yes",
"count":1,
}
Auth section for AWS using
- Use following code for authorization with Amazon Web Services
"auth": {
"type": "iam",
"mount": "<%= env('<VAULT_AWS_AUTH_MOUNT>') %>",
"config": {
"role": "<%= env('VAULT_ROLE') %>",
"iam_server_id_header_value": "<%= env('VAULT_ADDR') %>"
}
}
Secret format for the mysql connection string
- For passing connection string from the secrets use following secret configuration:
{
"path": "secret/my_awesome_team_namespace/<%= env('ENVIRONMENT') %>/mysql/creds/rw",
"format": "server=<%= env('DATABASE_HOST') %>;port=<%= env('DATABASE_PORT') %>;database=<%= env('DATABASE_NAME') %>;uid=<%= username %>;pwd=<%= password %>",
"key": "ConnectionString",
"upcase": false
},
Secret format for ASP.NET Core configuration class
- If you have configuration class:
public class MyConfiguration
{
public string Secret1 {get;set;}
public string Secret2 {get;set;}
}
- You add secrets with following name:
$ vault write secret/data test=12 Secret1=secret Secret2=true
Success! Data written to: secret/data
- And create secret config section in config.json:
{
"path": "secret/my_awesome_team_namespace/<%= env('ENVIRONMENT') %>/config",
"format": "<%= key %>",
"upcase": false
}
- All configuration will be passed as environment variables.
Nested classes
- If you have nested configuration class:
public class MyConfiguration
{
public ItemClass Item { get; set; }
}
public class ItemClass
{
public string SubItem { get; set; }
}
- And create secret config section:
{
"path": "secret/my_awesome_team_namespace/<%= env('ENVIRONMENT') %>/config",
"format": "<%= value %>",
"key": "item__subitem",
"upcase": false
}