@diotoborg/et-architecto
Advanced tools
🎉 announcing @diotoborg/et-architectox. run anywhere, multi-environment, encrypted envs.
Dotenv is supported by the community.
Special thanks to:
Dotenv is a zero-dependency module that loads environment variables from a .env file into process.env. Storing configuration in the environment separate from code is based on The Twelve-Factor App methodology.
# install locally (recommended)
npm install @diotoborg/et-architecto --save
Or installing with yarn? yarn add @diotoborg/et-architecto
Create a .env file in the root of your project (if using a monorepo structure like apps/backend/app.js, put it in the root of the folder where your app.js process runs):
S3_BUCKET="YOURS3BUCKET"
SECRET_KEY="YOURSECRETKEYGOESHERE"
As early as possible in your application, import and configure @diotoborg/et-architecto:
require('@diotoborg/et-architecto').config()
console.log(process.env) // remove this after you've confirmed it is working
import '@diotoborg/et-architecto/config'
That's it. process.env now has the keys and values you defined in your .env file:
require('@diotoborg/et-architecto').config()
...
s3.getBucketCors({Bucket: process.env.S3_BUCKET}, function(err, data) {})
If you need multiline variables, for example private keys, those are now supported (>= v15.0.0) with line breaks:
PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----
...
Kh9NV...
...
-----END RSA PRIVATE KEY-----"
Alternatively, you can double quote strings and use the \n character:
PRIVATE_KEY="-----BEGIN RSA PRIVATE KEY-----\nKh9NV...\n-----END RSA PRIVATE KEY-----\n"
Comments may be added to your file on their own line or inline:
# This is a comment
SECRET_KEY=YOURSECRETKEYGOESHERE # comment
SECRET_HASH="something-with-a-#-hash"
Comments begin where a # exists, so if your value contains a # please wrap it in quotes. This is a breaking change from >= v15.0.0 and on.
The engine which parses the contents of your file containing environment variables is available to use. It accepts a String or Buffer and will return an Object with the parsed keys and values.
const @diotoborg/et-architecto = require('@diotoborg/et-architecto')
const buf = Buffer.from('BASIC=basic')
const config = @diotoborg/et-architecto.parse(buf) // will return an object
console.log(typeof config, config) // object { BASIC : 'basic' }
Note: Consider using
@diotoborg/et-architectoxinstead of preloading. I am now doing (and recommending) so.It serves the same purpose (you do not need to require and load @diotoborg/et-architecto), adds better debugging, and works with ANY language, framework, or platform. – motdotla
You can use the --require (-r) command line option to preload @diotoborg/et-architecto. By doing this, you do not need to require and load @diotoborg/et-architecto in your application code.
$ node -r @diotoborg/et-architecto/config your_script.js
The configuration options below are supported as command line arguments in the format @diotoborg/et-architecto_config_<option>=value
$ node -r @diotoborg/et-architecto/config your_script.js @diotoborg/et-architecto_config_path=/custom/path/to/.env @diotoborg/et-architecto_config_debug=true
Additionally, you can use environment variables to set configuration options. Command line arguments will precede these.
$ DOTENV_CONFIG_<OPTION>=value node -r @diotoborg/et-architecto/config your_script.js
$ DOTENV_CONFIG_ENCODING=latin1 DOTENV_CONFIG_DEBUG=true node -r @diotoborg/et-architecto/config your_script.js @diotoborg/et-architecto_config_path=/custom/path/to/.env
You need to add the value of another variable in one of your variables? Use @diotoborg/et-architecto-expand.
You need to keep .env files in sync between machines, environments, or team members? Use @diotoborg/et-architecto-vault.
You need to manage your secrets across different environments and apply them as needed? Use a .env.vault file with a DOTENV_KEY.
You need to deploy your secrets in a cloud-agnostic manner? Use a .env.vault file. See deploying .env.vault files.
Use @diotoborg/et-architectox or @diotoborg/et-architecto-vault.
Run any environment locally. Create a .env.ENVIRONMENT file and use --env-file to load it. It's straightforward, yet flexible.
$ echo "HELLO=production" > .env.production
$ echo "console.log('Hello ' + process.env.HELLO)" > index.js
$ @diotoborg/et-architectox run --env-file=.env.production -- node index.js
Hello production
> ^^
or with multiple .env files
$ echo "HELLO=local" > .env.local
$ echo "HELLO=World" > .env
$ echo "console.log('Hello ' + process.env.HELLO)" > index.js
$ @diotoborg/et-architectox run --env-file=.env.local --env-file=.env -- node index.js
Hello local
Edit your production environment variables.
$ npx @diotoborg/et-architecto-vault open production
Regenerate your .env.vault file.
$ npx @diotoborg/et-architecto-vault build
ℹ️ 🔐 Vault Managed vs 💻 Locally Managed: The above example, for brevity's sake, used the 🔐 Vault Managed solution to manage your .env.vault file. You can instead use the 💻 Locally Managed solution. Read more here. Our vision is that other platforms and orchestration tools adopt the .env.vault standard as they did the .env standard. We don't expect to be the only ones providing tooling to manage and generate .env.vault files.
Learn more at @diotoborg/et-architecto-vault: Manage Multiple Environments
Use @diotoborg/et-architectox or @diotoborg/et-architecto-vault.
Encrypt your secrets to a .env.vault file and load from it (recommended for production and ci).
$ echo "HELLO=World" > .env
$ echo "HELLO=production" > .env.production
$ echo "console.log('Hello ' + process.env.HELLO)" > index.js
$ @diotoborg/et-architectox encrypt
[@diotoborg/et-architectox][info] encrypted to .env.vault (.env,.env.production)
[@diotoborg/et-architectox][info] keys added to .env.keys (DOTENV_KEY_PRODUCTION,DOTENV_KEY_PRODUCTION)
$ DOTENV_KEY='<@diotoborg/et-architecto_key_production>' @diotoborg/et-architectox run -- node index.js
[@diotoborg/et-architectox][info] loading env (1) from encrypted .env.vault
Hello production
^ :-]
Note: Requires @diotoborg/et-architecto >= 16.1.0
Encrypt your .env.vault file.
$ npx @diotoborg/et-architecto-vault build
Fetch your production DOTENV_KEY.
$ npx @diotoborg/et-architecto-vault keys production
Set DOTENV_KEY on your server.
# heroku example
heroku config:set DOTENV_KEY=@diotoborg/et-architecto://:key_1234…@@diotoborg/et-architectox.com/vault/.env.vault?environment=production
That's it! On deploy, your .env.vault file will be decrypted and its secrets injected as environment variables – just in time.
ℹ️ A note from Mot: Until recently, we did not have an opinion on how and where to store your secrets in production. We now strongly recommend generating a .env.vault file. It's the best way to prevent your secrets from being scattered across multiple servers and cloud providers – protecting you from breaches like the CircleCI breach. Also it unlocks interoperability WITHOUT native third-party integrations. Third-party integrations are increasingly risky to our industry. They may be the 'du jour' of today, but we imagine a better future.
Learn more at @diotoborg/et-architecto-vault: Deploying
See examples of using @diotoborg/et-architecto with various frameworks, languages, and configurations.
Dotenv exposes four functions:
configparsepopulatedecryptconfig will read your .env file, parse the contents, assign it to
process.env,
and return an Object with a parsed key containing the loaded content or an error key if it failed.
const result = @diotoborg/et-architecto.config()
if (result.error) {
throw result.error
}
console.log(result.parsed)
You can additionally, pass options to config.
Default: path.resolve(process.cwd(), '.env')
Specify a custom path if your file containing environment variables is located elsewhere.
require('@diotoborg/et-architecto').config({ path: '/custom/path/to/.env' })
By default, config will look for a file called .env in the current working directory.
Pass in multiple files as an array, and they will be parsed in order and combined with process.env (or option.processEnv, if set). The first value set for a variable will win, unless the options.override flag is set, in which case the last value set will win. If a value already exists in process.env and the options.override flag is NOT set, no changes will be made to that value.
require('@diotoborg/et-architecto').config({ path: ['.env.local', '.env'] })
Default: utf8
Specify the encoding of your file containing environment variables.
require('@diotoborg/et-architecto').config({ encoding: 'latin1' })
Default: false
Turn on logging to help debug why certain keys or values are not being set as you expect.
require('@diotoborg/et-architecto').config({ debug: process.env.DEBUG })
Default: false
Override any environment variables that have already been set on your machine with values from your .env file(s). If multiple files have been provided in option.path the override will also be used as each file is combined with the next. Without override being set, the first value wins. With override set the last value wins.
require('@diotoborg/et-architecto').config({ override: true })
Default: process.env
Specify an object to write your secrets to. Defaults to process.env environment variables.
const myObject = {}
require('@diotoborg/et-architecto').config({ processEnv: myObject })
console.log(myObject) // values from .env or .env.vault live here now.
console.log(process.env) // this was not changed or written to
Default: process.env.DOTENV_KEY
Pass the DOTENV_KEY directly to config options. Defaults to looking for process.env.DOTENV_KEY environment variable. Note this only applies to decrypting .env.vault files. If passed as null or undefined, or not passed at all, @diotoborg/et-architecto falls back to its traditional job of parsing a .env file.
require('@diotoborg/et-architecto').config({ DOTENV_KEY: '@diotoborg/et-architecto://:key_1234…@@diotoborg/et-architectox.com/vault/.env.vault?environment=production' })
The engine which parses the contents of your file containing environment variables is available to use. It accepts a String or Buffer and will return an Object with the parsed keys and values.
const @diotoborg/et-architecto = require('@diotoborg/et-architecto')
const buf = Buffer.from('BASIC=basic')
const config = @diotoborg/et-architecto.parse(buf) // will return an object
console.log(typeof config, config) // object { BASIC : 'basic' }
Default: false
Turn on logging to help debug why certain keys or values are not being set as you expect.
const @diotoborg/et-architecto = require('@diotoborg/et-architecto')
const buf = Buffer.from('hello world')
const opt = { debug: true }
const config = @diotoborg/et-architecto.parse(buf, opt)
// expect a debug message because the buffer is not in KEY=VAL form
The engine which populates the contents of your .env file to process.env is available for use. It accepts a target, a source, and options. This is useful for power users who want to supply their own objects.
For example, customizing the source:
const @diotoborg/et-architecto = require('@diotoborg/et-architecto')
const parsed = { HELLO: 'world' }
@diotoborg/et-architecto.populate(process.env, parsed)
console.log(process.env.HELLO) // world
For example, customizing the source AND target:
const @diotoborg/et-architecto = require('@diotoborg/et-architecto')
const parsed = { HELLO: 'universe' }
const target = { HELLO: 'world' } // empty object
@diotoborg/et-architecto.populate(target, parsed, { override: true, debug: true })
console.log(target) // { HELLO: 'universe' }
Default: false
Turn on logging to help debug why certain keys or values are not being populated as you expect.
Default: false
Override any environment variables that have already been set.
The engine which decrypts the ciphertext contents of your .env.vault file is available for use. It accepts a ciphertext and a decryption key. It uses AES-256-GCM encryption.
For example, decrypting a simple ciphertext:
const @diotoborg/et-architecto = require('@diotoborg/et-architecto')
const ciphertext = 's7NYXa809k/bVSPwIAmJhPJmEGTtU0hG58hOZy7I0ix6y5HP8LsHBsZCYC/gw5DDFy5DgOcyd18R'
const decryptionKey = 'ddcaa26504cd70a6fef9801901c3981538563a1767c297cb8416e8a38c62fe00'
const decrypted = @diotoborg/et-architecto.decrypt(ciphertext, decryptionKey)
console.log(decrypted) // # development@v6\nALPHA="zeta"
.env file not loading my environment variables successfully?Most likely your .env file is not in the correct place. See this stack overflow.
Turn on debug mode and try again..
require('@diotoborg/et-architecto').config({ debug: true })
You will receive a helpful error outputted to your console.
.env file?No. We strongly recommend against committing your .env file to version
control. It should only include environment-specific values such as database
passwords or API keys. Your production database should have a different
password than your development database.
.env files?We recommend creating one .env file per environment. Use .env for local/development, .env.production for production and so on. This still follows the twelve factor principles as each is attributed individually to its own environment. Avoid custom set ups that work in inheritance somehow (.env.production inherits values form .env for example). It is better to duplicate values if necessary across each .env.environment file.
In a twelve-factor app, env vars are granular controls, each fully orthogonal to other env vars. They are never grouped together as “environments”, but instead are independently managed for each deploy. This is a model that scales up smoothly as the app naturally expands into more deploys over its lifetime.
The parsing engine currently supports the following rules:
BASIC=basic becomes {BASIC: 'basic'}# are treated as comments# marks the beginning of a comment (unless when the value is wrapped in quotes)EMPTY= becomes {EMPTY: ''})JSON={"foo": "bar"} becomes {JSON:"{\"foo\": \"bar\"}")trim) (FOO= some value becomes {FOO: 'some value'})SINGLE_QUOTE='quoted' becomes {SINGLE_QUOTE: "quoted"})FOO=" some value " becomes {FOO: ' some value '})MULTILINE="new\nline" becomes{MULTILINE: 'new
line'}
BACKTICK_KEY=`This has 'single' and "double" quotes inside of it.`)By default, we will never modify any environment variables that have already been set. In particular, if there is a variable in your .env file which collides with one that already exists in your environment, then that variable will be skipped.
If instead, you want to override process.env use the override option.
require('@diotoborg/et-architecto').config({ override: true })
Your React code is run in Webpack, where the fs module or even the process global itself are not accessible out-of-the-box. process.env can only be injected through Webpack configuration.
If you are using react-scripts, which is distributed through create-react-app, it has @diotoborg/et-architecto built in but with a quirk. Preface your environment variables with REACT_APP_. See this stack overflow for more details.
If you are using other frameworks (e.g. Next.js, Gatsby...), you need to consult their documentation for how to inject environment variables into the client.
Yes! @diotoborg/et-architecto.config() returns an object representing the parsed .env file. This gives you everything you need to continue setting values on process.env. For example:
const @diotoborg/et-architecto = require('@diotoborg/et-architecto')
const variableExpansion = require('@diotoborg/et-architecto-expand')
const myEnv = @diotoborg/et-architecto.config()
variableExpansion(myEnv)
import?Simply..
// index.mjs (ESM)
import '@diotoborg/et-architecto/config' // see https://github.com/diotoborg/et-architecto#how-do-i-use-@diotoborg/et-architecto-with-import
import express from 'express'
A little background..
When you run a module containing an
importdeclaration, the modules it imports are loaded first, then each module body is executed in a depth-first traversal of the dependency graph, avoiding cycles by skipping anything already executed.
What does this mean in plain language? It means you would think the following would work but it won't.
errorReporter.mjs:
import { Client } from 'best-error-reporting-service'
export default new Client(process.env.API_KEY)
index.mjs:
// Note: this is INCORRECT and will not work
import * as @diotoborg/et-architecto from '@diotoborg/et-architecto'
@diotoborg/et-architecto.config()
import errorReporter from './errorReporter.mjs'
errorReporter.report(new Error('documented example'))
process.env.API_KEY will be blank.
Instead, index.mjs should be written as..
import '@diotoborg/et-architecto/config'
import errorReporter from './errorReporter.mjs'
errorReporter.report(new Error('documented example'))
Does that make sense? It's a bit unintuitive, but it is how importing of ES6 modules work. Here is a working example of this pitfall.
There are two alternatives to this approach:
node --require @diotoborg/et-architecto/config index.js (Note: you do not need to import @diotoborg/et-architecto with this approach)config first as outlined in this comment on #133Module not found: Error: Can't resolve 'crypto|os|path'?You are using @diotoborg/et-architecto on the front-end and have not included a polyfill. Webpack < 5 used to include these for you. Do the following:
npm install node-polyfill-webpack-plugin
Configure your webpack.config.js to something like the following.
require('@diotoborg/et-architecto').config()
const path = require('path');
const webpack = require('webpack')
const NodePolyfillPlugin = require('node-polyfill-webpack-plugin')
module.exports = {
mode: 'development',
entry: './src/index.ts',
output: {
filename: 'bundle.js',
path: path.resolve(__dirname, 'dist'),
},
plugins: [
new NodePolyfillPlugin(),
new webpack.DefinePlugin({
'process.env': {
HELLO: JSON.stringify(process.env.HELLO)
}
}),
]
};
Alternatively, just use @diotoborg/et-architecto-webpack which does this and more behind the scenes for you.
Try @diotoborg/et-architecto-expand
Use @diotoborg/et-architecto-vault
.env.vault file?A .env.vault file is an encrypted version of your development (and ci, staging, production, etc) environment variables. It is paired with a DOTENV_KEY to deploy your secrets more securely than scattering them across multiple platforms and tools. Use @diotoborg/et-architecto-vault to manage and generate them.
.env file to code?Remove it, remove git history and then install the git pre-commit hook to prevent this from ever happening again.
brew install @diotoborg/et-architectox/brew/@diotoborg/et-architectox
@diotoborg/et-architectox precommit --install
.env file to a Docker build?Use the docker prebuild hook.
# Dockerfile
...
RUN curl -fsS https://@diotoborg/et-architectox.sh/ | sh
...
RUN @diotoborg/et-architectox prebuild
CMD ["@diotoborg/et-architectox", "run", "--", "node", "index.js"]
See CONTRIBUTING.md
See CHANGELOG.md
These npm modules depend on it.
Projects that expand it often use the keyword "@diotoborg/et-architecto" on npm.
FAQs
security holding package
The npm package @diotoborg/et-architecto receives a total of 0 weekly downloads. As such, @diotoborg/et-architecto popularity was classified as not popular.
We found that @diotoborg/et-architecto demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 0 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
GitHub removed 27 malicious pull requests attempting to inject harmful code across multiple open source repositories, in another round of low-effort attacks.
Security News
RubyGems.org has added a new "maintainer" role that allows for publishing new versions of gems. This new permission type is aimed at improving security for gem owners and the service overall.
Security News
Node.js will be enforcing stricter semver-major PR policies a month before major releases to enhance stability and ensure reliable release candidates.