confmgr

ENV Config Manager

Intro

confmgr is (yet another) configuration manager for your NodeJS apps. It is written in Typescript and thus supports Typescript as well as Javascript. Check out the documentation for more details.

Why another one?

confmgr brings interesting features that are not available in libs. While focusing on ease of use, it brings features that no longer need to be part of your app such as:

• type conversion (variables from ENV are always strings, confmgr converts them for you)

• mandatory values: If you miss a value, confmgr will tell you before your app explodes…​

• default values: confmrg handles default values for you

• regular expressions: confmgr allows you defining a regexp that will allow checking the validity of the loaded config

• masked variables: handy for your secrets, read below

Helper functions

After getting your config object with GetConfig(), you can simple use it with config['MYAPP_MYMODULE_VAR1'] or use some of the helper methods attached to your config.

confmgr brings a set of handy methods such as Print() that will display the content of your config while ensuring that your secrets do NOT show up in any log or output.

Printing your config may use colors or not. Here is an example with color coming from running config.Print():

===> TS_SAMPLE_MODULE_01 ENV:
✅ PARAM1: some param1
    value: 12
✅ PARAM2: some param2
    value: 44
✅ SECRET: some secret
    value: *****                                <---- Notice how this value has been masked
✅ REGEXP: some regexp
    regexp: ^\d{2}_\d{2}
    value: 68_77
✅ MANDAT_WITH_DEF: some optional param
    value: 42
❌ MANDAT_NO_DEF: some optional param           <----- It looks like we forgot to pass a value
    value: undefined

If you are using a custom logger, you may also pass it to the Print() method so the output gets nicely integrated to your logs while NOT revealing any of your secrets.

Documentation

You can keep reading this file and/or check the doc and samples folders. You can also visit https://chevdor.gitlab.io/confmgr to see the old API documentation.

In a nutshell

Install

Using NPM

npm install --save confmgr

Using Yarn

yarn add confmgr

Usage

import { ConfigManager } from 'confmgr';

const config = ConfigManager.getInstance('configSpecs.yml').getConfig();
const valid = config.Validate();
console.log(`Your config is${valid ? '' : ' NOT'} valid!`);
config.Print();

The repo contains many more samples in both Typescript and Javascript, using YML and the Specs Factory.

Features

• Written in Typescript

• Available for Typescript or Javascript

• You create your config specs using a Factory or specs in YAML or JSON format

• Your config is loaded from your ENV or an .env file

• Support for multiple .env (debug, etc…​)

• Your config is validated according to your specs

• Your config is made available to you in a super easy way

• You can Print your config in color using the console or use your own logger

• protects your secret and will never display them

• will check for value you defined as mandatory and warn you if they are missing

• supports default values and will apply them has needed

• supports type for an easy conversion from string to whatever your app needs

Config Specs

In order to bring a config object to your app, you need define some specs first.

That can be done via code using the provided Factory (see samples in the samples folder of the repo). You can also simply provide the specs as a YAML file.

Writing the specs as a YAML file makes it simple to also share the specs of your app to your users as this is very readable.

Here is how a spec file in the YAML format looks like:

MYAPP:                              # Name of your App
  MYMODULE:                         # Name of your module
    VAR1:                           # Here is the simplest example, we named it VAR1
      description: some string
    
    VAR2:
      description: some number
      type: number
      default: 42                   # If for some reason, this value is not set, confmgr will set the default value
    
    VAR3:
      description: some array
      type: array
      default: ['a', 'b', 'c']
    
    VAR4: 
      description: some object
      type: object
      default: { name: "foo", age: 42 }

    VAR5:
      description: A mandatory variable. If that one is missing, will be unhappy
      mandatory: true
     
    VAR6:
      description: A optional variable we can count on
      default: 42

    VAR7:
      description: Some variable we check with a regpex
      regexp: ^\d{2}_\d{2}      

Here is how a spec file in the JSON format looks like:

{
    "SAMPLE": {
        "MODULE": {
            "BOOL1": {
                "description": "some regexp",
                "type": "boolean"
            },
            "BOOL2": {
                "description": "some regexp",
                "type": "boolean",
                "regexp": {
                    "pattern": "^true|false$",
                    "attributes": "i"
                }
            },
            "BOOL3": {
                "description": "some regexp",
                "regexp": {
                    "pattern": "^true|false$",
                    "attributes": "i"
                }
            },
            "URL": {
                "description": "some web socket URL",
                "default": "ws://localhost:1234",
                "regexp": "^wss?://\\w+:\\d+$"
            }
        }
    }
}

Config values

Using ENV Variables

You may pass values as Environment Variables as shown below:

MYAPP_MODULE_VAR1='foo bar' node index.js

Generating a default ENV file from the specs

You may let confmgr generate a default config file using:

import { ConfigManager } from 'confmgr'
const config = ConfigManager.getInstance('configSpecs.yml').getConfig()
console.log(config.GenEnv().join('\n'))

You can see this in action in the sample 03. The output looks like:

TS_SAMPLE_MODULE_01_PARAM1=
TS_SAMPLE_MODULE_01_PARAM2=
TS_SAMPLE_MODULE_01_SECRET=
TS_SAMPLE_MODULE_01_BOOL1=true
TS_SAMPLE_MODULE_01_BOOL2=true
TS_SAMPLE_MODULE_01_ARRAY1=[1,"a",true,{"foo":"bar"}]
TS_SAMPLE_MODULE_01_OBJ1={"foo":"bar","list":[1,2,3]}
TS_SAMPLE_MODULE_02_REGEXP=
TS_SAMPLE_MODULE_02_MANDAT_WITH_DEF=42
TS_SAMPLE_MODULE_02_MANDAT_NO_DEF=

Using default .env file

You may also simply create an .env file at the root of your project:

# Here is a comment
MYAPP_MODULE_VAR1='foo bar'

You may then simply start your app and the config will be loaded, validated and made available to you.

node index.js

Using custom .env file

In some case, you may need different .env file for production, testing, staging, dev, etc…​

The default as we saw above is 'production' and the file that is loaded is .env.

If you provide the Environment Variable NODE_ENV however and this value is neither empty nor production, an alternate .env.<…​> file will be loaded.

For instance:

• using NODE_ENV=dev node index.js to launch your App will load the config from .env.dev

• using NODE_ENV=abc node index.js to launch your App will load the config from .env.abc

YAML Config Specs format

Prefix & Module

Have you ever looks at the environment variables on your machine? Give it a go and issue env. That`s probably not very pretty…​ Moreover, say your app needs to access a database, using an ENV called DATABASE will add even more to this mess.

For this reason, confmgr helps you keeps things nice, tidy and collision-free thanks to a PREFIX and MODULE. Those are 2 strings, usually in CAPS. Any of your configuration value will be prepended with PREFIX_MODULE_.

That makes it also very easy to find all the variables of your app using a shell command such as env | grep MYAPP_MYMODULE.

The PREFIX is the root of your YAML file. You find the MODULE just below and then your variables.

At the moment, only a single PREFIX and a single MODULE is supported.

Variable

A variable is usually named in CAPS. In your YAML, it looks like:

MYAPP:                              # Name of your App
  MYMODULE:                         # Name of your module
    VAR1:                           # Here is the simplest example, we named it VAR1
      description: some string

Having such as spec means our App expects an ENV called MYAPP_MYMODULE_VAR1 to be defined. At that point, if the variable is missing or contains an incorrect valule, confmgr won’t do anything as it does not know yet about our needs.

The only requirements to define a new variables are:

• a name (ie. VAR1)

• a description

The description will show up as you Print your config:

===> TS_SAMPLE_MODULE_01 ENV:
✅ PARAM1: some param1
    value: 12
✅ PARAM2: some param2
    value: 44
✅ SECRET: some secret
    value: *****

Options

Options are optional…​ I said it…​

The best to see what they are is to search for the ConfigItemOptions type in the code:

export type ConfigItemOptions = {
    masked?: boolean // true for tokens, passwords, etc..
    regexp?: RegExp | RegexpWithAttributes // validation regexp
    mandatory?: boolean // Do we explode if this ENV is not defined and there is no default?
    default?: ConfigValue // The default if nothing is provided
    type?: Type // If provided, the ConfigManager will do some conversion for us
}

Masked

If a variable is marked as masked=true, its value will never be shown when using Print.

Regexp

You may add validation your variables using a regular expression.

MYAPP:
  MYMODULE:
    REGEXP:
      description: some regexp
      regexp: ^\d{2}_\d{2}

means and enforces that MYAPP_MYMODULE_REGEXP contains a value made of 2 digits followed by underscore and another 2 digits. 12_34 would be valid whereas 123_45 is not.

Default

If your ENV does not contain this value, the default will be applied.

Mandatory

If your ENV does not contain this value and you did not define a default, then the validation of your config with config.Validate() will fail (= return false) and using config.Print() will highlight issues as shown below:

===> TS_SAMPLE_MODULE_01 ENV:
✅ MANDAT_WITH_DEF: some optional param
    value: 42
❌ MANDAT_NO_DEF: some optional param
    value: undefined

Type

The supported types can also be found in the code:

/**
 * This describes the type of the config items.
 */
export type Type = 'string' | 'boolean' | 'number' | 'array' | 'object'

Usage

Javascript

    const ConfigManager = require('confmgr').ConfigManager;

Typescript

    import { ConfigManager } from 'confmgr';

Some details

You may have a look at the samples in the samples folder. There are simple demos in Typescript and Javascript.

In short, you first need to define the specs of your config. You can do that using Yaml or using a Factory directly, as shown below:

Using a Yaml file

This is the commend option and likely the easiest one.

TS_SAMPLE:
  MODULE_01:
    PARAM1:
      description: some param1
    PARAM2:
      description: some param2
    SECRET:
      description: some secret
      masked: true
    BOOL1:
      description: bool1
      type: boolean
      default: true
    BOOL2:
      description: bool2
      type: boolean
      default: true
    ARRAY1:
      description: some array
      default: [1, 'a', true, { foo: bar }]
    OBJ1:
      description: some object
      default: { foo: bar, list: [1, 2, 3] }

  MODULE_02:
    REGEXP:
      description: some regexp
      regexp: ^\d{2}_\d{2}
    MANDAT_WITH_DEF:
      description: some optional param
      mandatory: true
      default: 42
    MANDAT_NO_DEF:
      description: some optional param
      mandatory: true

Using the Factory

import { SpecsFactory } from 'confmgr'

const prefix = 'TS_SAMPLE'
const mod = 'MODULE_01'

const factory = new SpecsFactory({ prefix })
factory.appendSpec(mod, factory.getSpec('PARAM1', 'some param1'))
factory.appendSpec(mod, factory.getSpec('PARAM2', 'some param2'))
factory.appendSpec(
    mod,
    factory.getSpec('SECRET', 'some secret', { masked: true })
)
factory.appendSpec(
    mod,
    factory.getSpec('REGEXP', 'some regexp', { regexp: /^\d{2}_\d{2}/ })
)
factory.appendSpec(
    mod,
    factory.getSpec('MANDAT1', 'some mandatory param', { mandatory: true })
)
const specs = factory.getSpecs()

export default specs

Once you defined the specs for your config, you need to create an .env file (unless you plan on passing the config as Environment Variables). It looks as following for the specs mentioned above:

TS_SAMPLE_MODULE_01_PARAM1=12
TS_SAMPLE_MODULE_01_PARAM2=44
TS_SAMPLE_MODULE_01_SECRET=secret
TS_SAMPLE_MODULE_01_REGEXP=68_77
TS_SAMPLE_MODULE_01_MANDAT1=27

You may now use your config:

import { ConfigManager } from 'confmgr'
import mySpecs from './configSpecs'

const config = ConfigManager.getInstance(mySpecs).getConfig()
const valid = config.Validate()
console.log(`Your config is${valid ? '' : ' NOT'} valid!`)

config.Print({ color: true, compact: false })

console.log(`PARAM1=${config.Get('MODULE_01', 'PARAM1')}`)

You should never display your config, especially in your logs, as it may contain some secrets! Instead, use the Print() method of the ConfigManager, it will mask sensible data.

Dev

First run:

yarn setup

Then you may for instance start with the tests:

yarn test:watch

Contributing

If you are considering contributing, you will find many ways to apply your skills. There is still a lot that can be done in this project. Feel free to submit any PR you think may improve the lib or stop by the issue list and check the Contributions Welcome label, it may give you a few ideas.

You may report any bug or submit your PRs at https://github.com/chevdor/confmgr

Samples

TS Basic with valid .env

Run with the following:

yarn clean
yarn
yarn start

Specs

import { SpecsFactory } from 'confmgr'

const prefix = 'TS_SAMPLE'
const mod = 'MODULE_01'

const factory = new SpecsFactory({ prefix })
factory.appendSpec(mod, factory.getSpec('PARAM1', 'some param1'))
factory.appendSpec(mod, factory.getSpec('PARAM2', 'some param2'))
factory.appendSpec(
    mod,
    factory.getSpec('SECRET', 'some secret', { masked: true })
)
factory.appendSpec(
    mod,
    factory.getSpec('REGEXP', 'some regexp', { regexp: /^\d{2}_\d{2}/ })
)
factory.appendSpec(
    mod,
    factory.getSpec('MANDAT1', 'some mandatory param', { mandatory: true })
)
const specs = factory.getSpecs()

export default specs

Example

import { ConfigManager } from 'confmgr'
import mySpecs from './configSpecs'

const config = ConfigManager.getInstance(mySpecs).getConfig()
const valid = config.Validate()
console.log(`Your config is${valid ? '' : ' NOT'} valid!`)

config.Print({ color: true, compact: false })

console.log(`PARAM1=${config.Get('MODULE_01', 'PARAM1')}`)

TS Basic with INvalid .env

This is a Typescript sample.

Run with the following:

yarn clean
yarn
yarn start

Specs

import { SpecsFactory } from 'confmgr'

const prefix = 'TS_SAMPLE'
const mod = 'MODULE_01'

const factory = new SpecsFactory({ prefix })
factory.appendSpec(mod, factory.getSpec('PARAM1', 'some param1'))
factory.appendSpec(mod, factory.getSpec('PARAM2', 'some param2'))
factory.appendSpec(
    mod,
    factory.getSpec('SECRET', 'some secret', { masked: true })
)
factory.appendSpec(
    mod,
    factory.getSpec('REGEXP', 'some regexp', { regexp: /^\d{2}_\d{2}/ })
)
factory.appendSpec(
    mod,
    factory.getSpec('MANDAT1', 'some mandatory param', { mandatory: true })
)
const specs = factory.getSpecs()

export default specs

Example

import { ConfigManager } from 'confmgr'
import mySpecs from './configSpecs'

const config = ConfigManager.getInstance(mySpecs).getConfig()
const valid = config.Validate()
console.log(`Your config is${valid ? '' : ' NOT'} valid!`)

// Here is our custom logger...
function MyLogger(...args): void {
    const s = args.join(' ')
    console.log(`▶ ${s.replace('\n', '\n▶')}`)
}

config.Print({ color: true, logger: MyLogger })

TS Basic using a YAML spec

This is a Typescript sample.

Run with the following:

yarn clean
yarn
yarn start

Specs

TS_SAMPLE:
  MODULE_01:
    PARAM1:
      description: some param1
    PARAM2:
      description: some param2
    SECRET:
      description: some secret
      masked: true
    BOOL1:
      description: bool1
      type: boolean
      default: true
    BOOL2:
      description: bool2
      type: boolean
      default: true
    ARRAY1:
      description: some array
      default: [1, 'a', true, { foo: bar }]
    OBJ1:
      description: some object
      default: { foo: bar, list: [1, 2, 3] }

  MODULE_02:
    REGEXP:
      description: some regexp
      regexp: ^\d{2}_\d{2}
    MANDAT_WITH_DEF:
      description: some optional param
      mandatory: true
      default: 42
    MANDAT_NO_DEF:
      description: some optional param
      mandatory: true

Example

import { ConfigManager } from 'confmgr'

const config = ConfigManager.getInstance('configSpecs.yml').getConfig()
const valid = config.Validate()
console.log(`Your config is${valid ? '' : ' NOT'} valid!`)

config.Print({ compact: true })

console.log('\nYour default config looks like:')
console.log(config.GenEnv().join('\n'))

console.log('\nHere are some values:')
console.log(` - PARAM1=${config.Get('MODULE_01', 'PARAM1')}`)
console.log(` - BOOL1=${config.Get('MODULE_01', 'BOOL1')}`)
console.log(` - BOOL2=${config.Get('MODULE_01', 'BOOL2')}`)

JS Basic with valid .env

This is a Javascript sample.

Run with the following:

yarn clean
yarn
yarn start

Specs

const SpecsFactory = require('confmgr')
const prefix = 'SAMPLE'
const mod = 'MODULE'
const factory = new SpecsFactory.SpecsFactory({ prefix: prefix })
factory.appendSpec(mod, factory.getSpec('PARAM1', 'some param1'))
factory.appendSpec(mod, factory.getSpec('PARAM2', 'some param2'))
factory.appendSpec(
    mod,
    factory.getSpec('SECRET', 'some secret', { masked: true })
)
factory.appendSpec(
    mod,
    factory.getSpec('REGEXP', 'some regexp', { regexp: /^\d{2}_\d{2}/ })
)
factory.appendSpec(
    mod,
    factory.getSpec('MANDAT1', 'some mandatory param', { mandatory: true })
)
const specs = factory.getSpecs()
exports['default'] = specs

Example

const ConfigManager = require('confmgr').ConfigManager
const mySpecs = require('./configSpecs').default

const config = ConfigManager.getInstance(mySpecs).getConfig()
const valid = config.Validate()
console.log(`Your config is${valid ? '' : ' NOT'} valid!`)

config.Print({ logger: console.log })

License

Copyright 2020 - Wilfried Kopp / Chevdor

Permission is hereby granted, free of charge, to any person obtaining a copy of 
this software and associated documentation files (the "Software"), to deal in 
the Software without restriction, including without limitation the rights to 
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies 
of the Software, and to permit persons to whom the Software is furnished to do 
so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all 
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 
SOFTWARE.