`onion_config` is a Python package designed for easy configuration management. It supports loading and validating configuration data from environment variables and configuration files in JSON and YAML formats. It is a `Pydantic` based custom configuration package for Python projects.
onion_config is a Python package designed for easy configuration management. It supports loading and validating configuration data from environment variables and configuration files in JSON and YAML formats. It is a Pydantic based custom configuration package for Python projects.
extra_dir directory)BaseConfig)ConfigLoader)[OPTIONAL] For DEVELOPMENT environment:
[!TIP] Skip this step, if you're going to install the package directly from PyPi or GitHub repository.
2.1. Prepare projects directory (if not exists):
# Create projects directory:
mkdir -pv ~/workspaces/projects
# Enter into projects directory:
cd ~/workspaces/projects
2.2. Follow one of the below options [A], [B] or [C]:
OPTION A. Clone the repository:
git clone https://github.com/bybatkhuu/module.python-config.git && \
cd module.python-config
OPTION B. Clone the repository (for DEVELOPMENT: git + ssh key):
git clone git@github.com:bybatkhuu/module.python-config.git && \
cd module.python-config
OPTION C. Download source code:
[!NOTE] Choose one of the following methods to install the package [A ~ E]:
OPTION A. [RECOMMENDED] Install from PyPi:
[!WARNING] If you wanted to use Pydantic-v1, but if you already installed
pydantic-settingsandpydantic-core, remove it before installing Pydantic-v1:
pip uninstall -y pydantic-settings
pip uninstall -y pydantic-core
# Then install with Pydantic-v1:
pip install -U onion-config[pydantic-v1]
[!WARNING] If you wanted to use Pydantic-v2, but if you already installed
onion-configpackage just by
pip install -U onion-configcommand, and this will not installpydantic-settings.
For this case, 'env_prefix' WILL NOT WORK forBaseConfigorBaseSettingswithoutpydantic-settings! This is Pydantic-v2's problem, and there could be some other problems.
So fix these issues re-installonion-configwithpydantic-settings:
# Install with pydantic-settings for Pydantic-v2:
pip install -U onion-config[pydantic-settings]
OPTION B. Install latest version directly from GitHub repository:
# Pydantic-v1:
pip install git+https://github.com/bybatkhuu/module.python-config.git[pydantic-v1]
# Pydantic-v2:
pip install git+https://github.com/bybatkhuu/module.python-config.git[pydantic-settings]
OPTION C. Install from the downloaded source code:
# Install directly from the source code:
# Pydantic-v1:
pip install .[pydantic-v1]
# Pydantic-v2:
pip install .[pydantic-settings]
# Or install with editable mode (for DEVELOPMENT):
# Pydantic-v1:
pip install -e .[pydantic-v1]
# Pydantic-v2:
pip install -e .[pydantic-settings]
OPTION D. Install from pre-built release files:
.whl or .tar.gz file from releases# Pydantic-v1:
# Install from .whl file:
pip install ./onion_config-[VERSION]-py3-none-any.whl[pydantic-v1]
# Or install from .tar.gz file:
pip install ./onion_config-[VERSION].tar.gz[pydantic-v1]
# Pydantic-v2:
# Install from .whl file:
pip install ./onion_config-[VERSION]-py3-none-any.whl[pydantic-settings]
# Or install from .tar.gz file:
pip install ./onion_config-[VERSION].tar.gz[pydantic-settings]
OPTION E. Copy the module into the project directory (for testing):
# Install python dependencies:
pip install -r ./requirements/requirements.core.txt
# Pydantic-v1:
pip install -r ./requirements/requirements.pydantic-v1.txt
# Pydantic-v2:
pip install -r ./requirements/requirements.pydantic-settings.txt
# Copy the module source code into the project:
cp -r ./src/onion_config [PROJECT_DIR]
# For example:
cp -r ./src/onion_config /some/path/project/
ENV=production
examples/simple/configs/1.base.yml:
env: test
app:
name: "My App"
version: "0.0.1"
nested:
key: "value"
examples/simple/configs/2.extra.yml:
app:
name: "New App"
nested:
some: "value"
description: "Description of my app."
another_val:
extra: 1
import pprint
from loguru import logger
try:
import pydantic_settings
_has_pydantic_settings = True
except ImportError:
_has_pydantic_settings = False
from onion_config import ConfigLoader, BaseConfig
class ConfigSchema(BaseConfig):
env: str = "local"
try:
config: ConfigSchema = ConfigLoader(config_schema=ConfigSchema).load()
except Exception:
logger.exception("Failed to load config:")
exit(2)
if __name__ == "__main__":
logger.info(f"All: {config}")
logger.info(f"App name: {config.app['name']}")
if _has_pydantic_settings:
# Pydantic-v2:
logger.info(f"Config:\n{pprint.pformat(config.model_dump())}\n")
else:
# Pydantic-v1:
logger.info(f"Config:\n{pprint.pformat(config.dict())}\n")
Run the examples/simple:
cd ./examples/simple
python ./main.py
Output:
2023-09-01 00:00:00.000 | INFO | __main__:<module>:29 - All: env='production' another_val={'extra': 1} app={'name': 'New App', 'version': '0.0.1', 'nested': {'key': 'value', 'some': 'value'}, 'description': 'Description of my app.'}
2023-09-01 00:00:00.000 | INFO | __main__:<module>:30 - App name: New App
2023-09-01 00:00:00.000 | INFO | __main__:<module>:35 - Config:
{'another_val': {'extra': 1},
'app': {'description': 'Description of my app.',
'name': 'New App',
'nested': {'key': 'value', 'some': 'value'},
'version': '0.0.1'},
'env': 'production'}
ENV=development
DEBUG=true
APP_NAME="Old App"
ONION_CONFIG_EXTRA_DIR="extra_configs"
ENV=production
APP_NAME="New App"
APP_SECRET="my_secret"
examples/advanced/configs/config.yml:
env: local
app:
name: "My App"
port: 9000
bind_host: "0.0.0.0"
version: "0.0.1"
ignore_val: "Ignore me"
logger:
output: "file"
examples/advanced/configs/logger.json:
{
"logger": {
"level": "info",
"output": "stdout"
}
}
examples/advanced/configs_2/config.yml:
extra:
config:
key1: 1
examples/advanced/configs_2/config_2.yml:
extra:
config:
key2: 2
examples/advanced/extra_configs/extra.json:
{
"extra": {
"type": "json"
}
}
from enum import Enum
from typing import Union
import pydantic
from pydantic import Field, SecretStr
_has_pydantic_settings = False
if "2.0.0" <= pydantic.__version__:
try:
from pydantic_settings import SettingsConfigDict
_has_pydantic_settings = True
except ImportError:
pass
from onion_config import BaseConfig
# Environments as Enum:
class EnvEnum(str, Enum):
LOCAL = "local"
DEVELOPMENT = "development"
TEST = "test"
DEMO = "demo"
STAGING = "staging"
PRODUCTION = "production"
# App config schema:
class AppConfig(BaseConfig):
name: str = Field("App", min_length=2, max_length=32)
bind_host: str = Field("localhost", min_length=2, max_length=128)
port: int = Field(8000, ge=80, lt=65536)
secret: SecretStr = Field(..., min_length=8, max_length=64)
version: str = Field(..., min_length=5, max_length=16)
description: Union[str, None] = Field(None, min_length=4, max_length=64)
if _has_pydantic_settings:
# Pydantic-v2:
model_config = SettingsConfigDict(extra="ignore", env_prefix="APP_")
else:
# Pydantic-v1:
class Config:
extra = "ignore"
env_prefix = "APP_"
# Main config schema:
class ConfigSchema(BaseConfig):
env: EnvEnum = Field(EnvEnum.LOCAL)
debug: bool = Field(False)
app: AppConfig = Field(...)
from loguru import logger
from onion_config import ConfigLoader
from schema import ConfigSchema
# Pre-load function to modify config data before loading and validation:
def _pre_load_hook(config_data: dict) -> dict:
config_data["app"]["port"] = "80"
config_data["extra_val"] = "Something extra!"
return config_data
config = None
try:
_config_loader = ConfigLoader(
config_schema=ConfigSchema,
configs_dirs=["configs", "configs_2", "/not_exists/path/configs_3"],
env_file_paths=[".env", ".env.base", ".env.prod"],
pre_load_hook=_pre_load_hook,
config_data={"base": "start_value"},
warn_mode="ALWAYS",
)
# Main config object:
config: ConfigSchema = _config_loader.load()
except Exception:
logger.exception("Failed to load config:")
exit(2)
import pprint
from loguru import logger
try:
import pydantic_settings
_has_pydantic_settings = True
except ImportError:
_has_pydantic_settings = False
from config import config
if __name__ == "__main__":
logger.info(f"All: {config}")
logger.info(f"ENV: {config.env}")
logger.info(f"DEBUG: {config.debug}")
logger.info(f"Extra: {config.extra_val}")
logger.info(f"Logger: {config.logger}")
logger.info(f"App: {config.app}")
logger.info(f"Secret: '{config.app.secret.get_secret_value()}'\n")
if _has_pydantic_settings:
# Pydantic-v2:
logger.info(f"Config:\n{pprint.pformat(config.model_dump())}\n")
else:
# Pydantic-v1:
logger.info(f"Config:\n{pprint.pformat(config.dict())}\n")
try:
# This will raise ValidationError
config.app.port = 8443
except Exception as e:
logger.error(f"{e}\n")
Run the examples/advanced:
cd ./examples/advanced
python ./main.py
Output:
2023-09-01 00:00:00.000 | INFO | onion_config._base:load:143 - Loading all configs...
2023-09-01 00:00:00.000 | WARNING | onion_config._base:_load_dotenv_file:201 - '/home/user/workspaces/projects/onion_config/examples/advanced/.env' file is not exist!
2023-09-01 00:00:00.000 | WARNING | onion_config._base:_load_configs_dir:257 - '/not_exists/path/configs_3' directory is not exist!
2023-09-01 00:00:00.000 | SUCCESS | onion_config._base:load:171 - Successfully loaded all configs!
2023-09-01 00:00:00.000 | INFO | __main__:<module>:19 - All: env=<EnvEnum.PRODUCTION: 'production'> debug=True app=AppConfig(name='New App', bind_host='0.0.0.0', port=80, secret=SecretStr('**********'), version='0.0.1', description=None) extra={'config': {'key1': 1, 'key2': 2}, 'type': 'json'} extra_val='Something extra!' logger={'output': 'stdout', 'level': 'info'} base='start_value'
2023-09-01 00:00:00.000 | INFO | __main__:<module>:20 - ENV: production
2023-09-01 00:00:00.000 | INFO | __main__:<module>:21 - DEBUG: True
2023-09-01 00:00:00.000 | INFO | __main__:<module>:22 - Extra: Something extra!
2023-09-01 00:00:00.000 | INFO | __main__:<module>:23 - Logger: {'output': 'stdout', 'level': 'info'}
2023-09-01 00:00:00.000 | INFO | __main__:<module>:24 - App: name='New App' bind_host='0.0.0.0' port=80 secret=SecretStr('**********') version='0.0.1' description=None
2023-09-01 00:00:00.000 | INFO | __main__:<module>:25 - Secret: 'my_secret'
2023-09-01 00:00:00.000 | INFO | __main__:<module>:30 - Config:
{'app': {'bind_host': '0.0.0.0',
'description': None,
'name': 'New App',
'port': 80,
'secret': SecretStr('**********'),
'version': '0.0.1'},
'base': 'start_value',
'debug': True,
'env': <EnvEnum.PRODUCTION: 'production'>,
'extra': {'config': {'key1': 1, 'key2': 2}, 'type': 'json'},
'extra_val': 'Something extra!',
'logger': {'level': 'info', 'output': 'stdout'}}
2023-09-01 00:00:00.000 | ERROR | __main__:<module>:36 - "AppConfig" is immutable and does not support item assignment
👍
# ENV=development
# DEBUG=true
ONION_CONFIG_EXTRA_DIR="./extra_configs"
To run tests, run the following command:
# Install core dependencies:
pip install -r ./requirements/requirements.core.txt
# Pydantic-v1:
pip install -r ./requirements/requirements.pydantic-v1.txt
# Pydantic-v2:
pip install -r ./requirements/requirements.pydantic-settings.txt
# Install python test dependencies:
pip install -r ./requirements.test.txt
# Run tests:
python -m pytest -sv -o log_cli=true
# Or use the test script:
./scripts/test.sh -l -v -c
To build the python package, run the following command:
# Install python build dependencies:
pip install -r ./requirements/requirements.build.txt
# Build python package:
python -m build
# Or use the build script:
./scripts/build.sh
To build the documentation, run the following command:
# Install python documentation dependencies:
pip install -r ./requirements/requirements.docs.txt
# Serve documentation locally (for development):
mkdocs serve
# Or use the docs script:
./scripts/docs.sh
# Or build documentation:
mkdocs build
# Or use the docs script:
./scripts/docs.sh -b
FAQs
`onion_config` is a Python package designed for easy configuration management. It supports loading and validating configuration data from environment variables and configuration files in JSON and YAML formats. It is a `Pydantic` based custom configuration package for Python projects.
We found that onion-config demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
/Security News
A flawed sandbox in @nestjs/devtools-integration lets attackers run code on your machine via CSRF, leading to full Remote Code Execution (RCE).
Product
Customize license detection with Socket’s new license overlays: gain control, reduce noise, and handle edge cases with precision.
Product
Socket now supports Rust and Cargo, offering package search for all users and experimental SBOM generation for enterprise projects.