@push.rocks/npmextra

@push.rocks/npmextra 🚀

Supercharge your npm projects with powerful configuration management, tool orchestration, and persistent key-value storage.

Install 📦

# Using npm
npm install @push.rocks/npmextra --save

# Using pnpm (recommended)
pnpm add @push.rocks/npmextra

Overview 🎯

@push.rocks/npmextra is your Swiss Army knife for npm project configuration. It eliminates configuration sprawl by centralizing tool settings, providing intelligent key-value storage, and offering powerful environment variable mapping with automatic type conversions.

Why npmextra?

• 🎛️ Centralized Configuration: Manage all your tool configs in one npmextra.json file
• 💾 Persistent Storage: Smart key-value store with multiple storage strategies
• 🔐 Environment Mapping: Sophisticated env var handling with automatic type conversion
• 🏗️ TypeScript First: Full type safety and IntelliSense support
• ⚡ Zero Config: Works out of the box with sensible defaults
• 🔄 Reactive: Built-in change detection and observables

Core Concepts 🏗️

1. Npmextra Configuration Management

Stop scattering configuration across dozens of files. Centralize everything in npmextra.json:

import { Npmextra } from '@push.rocks/npmextra';

// Initialize with current directory
const npmextra = new Npmextra();

// Or specify a custom path
const npmextra = new Npmextra('/path/to/project');

// Get merged configuration for any tool
const eslintConfig = npmextra.dataFor<EslintConfig>('eslint', {
  // Default values if not in npmextra.json
  extends: 'standard',
  rules: {}
});

npmextra.json example:

{
  "eslint": {
    "extends": "@company/eslint-config",
    "rules": {
      "no-console": "warn"
    }
  },
  "prettier": {
    "semi": false,
    "singleQuote": true
  }
}

2. KeyValueStore - Persistent Data Storage

A flexible key-value store that persists data between script executions:

import { KeyValueStore } from '@push.rocks/npmextra';

interface UserSettings {
  username: string;
  apiKey: string;
  preferences: {
    theme: 'light' | 'dark';
    notifications: boolean;
  };
}

// Different storage strategies
const kvStore = new KeyValueStore<UserSettings>({
  typeArg: 'userHomeDir',  // Store in user's home directory
  identityArg: 'myApp',
  mandatoryKeys: ['username', 'apiKey']
});

// Write operations
await kvStore.writeKey('username', 'john_doe');
await kvStore.writeKey('preferences', {
  theme: 'dark',
  notifications: true
});

// Read operations
const username = await kvStore.readKey('username');
const allData = await kvStore.readAll();

// Check for missing mandatory keys
const missingKeys = await kvStore.getMissingMandatoryKeys();
if (missingKeys.length > 0) {
  console.log('Missing required configuration:', missingKeys);
}

// Wait for keys to be present
await kvStore.waitForKeysPresent(['apiKey']);

Storage Types:

• userHomeDir: Store in user's home directory
• custom: Specify your own path
• ephemeral: In-memory only (perfect for testing)

3. AppData - Advanced Environment Management 🌟

The crown jewel of npmextra - sophisticated environment variable mapping with automatic type conversion:

import { AppData } from '@push.rocks/npmextra';

interface AppConfig {
  apiUrl: string;
  apiKey: string;
  port: number;
  features: {
    analytics: boolean;
    payment: boolean;
  };
  cache: {
    ttl: number;
    redis: {
      host: string;
      password: string;
    };
  };
}

const appData = await AppData.createAndInit<AppConfig>({
  dirPath: '/app/config',  // Optional: defaults to smart path selection
  requiredKeys: ['apiKey', 'apiUrl'],
  envMapping: {
    apiUrl: 'API_URL',                    // Simple mapping
    apiKey: 'hard:development-key-123',   // Hardcoded value
    port: 'hard:3000',                     // Hardcoded number
    features: {
      analytics: 'boolean:ENABLE_ANALYTICS',  // Force boolean conversion
      payment: 'hard_boolean:true'            // Hardcoded boolean
    },
    cache: {
      ttl: 'json:CACHE_CONFIG',  // Parse JSON from env var
      redis: {
        host: 'REDIS_HOST',
        password: 'base64:REDIS_PASSWORD_B64'  // Decode base64
      }
    }
  },
  overwriteObject: {
    // Force these values regardless of env vars
    port: 8080
  }
});

const store = await appData.getKvStore();
const apiUrl = await store.readKey('apiUrl');

AppData Special Cases & Conversions 🎯

Environment Variable Prefixes

AppData supports sophisticated type conversion through prefixes:

Prefix	Description	Example	Result
hard:	Hardcoded value	hard:myvalue	"myvalue"
hard_boolean:	Hardcoded boolean	hard_boolean:true	true
hard_json:	Hardcoded JSON	hard_json:{"key":"value"}	{key: "value"}
hard_base64:	Hardcoded base64	hard_base64:SGVsbG8=	"Hello"
boolean:	Env var as boolean	boolean:FEATURE_FLAG	true/false
json:	Parse env var as JSON	json:CONFIG_JSON	Parsed object
base64:	Decode env var from base64	base64:SECRET_B64	Decoded string

Automatic Suffix Detection

Variables ending with certain suffixes get automatic conversion:

{
  envMapping: {
    // Automatically parsed as JSON if MY_CONFIG_JSON="{"enabled":true}"
    config: 'MY_CONFIG_JSON',
    
    // Automatically decoded from base64 if SECRET_KEY_BASE64="SGVsbG8="
    secret: 'SECRET_KEY_BASE64'
  }
}

Complex Examples

const appData = await AppData.createAndInit({
  envMapping: {
    // Simple environment variable
    apiUrl: 'API_URL',
    
    // Hardcoded values with type conversion
    debugMode: 'hard_boolean:false',
    maxRetries: 'hard:5',
    defaultConfig: 'hard_json:{"timeout":30,"retries":3}',
    
    // Environment variables with conversion
    features: 'json:FEATURE_FLAGS',  // Expects: {"feature1":true,"feature2":false}
    isProduction: 'boolean:IS_PROD', // Expects: "true" or "false"
    apiSecret: 'base64:API_SECRET',  // Expects: base64 encoded string
    
    // Nested structures
    database: {
      host: 'DB_HOST',
      port: 'hard:5432',
      credentials: {
        user: 'DB_USER',
        password: 'base64:DB_PASSWORD_ENCODED',
        ssl: 'boolean:DB_USE_SSL'
      }
    }
  },
  
  // Override any env mappings
  overwriteObject: {
    debugMode: true,  // Force debug mode regardless of env
    database: {
      host: 'localhost'  // Force localhost for development
    }
  }
});

Boolean Conversion Rules

AppData intelligently handles boolean conversions:

• String "true"/"false": Converted to boolean
• With boolean: prefix: Any env var value is converted ("true" → true, anything else → false)
• With hard_boolean: prefix: Hardcoded boolean value
• Regular env vars: Strings remain strings unless prefixed

// Environment: FEATURE_A="true", FEATURE_B="yes", FEATURE_C="1"
{
  envMapping: {
    featureA: 'FEATURE_A',           // Result: "true" (string)
    featureB: 'boolean:FEATURE_B',   // Result: false (only "true" → true)
    featureC: 'boolean:FEATURE_C',   // Result: false (only "true" → true)
    featureD: 'hard_boolean:true'    // Result: true (hardcoded)
  }
}

Static Helper Functions

AppData provides convenient static methods for directly accessing and converting environment variables without creating an instance:

import { AppData } from '@push.rocks/npmextra';

// Get environment variable as boolean
const isEnabled = await AppData.valueAsBoolean('FEATURE_ENABLED');
// Returns: true if "true", false otherwise

// Get environment variable as parsed JSON
interface Config {
  timeout: number;
  retries: number;
}
const config = await AppData.valueAsJson<Config>('SERVICE_CONFIG');
// Returns: Parsed object or undefined

// Get environment variable as base64 decoded string
const secret = await AppData.valueAsBase64('ENCODED_SECRET');
// Returns: Decoded string or undefined

// Get environment variable as string
const apiUrl = await AppData.valueAsString('API_URL');
// Returns: String value or undefined

// Get environment variable as number
const port = await AppData.valueAsNumber('PORT');
// Returns: Number value or undefined

These static methods are perfect for:

• Quick environment variable access without setup
• Simple type conversions in utility functions
• One-off configuration checks
• Scenarios where you don't need the full AppData instance

Advanced Patterns 🎨

Reactive Configuration

Subscribe to configuration changes:

const kvStore = new KeyValueStore<Config>({
  typeArg: 'custom',
  identityArg: 'myApp'
});

// Subscribe to changes
kvStore.changeSubject.subscribe((newData) => {
  console.log('Configuration changed:', newData);
});

// Changes trigger notifications
await kvStore.writeKey('theme', 'dark');

Testing with Ephemeral Storage

Perfect for unit tests - no file system pollution:

const testStore = new KeyValueStore<TestData>({
  typeArg: 'ephemeral',
  identityArg: 'test'
});

// All operations work normally, but nothing persists to disk
await testStore.writeKey('testKey', 'testValue');

Smart Path Resolution

AppData automatically selects the best storage location:

• Checks for /app/data (containerized environments)
• Falls back to /data (alternate container path)
• Uses .nogit/appdata (local development)

// Automatic path selection
const appData = await AppData.createAndInit({
  // No dirPath specified - smart detection
  requiredKeys: ['apiKey']
});

// Or force ephemeral for testing
const testData = await AppData.createAndInit({
  ephemeral: true,  // No disk persistence
  requiredKeys: ['testKey']
});

Waiting for Configuration

Block until required configuration is available:

const appData = await AppData.createAndInit<Config>({
  requiredKeys: ['apiKey', 'apiUrl']
});

// Wait for specific key
const apiKey = await appData.waitForAndGetKey('apiKey');

// Check missing keys
const missingKeys = await appData.logMissingKeys();
// Logs: "The following mandatory keys are missing in the appdata:
//        -> apiKey,
//        -> apiUrl"

Real-World Example 🌍

Here's a complete example of a CLI tool using npmextra:

import { Npmextra, AppData, KeyValueStore } from '@push.rocks/npmextra';

interface CliConfig {
  githubToken: string;
  openaiKey: string;
  model: 'gpt-3' | 'gpt-4';
  cache: {
    enabled: boolean;
    ttl: number;
  };
}

class MyCLI {
  private npmextra: Npmextra;
  private appData: AppData<CliConfig>;
  private cache: KeyValueStore<{[key: string]: any}>;

  async initialize() {
    // Load tool configuration
    this.npmextra = new Npmextra();
    const config = this.npmextra.dataFor<any>('mycli', {
      defaultModel: 'gpt-3'
    });

    // Setup app data with env mapping
    this.appData = await AppData.createAndInit<CliConfig>({
      requiredKeys: ['githubToken', 'openaiKey'],
      envMapping: {
        githubToken: 'GITHUB_TOKEN',
        openaiKey: 'base64:OPENAI_KEY_ENCODED',
        model: 'hard:gpt-4',
        cache: {
          enabled: 'boolean:ENABLE_CACHE',
          ttl: 'hard:3600'
        }
      }
    });

    // Initialize cache
    this.cache = new KeyValueStore({
      typeArg: 'userHomeDir',
      identityArg: 'mycli-cache'
    });

    // Check for missing configuration
    const missingKeys = await this.appData.logMissingKeys();
    if (missingKeys.length > 0) {
      console.error('Please configure the missing keys');
      process.exit(1);
    }
  }

  async run() {
    await this.initialize();
    
    const config = await this.appData.getKvStore();
    const settings = await config.readAll();
    
    console.log(`Using model: ${settings.model}`);
    console.log(`Cache enabled: ${settings.cache.enabled}`);
    
    // Use the configuration...
  }
}

// Run the CLI
const cli = new MyCLI();
cli.run();

API Reference 📚

Npmextra Class

new Npmextra(cwdArg?: string)

• cwdArg: Optional working directory path

Methods:

• dataFor<T>(toolName: string, defaultOptions: T): T - Get merged configuration

KeyValueStore Class

new KeyValueStore<T>(options: {
  typeArg: 'custom' | 'userHomeDir' | 'ephemeral';
  identityArg: string;
  customPath?: string;
  mandatoryKeys?: Array<keyof T>;
})

Methods:

• readKey(key: string): Promise<T> - Read single value
• writeKey(key: string, value: T): Promise<void> - Write single value
• readAll(): Promise<T> - Read all values
• writeAll(data: T): Promise<void> - Write all values
• deleteKey(key: string): Promise<void> - Delete a key
• getMissingMandatoryKeys(): Promise<string[]> - Check missing required keys
• waitForKeysPresent(keys: string[]): Promise<void> - Wait for keys

AppData Class

await AppData.createAndInit<T>(options: {
  dirPath?: string;
  requiredKeys?: Array<keyof T>;
  ephemeral?: boolean;
  envMapping?: PartialDeep<T>;
  overwriteObject?: PartialDeep<T>;
})

Methods:

• getKvStore(): Promise<KeyValueStore<T>> - Get underlying store
• logMissingKeys(): Promise<Array<keyof T>> - Log and return missing keys
• waitForAndGetKey<K>(key: K): Promise<T[K]> - Wait for and retrieve key

License and Legal Information

This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the license file within this repository.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.

Company Information

Task Venture Capital GmbH
Registered at District court Bremen HRB 35230 HB, Germany

For any legal inquiries or if you require further information, please contact us via email at [e-mail].

By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.