serverless-plugin-module-registry

Serverless Module Registry Plugin

A Serverless Framework plugin that scans module registry files, discovers endpoints, and stores module feature mappings in DynamoDB for cross-module discovery and access control.

🎯 Purpose

This plugin enables:

• Module Discovery: Automatically catalog all modules and their features
• Endpoint Mapping: Track which API endpoints belong to which features
• Access Control: Store IAM policies for cross-module permission management
• Service Registry: Provide a central registry for module-to-module communication

📦 Installation

npm install --save-dev serverless-plugin-module-registry

🚀 Usage

Basic Configuration

Add the plugin to your serverless.yml:

plugins:
  - '@hyperdrive.bot/serverless-composer'      # Must be loaded first
  - 'serverless-plugin-module-registry'        # Load after composer

custom:
  moduleRegistry:
    tableName: ModuleRegistry                   # Optional: Custom table name
    region: us-east-1                          # Optional: Custom region
    skipDynamoDB: false                        # Optional: Skip DynamoDB operations

Module Registry Structure

Each module should have a registry/ folder:

serverless/modules/my-module/
├── functions/                  # Existing serverless functions
├── resources/                  # Existing serverless resources
└── registry/                   # NEW: Registry definitions
    ├── module.yml              # Module metadata
    └── features/               # Feature definitions
        ├── user-management.yml
        └── notification-system.yml

Module Metadata (registry/module.yml)

name: "My Module"
version: "1.0.0"
description: "Sample module for testing"
maintainer: "DevSquad Team"
tags:
  - "sample"
  - "testing"

Feature Definition (registry/features/user-management.yml)

name: "User Management"
description: "CRUD operations for user lifecycle management"
version: "1.2.0"

endpoints:
  - "POST/users"
  - "GET/users"
  - "GET/users/*"
  - "PUT/users/*"
  - "DELETE/users/*"

customPolicies:
  - Effect: Allow
    Action:
      - "execute-api:Invoke"
    Resource:
      - "arn:aws:execute-api:*:*:*/*/POST/users"
      - "arn:aws:execute-api:*:*:*/*/GET/users"
      - "arn:aws:execute-api:*:*:*/*/GET/users/*"
  - Effect: Allow
    Action:
      - "dynamodb:PutItem"
      - "dynamodb:GetItem"
      - "dynamodb:Query"
    Resource:
      - "arn:aws:dynamodb:*:*:table/Users"

🗄️ DynamoDB Table Schema

The plugin creates an intermodular ModuleRegistry table using AWS SDK v3 (not CloudFormation) with the following structure:

🔄 Why SDK v3 Instead of CloudFormation?

✅ Intermodular Independence: The table exists independently of any specific module's deployment lifecycle

✅ Cross-Module Sharing: Multiple modules can write to the same registry without ownership conflicts

✅ Deployment Flexibility: The table persists even if individual modules are removed or redeployed

✅ No Stack Dependencies: No CloudFormation stack owns the table, preventing accidental deletion

📋 Table Management

• Creation: Automatic during first deployment with registry data
• Naming: {service}-{stage}-module-registry (e.g., ds-api-live-module-registry)
• Compliance: Follows @dynamodb-tables.md standards (point-in-time recovery, tags)
• Lifecycle: Independent of individual module deployments

Primary Key

• PK: MODULE#{moduleName} (e.g., MODULE#sign)
• SK: FEATURE#{featureName} (e.g., FEATURE#user-management)

Global Secondary Index (GSI1)

• GSI1PK: MODULES (for listing all modules)
• GSI1SK: MODULE#{moduleName} (for grouping by module)

Global Secondary Index (GSI2)

• GSI2PK: FEATURES (for listing all features)
• GSI2SK: FEATURE#{featureId} (for direct feature lookup by ID)

Data Structure

{
  "PK": "MODULE#sign",
  "SK": "FEATURE#user-management", 
  "GSI1PK": "MODULES",
  "GSI1SK": "MODULE#sign",
  "moduleName": "sign",
  "featureName": "user-management",
  "description": "CRUD operations for user lifecycle",
  "version": "1.2.0",
  "endpoints": [
    "POST/users",
    "GET/users",
    "GET/users/*"
  ],
  "customPolicies": [...],
  "lastUpdated": "2025-01-01T00:00:00Z"
}

🔄 How It Works

• After Composer: Plugin hooks after composer processes modules
• Registry Scanning: Scans each module's registry/ folder
• Feature Processing: Reads and validates feature definitions
• Table Management: Creates intermodular DynamoDB table via AWS SDK v3 (if doesn't exist)
• Standards Compliance: Enables point-in-time recovery and applies compliance tags
• Data Storage: Batch writes registry data to DynamoDB with optimized structure

📋 Access Patterns

The stored data supports these query patterns:

List All Modules

// Query GSI1 with GSI1PK = "MODULES"
const modules = await dynamoClient.query({
  IndexName: 'GSI1',
  KeyConditionExpression: 'GSI1PK = :pk',
  ExpressionAttributeValues: {
    ':pk': 'MODULES'
  }
})

Get Module Features

// Query main table with PK = "MODULE#{name}"
const features = await dynamoClient.query({
  KeyConditionExpression: 'PK = :pk',
  ExpressionAttributeValues: {
    ':pk': `MODULE#${moduleName}`
  }
})

Get Specific Feature

// Get item with PK + SK
const feature = await dynamoClient.getItem({
  Key: {
    PK: `MODULE#${moduleName}`,
    SK: `FEATURE#${featureName}`
  }
})

Get Feature by ID (GSI2)

// Query GSI2 with featureId
const feature = await dynamoClient.query({
  IndexName: 'GSI2',
  KeyConditionExpression: 'GSI2PK = :pk AND GSI2SK = :sk',
  ExpressionAttributeValues: {
    ':pk': 'FEATURES',
    ':sk': `FEATURE#${featureId}`
  }
})

⚙️ Configuration Options

Option	Type	Default	Description
tableName	string	{service}-{stage}-module-registry	DynamoDB table name
region	string	provider.region	AWS region for DynamoDB
skipDynamoDB	boolean	false	Skip DynamoDB operations
strict	boolean	false	Enforce registry folders on all modules

🔒 Strict Mode (strict: true)

Controls whether the plugin should enforce registry compliance:

strict: false (Default - Graceful Mode)

custom:
  moduleRegistry:
    strict: false  # Graceful: skip modules without registry folders

• ✅ Graceful Migration: Modules without registry folders are silently skipped
• ✅ Backwards Compatible: Existing modules continue to work unchanged
• ✅ Gradual Adoption: Teams can add registry folders at their own pace
• ✅ Production Safe: No deployment failures

strict: true (Enforcement Mode)

custom:
  moduleRegistry:
    strict: true   # Strict: error if modules are missing registry folders

• 🚨 Enforces Compliance: Deployment fails if modules don't have registry folders
• 🔒 Quality Gate: Ensures all modules participate in the registry
• 📋 Documentation Requirement: Forces teams to document their module features
• ⚡ Use After Migration: Enable once all modules have been updated

🎯 When to Use Each Mode

Use strict: false when:

• 🚀 Initial Rollout: Adding the plugin to existing systems
• 🔄 Migration Phase: Teams are gradually adding registry folders
• 🧪 Testing Phase: Experimenting with the registry system
• 🏗️ Development: Working with modules that aren't fully documented yet

Use strict: true when:

• ✅ Full Adoption: All modules have been migrated to use registry folders
• 🔒 Governance: You want to enforce registry compliance as a quality gate
• 📋 Documentation Standards: Registry definitions are required for all modules
• 🚀 Production Systems: Where complete module documentation is mandatory

🐛 Debugging

The plugin provides detailed logging with [module-registry] prefix:

serverless deploy --verbose

Example output:

[module-registry] 🔍 Scanning modules for registry definitions...
[module-registry] Found 3 modules: sample, sign, agents
[module-registry] 📋 Processing registry for module: sample
[module-registry]    Found 2 feature definitions
[module-registry]    ✓ user-management: 5 endpoints
[module-registry]    ✓ notification-system: 9 endpoints
[module-registry] ✅ Registry processing complete. Found 4 features across 3 modules

🤝 Integration with Handlers - Direct Import Pattern

No need for separate service files! Import service functions directly from the plugin:

TypeScript/ES6 Imports

// In your handlers - import from the generated service package
import {
  listAllModules,
  getModuleFeatures,
  getFeatureDetails,
  createModuleRegistryLogger,
  type ModuleInfo,
  type FeatureInfo
} from 'module-registry'

const logger = createModuleRegistryLogger('my-handler')

export const myHandler = async (event: any) => {
  const modules = await listAllModules()
  logger.info(`Found ${modules.length} modules`)
  return { modules }
}

// Example: Get feature by ID without knowing module
export const getFeatureHandler = async (event: any) => {
  const { featureId } = event.pathParameters
  const feature = await getFeatureById(featureId)
  
  if (!feature) {
    return { statusCode: 404, body: { error: 'Feature not found' } }
  }
  
  return { statusCode: 200, body: feature }
}

CommonJS/Node.js Require

// In your handlers - require from the generated service package
const {
  listAllModules,
  getModuleFeatures,
  createModuleRegistryLogger
} = require('module-registry')

const logger = createModuleRegistryLogger('my-handler')

exports.myHandler = async (event) => {
  const modules = await listAllModules()
  logger.info(`Found ${modules.length} modules`)
  return { modules }
}

Available Service Functions

Function	Description	Returns
listAllModules()	List all deployed modules	Promise<ModuleInfo[]>
getModuleFeatures(moduleName)	Get features for a module	Promise<FeatureInfo[]>
getFeatureDetails(moduleName, featureName)	Get feature details	Promise<FeatureDetails | null>
getFeatureById(featureId)	Get feature by ID only (uses GSI2)	Promise<FeatureDetails | null>
getModuleMetadata(moduleName)	Get module metadata only	Promise<ModuleInfo | null>
getAllEndpoints()	Get all endpoints across modules	Promise<EndpointInfo[]>
createModuleRegistryLogger(context)	Create logger for service functions	Logger

Example Handler Implementation

// src/handlers/core/module-registry/listModules.js
const { http } = require('../../middlewares/http')
const { listAllModules, createModuleRegistryLogger } = require('serverless-plugin-module-registry')

const logger = createModuleRegistryLogger('list-modules-handler')

export const listModules = http(async (event) => {
  logger.info('Listing all deployed modules', { 
    userSub: event.requestContext.authorizer?.claims?.sub 
  })

  try {
    const modules = await listAllModules()
    
    logger.info(`Successfully retrieved ${modules.length} modules`)
    
    return {
      statusCode: 200,
      body: {
        modules,
        count: modules.length,
        timestamp: new Date().toISOString()
      }
    }
  } catch (error) {
    logger.error('Error listing modules:', error.message)
    
    return {
      statusCode: 500,
      body: {
        error: 'Failed to list modules',
        message: error.message
      }
    }
  }
})

🚀 CLI Commands

The plugin provides powerful CLI commands for managing module registries:

Generate Registry with AI (registryGenerate)

Automatically generate registry files for a module using AI analysis:

# Generate registry for a specific module
serverless registryGenerate --module workforce

# Force overwrite existing registry files
serverless registryGenerate --module workforce --force

Prerequisites:

• Set OPENROUTER_API_KEY environment variable for AI generation
• Module must have function descriptions in serverless function definitions
• Only functions with aws_iam authorizer are included in registry

What it does:

• Analyzes module structure (functions, resources, endpoints)
• Uses AI (Claude 3.5 Sonnet) to generate AWS-style feature groupings
• Creates registry/module.yml and registry/features/*.yml files
• Follows AWS IAM naming conventions (e.g., EmployeeReadAccess, UserSelfService)

Generate Service Package (registryGeneratePackage)

Generate the virtual service package for importing registry functions:

# Generate service package
serverless registryGeneratePackage

# Force regeneration
serverless registryGeneratePackage --force --verbose

What it creates:

• node_modules/module-registry/service.js - Service functions
• node_modules/module-registry/service.d.ts - TypeScript definitions
• node_modules/module-registry/env.js - Environment configuration
• node_modules/module-registry/package.json - Package manifest

🛠️ Development

Prerequisites

• Node.js ≥14.0.0
• TypeScript ≥5.0
• Serverless Framework ≥2.0.0

Setup

# Install dependencies
npm install

# Build the plugin
npm run build

# Watch for changes during development
npm run watch

# Run type checking
npm run typecheck

# Run linting
npm run lint

Build Configuration

The project uses tsup for building:

• Entry: src/index.ts (main plugin), src/service.ts (service functions)
• Output: dist/ directory with CommonJS and TypeScript definitions
• Target: Node.js 14+ compatibility
• External: Serverless Framework excluded from bundle

Testing

Tests are currently disabled but the framework is set up with Jest:

# Tests are disabled - would run with:
npm test  # Currently outputs: "Tests disabled"

🔧 Troubleshooting

Common Issues

Plugin not found

Error: Plugin "serverless-plugin-module-registry" not found

Solution: Ensure plugin is installed and listed in serverless.yml plugins section

Variable resolution errors

Module Registry: Unresolved variables in configuration

Solution: Check that all variables in custom.moduleRegistry section can be resolved

DynamoDB table creation fails

Failed to create table: AccessDenied

Solution: Ensure AWS credentials have DynamoDB permissions:

• dynamodb:CreateTable
• dynamodb:DescribeTable
• dynamodb:UpdateContinuousBackups
• dynamodb:TagResource

AI generation fails

OPENROUTER_API_KEY environment variable is required

Solution: Set OpenRouter API key: export OPENROUTER_API_KEY=your_key_here

Registry generation requires function descriptions

Function 'myFunction' is missing required 'description' field

Solution: Add descriptions to all functions in your serverless function definitions

Debug Mode

Enable detailed logging:

serverless deploy --verbose

Look for [module-registry] prefixed logs for plugin-specific information.

📊 Performance Considerations

DynamoDB Costs

• Plugin creates one table per service/stage combination
• Uses Pay-Per-Request billing mode
• Table persists across deployments (not recreated)
• Point-in-time recovery enabled by default

Batch Operations

• Registry updates use batch writes (25 items per batch)
• Automatic cleanup of stale entries during deployment
• Optimized for sparse access patterns

CloudFormation Policies

• IAM policies generated as CloudFormation resources
• Policy ARNs resolved at deployment time
• No runtime AWS API calls for policy management

🔐 Tenant Role Creation

The module registry provides a utility function to create IAM roles for tenant onboarding with ABAC (Attribute-Based Access Control) tags.

Usage

import { createTenantRoles } from 'serverless-plugin-module-registry/tenant'

// During tenant onboarding
const roleArns = await createTenantRoles({
  tenantId: 'acme',
  identityPoolId: 'us-east-1:12345678-1234-1234-1234-123456789012',
  modules: ['sign', 'workforce'],  // Optional: filter to specific modules
  config: {
    tableName: 'ModuleRegistry',
    region: 'us-east-1',
    policyPrefix: 'api'
  },
  logger: {
    info: (msg, data) => console.log(msg, data),
    warning: (msg, data) => console.warn(msg, data),
    error: (msg, err) => console.error(msg, err)
  }
})

console.log(roleArns)
// {
//   authenticated: "arn:aws:iam::123456789012:role/api-acme-authenticated",
//   unauthenticated: "arn:aws:iam::123456789012:role/api-acme-unauthenticated"
// }

What It Does

• Discovers Roles: Queries DynamoDB to find all role types (authenticated, unauthenticated, admin, etc.)
• Collects Features: For each role, gathers all module features and builds ABAC tags
• Creates IAM Roles: Creates roles with proper Cognito trust policies and ABAC tags
• Attaches Policies: Attaches module ManagedPolicies to roles
• Registers with Cognito: Attaches roles to the Cognito Identity Pool

Role Naming Convention

Roles follow the pattern: {policyPrefix}-{tenantId}-{roleType}

Examples:

• api-acme-authenticated
• api-acme-unauthenticated
• api-acme-admin

ABAC Tags

Each role is tagged with module features in the format:

{moduleName}Features: "feature1:feature2:feature3"

Example tags:

• signFeatures: "7tmARMbS:BRUBT2SN:F8d3wY_v"
• workforceFeatures: "abc123:def456:ghi789"

Idempotency

The function is idempotent - it can be called multiple times safely:

• Existing roles are updated with new tags and policies
• No duplicate roles are created
• Cognito attachment is updated if needed

🔒 Security Considerations

IAM Policies

• Generated policies follow least privilege principle
• Endpoint-specific resource ARNs (not wildcards)
• Custom policies allow fine-grained permission control
• Tenant roles use Cognito Identity Pool trust policies

Access Control

• Registry table access requires DynamoDB permissions
• Cross-module access controlled via IAM policy ARNs
• Service functions inherit Lambda execution role permissions

Best Practices

• Review generated policies before deployment
• Use strict mode (strict: true) in production
• Regularly audit endpoint permissions
• Implement proper IAM role boundaries

🤝 Contributing

Development Workflow

• Fork the repository
• Create a feature branch: git checkout -b feature/my-feature
• Make changes and test thoroughly
• Update documentation if needed
• Submit a pull request

Code Standards

• Follow TypeScript best practices
• Use provided ESLint configuration
• Add JSDoc comments for public APIs
• Maintain backward compatibility

Testing

When contributing:

• Add test cases for new features
• Ensure existing functionality isn't broken
• Test with multiple Serverless Framework versions

📋 Version Compatibility

Plugin Version	Serverless Framework	Node.js
1.0.x	2.x, 3.x, 4.x	≥14.0

Serverless Framework Features

• v2: Basic hook support, CloudFormation resources
• v3: Enhanced variable resolution, improved logging
• v4: Full ESM support, performance optimizations

📄 License

MIT License - see the LICENSE file for details.