zync-nest-data-module

Zync NestJS Data Module

A comprehensive NestJS data module providing database utilities, backup services, and base service classes for modern web applications built with MongoDB and Mongoose.

📦 Features

• Database Module: Complete MongoDB/Mongoose integration with repositories, transactions, and utilities
• Database Backup Service: Automated MongoDB backup with cloud storage support
• Base Service Class: Generic service class with CRUD operations and pagination
• Database Utilities: Helper functions for database operations, unique ID generation, and data synchronization
• Transaction Management: Built-in transaction support for complex database operations
• Repository Pattern: Abstract base repository with common database operations
• Database Schema Utilities: Schema management and validation helpers

🚀 Installation

Local Development

# Clone the repository
git clone https://github.com/zynctech/zync-nest-data-module.git
cd zync-nest-data-module

# Install dependencies
pnpm install

# Build the library
pnpm run build

Using in Other Projects

Method 1: npm/pnpm install from registry

# Install from AsyncTech registry
npm install zync-nest-data-module --registry https://registry.asynctechs.com/
# or
pnpm add zync-nest-data-module --registry https://registry.asynctechs.com/

Method 2: File Path Dependency (For local development)

In your project's package.json:

{
  "dependencies": {
    "zync-nest-data-module": "file:../../zync-library/zync-nest-data-module"
  }
}

Then run:

pnpm install

Method 3: npm link (For npm users)

# In the library directory
npm link

# In your consuming project
npm link zync-nest-data-module

Method 4: pnpm link (Alternative for pnpm)

# Setup pnpm global bin directory (one-time setup)
pnpm setup

# In the library directory
pnpm link --global

# In your consuming project
pnpm link --global zync-nest-data-module

📚 Usage

Basic Setup

Import the required modules in your NestJS application:

import { Module } from '@nestjs/common';
import { 
  DatabaseModule,
  BackupModule,
  BaseService
} from 'zync-nest-data-module';

@Module({
  imports: [
    DatabaseModule,
    BackupModule,
    // ... other modules
  ],
})
export class AppModule {}

Database Module

The DatabaseModule provides comprehensive MongoDB/Mongoose integration with repositories, transactions, and utilities.

Database Schema

The module includes a BaseSchema class that provides common fields for all documents:

import { BaseSchema } from 'zync-nest-data-module';

@Schema()
export class MyDocument extends BaseSchema {
  @Prop({ required: true })
  name: string;
  
  @Prop()
  description: string;
}

Repository Pattern

Create repositories by extending AbstractBaseRepository:

import { Injectable } from '@nestjs/common';
import { AbstractBaseRepository, IPageParams } from 'zync-nest-data-module';

@Injectable()
export class MyRepository extends AbstractBaseRepository<MyDocument> {
  constructor(@InjectModel(MyDocument.name) model: SoftDeleteModel<MyDocument>) {
    super(model);
  }

  protected buildQuery(query: Partial<MyDocument>): any {
    return query;
  }

  public async mapData(data: any, isCreate: boolean): Promise<MyDocument> {
    return data;
  }

  // Additional custom methods
  async findByName(name: string): Promise<MyDocument[]> {
    return this.find({ name });
  }
}

Transaction Support

Use transactions for complex operations:

import { Injectable } from '@nestjs/common';
import { TransactionManager } from 'zync-nest-data-module';

@Injectable()
export class MyService {
  constructor(
    private readonly transactionManager: TransactionManager,
    private readonly myRepository: MyRepository
  ) {}

  async complexOperation() {
    return await this.transactionManager.runTransaction(async (session) => {
      this.myRepository.session = session;
      
      const doc1 = await this.myRepository.create({ name: 'Document 1' });
      const doc2 = await this.myRepository.create({ name: 'Document 2' });
      
      return { doc1, doc2 };
    });
  }
}

Pagination

Use built-in pagination functionality:

async getPaginatedData(pageParams: IPageParams) {
  return await this.myRepository.page({
    ...pageParams,
    // additional query filters
  });
}

Database Backup Service

The BackupService provides automated MongoDB backup functionality with cloud storage support.

Configuration

Configure backup settings using environment variables:

# Database Backup Configuration
app_env=production
mongodb_backup_connection_strings=mongodb://localhost:27017/mydb
db_backup_enabled=true
db_backup_dir=./backups
db_backup_max=10

# Cloud Storage Configuration (DigitalOcean Spaces / AWS S3)
aws_bucket=my-backup-bucket
aws_s3_region=nyc3
aws_access_key_id=your-access-key
aws_secret_access_key=your-secret-key
s3_spaces_endpoint=https://nyc3.digitaloceanspaces.com
s3_spaces_dir=db-backups

Usage

import { Injectable } from '@nestjs/common';
import { BackupService } from 'zync-nest-data-module';

@Injectable()
export class MyBackupService {
  constructor(private readonly backupService: BackupService) {}

  async createBackup() {
    await this.backupService.backup();
  }

  async scheduleBackups() {
    // Set up automated backups (runs every 6 hours by default)
    this.backupService.startBackupSchedule();
  }
}

Base Service Class

The BaseService provides a generic service class with common CRUD operations:

import { Injectable } from '@nestjs/common';
import { BaseService } from 'zync-nest-data-module';

@Injectable()
export class MyService extends BaseService<MyDocument> {
  constructor(myRepository: MyRepository) {
    super(myRepository);
  }

  // Inherits all CRUD operations:
  // - create(data)
  // - update(_id, data)
  // - findById(_id)
  // - findOne(query)
  // - find(query)
  // - delete(_id)
  // - page(query)

  // Add custom business logic
  async findActiveDocuments(): Promise<MyDocument[]> {
    return this.find({ deleted: false });
  }
}

Database Utilities

The module includes various utility functions:

import { 
  DbUtils,
  ApUniqueIdGenerator,
  UniqueKeyTypes 
} from 'zync-nest-data-module';

// Generate unique IDs
const uniqueId = await uniqueIdGenerator.generate({
  prefix: 'DOC',
  key: UniqueKeyTypes.DOCUMENT,
  filter: { status: 'active' }
});

// Database utilities
const isValidObjectId = DbUtils.isValidObjectId('507f1f77bcf86cd799439011');
const objectId = DbUtils.toObjectId('507f1f77bcf86cd799439011');

🔧 Configuration

Environment Variables

# Application Environment
app_env=production
NODE_ENV=production
is_docker=false

# MongoDB Configuration
mongodb_url=mongodb://localhost:27017/mydb
mongodb_backup_connection_strings=mongodb://localhost:27017/mydb

# Database Backup Configuration
db_backup_enabled=true
db_backup_dir=./backups
db_backup_max=10

# Cloud Storage Configuration (DigitalOcean Spaces / AWS S3)
aws_bucket=my-backup-bucket
aws_s3_region=nyc3
aws_access_key_id=your-access-key
aws_secret_access_key=your-secret-key
aws_s3_endpoint=https://nyc3.digitaloceanspaces.com
aws_base_key=development
s3_spaces_dir=db-backups
s3_spaces_endpoint=https://sgp1.digitaloceanspaces.com

🛠️ Development

Scripts

# Build the library
pnpm run build

# Build library with clean (removes dist first)
pnpm run build:lib

# Build in watch mode
pnpm run build:watch

# Clean build artifacts
pnpm run clean

# Start the application (builds first)
pnpm run start

# Start in development mode (builds in watch mode + nodemon)
pnpm run start:dev

# Publish to AsyncTech registry
pnpm run publish:update

# Update links in consuming projects
pnpm run update-links

# Update links with version bump (minor)
pnpm run update-links:minor

# Update links with version bump (major)
pnpm run update-links:major

# Update links without building
pnpm run update-links:no-build

# Update links without version bump or build
pnpm run update-links:no-version

Project Structure

libs/
├── src/
│   ├── backup/              # Database backup functionality
│   │   ├── backup.config.ts
│   │   ├── backup.interface.ts
│   │   ├── backup.module.ts
│   │   ├── backup.service.ts
│   │   └── index.ts
│   ├── database/            # Database utilities and patterns
│   │   ├── database.module.ts
│   │   ├── database.repository.ts
│   │   ├── database.scheme.ts
│   │   ├── database.service.ts
│   │   ├── database.sync.ts
│   │   ├── database.transaction.ts
│   │   ├── database.uniqueId.ts
│   │   ├── database.utils.ts
│   │   └── index.ts
│   ├── service/             # Base service class
│   │   ├── service.ts
│   │   └── index.ts
│   ├── test/                # Example/test implementation
│   │   ├── test.dto.ts
│   │   ├── test.module.ts
│   │   ├── test.repository.ts
│   │   ├── test.resolver.ts
│   │   ├── test.schema.ts
│   │   └── test.service.ts
│   ├── app.controller.ts
│   ├── app.module.ts
│   ├── main.ts
│   └── index.ts             # Main exports
└── tsconfig.lib.json

Common Issues

1. pnpm link not working

If pnpm link doesn't work, use the file path dependency method instead:

{
  "dependencies": {
    "zync-nest-data-module": "file:../../zync-library/zync-nest-data-module"
  }
}

2. Peer dependency warnings

If you see peer dependency warnings (e.g., @nestjs/core version mismatch), you can:

• Update your project's NestJS version to match
• Or add to your package.json:

{
  "pnpm": {
    "peerDependencyRules": {
      "ignoreMissing": ["@nestjs/core"]
    }
  }
}

3. Build errors after linking

Make sure to build the library first:

cd zync-nest-data-module
pnpm run build

4. Changes not reflected

After making changes to the library:

# In library directory
pnpm run build

# No need to reinstall - changes are automatically available in linked projects

5. Registry access issues

If you encounter issues accessing the AsyncTech registry, ensure you have proper authentication:

# Login to AsyncTech registry
npm login --registry https://registry.asynctechs.com/

🐛 Issues

If you encounter any issues or have questions, please file an issue on the project repository.