@yofix/storage

@yofix/storage

Universal storage manager for handling file operations across multiple cloud providers. Upload files to GitHub Actions Artifacts, Firebase Storage, AWS S3, or local filesystem with a unified API.

Features

• 🚀 Multiple Providers: Support for GitHub Actions Artifacts, Firebase Storage, AWS S3, and local filesystem
• 📦 Unified API: Same interface across all providers
• 🔄 Glob Patterns: Upload multiple files using glob patterns
• 📊 Progress Tracking: Real-time upload progress callbacks
• 🎯 Type Safe: Full TypeScript support with comprehensive types
• ⚡ Fast: Efficient file uploads with streaming support
• 🔌 GitHub Action: Ready-to-use as a GitHub Action
• 🛠️ File Management: Check existence, delete, and list files across providers

Installation

npm install @yofix/storage

or

yarn add @yofix/storage

Usage

As a Library

Local Storage

import { uploadFiles } from '@yofix/storage'

const result = await uploadFiles({
  storage: {
    provider: 'local',
    config: {
      directory: './uploads',
      createIfNotExists: true
    }
  },
  files: [
    'package.json',
    'dist/**/*',
    {
      path: 'README.md',
      destination: 'docs/README.md'
    }
  ],
  verbose: true
})

console.log(`Uploaded ${result.metadata.totalFiles} files`)

GitHub Actions Artifacts

const result = await uploadFiles({
  storage: {
    provider: 'github',
    config: {
      artifactName: 'build-artifacts',
      retentionDays: 30
    }
  },
  files: ['dist/**/*', 'coverage/**/*'],
  verbose: true
})

Firebase Storage

const result = await uploadFiles({
  storage: {
    provider: 'firebase',
    config: {
      bucket: 'your-project.appspot.com',
      credentials: JSON.parse(process.env.FIREBASE_CREDENTIALS),
      basePath: 'uploads'
    }
  },
  files: ['images/**/*.{jpg,png}'],
  verbose: true
})

AWS S3

const result = await uploadFiles({
  storage: {
    provider: 's3',
    config: {
      region: 'us-east-1',
      bucket: 'your-bucket',
      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
      basePath: 'uploads',
      acl: 'public-read'
    }
  },
  files: [
    {
      path: 'build/app.js',
      destination: 'static/app.js',
      metadata: {
        version: '1.0.0'
      }
    }
  ],
  onProgress: (progress) => {
    console.log(`${progress.filesUploaded}/${progress.totalFiles}`)
  }
})

As a GitHub Action

name: Upload Artifacts

on:
  push:
    branches: [main]

jobs:
  upload:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Build
        run: npm run build

      - name: Upload to GitHub Artifacts
        uses: yofix/storage@main
        with:
          provider: github
          files: 'dist/**/*,coverage/**/*'
          artifact-name: 'build-artifacts'
          retention-days: 30
          verbose: true

      - name: Upload to S3
        uses: yofix/storage@main
        with:
          provider: s3
          files: 'dist/**/*'
          s3-region: us-east-1
          s3-bucket: my-bucket
          s3-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          s3-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          s3-base-path: builds/${{ github.sha }}
          s3-acl: public-read
          verbose: true

API

uploadFiles(options: StorageOptions): Promise<StorageResult>

Main function to upload files to storage.

StorageOptions

interface StorageOptions {
  storage: ProviderConfig
  files?: (string | FileToUpload)[]
  overwrite?: boolean
  verbose?: boolean
  onProgress?: (progress: UploadProgress) => void
}

ProviderConfig

type ProviderConfig =
  | { provider: 'github'; config?: GitHubConfig }
  | { provider: 'firebase'; config: FirebaseConfig }
  | { provider: 's3'; config: S3Config }
  | { provider: 'local'; config: LocalConfig }

StorageResult

interface StorageResult {
  success: boolean
  files: UploadedFile[]
  metadata: {
    provider: StorageProvider
    totalFiles: number
    totalSize: number
    duration: number
    startedAt: string
    completedAt: string
    artifactInfo?: {
      artifactName: string
      artifactId?: string
      size: number
    }
    bucketInfo?: {
      bucket: string
      basePath?: string
      region?: string
    }
  }
  errors?: StorageError[]
}

Provider Configuration

GitHub Actions

interface GitHubConfig {
  artifactName?: string // Default: 'storage-artifacts'
  retentionDays?: number // 1-90, Default: 90
}

Firebase Storage

interface FirebaseConfig {
  projectId?: string
  credentials?: string | object // Service account JSON
  bucket: string // Required
  basePath?: string
}

AWS S3

interface S3Config {
  region: string // Required
  bucket: string // Required
  accessKeyId?: string
  secretAccessKey?: string
  basePath?: string
  acl?: string // e.g., 'public-read', 'private'
  endpoint?: string // For S3-compatible services
}

Local Storage

interface LocalConfig {
  directory: string // Required
  createIfNotExists?: boolean // Default: true
}

Examples

See the demo.js file for complete examples:

# Install dependencies
npm install

# Build the project
npm run build

# Run demos
npm run demo        # Local storage (default)
node demo.js s3     # S3 storage
node demo.js firebase  # Firebase storage
node demo.js github    # GitHub Actions (requires GITHUB_ACTIONS env)
node demo.js all    # Run all demos

Advanced Usage

Custom File Destinations

const result = await uploadFiles({
  storage: { provider: 'local', config: { directory: './uploads' } },
  files: [
    {
      path: 'src/app.js',
      destination: 'build/app.min.js',
      contentType: 'application/javascript',
      metadata: {
        version: '1.0.0',
        environment: 'production'
      }
    }
  ]
})

Progress Tracking

const result = await uploadFiles({
  storage: { provider: 's3', config: {...} },
  files: ['large-files/**/*'],
  onProgress: (progress) => {
    const percent = (progress.filesUploaded / progress.totalFiles) * 100
    console.log(`Progress: ${percent.toFixed(1)}%`)
    console.log(`Current: ${progress.currentFile}`)
  }
})

Using Storage Manager Class

import { StorageManager } from '@yofix/storage'

const manager = new StorageManager({
  storage: { provider: 'local', config: { directory: './uploads' } },
  files: ['*.json'],
  verbose: true
})

// Upload files
const result = await manager.upload()

// Check if file exists
const exists = await manager.fileExists('package.json')

// List files
const files = await manager.listFiles()

// Delete file
await manager.deleteFile('old-file.txt')

Environment Variables

For convenience, you can use environment variables:

# AWS S3
export AWS_ACCESS_KEY_ID=your-access-key
export AWS_SECRET_ACCESS_KEY=your-secret-key

# Firebase
export FIREBASE_CREDENTIALS='{"type":"service_account",...}'

# Or use in code
const result = await uploadFiles({
  storage: {
    provider: 's3',
    config: {
      region: 'us-east-1',
      bucket: 'my-bucket',
      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
    }
  },
  files: ['dist/**/*']
})

TypeScript

Full TypeScript support with comprehensive type definitions:

import type {
  StorageOptions,
  StorageResult,
  ProviderConfig,
  UploadedFile,
  StorageError
} from '@yofix/storage'

Error Handling

try {
  const result = await uploadFiles({...})

  if (!result.success) {
    result.errors?.forEach(error => {
      console.error(`[${error.code}] ${error.message}`)
      if (error.file) {
        console.error(`File: ${error.file}`)
      }
    })
  }
} catch (error) {
  console.error('Upload failed:', error)
}

GitHub Repository

• Repository: yofix/storage
• Issues: Report an issue
• Pull Requests: Contribute

License

MIT

Contributing

Contributions are welcome! Please open an issue or submit a pull request on GitHub.