@synet/fs-github
Advanced tools
GitHub FileSystem Adapter - Transform GitHub repositories into powerful file storage systems with built-in version control, collaboration, and global CDN distribution.
Turn any GitHub repository into a fully functional filesystem with automatic versioning, instant global availability, and enterprise-grade security. Perfect for configuration management, documentation systems, and collaborative data storage.
# npm
npm install @synet/fs-github
Navigate to GitHub Settings:
Generate New Token:
SYNET FS GitHub PackageRequired Permissions:
repo (Full control of private repositories)
├── repo:status - Access commit status
├── repo_deployment - Access deployment status
├── public_repo - Access public repositories
├── repo:invite - Access repository invitations
└── security_events - Read and write security events
Copy Token:
ghp_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxQuick Link: Create Token
Create a repository where files will be stored:
# Via GitHub CLI
gh repo create my-app-storage --private
# Or via GitHub web interface
# https://github.com/new
Test your setup:
import { GitHubFileSystem } from '@synet/fs-github';
const github = new GitHubFileSystem({
token: 'your_token_here',
owner: 'your-username',
repo: 'your-repo',
branch: 'main'
});
// Test connection
const info = github.getRepositoryInfo();
console.log(`Connected to: ${info.owner}/${info.repo}`);
import { GitHubFileSystem } from '@synet/fs-github';
const githubFs = new GitHubFileSystem({
token: process.env.GITHUB_TOKEN!,
owner: 'your-username',
repo: 'app-config',
branch: 'main',
authorName: 'Config Manager',
authorEmail: 'config@yourcompany.com'
});
// Write configuration files
await githubFs.writeFile('/config/production.json', JSON.stringify({
database: { host: 'prod-db.com', port: 5432 },
api: { url: 'https://api.yourcompany.com' }
}));
// Read configuration
const config = JSON.parse(await githubFs.readFile('/config/production.json'));
// List files
const files = await githubFs.readDir('/config');
console.log('Config files:', files);
# .env file
GITHUB_TOKEN=ghp_your_token_here
GITHUB_OWNER=your-username
GITHUB_REPO=app-storage
GITHUB_BRANCH=main
import { GitHubFileSystem } from '@synet/fs-github';
const githubFs = new GitHubFileSystem({
token: process.env.GITHUB_TOKEN!,
owner: process.env.GITHUB_OWNER!,
repo: process.env.GITHUB_REPO!,
branch: process.env.GITHUB_BRANCH || 'main',
authorName: 'Application',
authorEmail: 'app@yourcompany.com'
});
import { GitHubFileSystem } from '@synet/fs-github';
class ConfigManager {
private environments = new Map<string, GitHubFileSystem>();
constructor() {
// Production environment (main branch)
this.environments.set('production', new GitHubFileSystem({
token: process.env.GITHUB_TOKEN!,
owner: 'your-company',
repo: 'app-config',
branch: 'main',
authorName: 'Production Deploy',
authorEmail: 'deploy@yourcompany.com'
}));
// Staging environment (staging branch)
this.environments.set('staging', new GitHubFileSystem({
token: process.env.GITHUB_TOKEN!,
owner: 'your-company',
repo: 'app-config',
branch: 'staging',
authorName: 'Staging Deploy',
authorEmail: 'deploy@yourcompany.com'
}));
// Development environment (dev branch)
this.environments.set('development', new GitHubFileSystem({
token: process.env.GITHUB_TOKEN!,
owner: 'your-company',
repo: 'app-config',
branch: 'development',
authorName: 'Dev Team',
authorEmail: 'dev@yourcompany.com'
}));
}
getConfig(environment: string): GitHubFileSystem {
const fs = this.environments.get(environment);
if (!fs) {
throw new Error(`Unknown environment: ${environment}`);
}
return fs;
}
async deployConfig(source: string, target: string, configPath: string): Promise<void> {
const sourceFs = this.getConfig(source);
const targetFs = this.getConfig(target);
const config = await sourceFs.readFile(configPath);
await targetFs.writeFile(configPath, config);
}
}
// Usage
const configManager = new ConfigManager();
const prodConfig = configManager.getConfig('production');
await prodConfig.writeFile('/api/settings.json', JSON.stringify({
rateLimiting: { enabled: true, maxRequests: 1000 },
features: { betaFeatures: false }
}));
import { GitHubFileSystem } from '@synet/fs-github';
class DocumentationManager {
private docsFs = new GitHubFileSystem({
token: process.env.GITHUB_TOKEN!,
owner: 'your-company',
repo: 'documentation',
branch: 'main',
authorName: 'Documentation Bot',
authorEmail: 'docs@yourcompany.com'
});
async publishDocs(category: string, title: string, content: string): Promise<void> {
const slug = title.toLowerCase().replace(/\s+/g, '-');
const path = `/docs/${category}/${slug}.md`;
const markdown = `# ${title}
${content}
---
*Last updated: ${new Date().toISOString()}*
*Published via @synet/fs-github*
`;
await this.docsFs.writeFile(path, markdown);
}
async getDocs(category: string): Promise<string[]> {
return await this.docsFs.readDir(`/docs/${category}`);
}
async getDocContent(category: string, slug: string): Promise<string> {
return await this.docsFs.readFile(`/docs/${category}/${slug}.md`);
}
}
// Usage
const docs = new DocumentationManager();
await docs.publishDocs('api', 'Authentication Guide', `
## Overview
Our API uses OAuth 2.0 for authentication...
## Quick Start
1. Register your application
2. Get your client credentials
3. Implement the OAuth flow
`);
import { GitHubFileSystem } from '@synet/fs-github';
class VersionedConfig {
private fs: GitHubFileSystem;
constructor() {
this.fs = new GitHubFileSystem({
token: process.env.GITHUB_TOKEN!,
owner: 'your-company',
repo: 'versioned-config',
branch: 'main',
authorName: 'Config Manager',
authorEmail: 'config@yourcompany.com'
});
}
async updateConfig(
path: string,
config: any,
reason: string
): Promise<void> {
const content = JSON.stringify(config, null, 2);
const commitMessage = `Update ${path}: ${reason}`;
// The GitHub filesystem automatically creates commits
await this.fs.writeFile(path, content);
console.log(`✅ Configuration updated: ${commitMessage}`);
}
async rollbackConfig(path: string, commitSha: string): Promise<void> {
// In a real implementation, you'd use GitHub API to get file content at specific commit
const history = await this.fs.getFileHistory(path);
const targetCommit = history.find(commit => commit.sha === commitSha);
if (targetCommit) {
await this.fs.writeFile(path, targetCommit.content);
console.log(`✅ Rolled back ${path} to ${commitSha}`);
}
}
async getConfigHistory(path: string): Promise<any[]> {
return await this.fs.getFileHistory(path);
}
}
interface GitHubFileSystemOptions {
token: string; // GitHub personal access token
owner: string; // Repository owner (username or organization)
repo: string; // Repository name
branch?: string; // Target branch (default: 'main')
authorName?: string; // Commit author name (default: 'GitHubFileSystem')
authorEmail?: string; // Commit author email (default: 'noreply@github.com')
autoCommit?: boolean; // Auto-commit on writes (default: true)
}
writeFile(path: string, content: string): Promise<void> - Write file and create commitreadFile(path: string): Promise<string> - Read file contentdeleteFile(path: string): Promise<void> - Delete file and create commitexists(path: string): Promise<boolean> - Check if file existsensureDir(path: string): Promise<void> - No-op (GitHub doesn't have directories)deleteDir(path: string): Promise<void> - Not supported (throws error)readDir(path: string): Promise<string[]> - List files with path prefixstat(path: string): Promise<FileStats> - Get file metadatagetFileHistory(path: string): Promise<CommitInfo[]> - Get file commit historygetRepositoryInfo(): RepositoryInfo - Get repository informationclearCache(): void - Clear internal file cachegetCacheStats(): CacheStats - Get cache statisticsinterface FileStats {
isFile(): boolean; // Always true for GitHub files
isDirectory(): boolean; // Always false for GitHub files
isSymbolicLink(): boolean; // Always false
size: number; // File size in bytes
mtime: Date; // Last modified time (commit date)
ctime: Date; // Creation time (first commit date)
atime: Date; // Access time (same as mtime)
mode: number; // File permissions (644)
}
interface RepositoryInfo {
owner: string; // Repository owner
repo: string; // Repository name
branch: string; // Current branch
url: string; // Repository URL
}
# Run all tests
npm test
# Run tests in watch mode
npm run dev:test
# Run tests with coverage
npm run coverage
# Run demo
npm run demo
Create test configuration:
// test-config.ts
export const testConfig = {
token: process.env.GITHUB_TEST_TOKEN!,
owner: 'test-org',
repo: 'test-repo',
branch: 'test-branch',
authorName: 'Test Bot',
authorEmail: 'test@example.com'
};
import { GitHubFileSystem } from '@synet/fs-github';
import { testConfig } from './test-config';
describe('GitHub FileSystem Integration', () => {
let githubFs: GitHubFileSystem;
beforeEach(() => {
githubFs = new GitHubFileSystem(testConfig);
});
afterEach(async () => {
// Clean up test files
try {
const files = await githubFs.readDir('/test');
for (const file of files) {
await githubFs.deleteFile(`/test/${file}`);
}
} catch {
// Ignore cleanup errors
}
});
it('should write and read files', async () => {
const path = '/test/integration.json';
const content = JSON.stringify({ test: 'data', timestamp: Date.now() });
await githubFs.writeFile(path, content);
const retrieved = await githubFs.readFile(path);
expect(retrieved).toBe(content);
});
it('should maintain file history', async () => {
const path = '/test/versioned.txt';
await githubFs.writeFile(path, 'Version 1');
await githubFs.writeFile(path, 'Version 2');
const history = await githubFs.getFileHistory(path);
expect(history).toHaveLength(2);
expect(history[0].message).toContain('versioned.txt');
});
});
const configFs = new GitHubFileSystem({
token: process.env.GITHUB_TOKEN!,
owner: 'company',
repo: 'app-config',
branch: 'production'
});
// Store feature flags
await configFs.writeFile('/features.json', JSON.stringify({
enableNewDashboard: true,
betaFeatures: false,
maintenanceMode: false
}));
// Store API configurations
await configFs.writeFile('/api/endpoints.json', JSON.stringify({
user: 'https://api.company.com/users',
payments: 'https://api.company.com/payments',
analytics: 'https://analytics.company.com/events'
}));
const docsFs = new GitHubFileSystem({
token: process.env.GITHUB_TOKEN!,
owner: 'company',
repo: 'documentation',
branch: 'main'
});
// Publish API documentation
await docsFs.writeFile('/api/users.md', `
# User API
## GET /api/users
Returns a list of users...
## POST /api/users
Creates a new user...
`);
// Update changelog
const changelog = await docsFs.readFile('/CHANGELOG.md');
const updated = `## v2.1.0 - ${new Date().toISOString().split('T')[0]}
- Added user management API
- Fixed authentication bug
${changelog}`;
await docsFs.writeFile('/CHANGELOG.md', updated);
class GitHubCMS {
private fs: GitHubFileSystem;
constructor() {
this.fs = new GitHubFileSystem({
token: process.env.GITHUB_TOKEN!,
owner: 'blog',
repo: 'content',
branch: 'main',
authorName: 'CMS Bot',
authorEmail: 'cms@blog.com'
});
}
async publishPost(slug: string, frontmatter: any, content: string): Promise<void> {
const post = `---
${Object.entries(frontmatter)
.map(([key, value]) => `${key}: ${JSON.stringify(value)}`)
.join('\n')}
---
${content}`;
await this.fs.writeFile(`/posts/${slug}.md`, post);
}
async getPosts(): Promise<string[]> {
return await this.fs.readDir('/posts');
}
async getPost(slug: string): Promise<string> {
return await this.fs.readFile(`/posts/${slug}.md`);
}
}
// Usage
const cms = new GitHubCMS();
await cms.publishPost('hello-world', {
title: 'Hello World',
date: '2025-08-10',
author: 'Admin',
tags: ['announcement', 'welcome']
}, 'Welcome to our new blog platform!');
class GitHubBackup {
private backupFs: GitHubFileSystem;
constructor() {
this.backupFs = new GitHubFileSystem({
token: process.env.GITHUB_TOKEN!,
owner: 'company',
repo: 'app-backups',
branch: 'main',
authorName: 'Backup Service',
authorEmail: 'backup@company.com'
});
}
async backupDatabase(data: any): Promise<void> {
const timestamp = new Date().toISOString();
const filename = `/backups/db-${timestamp.split('T')[0]}.json`;
await this.backupFs.writeFile(filename, JSON.stringify({
timestamp,
data,
metadata: {
source: 'production-db',
type: 'full-backup',
size: JSON.stringify(data).length
}
}, null, 2));
}
async listBackups(): Promise<string[]> {
return await this.backupFs.readDir('/backups');
}
async restoreBackup(filename: string): Promise<any> {
const backup = await this.backupFs.readFile(`/backups/${filename}`);
return JSON.parse(backup);
}
}
class TenantConfigManager {
private fs: GitHubFileSystem;
constructor() {
this.fs = new GitHubFileSystem({
token: process.env.GITHUB_TOKEN!,
owner: 'saas-company',
repo: 'tenant-configs',
branch: 'main'
});
}
async setTenantConfig(tenantId: string, config: any): Promise<void> {
const path = `/tenants/${tenantId}/config.json`;
await this.fs.writeFile(path, JSON.stringify(config, null, 2));
}
async getTenantConfig(tenantId: string): Promise<any> {
const path = `/tenants/${tenantId}/config.json`;
if (await this.fs.exists(path)) {
const content = await this.fs.readFile(path);
return JSON.parse(content);
}
return this.getDefaultConfig();
}
async listTenants(): Promise<string[]> {
return await this.fs.readDir('/tenants');
}
private getDefaultConfig() {
return {
features: { advanced: false },
limits: { users: 10, storage: '1GB' },
theme: 'default'
};
}
}
class OptimizedGitHubFS {
private fs: GitHubFileSystem;
private localCache = new Map<string, { content: string; timestamp: number }>();
private cacheTTL = 5 * 60 * 1000; // 5 minutes
constructor(options: GitHubFileSystemOptions) {
this.fs = new GitHubFileSystem(options);
}
async readFile(path: string): Promise<string> {
// Check local cache first
const cached = this.localCache.get(path);
const now = Date.now();
if (cached && (now - cached.timestamp) < this.cacheTTL) {
return cached.content;
}
// Fetch from GitHub
const content = await this.fs.readFile(path);
// Update cache
this.localCache.set(path, { content, timestamp: now });
return content;
}
async writeFile(path: string, content: string): Promise<void> {
await this.fs.writeFile(path, content);
// Update cache
this.localCache.set(path, { content, timestamp: Date.now() });
}
clearLocalCache(): void {
this.localCache.clear();
}
}
class BatchGitHubOperations {
private fs: GitHubFileSystem;
constructor(options: GitHubFileSystemOptions) {
this.fs = new GitHubFileSystem(options);
}
async batchWriteFiles(files: Array<{ path: string; content: string }>): Promise<void> {
// Process in batches to avoid rate limiting
const batchSize = 5;
const batches = [];
for (let i = 0; i < files.length; i += batchSize) {
batches.push(files.slice(i, i + batchSize));
}
for (const batch of batches) {
await Promise.all(
batch.map(file => this.fs.writeFile(file.path, file.content))
);
// Small delay between batches to respect rate limits
await new Promise(resolve => setTimeout(resolve, 1000));
}
}
async batchReadFiles(paths: string[]): Promise<Map<string, string>> {
const results = new Map<string, string>();
const readPromises = paths.map(async (path) => {
try {
const content = await this.fs.readFile(path);
results.set(path, content);
} catch (error) {
console.warn(`Failed to read ${path}:`, error);
}
});
await Promise.all(readPromises);
return results;
}
}
import { GitHubFileSystem } from '@synet/fs-github';
class ResilientGitHubFS {
private fs: GitHubFileSystem;
private maxRetries = 3;
private retryDelay = 1000;
constructor(options: GitHubFileSystemOptions) {
this.fs = new GitHubFileSystem(options);
}
async writeFileWithRetry(path: string, content: string): Promise<void> {
let lastError: Error;
for (let attempt = 1; attempt <= this.maxRetries; attempt++) {
try {
await this.fs.writeFile(path, content);
return; // Success
} catch (error: any) {
lastError = error;
// Don't retry authentication errors
if (error.status === 401 || error.status === 403) {
throw error;
}
// Don't retry if it's the last attempt
if (attempt === this.maxRetries) {
break;
}
// Wait before retrying
await this.delay(this.retryDelay * attempt);
}
}
throw new Error(`Failed to write file after ${this.maxRetries} attempts: ${lastError.message}`);
}
async readFileWithFallback(path: string, fallbackContent?: string): Promise<string> {
try {
return await this.fs.readFile(path);
} catch (error: any) {
if (error.status === 404 && fallbackContent !== undefined) {
return fallbackContent;
}
throw error;
}
}
private delay(ms: number): Promise<void> {
return new Promise(resolve => setTimeout(resolve, ms));
}
}
// Error handling patterns
try {
await githubFs.writeFile('/config.json', JSON.stringify(config));
} catch (error: any) {
if (error.status === 401) {
console.error('Authentication failed - check your token');
} else if (error.status === 403) {
console.error('Permission denied - check token permissions');
} else if (error.status === 404) {
console.error('Repository not found - check owner/repo');
} else if (error.status === 422) {
console.error('Validation failed - check file content');
} else {
console.error('Unexpected error:', error.message);
}
}
class RateLimitedGitHubFS {
private fs: GitHubFileSystem;
private requestQueue: Array<() => Promise<any>> = [];
private processing = false;
private requestsPerMinute = 60; // GitHub rate limit
private requestInterval = 60000 / this.requestsPerMinute;
constructor(options: GitHubFileSystemOptions) {
this.fs = new GitHubFileSystem(options);
}
async queueRequest<T>(operation: () => Promise<T>): Promise<T> {
return new Promise((resolve, reject) => {
this.requestQueue.push(async () => {
try {
const result = await operation();
resolve(result);
} catch (error) {
reject(error);
}
});
this.processQueue();
});
}
private async processQueue(): Promise<void> {
if (this.processing || this.requestQueue.length === 0) {
return;
}
this.processing = true;
while (this.requestQueue.length > 0) {
const operation = this.requestQueue.shift()!;
await operation();
// Wait between requests to respect rate limits
if (this.requestQueue.length > 0) {
await this.delay(this.requestInterval);
}
}
this.processing = false;
}
private delay(ms: number): Promise<void> {
return new Promise(resolve => setTimeout(resolve, ms));
}
}
// ✅ Good: Use environment variables
const githubFs = new GitHubFileSystem({
token: process.env.GITHUB_TOKEN!,
owner: 'company',
repo: 'secure-config'
});
// ❌ Avoid: Hardcoding tokens
const insecureFs = new GitHubFileSystem({
token: 'ghp_hardcoded_token_here', // Never do this!
owner: 'company',
repo: 'config'
});
// ✅ Good: Validate environment variables
function validateEnvironment(): void {
const required = ['GITHUB_TOKEN', 'GITHUB_OWNER', 'GITHUB_REPO'];
const missing = required.filter(key => !process.env[key]);
if (missing.length > 0) {
throw new Error(`Missing required environment variables: ${missing.join(', ')}`);
}
}
// ✅ Good: Use scoped tokens
// Create tokens with minimal required permissions
// Separate tokens for different environments
class SecureConfigManager {
private fs: GitHubFileSystem;
private allowedPaths: string[] = ['/config/', '/settings/', '/features/'];
constructor() {
validateEnvironment();
this.fs = new GitHubFileSystem({
token: process.env.GITHUB_TOKEN!,
owner: process.env.GITHUB_OWNER!,
repo: process.env.GITHUB_REPO!
});
}
async writeFile(path: string, content: string): Promise<void> {
this.validatePath(path);
this.validateContent(content);
await this.fs.writeFile(path, content);
}
async readFile(path: string): Promise<string> {
this.validatePath(path);
return await this.fs.readFile(path);
}
private validatePath(path: string): void {
const isAllowed = this.allowedPaths.some(allowed => path.startsWith(allowed));
if (!isAllowed) {
throw new Error(`Access denied: Path '${path}' is not allowed`);
}
}
private validateContent(content: string): void {
try {
JSON.parse(content);
} catch {
throw new Error('Invalid JSON content');
}
if (content.includes('password') || content.includes('secret')) {
throw new Error('Sensitive data detected in content');
}
}
}
npm run build
npm run lint
npm run lint:fix
npm run format
npm run demo
| Error | Cause | Solution |
|---|---|---|
401 Unauthorized | Invalid token | Check token validity and regenerate if needed |
403 Forbidden | Insufficient permissions | Ensure token has repo scope |
404 Not Found | Repository doesn't exist | Create repository or check owner/repo names |
422 Unprocessable Entity | Invalid file content | Check file content and encoding |
409 Conflict | Concurrent modifications | Implement retry logic with exponential backoff |
// Enable debug logging
process.env.DEBUG = 'github-fs:*';
const githubFs = new GitHubFileSystem({
token: process.env.GITHUB_TOKEN!,
owner: 'test-owner',
repo: 'test-repo'
});
// Operations will now log detailed information
await githubFs.writeFile('/debug.txt', 'test content');
async function healthCheck(): Promise<boolean> {
try {
const githubFs = new GitHubFileSystem({
token: process.env.GITHUB_TOKEN!,
owner: process.env.GITHUB_OWNER!,
repo: process.env.GITHUB_REPO!
});
// Test basic operations
const testPath = '/.health-check';
const testContent = JSON.stringify({ timestamp: Date.now() });
await githubFs.writeFile(testPath, testContent);
const retrieved = await githubFs.readFile(testPath);
await githubFs.deleteFile(testPath);
return retrieved === testContent;
} catch (error) {
console.error('Health check failed:', error);
return false;
}
}
MIT License - see LICENSE file for details.
Contributions are welcome! Please read our Contributing Guide for details on our code of conduct and the process for submitting pull requests.
git clone https://github.com/synthetism/fs-github.git
cd fs-github
npm install
# Set up test environment
cp .env.example .env
# Edit .env with your test GitHub token and repository
npm test
Built with ❤️ by the Synet Team
FAQs
Github as filesystem abstraction following FileSystem pattern
We found that @synet/fs-github demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
Five malicious NuGet packages impersonate Chinese .NET libraries to deploy a stealer targeting browser credentials, crypto wallets, SSH keys, and local files.
Security News
pnpm 11 turns on a 1-day Minimum Release Age and blocks exotic subdeps by default, adding safeguards against fast-moving supply chain attacks.
Security News
The remediated findings include organization permission bugs, stale project access after transfers, OIDC replay edge cases, audit logging gaps, and an IDOR in API token deletion.