@dataformer/gcp-auth

@dataformer/gcp-auth

GCP Authentication Service for Dataformer - provides unified authentication for all Google Cloud Platform services across local development and deployed environments.

Overview

This service automatically detects the environment and uses the appropriate authentication method, eliminating the need for manual authentication configuration in each Google Cloud client.

Authentication Methods

Local Development

• Method: Application Default Credentials (ADC) from gcloud CLI
• Setup Required:

  # Install Google Cloud SDK
  # https://cloud.google.com/sdk/docs/install-sdk
  
  # Initialize gcloud (if not already done)
  gcloud init
  
  # Set up application default credentials
  gcloud auth application-default login
  
  # Verify configuration
  gcloud config list
  gcloud auth list
  

Deployed Environments (Cloud Run, Cloud Functions, etc.)

• Method: Service account attached to the compute resource
• Setup Required:
  • Service account with appropriate IAM permissions
  • Service account attached to the deployment resource
  • No additional configuration needed in code

Manual Service Account (Optional)

• Method: Service account key file
• Setup Required:

  # Set environment variable pointing to service account key JSON
  export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account-key.json"
  

Usage

Basic Usage

import { GcpAuthService } from '@dataformer/gcp-auth';

// Create auth service with default configuration
const auth = await GcpAuthService.create();

// Get configuration for Google Cloud client libraries
const config = await auth.getClientConfig();

With Custom Configuration

import { GcpAuthService } from '@dataformer/gcp-auth';

const auth = await GcpAuthService.create({
  projectId: 'my-custom-project',
  scopes: ['https://www.googleapis.com/auth/bigquery'],
  keyFilename: '/path/to/custom-service-account.json'
});

Validation and Debugging

import { GcpAuthService } from '@dataformer/gcp-auth';

// This will validate auth and log details  
const auth = await GcpAuthService.create();

// Get detailed auth information
const authInfo = await auth.getAuthInfo();
console.log('Project ID:', authInfo.projectId);
console.log('Auth Type:', authInfo.authType);
console.log('Service Account:', authInfo.serviceAccountEmail);

Integration with Dataformer Clients

All @dataformer/*-client packages that interact with Google Cloud services now use this authentication service automatically:

• @dataformer/bigquery-client
• @dataformer/cloud-storage-client
• @dataformer/firestore-client
• @dataformer/secret-manager-client
• @dataformer/pubsub-client
• @dataformer/bigtable-client

Example: Using BigQuery Client

import { BigQueryClient } from '@dataformer/bigquery-client';

// Client automatically uses proper authentication
const client = await BigQueryClient.create({
    projectId: 'my-project',
    datasetId: 'my-dataset'
});
const results = await client.queryRows('SELECT * FROM my_table LIMIT 10');

Environment Variables

The service respects these environment variables:

• GOOGLE_CLOUD_PROJECT - GCP project ID
• GCLOUD_PROJECT - Alternative GCP project ID
• GOOGLE_APPLICATION_CREDENTIALS - Path to service account key JSON

IAM Permissions

Ensure your service accounts have the appropriate permissions for the services you're using:

Common Roles

• roles/bigquery.user - For BigQuery operations
• roles/storage.admin - For Cloud Storage operations
• roles/secretmanager.admin - For Secret Manager operations
• roles/pubsub.editor - For Pub/Sub operations
• roles/datastore.user - For Firestore operations
• roles/bigtable.user - For Bigtable operations

Development Permissions

For local development, your gcloud user account needs:

• roles/resourcemanager.projectViewer - To list and access projects
• Plus any service-specific roles listed above

Troubleshooting

Common Issues

• "Could not determine GCP project ID"

  • Ensure gcloud config set project YOUR_PROJECT_ID
  • Or set GOOGLE_CLOUD_PROJECT environment variable

• "Failed to get GCP credentials"

  • Run gcloud auth application-default login
  • Or verify service account key file exists and is readable

• Permission Denied Errors

  • Check IAM permissions for your user/service account
  • Verify correct project is selected

Debug Authentication

import { GcpAuthService } from '@dataformer/gcp-auth';

const auth = await GcpAuthService.create();
try {
  await auth.validateAuth();
  console.log('✅ Authentication working');
} catch (error) {
  console.error('❌ Authentication failed:', error.message);
}

API Reference

GcpAuthService

Static Factory Method

GcpAuthService.create(config?: GcpAuthConfig)

Methods

• getCredentials() - Get auth client for Google Cloud libraries
• getProjectId() - Get current GCP project ID
• getClientConfig() - Get config object for Google Cloud clients
• getAuthInfo() - Get detailed authentication information
• validateAuth() - Validate and log authentication status

Types

• GcpAuthConfig - Configuration options
• GcpAuthInfo - Authentication status information