aws-workflow

aws-workflow

⚠️ EXPERIMENTAL & BETA: This package is in active development and should be used with caution in production.

AWS World implementation for Workflow DevKit - Run durable, resumable workflows on AWS Lambda with DynamoDB, SQS, and S3.

What is Workflow DevKit?

Workflow DevKit brings durability, reliability, and observability to async JavaScript. Build workflows and AI Agents that can suspend, resume, and maintain state with ease - all with simple TypeScript functions.

aws-workflow is a World implementation that runs your workflows on AWS infrastructure, providing:

• ✅ Serverless execution on AWS Lambda
• ✅ State persistence with DynamoDB
• ✅ Message queuing with SQS
• ✅ Large payload storage with S3
• ✅ Automatic retries and error handling
• ✅ No vendor lock-in - same code runs locally or on any cloud

Quick Start

Prerequisites

• Node.js 18+
• AWS CLI configured with credentials
• A Next.js 14+ application

1. Install

npm install aws-workflow workflow

2. Bootstrap AWS Resources

This creates the required AWS infrastructure (DynamoDB tables, SQS queues, S3 bucket, Lambda function):

npx aws-workflow bootstrap -y

What this does:

• Creates 5 DynamoDB tables (workflow runs, steps, events, hooks, stream chunks)
• Creates 2 SQS queues (workflow queue, step queue)
• Creates 1 S3 bucket for large payload storage
• Deploys Lambda worker function
• Outputs environment variables to .env.aws

Cost estimate: Free tier eligible. Typical cost: $5-20/month for moderate usage.

3. Configure Your Next.js App

Copy the generated environment variables from .env.aws to your Next.js .env.local:

# From bootstrap output
WORKFLOW_QUEUE_URL=https://sqs.us-east-1.amazonaws.com/...
WORKFLOW_STEP_QUEUE_URL=https://sqs.us-east-1.amazonaws.com/...
WORKFLOW_RUNS_TABLE=workflow_runs
WORKFLOW_STEPS_TABLE=workflow_steps
WORKFLOW_EVENTS_TABLE=workflow_events
WORKFLOW_HOOKS_TABLE=workflow_hooks
WORKFLOW_STREAM_CHUNKS_TABLE=workflow_stream_chunks
WORKFLOW_STREAM_BUCKET=workflow-streams-...

# Add your AWS credentials
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=your-access-key
AWS_SECRET_ACCESS_KEY=your-secret-key

Add to your next.config.ts:

import { withWorkflow } from 'workflow/next';

export default withWorkflow({
  experimental: {
    serverActions: {
      bodySizeLimit: '10mb',
    },
  },
});

4. Write Your First Workflow

Create workflows/user-signup.ts:

import { sleep } from 'workflow';

export async function handleUserSignup(email: string) {
  'use workflow';

  // Step 1: Create user
  const user = await createUser(email);
  
  // Step 2: Send welcome email
  await sendWelcomeEmail(email);
  
  // Step 3: Wait 7 days (workflow suspends - no resources consumed!)
  await sleep('7 days');
  
  // Step 4: Send follow-up
  await sendFollowUpEmail(email);
  
  return { userId: user.id, status: 'completed' };
}

async function createUser(email: string) {
  'use step';
  // Your user creation logic
  return { id: '123', email };
}

async function sendWelcomeEmail(email: string) {
  'use step';
  // Send email via Resend, SendGrid, etc.
}

async function sendFollowUpEmail(email: string) {
  'use step';
  // Send follow-up email
}

5. Deploy to AWS Lambda

Whenever you add or update workflows, deploy them:

npm run deploy

What this does:

• Compiles your TypeScript workflows
• Builds Next.js to generate workflow bundles
• Packages Lambda handler with your workflows
• Deploys to AWS Lambda (no Docker required!)

6. Trigger Your Workflow

From your Next.js API route or Server Action:

import { handleUserSignup } from '@/workflows/user-signup';

export async function POST(request: Request) {
  const { email } = await request.json();
  
  // Start the workflow
  const handle = await handleUserSignup(email);
  
  return Response.json({
    workflowId: handle.id,
    status: 'started'
  });
}

That's it! Your workflow is now running on AWS Lambda. 🚀

Architecture

┌─────────────────┐
│   Next.js App   │
│  (Your Code)    │
└────────┬────────┘
         │
         │ Triggers workflow
         ▼
┌─────────────────┐      ┌──────────────┐
│   SQS Queues    │─────▶│ Lambda Worker│
│ (Orchestration) │      │ (Executes)   │
└─────────────────┘      └──────┬───────┘
                                │
                    ┌───────────┴───────────┐
                    │                       │
         ┌──────────▼────────┐   ┌─────────▼────────┐
         │   DynamoDB        │   │   S3 Bucket      │
         │ (State & Runs)    │   │ (Large Payloads) │
         └───────────────────┘   └──────────────────┘

Key Features

🔄 Automatic Retries

Steps automatically retry on failure with exponential backoff.

💾 State Persistence

Workflow state is persisted to DynamoDB - resume from any point.

⏸️ Sleep & Wait

Use sleep() to pause workflows for minutes, hours, or days without consuming resources.

📊 Observability

Query workflow status, inspect step execution, view history:

import { getWorkflowRun } from 'aws-workflow';

const run = await getWorkflowRun(workflowId);
console.log(run.status); // 'running' | 'completed' | 'failed'

🔀 Parallel Execution

Run multiple steps concurrently:

export async function processOrder(orderId: string) {
  'use workflow';
  
  const [payment, inventory, shipping] = await Promise.all([
    processPayment(orderId),
    reserveInventory(orderId),
    calculateShipping(orderId),
  ]);
  
  return { payment, inventory, shipping };
}

Commands

# Bootstrap AWS infrastructure (first time only)
npm run bootstrap

# Deploy workflows to Lambda
npm run deploy

# View Lambda logs in real-time
npm run logs

# Tear down all AWS resources
npm run teardown

# Get current AWS resource info
npm run outputs

Configuration

Environment Variables

Variable	Description	Required
WORKFLOW_QUEUE_URL	SQS queue URL for workflow orchestration	✅
WORKFLOW_STEP_QUEUE_URL	SQS queue URL for step execution	✅
WORKFLOW_RUNS_TABLE	DynamoDB table for workflow runs	✅
WORKFLOW_STEPS_TABLE	DynamoDB table for step execution	✅
WORKFLOW_STREAM_BUCKET	S3 bucket for large payloads	✅
AWS_REGION	AWS region	✅
AWS_ACCESS_KEY_ID	AWS access key (local dev)	✅*
AWS_SECRET_ACCESS_KEY	AWS secret key (local dev)	✅*

*Not required when running on AWS (uses IAM roles)

Cost Optimization

• Lambda executions are short-lived (typically <500ms per step)
• DynamoDB uses on-demand pricing (no upfront cost)
• SQS has free tier of 1M requests/month
• S3 is only used for payloads >256KB

Typical monthly cost for moderate usage: $5-20

Limitations & Caveats

⚠️ Beta Software Warnings:

• API Stability: APIs may change between versions
• Production Use: Test thoroughly before production deployment
• Error Handling: Custom error handling may be needed for edge cases
• Concurrent Execution: Lambda concurrency limits apply (default: 1000)
• DynamoDB Limits: Write capacity may need adjustment for high throughput
• Payload Size: SQS messages limited to 256KB (larger payloads use S3)

Known Issues

Multi-Step Workflow Resume Bug

Status: 🔴 Critical runtime issue in @workflow/core (not AWS-specific)

Symptoms:

• First step in a workflow executes successfully
• Subsequent steps fail with Runtime.NodeJsExit error when workflow attempts to resume
• Workflow status remains "running" indefinitely

Root Cause: The workflow runtime's replay mechanism has a bug that causes promise rejections when resuming execution after a step completes. This is a core runtime issue affecting all environments, not specific to AWS infrastructure.

Workaround: Use single-step workflows for production:

// ✅ Works - Single step
export async function processPayment(orderId: string) {
  'use workflow';

  const result = await handlePayment(orderId);
  return result;
}

// ❌ Broken - Multiple steps
export async function processOrder(orderId: string) {
  'use workflow';

  const payment = await processPayment(orderId);  // ✅ First step works
  const shipping = await calculateShipping(orderId); // ❌ Crashes on resume
}

For complex workflows, chain single-step workflows using AWS Step Functions or your own orchestration.

Troubleshooting

Deployment fails with "Cannot find asset"

# Clean and rebuild
rm -rf cdk.out .next node_modules/.cache
npm run deploy

Workflows not executing

# Check Lambda logs
npm run logs

# Verify environment variables
npm run outputs

"Module not found" errors

Ensure your Next.js app uses npm (not pnpm) for flat node_modules structure:

rm -rf node_modules pnpm-lock.yaml
npm install

Examples

Check out the example Next.js app for a complete implementation including:

• User signup workflow with email sequence
• Multi-step ordering process
• Error handling and retries
• Webhook integrations

Documentation

• Workflow DevKit Docs
• API Reference
• Architecture Guide
• Migration Guide

Contributing

Contributions are welcome! Please read our Contributing Guide first.

License

Apache-2.0 - see LICENSE.md

Built with ❤️ by Langtrace

GitHub: https://github.com/karthikscale3/aws-workflow

Part of the Workflow DevKit ecosystem