@amazon-codecatalyst/blueprints.gen-ai-chatbot

This Blueprint

This blueprint generates a sample chatbot using the Anthropic company's LLM Claude 2, one of the foundational models provided by Amazon Bedrock for generative AI.

Basic Conversation

Bot Personalization

Add your own instruction and give external knowledge as URL or files (a.k.a RAG). The bot can be shared among application users.

📚 Supported Languages

• English 💬
• 日本語 💬 (ドキュメントはこちら)
• 한국어 💬
• 中文 💬

🚀 Super-easy Deployment

• On us-east-1, open Bedrock Model access > Manage model access > Check Anthropic / Claude, Anthropic / Claude Instant and Cohere / Embed Multilingual then Save changes.

Screenshot

Architecture

It's an architecture built on AWS managed services, eliminating the need for infrastructure management. Utilizing Amazon Bedrock, there's no need to communicate with APIs outside of AWS. This enables deploying scalable, reliable, and secure applications.

• Amazon DynamoDB: NoSQL database for conversation history storage
• Amazon API Gateway + AWS Lambda: Backend API endpoint (AWS Lambda Web Adapter, FastAPI)
• Amazon SNS: Used to decouple streaming calls between API Gateway and Bedrock because streaming responses can take over 30 seconds in total, exceeding the limitations of HTTP integration (See quota).
• Amazon CloudFront + S3: Frontend application delivery (React, Tailwind CSS)
• AWS WAF: IP address restriction
• Amazon Cognito: User authentication
• Amazon Bedrock: Managed service to utilize foundational models via APIs. Claude is used for chat response and Cohere for vector embedding
• Amazon EventBridge Pipes: Receiving event from DynamoDB stream and launching ECS task to embed external knowledge
• Amazon Elastic Container Service: Run crawling, parsing and embedding tasks. Cohere Multilingual is the model used for embedding.
• Amazon Aurora PostgreSQL: Scalable vector store with pgvector plugin

Connections and permissions

This blueprint requires a custom development role. To create one, click "Add an existing IAM role" from the add IAM role options. The IAM role needs to contain the CodeCatalyst trust policy, as well as the following permissions:

{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Effect": "Allow",
          "Action": [
            "iam:DeleteRole",
            "iam:GetRole",
            "iam:TagRole",
            "iam:CreateRole",
            "iam:AttachRolePolicy",
            "iam:DetachRolePolicy",
            "iam:DeleteRolePolicy",
            "cloudformation:*",
            "lambda:*",
            "apigateway:*",
            "ecr:*",
            "ssm:PutParameter",
            "ssm:DeleteParameter",
            "iam:PutRolePolicy",
            "s3:*",
            "ssm:GetParameter",
            "ssm:GetParameters"
          ],
          "Resource": "*"
      },
      {
          "Effect": "Allow",
          "Action": [
              "sts:AssumeRole"
          ],
          "Resource": [
              "arn:aws:iam::*:role/cdk-*"
          ]
      }
  ]
}

The IAM roles also require the Amazon CodeCatalyst service principals codecatalyst.amazonaws.com and codecatalyst-runner.amazonaws.com.

Required IAM role trust policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "codecatalyst.amazonaws.com",
                    "codecatalyst-runner.amazonaws.com"
                ]
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

Features and Roadmap

Basic chat features

• Authentication (Sign-up, Sign-in)
• Creation, storage, and deletion of conversations
• Copying of chatbot replies
• Automatic subject suggestion for conversations
• Syntax highlighting for code
• Rendering of Markdown
• Streaming Response
• IP address restriction
• Edit message & re-send
• I18n
• Model switch (Claude Instant / Claude)

Customized bot features

• Customized bot creation
• Customized bot sharing

RAG features

• Web (html)
• Text data (txt, csv, markdown and etc)
• PDF
• Microsoft office files (pptx, docx, xlsx)
• Youtube transcript

Admin features

• Admin console to analyze user usage