uc3-sam-sceptre

UC3 SAM Sceptre gem

This gem provides functionality that can be used to help Sceptre trigger a SAM build and/or SAM deploy to create an image and push it to an ECR repository or build the ZIP archive and put it into an S3 bucket and then run the CloudFormation create/update/delete.

You can run the script manually and via a Sceptre hook when you are initialiiy creating your stacks.

Prerequisites

This gem assumes that you have organized your Sceptre config by environment! For example:

my_project
   |
    ------ config
   |         |
   |          ---- dev
   |         |      |
   |         |       ----- s3.yaml
   |         |
   |          ---- prd
   |                |
   |                 ----- s3.yaml
   |
    ------ templates
              |
               ------ s3.yaml

This gem assumes that you have access to run CloudFormation tasks in 2 AWS accounts: dev and prd

If you specify env: prd or env: stg when initializing the Proxy, your resources will be constucted in the prd AWS account otherwise they will be created in the dev account.

Since SAM is not directly connected to Sceptre and cannot therefore access any of Sceptre's resolvers, you will need to ensure access to any SAM template parameters defined in your template.

This gem will retrieve values from either the SSM parameter store or exported CloudFormation stack outputs. The SSM parameters and stack outputs MUST live within the same region that you are running your SAM deploy!

For example, given the following SAM template:

AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'

Parameters:
  VpcId:
    Type: 'AWS::EC2::VPC::Id'

You would need to make the VpcId available as an SSM parameter or as an exported output from one of your other Sceptre managed CloudFormation stacks like this:

Outputs:
  VpcId:
    Value: !Ref Vpc
    Export:
      Name: VpcId

You should place your SAM template and the Lambda code under the Sceptre project's templates directory. For example:

my_project
    |
     ------ config
    |          |
    |           ----- dev
    |                   |
    |                    ----- api-gateway.yaml                     # Your Sceptre config for the API Gateway that
    |                                                               # includes a hook to run the SAM build + deploy
    |
     ------ templates
               |
                ------- api-gateway.yaml                            # Your SAM template that builds the API Gateway
               |
               |
                ------- lambdas
                         |
                          ----- src
                                 |
                                  ------ my_lambda
                                 |          |
                                 |           ------ file(s)         # Your Lambda source
                                 |
                                  ------- Gemfile                   # Gemfile that includes this gem
                                 |
                                  ------- sam_build_deploy.rb       # The Ruby script that initializes and uses this gem
                                 |
                                  ------- template.yaml             # Your SAM template for the Lambda(s)

Usage

This gem is intended to be invoked from a Ruby script that you create and store in the same directory as your SAM template. The script can be run manually at any time and via a Sceptre hook.

Since CloudFormation requires your Lambda code to exist in an S3 bucket or an ECR when you create your stack, we advise that you at least add this script as a hook to an appropriate Sceptre config. For example as an after_create hook on the Sceptre config for the S3 bucket or ECR that will be used to store your Lambda code.

Build a Ruby script

Create a Gemfile that is adjacent to your SAM template (see the location of the Gemfile file in the directory structure outlined above). This file should look like this:

# frozen_string_literal: true

source 'https://rubygems.org'

gem 'uc3-sam-sceptre'

Create a Ruby script that is adjacent to your SAM template (see the location of the sam_build_deploy.rb file in the directory structure outlined above). (See the exmaples below)

The following arguments can be used in your Ruby script when initializing the SAM proxy:

# REQUIRED arguments:
# ------------------------
service: 'dmp'                                # REQUIRED - The service name
subservice: 'hub'                             # REQUIRED - The subservice name
git_repo: 'https://github.com/MYORG/my-repo'  # REQUIRED - The location of your SAM code
stack_suffix: 'docker-lambdas'                # REQUIRED - Used to generate the CloudFormation stack name
admin_email_key: 'AdminEmail'                 # REQUIRED - An SSM parameter name or a CF stack export name

# Include one of the following:
ecr_uri_key: 'MyEcrRepoUri                    # REQUIRED if Lambda is a Docker Image - An SSM parameter name or a CF stack export name
s3_arn_key: 'S3PrivateBucketArn'              # REQUIRED if Lambda is NOT a Docker Image - An SSM parameter name or a CF stack export name

# An entry for each of your SAM template parameters.
#   - Fetchable will be looked up in either the SSM paramter store or available CloudFormation stack exports
#   - Static will be used as-is
#
fetchable_cf_params: [
  {
    stack_export_name: 'CloudFrontDistroId'   # REQUIRED - An SSM parameter name or a CF stack export name
    template_param_name: 'CFDistributionId'   # (defaults to :stack_export_name) The name of the parameter your SAM template is looking for
  }
],
static_cf_params: [
  {
    template_param_name: 'Version'          # REQUIRED - The name of the parameter your SAM template is looking for
    value: 'v0'                             # REQUIRED - The value of the parameter
  }
]

# OPTIONAL arguments:
# ------------------------
program: 'uc3'                                # The group that owns the service (default: 'uc3')
env: 'stg'                                    # The environment (default: 'dev')
region: 'us-east-1'                           # The AWS region (default: 'us-west-2')
auto: false                                   # Whether or not to run the SAM deploy in `--guided` mode (default: true)

The :program, :service, :subservice, and :env are used to:

• Create the SAM CloudFormation stack name (along with the :stack_suffix) For example: uc3-dmp-hub-dev-foo-bar
• Help find the values of parameters in the :fetchable_cf_params by limiting the stacks to those that share your Sceptre project's naming conventions. For example: uc3-dmp-hub-dev
• Create resource tags along with the :git_repo and the value found for the :admin_email_key to tag all resources created by SAM with tags for Program, Service, Subservice, Environment, CodeRepo and Contact

The entries within the :fetchable_cf_params and :static_cf_params will all be passed to the SAM deploy as --parameter-overrides

This gem will automatically append the :program + :service + :subservice + :env to the names of the SSM parameter keys you provide (unless you provided the fully qulified key name). So for example if you specify /uc3/dmp/hub/dev/AdminEmail, it will be used as-is; if you provide AdminEmail then the gem will prepend /uc3/dmp/hub/dev to the name.

Build the bundle

You will need to run bundle install so that this gem is installed and ready

Add the Sceptre hook

Recommended but optional.

Add a hook to an existing Sceptre config that will run this script. Since CloudFormation requires your Lambda code to already exist in an S3 bucket or an ECR when you are creating your SAM stacks. For example here is a hook that will run a SAM build and deploy immediately after an S3 bucket is created:

# config/dev/s3.yaml
template:
  path: s3.yaml
  type: file

parameters:
  LogBucketObjectLifeSpan: '30'

hooks:
  after_create:
    # Build the Lambda code and store in the bucket so it is available downstream
    - !cmd './templates/lambdas/src/sam_build_deploy.rb true true'

Examples Ruby scripts:

The following is an example of deploying to a Lambda that is a Docker image into an ECR.

# frozen_string_literal: true

require 'uc3-sam-sceptre'

if ARGV.length == 2
  params = {
    service: 'dmp',
    subservice: 'hub',
    git_repo: 'https://github.com/CDLUC3/dmp-hub-cfn',
    stack_suffix: 'docker-lambdas',
    admin_email_key: 'AdminEmail',

    ecr_uri_key: 'EcrRepositoryUri',

    fetchable_cf_params: [
      { stack_export_name: 'VpcId' },
      { stack_export_name: 'SubnetA' },
      { stack_export_name: 'SubnetB' },
      { stack_export_name: 'SubnetC' },
      { stack_export_name: 'DeadLetterQueueArn' },
      { stack_export_name: 'LambdaSecGroupId' },
      { stack_export_name: 'SnsTopicEmailArn', template_param_name: 'SnsEmailTopicArn' },
      { stack_export_name: 'DomainName' },
      { stack_export_name: 'RdsHost' },
      { stack_export_name: 'RdsPort' },
      { stack_export_name: 'RdsDbName' }
    ],
    static_cf_params: [
      { template_param_name: 'Version', value: 'v0' }
    ]
  }
  proxy = Uc3SamSceptre::Proxy.new(params)

  if ARGV[0].to_s.downcase.strip == 'false' && ARGV[1].to_s.downcase.strip == 'false'
    proxy.delete
  else
    proxy.build if ARGV[0].to_s.downcase.strip == 'true'
    proxy.deploy if ARGV[1].to_s.downcase.strip == 'true'
  end
else
  p 'Expected 2 arguments: run build? and run deploy? (For example: `ruby sam_build_deploy.rb true true`).'
  p 'Setting both arguments to false will trigger a `sam delete`.'
end

The following is an example of deploying a Lambda that is NOT a Docker image into a ZIP archive stored in S3

# frozen_string_literal: true

require 'uc3-sam-sceptre'

if ARGV.length == 2
  params = {
    service: 'dmp',
    subservice: 'hub',
    git_repo: 'https://github.com/CDLUC3/dmp-hub-cfn',
    ecr_uri_key: 'EcrRepositoryUri',
    stack_suffix: 'docker-lambdas',
    admin_email_key: 'AdminEmail',

    s3_arn_key: 'S3PrivateBucketArn',

    fetchable_cf_params: [
      { stack_export_name: 'CloudFrontDistroId' },
      { stack_export_name: 'S3PublicBucketArn' },
      { stack_export_name: 'DynamoTableName' },
      { stack_export_name: 'SnsTopicEmailArn', template_param_name: 'SnsEmailTopicArn' },
      { stack_export_name: 'DomainName' }
    ],
    static_cf_params: [
      { template_param_name: 'Version', value: 'v0' }
    ]
  }

  proxy = Uc3SamSceptre::Proxy.new(params)

  if ARGV[0].to_s.downcase.strip == 'false' && ARGV[1].to_s.downcase.strip == 'false'
    proxy.delete
  else
    proxy.build if ARGV[0].to_s.downcase.strip == 'true'
    proxy.deploy if ARGV[1].to_s.downcase.strip == 'true'
  end
else
  p 'Expected 2 arguments: run build? and run deploy? (For example: `ruby sam_build_deploy.rb true true`)'
  p 'Setting both arguments to false will trigger a `sam delete`.'
end