@cumulus/deployment

@cumulus/deployment

@cumulus/deployment includes cloudformation templates needed for a successful deployment of a Cumulus Instance. The templates can be used with kes, a node CLI helper for AWS CloudFormation.

About Cumulus

Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.

Cumulus Documentation

Usage

See the Cumulus deployment documentation for a detailed overview of how to deploy Cumulus.

Contributing

To make a contribution, please see our contributing guidelines.

Changelog

[v1.21.0] 2020-03-30

PLEASE NOTE

• CUMULUS-1762: the messageConsumer for sns and kinesis-type rules now fetches the collection information from the message. You should ensure that your rule's collection name and version match what is in the message for these ingest messages to be processed. If no matching rule is found, an error will be thrown and logged in the messageConsumer Lambda function's log group.

Added

• CUMULUS-1629`

  • Updates discover-granules task to respect/utilize duplicateHandling configuration such that
    • skip: Duplicates will be filtered from the granule list
    • error: Duplicates encountered will result in step failure
    • replace, version: Duplicates will be ignored and handled as normal.
  • Adds a new copy of the API lambda PrivateApiLambda() which is configured to not require authentication. This Lambda is not connected to an API gateway
  • Adds @cumulus/api-client with functions for use by workflow lambdas to call the API when needed

• CUMULUS-1732

  • Added Python task/activity workflow and integration test (PythonReferenceSpec) to test cumulus-message-adapter-pythonand cumulus-process-py integration.

• CUMULUS-1795

  • Added an IAM policy on the Cumulus EC2 creation to enable SSM when the deploy_to_ngap flag is true

Changed

• CUMULUS-1762
  • the messageConsumer for sns and kinesis-type rules now fetches the collection information from the message.

Deprecated

• CUMULUS-1629
  • Deprecate granulesApi, rulesApi, emsApi, executionsAPI from @cumulus/integration-test/api in favor of code moved to @cumulus/api-client

Removed

• CUMULUS-1799: Deprecated code removals
  • Removed deprecated method @cumulus/api/models/Granule.createGranulesFromSns()
  • Removed deprecated method @cumulus/api/models/Granule.removeGranuleFromCmr()
  • Removed from @cumulus/common/aws:
    • apigateway()
    • buildS3Uri()
    • calculateS3ObjectChecksum()
    • cf()
    • cloudwatch()
    • cloudwatchevents()
    • cloudwatchlogs()
    • createAndWaitForDynamoDbTable()
    • createQueue()
    • deleteSQSMessage()
    • describeCfStackResources()
    • downloadS3File()
    • downloadS3Files()
    • DynamoDbSearchQueue class
    • dynamodbstreams()
    • ec2()
    • ecs()
    • fileExists()
    • findResourceArn()
    • fromSfnExecutionName()
    • getFileBucketAndKey()
    • getJsonS3Object()
    • getQueueUrl()
    • getObjectSize()
    • getS3ObjectReadStream()
    • getSecretString()
    • getStateMachineArn()
    • headObject()
    • isThrottlingException()
    • kinesis()
    • lambda()
    • listS3Objects()
    • promiseS3Upload()
    • publishSnsMessage()
    • putJsonS3Object()
    • receiveSQSMessages()
    • s3CopyObject()
    • s3GetObjectTagging()
    • s3Join()
    • S3ListObjectsV2Queue class
    • s3TagSetToQueryString()
    • s3PutObjectTagging()
    • secretsManager()
    • sendSQSMessage()
    • sfn()
    • sns()
    • sqs()
    • sqsQueueExists()
    • toSfnExecutionName()
    • uploadS3FileStream()
    • uploadS3Files()
    • validateS3ObjectChecksum()
  • Removed @cumulus/common/CloudFormationGateway class
  • Removed @cumulus/common/concurrency/Mutex class
  • Removed @cumulus/common/errors
  • Removed @cumulus/common/sftp
  • Removed @cumulus/common/string.unicodeEscape
  • Removed @cumulus/cmrjs/cmr-utils.getGranuleId()
  • Removed @cumulus/cmrjs/cmr-utils.getCmrFiles()
  • Removed @cumulus/cmrjs/cmr/CMR class
  • Removed @cumulus/cmrjs/cmr/CMRSearchConceptQueue class
  • Removed @cumulus/cmrjs/utils.getHost()
  • Removed @cumulus/cmrjs/utils.getIp()
  • Removed @cumulus/cmrjs/utils.hostId()
  • Removed @cumulus/cmrjs/utils/ummVersion()
  • Removed @cumulus/cmrjs/utils.updateToken()
  • Removed @cumulus/cmrjs/utils.validateUMMG()
  • Removed @cumulus/ingest/aws.getEndpoint()
  • Removed @cumulus/ingest/aws.getExecutionUrl()
  • Removed @cumulus/ingest/aws/invoke()
  • Removed @cumulus/ingest/aws/CloudWatch class
  • Removed @cumulus/ingest/aws/ECS class
  • Removed @cumulus/ingest/aws/Events class
  • Removed @cumulus/ingest/aws/SQS class
  • Removed @cumulus/ingest/aws/StepFunction class
  • Removed @cumulus/ingest/util.normalizeProviderPath()
  • Removed @cumulus/integration-tests/index.listCollections()
  • Removed @cumulus/integration-tests/index.listProviders()
  • Removed @cumulus/integration-tests/index.rulesList()
  • Removed @cumulus/integration-tests/api/api.addCollectionApi()