@cumulus/common/aws - Utilities for working with AWS. For ease of setup, testing, and credential management, code should obtain AWS client objects from helpers in this module.
@cumulus/common/concurrency - Implementations of distributed concurrency primitives (mutex, semaphore) using DynamoDB
@cumulus/common/errors - Classes for thrown errors
@cumulus/common/local-helpers - Provides methods for setting up message payloads for use in development / local testing
@cumulus/common/log - muting or potentially shipping logs to alternative locations
@cumulus/common/string - Utilities for manipulating strings
@cumulus/common/test-utils - Utilities for writing tests
@cumulus/common/URLUtils - a collection of utilities for working with URLs
@cumulus/common/util - Other misc general utilities

Contributing

To make a contribution, please see our contributing guidelines.

[v1.18.0] 2020-02-03

BREAKING CHANGES

CUMULUS-1686
- ecs_cluster_instance_image_id is now a required variable of the cumulus module, instead of optional.
CUMULUS-1698
- Change variable saml_launchpad_metadata_path to saml_launchpad_metadata_url in the tf-modules/cumulus Terraform module.
CUMULUS-1703
- Remove the unused forceDownload option from the sync-granule tasks's config
- Remove the @cumulus/ingest/granule.Discover class
- Remove the @cumulus/ingest/granule.Granule class
- Remove the @cumulus/ingest/pdr.Discover class
- Remove the @cumulus/ingest/pdr.Granule class
- Remove the @cumulus/ingest/parse-pdr.parsePdr function

Added

CUMULUS-1040
- Added @cumulus/aws-client package to provide utilities for working with AWS services and the Node.js AWS SDK
- Added @cumulus/errors package which exports error classes for use in Cumulus workflow code
- Added @cumulus/integration-tests/sfnStep to provide utilities for parsing step function execution histories
CUMULUS-1102
- Adds functionality to the @cumulus/api package for better local testing.
  - Adds data seeding for @cumulus/api's localAPI.
    - seed functions allow adding collections, executions, granules, pdrs, providers, and rules to a Localstack Elasticsearch and DynamoDB via addCollections, addExecutions, addGranules, addPdrs, addProviders, and addRules.
  - Adds eraseDataStack function to local API server code allowing resetting of local datastack for testing (ES and DynamoDB).
  - Adds optional parameters to the @cumulus/api bin serve to allow for launching the api without destroying the current data.
CUMULUS-1697
- Added the @cumulus/tf-inventory package that provides command line utilities for managing Terraform resources in your AWS account
CUMULUS-1703
- Add @cumulus/aws-client/S3.createBucket function
- Add @cumulus/aws-client/S3.putFile function
- Add @cumulus/common/string.isNonEmptyString function
- Add @cumulus/ingest/FtpProviderClient class
- Add @cumulus/ingest/HttpProviderClient class
- Add @cumulus/ingest/S3ProviderClient class
- Add @cumulus/ingest/SftpProviderClient class
- Add @cumulus/ingest/providerClientUtils.buildProviderClient function
- Add @cumulus/ingest/providerClientUtils.fetchTextFile function
CUMULUS-1731
- Add new optional input variables to the Cumulus Terraform module to support TEA upgrade:
  - thin_egress_cookie_domain - Valid domain for Thin Egress App cookie
  - thin_egress_domain_cert_arn - Certificate Manager SSL Cert ARN for Thin Egress App if deployed outside NGAP/CloudFront
  - thin_egress_download_role_in_region_arn - ARN for reading of Thin Egress App data buckets for in-region requests
  - thin_egress_jwt_algo - Algorithm with which to encode the Thin Egress App JWT cookie
  - thin_egress_jwt_secret_name - Name of AWS secret where keys for the Thin Egress App JWT encode/decode are stored
  - thin_egress_lambda_code_dependency_archive_key - Thin Egress App - S3 Key of packaged python modules for lambda dependency layer
CUMULUS-1733
- Add discovery-filtering operator doc to document previously undocumented functionality.
CUMULUS-1737
- Added the cumulus-test-cleanup module to run a nightly cleanup on resources left over from the integration tests run from the example/spec directory.

Changed

CUMULUS-1102
- Updates @cumulus/api/auth/testAuth to use JWT instead of random tokens.
- Updates the default AMI for the ecs_cluster_instance_image_id.
CUMULUS-1622
- Mutex class has been deprecated in @cumulus/common/concurrency and will be removed in a future release.
CUMULUS-1686
- Changed ecs_cluster_instance_image_id to be a required variable of the cumulus module and removed the default value. The default was not available across accounts and regions, nor outside of NGAP and therefore not particularly useful.
CUMULUS-1688
- Updated @cumulus/aws.receiveSQSMessages not to replace message.Body with a parsed object. This behavior was undocumented and confusing as received messages appeared to contradict AWS docs that state message.Body is always a string.
- Replaced sf_watcher CloudWatch rule from cloudwatch-events.tf with an EventSourceMapping on sqs2sf mapped to the start_sf SQS queue (in event-sources.tf).
- Updated sqs2sf with an EventSourceMapping handler and unit test.
CUMULUS-1698
- Change variable saml_launchpad_metadata_path to saml_launchpad_metadata_url in the tf-modules/cumulus Terraform module.
- Updated @cumulus/api/launchpadSaml to download launchpad IDP metadata from configured location when the metadata in s3 is not valid, and to work with updated IDP metadata and SAML response.
CUMULUS-1731
- Upgrade the version of the Thin Egress App deployed by Cumulus to v48
  - Note: New variables available, see the 'Added' section of this changelog.

Fixed

CUMULUS-1664
- Updated dbIndexer Lambda to remove hardcoded references to DynamoDB table names.
CUMULUS-1733
- Fixed granule discovery recursion algorithm used in S/FTP protocols.

Removed

CUMULUS-1481
- removed process config and output from PostToCmr as it was not required by the task nor downstream steps, and should still be in the output message's meta regardless.

Deprecated

CUMULUS-1040
- Deprecated the following code. For cases where the code was moved into another package, the new code location is noted:
  - @cumulus/common/CloudFormationGateway -> @cumulus/aws-client/CloudFormationGateway
  - @cumulus/common/DynamoDb -> @cumulus/aws-client/DynamoDb
  - @cumulus/common/errors -> @cumulus/errors
  - @cumulus/common/StepFunctions -> @cumulus/aws-client/StepFunctions
  - All of the exported functions in @cumulus/commmon/aws (moved into @cumulus/aws-client), except:
    - @cumulus/common/aws/isThrottlingException -> @cumulus/errors/isThrottlingException
    - @cumulus/common/aws/improveStackTrace (not deprecated)
    - @cumulus/common/aws/retryOnThrottlingException (not deprecated)
  - @cumulus/common/sfnStep/SfnStep.parseStepMessage -> @cumulus/integration-tests/sfnStep/SfnStep.parseStepMessage
  - @cumulus/common/sfnStep/ActivityStep -> @cumulus/integration-tests/sfnStep/ActivityStep
  - @cumulus/common/sfnStep/LambdaStep -> @cumulus/integration-tests/sfnStep/LambdaStep
  - @cumulus/common/string/unicodeEscape -> @cumulus/aws-client/StepFunctions.unicodeEscape
  - @cumulus/common/util/setErrorStack -> @cumulus/aws-client/util/setErrorStack
  - @cumulus/ingest/aws/invoke -> @cumulus/aws-client/Lambda/invoke
  - @cumulus/ingest/aws/CloudWatch.bucketSize
  - @cumulus/ingest/aws/CloudWatch.cw
  - @cumulus/ingest/aws/ECS.ecs
  - @cumulus/ingest/aws/ECS
  - @cumulus/ingest/aws/Events.putEvent -> @cumulus/aws-client/CloudwatchEvents.putEvent
  - @cumulus/ingest/aws/Events.deleteEvent -> @cumulus/aws-client/CloudwatchEvents.deleteEvent
  - @cumulus/ingest/aws/Events.deleteTarget -> @cumulus/aws-client/CloudwatchEvents.deleteTarget
  - @cumulus/ingest/aws/Events.putTarget -> @cumulus/aws-client/CloudwatchEvents.putTarget
  - @cumulus/ingest/aws/SQS.attributes -> @cumulus/aws-client/SQS.getQueueAttributes
  - @cumulus/ingest/aws/SQS.deleteMessage -> @cumulus/aws-client/SQS.deleteSQSMessage
  - @cumulus/ingest/aws/SQS.deleteQueue -> @cumulus/aws-client/SQS.deleteQueue
  - @cumulus/ingest/aws/SQS.getUrl -> @cumulus/aws-client/SQS.getQueueUrlByName
  - @cumulus/ingest/aws/SQS.receiveMessage -> @cumulus/aws-client/SQS.receiveSQSMessages
  - @cumulus/ingest/aws/SQS.sendMessage -> @cumulus/aws-client/SQS.sendSQSMessage
  - @cumulus/ingest/aws/StepFunction.getExecutionStatus -> @cumulus/aws-client/StepFunction.getExecutionStatus
  - @cumulus/ingest/aws/StepFunction.getExecutionUrl -> @cumulus/aws-client/StepFunction.getExecutionUrl

Keywords

FAQs

What is @cumulus/common?

Is @cumulus/common popular?

Is @cumulus/common well maintained?

Package last updated on 03 Feb 2020

Did you know?

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install