CUMULUS-1686
- ecs_cluster_instance_image_id is now a required variable of the cumulus module, instead of optional.
CUMULUS-1698
- Change variable saml_launchpad_metadata_path to saml_launchpad_metadata_url in the tf-modules/cumulus Terraform module.
CUMULUS-1703
- Remove the unused forceDownload option from the sync-granule tasks's config
- Remove the @cumulus/ingest/granule.Discover class
- Remove the @cumulus/ingest/granule.Granule class
- Remove the @cumulus/ingest/pdr.Discover class
- Remove the @cumulus/ingest/pdr.Granule class
- Remove the @cumulus/ingest/parse-pdr.parsePdr function

Added

CUMULUS-1040
- Added @cumulus/aws-client package to provide utilities for working with AWS services and the Node.js AWS SDK
- Added @cumulus/errors package which exports error classes for use in Cumulus workflow code
- Added @cumulus/integration-tests/sfnStep to provide utilities for parsing step function execution histories
CUMULUS-1102
- Adds functionality to the @cumulus/api package for better local testing.
  - Adds data seeding for @cumulus/api's localAPI.
    - seed functions allow adding collections, executions, granules, pdrs, providers, and rules to a Localstack Elasticsearch and DynamoDB via addCollections, addExecutions, addGranules, addPdrs, addProviders, and addRules.
  - Adds eraseDataStack function to local API server code allowing resetting of local datastack for testing (ES and DynamoDB).
  - Adds optional parameters to the @cumulus/api bin serve to allow for launching the api without destroying the current data.
CUMULUS-1697
- Added the @cumulus/tf-inventory package that provides command line utilities for managing Terraform resources in your AWS account
CUMULUS-1703
- Add @cumulus/aws-client/S3.createBucket function
- Add @cumulus/aws-client/S3.putFile function
- Add @cumulus/common/string.isNonEmptyString function
- Add @cumulus/ingest/FtpProviderClient class
- Add @cumulus/ingest/HttpProviderClient class
- Add @cumulus/ingest/S3ProviderClient class
- Add @cumulus/ingest/SftpProviderClient class
- Add @cumulus/ingest/providerClientUtils.buildProviderClient function
- Add @cumulus/ingest/providerClientUtils.fetchTextFile function
CUMULUS-1731
- Add new optional input variables to the Cumulus Terraform module to support TEA upgrade:
  - thin_egress_cookie_domain - Valid domain for Thin Egress App cookie
  - thin_egress_domain_cert_arn - Certificate Manager SSL Cert ARN for Thin Egress App if deployed outside NGAP/CloudFront
  - thin_egress_download_role_in_region_arn - ARN for reading of Thin Egress App data buckets for in-region requests
  - thin_egress_jwt_algo - Algorithm with which to encode the Thin Egress App JWT cookie
  - thin_egress_jwt_secret_name - Name of AWS secret where keys for the Thin Egress App JWT encode/decode are stored
  - thin_egress_lambda_code_dependency_archive_key - Thin Egress App - S3 Key of packaged python modules for lambda dependency layer
CUMULUS-1733
- Add discovery-filtering operator doc to document previously undocumented functionality.
CUMULUS-1737
- Added the cumulus-test-cleanup module to run a nightly cleanup on resources left over from the integration tests run from the example/spec directory.

Changed

CUMULUS-1102
- Updates @cumulus/api/auth/testAuth to use JWT instead of random tokens.
- Updates the default AMI for the ecs_cluster_instance_image_id.
CUMULUS-1622
- Mutex class has been deprecated in @cumulus/common/concurrency and will be removed in a future release.
CUMULUS-1686
- Changed ecs_cluster_instance_image_id to be a required variable of the cumulus module and removed the default value. The default was not available across accounts and regions, nor outside of NGAP and therefore not particularly useful.
CUMULUS-1688
- Updated @cumulus/aws.receiveSQSMessages not to replace message.Body with a parsed object. This behavior was undocumented and confusing as received messages appeared to contradict AWS docs that state message.Body is always a string.
- Replaced sf_watcher CloudWatch rule from cloudwatch-events.tf with an EventSourceMapping on sqs2sf mapped to the start_sf SQS queue (in event-sources.tf).
- Updated sqs2sf with an EventSourceMapping handler and unit test.
CUMULUS-1698
- Change variable saml_launchpad_metadata_path to saml_launchpad_metadata_url in the tf-modules/cumulus Terraform module.
- Updated @cumulus/api/launchpadSaml to download launchpad IDP metadata from configured location when the metadata in s3 is not valid, and to work with updated IDP metadata and SAML response.
CUMULUS-1731
- Upgrade the version of the Thin Egress App deployed by Cumulus to v48
  - Note: New variables available, see the 'Added' section of this changelog.

Fixed

CUMULUS-1664
- Updated dbIndexer Lambda to remove hardcoded references to DynamoDB table names.
CUMULUS-1733
- Fixed granule discovery recursion algorithm used in S/FTP protocols.

Removed

CUMULUS-1481
- removed process config and output from PostToCmr as it was not required by the task nor downstream steps, and should still be in the output message's meta regardless.

Deprecated

CUMULUS-1040
- Deprecated the following code. For cases where the code was moved into another package, the new code location is noted:
  - @cumulus/common/CloudFormationGateway -> @cumulus/aws-client/CloudFormationGateway
  - @cumulus/common/DynamoDb -> @cumulus/aws-client/DynamoDb
  - @cumulus/common/errors -> @cumulus/errors
  - @cumulus/common/StepFunctions -> @cumulus/aws-client/StepFunctions
  - All of the exported functions in @cumulus/commmon/aws (moved into @cumulus/aws-client), except:
    - @cumulus/common/aws/isThrottlingException -> @cumulus/errors/isThrottlingException
    - @cumulus/common/aws/improveStackTrace (not deprecated)
    - @cumulus/common/aws/retryOnThrottlingException (not deprecated)
  - @cumulus/common/sfnStep/SfnStep.parseStepMessage -> @cumulus/integration-tests/sfnStep/SfnStep.parseStepMessage
  - @cumulus/common/sfnStep/ActivityStep -> @cumulus/integration-tests/sfnStep/ActivityStep
  - @cumulus/common/sfnStep/LambdaStep -> @cumulus/integration-tests/sfnStep/LambdaStep
  - @cumulus/common/string/unicodeEscape -> @cumulus/aws-client/StepFunctions.unicodeEscape
  - @cumulus/common/util/setErrorStack -> @cumulus/aws-client/util/setErrorStack
  - @cumulus/ingest/aws/invoke -> @cumulus/aws-client/Lambda/invoke
  - @cumulus/ingest/aws/CloudWatch.bucketSize
  - @cumulus/ingest/aws/CloudWatch.cw
  - @cumulus/ingest/aws/ECS.ecs
  - @cumulus/ingest/aws/ECS
  - @cumulus/ingest/aws/Events.putEvent -> @cumulus/aws-client/CloudwatchEvents.putEvent
  - @cumulus/ingest/aws/Events.deleteEvent -> @cumulus/aws-client/CloudwatchEvents.deleteEvent
  - @cumulus/ingest/aws/Events.deleteTarget -> @cumulus/aws-client/CloudwatchEvents.deleteTarget
  - @cumulus/ingest/aws/Events.putTarget -> @cumulus/aws-client/CloudwatchEvents.putTarget
  - @cumulus/ingest/aws/SQS.attributes -> @cumulus/aws-client/SQS.getQueueAttributes
  - @cumulus/ingest/aws/SQS.deleteMessage -> @cumulus/aws-client/SQS.deleteSQSMessage
  - @cumulus/ingest/aws/SQS.deleteQueue -> @cumulus/aws-client/SQS.deleteQueue
  - @cumulus/ingest/aws/SQS.getUrl -> @cumulus/aws-client/SQS.getQueueUrlByName
  - @cumulus/ingest/aws/SQS.receiveMessage -> @cumulus/aws-client/SQS.receiveSQSMessages
  - @cumulus/ingest/aws/SQS.sendMessage -> @cumulus/aws-client/SQS.sendSQSMessage
  - @cumulus/ingest/aws/StepFunction.getExecutionStatus -> @cumulus/aws-client/StepFunction.getExecutionStatus
  - @cumulus/ingest/aws/StepFunction.getExecutionUrl -> @cumulus/aws-client/StepFunction.getExecutionUrl