@cumulus/common/aws - Utilities for working with AWS. For ease of setup, testing, and credential management, code should obtain AWS client objects from helpers in this module.
@cumulus/common/concurrency - Utilities for writing concurrent code
@cumulus/common/errors - Classes for thrown errors
@cumulus/common/log - muting or potentially shipping logs to alternative locations
@cumulus/common/string - Utilities for manipulating strings
@cumulus/common/test-utils - Utilities for writing tests
@cumulus/common/URLUtils - a collection of utilities for working with URLs
@cumulus/common/util - Other misc general utilities

Contributing

To make a contribution, please see our contributing guidelines.

[v1.23.2] 2020-05-22

BREAKING CHANGES

Updates to the Cumulus archive API:
- All endpoints now return a 401 response instead of a 403 for any request where the JWT passed as a Bearer token is invalid.
- POST /refresh and DELETE /token/<token> endpoints now return a 401 response for requests with expired tokens
CUMULUS-1894
- @cumulus/ingest/granule.handleDuplicateFile()
  - The copyOptions parameter has been removed
  - An ACL parameter has been added
- @cumulus/ingest/granule.renameS3FileWithTimestamp()
  - Now returns undefined
CUMULUS-1896 Updated all Cumulus core lambdas to utilize the new message adapter streaming interface via cumulus-message-adapter-js v1.2.0. Users of this version of Cumulus (or later) must utilize version 1.3.0 or greater of the cumulus-message-adapter to support core lambdas.
CUMULUS-1912
- @cumulus/api reconciliationReports list endpoint returns a list of reconciliationReport records instead of S3Uri.
CUMULUS-1969
- The DiscoverGranules task now expects provider_path to be provided at event.config.provider_path, not event.config.collection.provider_path
- config.provider_path is now a required parameter of the DiscoverGranules task

MIGRATION STEPS

To take advantage of the new TTL-based access token expiration implemented in CUMULUS-1777 (see notes below) and clear out existing records in your access tokens table, do the following:
1. Log out of any active dashboard sessions
2. Use the AWS console or CLI to delete your <prefix>-AccessTokensTable DynamoDB table
3. Re-deploy your data-persistence module, which should re-create the <prefix>-AccessTokensTable DynamoDB table
4. Return to using the Cumulus API/dashboard as normal
This release requires the Cumulus Message Adapter layer deployed with Cumulus Core to be at least 1.3.0, as the core lambdas have updated to cumulus-message-adapter-js v1.2.0 and the new CMA interface. As a result, users should:
1. Follow the Cumulus Message Adapter (CMA) deployment instructions and install a CMA layer version >=1.3.0
2. If you are using any custom Node.js Lambdas in your workflows and the Cumulus CMA layer/cumulus-message-adapter-js, you must update your lambda to use cumulus-message-adapter-js v1.2.0 and follow the migration instructions in the release notes. Prior versions of cumulus-message-adapter-js are not compatible with CMA >= 1.3.0.
Migrate existing s3 reconciliation report records to database (CUMULUS-1911):
- After update your data persistence module and Cumulus resources, run the command:
```
./node_modules/.bin/cumulus-api migrate --stack `<your-terraform-deployment-prefix>` --migrationVersion migration5
```

Added

Added a limit for concurrent Elasticsearch requests when doing an index from database operation
Added the es_request_concurrency parameter to the archive and cumulus Terraform modules
CUMULUS-1995
- Added the es_index_shards parameter to the archive and cumulus Terraform modules to configure the number of shards for the ES index
  - If you have an existing ES index, you will need to reindex and then change index to take advantage of shard updates
CUMULUS-1894
- Added @cumulus/aws-client/S3.moveObject()
CUMULUS-1911
- Added ReconciliationReports table
- Updated CreateReconciliationReport lambda to save Reconciliation Report records to database
- Updated dbIndexer and IndexFromDatabase lambdas to index Reconciliation Report records to Elasticsearch
- Added migration_5 to migrate existing s3 reconciliation report records to database and Elasticsearch
- Updated @cumulus/api package, tf-modules/archive and tf-modules/data-persistence Terraform modules
CUMULUS-1916
- Added util function for seeding reconciliation reports when running API locally in dashboard

Changed

CUMULUS-1777
- The expirationTime property is now a required field of the access tokens model.
- Updated the AccessTokens table to set a TTL on the expirationTime field in tf-modules/data-persistence/dynamo.tf. As a result, access token records in this table whose expirationTime has passed should be automatically deleted by DynamoDB.
- Updated all code creating access token records in the Dynamo AccessTokens table to set the expirationTime field value in seconds from the epoch.
CUMULUS-1912
- Updated reconciliationReports endpoints to query against Elasticsearch, delete report from both database and s3
- Added @cumulus/api-client/reconciliationReports
CUMULUS-1999
- Updated @cumulus/common/util.deprecate() so that only a single deprecation notice is printed for each name/version combination

Fixed

CUMULUS-1894
- The SyncGranule task can now handle files larger than 5 GB
CUMULUS-1987
- Remove granule from CMR operation in @cumulus/api now passes token to CMR when fetching granule metadata, allowing removal of private granules
CUMULUS-1993
- For a given queue, the sqs-message-consumer Lambda will now only schedule workflows for rules matching the queue and the collection information in each queue message (if any)
  - The consumer also now only reads each queue message once per Lambda invocation, whereas previously each message was read once per queue rule per Lambda invocation
- Fixed bug preventing the deletion of multiple SNS rules that share the same SNS topic

Deprecated

CUMULUS-1894
- @cumulus/ingest/granule.copyGranuleFile()
- @cumulus/ingest/granule.moveGranuleFile()
CUMULUS-1987 - Deprecated the following functions:
- @cumulus/cmrjs/getMetadata(cmrLink) -> @cumulus/cmr-client/CMR.getGranuleMetadata(cmrLink)
- @cumulus/cmrjs/getFullMetadata(cmrLink)

Keywords

FAQs

What is @cumulus/common?

Is @cumulus/common popular?

Is @cumulus/common well maintained?

Package last updated on 26 May 2020

Did you know?

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install