@cumulus/errors

@cumulus/errors

Package that exports error classes for use in Cumulus workflow code

Usage

  npm install @cumulus/errors

About Cumulus

Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.

Cumulus Documentation

Contributing

To make a contribution, please see our contributing guidelines.

Changelog

[v1.23.2] 2020-05-22

BREAKING CHANGES

• Updates to the Cumulus archive API:

  • All endpoints now return a 401 response instead of a 403 for any request where the JWT passed as a Bearer token is invalid.
  • POST /refresh and DELETE /token/<token> endpoints now return a 401 response for requests with expired tokens

• CUMULUS-1894

  • @cumulus/ingest/granule.handleDuplicateFile()
    • The copyOptions parameter has been removed
    • An ACL parameter has been added
  • @cumulus/ingest/granule.renameS3FileWithTimestamp()
    • Now returns undefined

• CUMULUS-1896 Updated all Cumulus core lambdas to utilize the new message adapter streaming interface via cumulus-message-adapter-js v1.2.0. Users of this version of Cumulus (or later) must utilize version 1.3.0 or greater of the cumulus-message-adapter to support core lambdas.

• CUMULUS-1912

  • @cumulus/api reconciliationReports list endpoint returns a list of reconciliationReport records instead of S3Uri.

• CUMULUS-1969

  • The DiscoverGranules task now expects provider_path to be provided at event.config.provider_path, not event.config.collection.provider_path
  • config.provider_path is now a required parameter of the DiscoverGranules task

MIGRATION STEPS

• To take advantage of the new TTL-based access token expiration implemented in CUMULUS-1777 (see notes below) and clear out existing records in your access tokens table, do the following:

  1. Log out of any active dashboard sessions
  2. Use the AWS console or CLI to delete your <prefix>-AccessTokensTable DynamoDB table
  3. Re-deploy your data-persistence module, which should re-create the <prefix>-AccessTokensTable DynamoDB table
  4. Return to using the Cumulus API/dashboard as normal

• This release requires the Cumulus Message Adapter layer deployed with Cumulus Core to be at least 1.3.0, as the core lambdas have updated to cumulus-message-adapter-js v1.2.0 and the new CMA interface. As a result, users should:

  1. Follow the Cumulus Message Adapter (CMA) deployment instructions and install a CMA layer version >=1.3.0
  2. If you are using any custom Node.js Lambdas in your workflows and the Cumulus CMA layer/cumulus-message-adapter-js, you must update your lambda to use cumulus-message-adapter-js v1.2.0 and follow the migration instructions in the release notes. Prior versions of cumulus-message-adapter-js are not compatible with CMA >= 1.3.0.

• Migrate existing s3 reconciliation report records to database (CUMULUS-1911):

  • After update your data persistence module and Cumulus resources, run the command:

  ./node_modules/.bin/cumulus-api migrate --stack `<your-terraform-deployment-prefix>` --migrationVersion migration5
  

Added

• Added a limit for concurrent Elasticsearch requests when doing an index from database operation

• Added the es_request_concurrency parameter to the archive and cumulus Terraform modules

• CUMULUS-1995

  • Added the es_index_shards parameter to the archive and cumulus Terraform modules to configure the number of shards for the ES index
    • If you have an existing ES index, you will need to reindex and then change index to take advantage of shard updates

• CUMULUS-1894

  • Added @cumulus/aws-client/S3.moveObject()

• CUMULUS-1911

  • Added ReconciliationReports table
  • Updated CreateReconciliationReport lambda to save Reconciliation Report records to database
  • Updated dbIndexer and IndexFromDatabase lambdas to index Reconciliation Report records to Elasticsearch
  • Added migration_5 to migrate existing s3 reconciliation report records to database and Elasticsearch
  • Updated @cumulus/api package, tf-modules/archive and tf-modules/data-persistence Terraform modules

• CUMULUS-1916

  • Added util function for seeding reconciliation reports when running API locally in dashboard

Changed

• CUMULUS-1777
  • The expirationTime property is now a required field of the access tokens model.
  • Updated the AccessTokens table to set a TTL on the expirationTime field in tf-modules/data-persistence/dynamo.tf. As a result, access token records in this table whose expirationTime has passed should be automatically deleted by DynamoDB.
  • Updated all code creating access token records in the Dynamo AccessTokens table to set the expirationTime field value in seconds from the epoch.
• CUMULUS-1912
  • Updated reconciliationReports endpoints to query against Elasticsearch, delete report from both database and s3
  • Added @cumulus/api-client/reconciliationReports
• CUMULUS-1999
  • Updated @cumulus/common/util.deprecate() so that only a single deprecation notice is printed for each name/version combination

Fixed

• CUMULUS-1894
  • The SyncGranule task can now handle files larger than 5 GB
• CUMULUS-1987
  • Remove granule from CMR operation in @cumulus/api now passes token to CMR when fetching granule metadata, allowing removal of private granules
• CUMULUS-1993
  • For a given queue, the sqs-message-consumer Lambda will now only schedule workflows for rules matching the queue and the collection information in each queue message (if any)
    • The consumer also now only reads each queue message once per Lambda invocation, whereas previously each message was read once per queue rule per Lambda invocation
  • Fixed bug preventing the deletion of multiple SNS rules that share the same SNS topic

Deprecated

• CUMULUS-1894

  • @cumulus/ingest/granule.copyGranuleFile()
  • @cumulus/ingest/granule.moveGranuleFile()

• CUMULUS-1987 - Deprecated the following functions:

  • @cumulus/cmrjs/getMetadata(cmrLink) -> @cumulus/cmr-client/CMR.getGranuleMetadata(cmrLink)
  • @cumulus/cmrjs/getFullMetadata(cmrLink)