Huge News!Announcing our $40M Series B led by Abstract Ventures.Learn More →
Socket
Sign inDemoInstall
Socket
Sign inDemoInstall

@cumulus/common

Package Overview
Dependencies
Maintainers
12
Versions
214
Alerts
File Explorer

Advanced tools

License
Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

@cumulus/common - npm Package Versions

1
…
…
22

2.0.6

Diff
lfrederick
 published 2.0.6 •

Changelog

Source

[v2.0.6] 2020-09-25 - [BACKPORT]

Fixed

  • CUMULUS-2168
    • Fixed issue where large number of documents (generally logs) in the cumulus elasticsearch index results in the collection granule stats queries failing for the collections list api endpoint

2.0.5

Diff
lfrederick
 published 2.0.5 •

Changelog

Source

[v2.0.5] 2020-09-15 - [BACKPORT]

Added
  • Added thin_egress_stack_name variable to cumulus and distribution Terraform modules to allow overriding the default Cloudformation stack name used for the thin-egress-app. Please note that if you change/set this value for an existing deployment, it will destroy and re-create your API gateway for the thin-egress-app.
Fixed
  • Fix collection list queries. Removed fixes to collection stats, which break queries for a large number of granules.

2.0.4

Diff
lfrederick
 published 2.0.4 •

Changelog

Source

[v2.0.4] 2020-09-08 - [BACKPORT]

Changed
  • Upgraded version of TEA deployed with Cumulus to build 88.

2.0.3

Diff
lfrederick
 published 2.0.3 •

Changelog

Source

[v2.0.3] 2020-09-02 - [BACKPORT]

Fixed

  • CUMULUS-1961

    • Fixed activeCollections query only returning 10 results

  • CUMULUS-2039

    • Fix issue causing SyncGranules task to run out of memory on large granules
CODE CHANGES
  • The @cumulus/aws-client/S3.getS3ObjectReadStreamAsync function has been removed. It read the entire S3 object into memory before returning a read stream, which could cause Lambdas to run out of memory. Use @cumulus/aws-client/S3.getObjectReadStream instead.

2.0.2

Diff
lfrederick
 published 2.0.2 •

Changelog

Source

v2.0.2+ Backport releases

Release v2.0.1 was the last release on the 2.0.x release series.

Changes after this version on the 2.0.x release series are limited security/requested feature patches and will not be ported forward to future releases unless there is a corresponding CHANGELOG entry.

For up-to-date CHANGELOG for the maintenance release branch see CHANGELOG.md from the 2.0.x branch.

For the most recent release information for the maintenance branch please see the release page

2.0.1

Diff
lfrederick
 published 2.0.1 •

Changelog

Source

[v2.0.1] 2020-07-28

Added

  • CUMULUS-1886
    • Added multiple sort keys support to @cumulus/api
  • CUMULUS-2099
    • @cumulus/message/Queue.getQueueUrl to get the queue URL specified in a Cumulus workflow message, if any.

Fixed

  • PR 1790
    • Fixed bug with request headers in @cumulus/launchpad-auth causing Launchpad token requests to fail

2.0.0

Diff
lfrederick
 published 2.0.0 •

Changelog

Source

[v2.0.0] 2020-07-23

BREAKING CHANGES

  • Changes to the @cumulus/api-client package
    • The CumulusApiClientError class must now be imported using const { CumulusApiClientError } = require('@cumulus/api-client/CumulusApiClientError')
  • The @cumulus/sftp-client/SftpClient class must now be imported using const { SftpClient } = require('@cumulus/sftp-client');
  • Instances of @cumulus/ingest/SftpProviderClient no longer implicitly connect when download, list, or sync are called. You must call connect on the provider client before issuing one of those calls. Failure to do so will result in a "Client not connected" exception being thrown.
  • Instances of @cumulus/ingest/SftpProviderClient no longer implicitly disconnect from the SFTP server when list is called.
  • Instances of @cumulus/sftp-client/SftpClient must now be explicitly closed by calling .end()
  • Instances of @cumulus/sftp-client/SftpClient no longer implicitly connect to the server when download, unlink, syncToS3, syncFromS3, and list are called. You must explicitly call connect before calling one of those methods.
  • Changes to the @cumulus/common package
    • cloudwatch-event.getSfEventMessageObject() now returns undefined if the message could not be found or could not be parsed. It previously returned null.
    • S3KeyPairProvider.decrypt() now throws an exception if the bucket containing the key cannot be determined.
    • S3KeyPairProvider.decrypt() now throws an exception if the stack cannot be determined.
    • S3KeyPairProvider.encrypt() now throws an exception if the bucket containing the key cannot be determined.
    • S3KeyPairProvider.encrypt() now throws an exception if the stack cannot be determined.
    • sns-event.getSnsEventMessageObject() now returns undefined if it could not be parsed. It previously returned null.
    • The aws module has been removed.
    • The BucketsConfig.buckets property is now read-only and private
    • The test-utils.validateConfig() function now resolves to undefined rather than true.
    • The test-utils.validateInput() function now resolves to undefined rather than true.
    • The test-utils.validateOutput() function now resolves to undefined rather than true.
    • The static S3KeyPairProvider.retrieveKey() function has been removed.
  • Changes to the @cumulus/cmrjs package
    • @cumulus/cmrjs.constructOnlineAccessUrl() and @cumulus/cmrjs/cmr-utils.constructOnlineAccessUrl() previously took a buckets parameter, which was an instance of @cumulus/common/BucketsConfig. They now take a bucketTypes parameter, which is a simple object mapping bucket names to bucket types. Example: { 'private-1': 'private', 'public-1': 'public' }
    • @cumulus/cmrjs.reconcileCMRMetadata() and @cumulus/cmrjs/cmr-utils.reconcileCMRMetadata() now take a required bucketTypes parameter, which is a simple object mapping bucket names to bucket types. Example: { 'private-1': 'private', 'public-1': 'public' }
    • @cumulus/cmrjs.updateCMRMetadata() and @cumulus/cmrjs/cmr-utils.updateCMRMetadata() previously took an optional inBuckets parameter, which was an instance of @cumulus/common/BucketsConfig. They now take a required bucketTypes parameter, which is a simple object mapping bucket names to bucket types. Example: { 'private-1': 'private', 'public-1': 'public' }
  • The minimum supported version of all published Cumulus packages is now Node 12.18.0
    • Tasks using the cumuluss/cumulus-ecs-task Docker image must be updated to cumuluss/cumulus-ecs-task:1.7.0. This can be done by updating the image property of any tasks defined using the cumulus_ecs_service Terraform module.
  • Changes to @cumulus/aws-client/S3
    • The signature of the getObjectSize function has changed. It now takes a params object with three properties:
      • s3: an instance of an AWS.S3 object
      • bucket
      • key
    • The getObjectSize function will no longer retry if the object does not exist
  • CUMULUS-1861
    • @cumulus/message/Collections.getCollectionIdFromMessage now throws a CumulusMessageError if collectionName and collectionVersion are missing from meta.collection. Previously this method would return 'undefined___undefined' instead
    • @cumulus/integration-tests/addCollections now returns an array of collections that were added rather than the count of added collections
  • CUMULUS-1930
    • The @cumulus/common/util.uuid() function has been removed
  • CUMULUS-1955
    • @cumulus/aws-client/S3.multipartCopyObject now returns an object with the AWS etag of the destination object
    • @cumulus/ingest/S3ProviderClient.list now sets a file object's path property to undefined instead of null when the file is at the top level of its bucket
    • The sync methods of the following classes in the @cumulus/ingest package now return an object with the AWS s3uri and etag of the destination file (they previously returned only a string representing the S3 URI)
      • FtpProviderClient
      • HttpProviderClient
      • S3ProviderClient
      • SftpProviderClient
  • CUMULUS-1958
    • The following methods exported from @cumulus/cmr-js/cmr-utils were made async, and added distributionBucketMap as a parameter:
      • constructOnlineAccessUrl
      • generateFileUrl
      • reconcileCMRMetadata
      • updateCMRMetadata
  • CUMULUS-1969
    • The DiscoverPdrs task now expects provider_path to be provided at event.config.provider_path, not event.config.collection.provider_path
    • event.config.provider_path is now a required parameter of the DiscoverPdrs task
    • event.config.collection is no longer a parameter to the DiscoverPdrs task
    • Collections no longer support the provider_path property. The tasks that relied on that property are now referencing config.meta.provider_path. Workflows should be updated accordingly.
  • CUMULUS-1977
    • Moved bulk granule deletion endpoint from /bulkDelete to /granules/bulkDelete
  • CUMULUS-1991
    • Updated CMR metadata generation to use "Download file.hdf" (where file.hdf is the filename of the given resource) as the resource description instead of "File to download"
    • CMR metadata updates now respect changes to resource descriptions (previously only changes to resource URLs were respected)

MIGRATION STEPS

  • Due to an issue with the AWS API Gateway and how the Thin Egress App Cloudformation template applies updates, you may need to redeploy your thin-egress-app-EgressGateway manually as a one time migration step. If your deployment fails with an error similar to:

    Error: Lambda function (<stack>-tf-TeaCache) returned error: ({"errorType":"HTTPError","errorMessage":"Response code 404 (Not Found)"})

    Then follow the AWS instructions to Redeploy a REST API to a stage for your egress API and re-run terraform apply.

Added

  • CUMULUS-2081

    • Add Integrator Guide section for onboarding
    • Add helpful tips documentation

  • CUMULUS-1902

    • Add Common Use Cases section under Operator Docs

  • CUMULUS-2058

    • Added lambda_processing_role_name as an output from the cumulus module to provide the processing role name

  • CUMULUS-1417

    • Added a checksumFor property to collection files config. Set this property on a checksum file's definition matching the regex of the target file. More details in the 'Data Cookbooks Setup' documentation.
    • Added checksumFor validation to collections model.

  • CUMULUS-1956

    • Added @cumulus/earthata-login-client package
    • The /s3credentials endpoint that is deployed as part of distribution now supports authentication using tokens created by a different application. If a request contains the EDL-ClientId and EDL-Token headers, authentication will be handled using that token rather than attempting to use OAuth.
    • @cumulus/earthata-login-client.getTokenUsername() now accepts an xRequestId argument, which will be included as the X-Request-Id header when calling Earthdata Login.
    • If the s3Credentials endpoint is invoked with an EDL token and an X-Request-Id header, that X-Request-Id header will be forwarded to Earthata Login.

  • CUMULUS-1957

    • If EDL token authentication is being used, and the EDL-Client-Name header is set, @the-client-name will be appended to the end of the Earthdata Login username that is used as the RoleSessionName of the temporary IAM credentials. This value will show up in the AWS S3 server access logs.

  • CUMULUS-1958

    • Add the ability for users to specify a bucket_map_key to the cumulus terraform module as an override for the default .yaml values that are passed to TEA by Core. Using this option requires that each configured Cumulus 'distribution' bucket (e.g. public/protected buckets) have a single TEA mapping. Multiple maps per bucket are not supported.
    • Updated Generating a distribution URL, the MoveGranules task and all CMR reconciliation functionality to utilize the TEA bucket map override.
    • Updated deploy process to utilize a bootstrap 'tea-map-cache' lambda that will, after deployment of Cumulus Core's TEA instance, query TEA for all protected/public buckets and generate a mapping configuration used internally by Core. This object is also exposed as an output of the Cumulus module as distribution_bucket_map.

  • CUMULUS-1961

    • Replaces DynamoDB for Elasticsearch for reconciliationReportForCumulusCMR comparisons between Cumulus and CMR.

  • CUMULUS-1970

    • Created the add-missing-file-checksums workflow task
    • Added @cumulus/aws-client/S3.calculateObjectHash() function
    • Added @cumulus/aws-client/S3.getObjectReadStream() function

  • CUMULUS-1887

    • Add additional fields to the granule CSV download file

  • CUMULUS-2019

    • Add infix search to es query builder @cumulus/api/es/es/queries to support partial matching of the keywords

Changed

  • CUMULUS-2032

    • Updated @cumulus/ingest/HttpProviderClient to utilize a configuration key httpListTimeout to set the default timeout for discovery HTTP/HTTPS requests, and updates the default for the provider to 5 minutes (300 seconds).
    • Updated the DiscoverGranules and DiscoverPDRs tasks to utilize the updated configuration value if set via workflow config, and updates the default for these tasks to 5 minutes (300 seconds).

  • CUMULUS-176

    • The API will now respond with a 400 status code when a request body contains invalid JSON. It had previously returned a 500 status code.

  • CUMULUS-1861

    • Updates Rule objects to no longer require a collection.
    • Changes the DLQ behavior for sfEventSqsToDbRecords and sfEventSqsToDbRecordsInputQueue. Previously failure to write a database record would result in lambda success, and an error log in the CloudWatch logs. The lambda has been updated to manually add a record to the sfEventSqsToDbRecordsDeadLetterQueue if the granule, execution, or pdr record fails to write, in addition to the previous error logging.

  • CUMULUS-1956

    • The /s3credentials endpoint that is deployed as part of distribution now supports authentication using tokens created by a different application. If a request contains the EDL-ClientId and EDL-Token headers, authentication will be handled using that token rather than attempting to use OAuth.

  • CUMULUS-1977

    • API endpoint POST /granules/bulk now returns a 202 status on a successful response instead of a 200 response
    • API endpoint DELETE /granules/<granule-id> now returns a 404 status if the granule record was already deleted
    • @cumulus/api/models/Granule.update() now returns the updated granule record
    • Implemented POST /granules/bulkDelete API endpoint to support deleting granules specified by ID or returned by the provided query in the request body. If the request is successful, the endpoint returns the async operation ID that has been started to remove the granules.
    • Added @cumulus/api/models/Granule.getRecord() method to return raw record from DynamoDB
    • Added @cumulus/api/models/Granule.delete() method which handles deleting the granule record from DynamoDB and the granule files from S3

  • CUMULUS-1982

    • The globalConnectionLimit property of providers is now optional and defaults to "unlimited"

  • CUMULUS-1997

    • Added optional launchpad configuration to @cumulus/hyrax-metadata-updates task config schema.

  • CUMULUS-1991

    • @cumulus/cmrjs/src/cmr-utils/constructOnlineAccessUrls() now throws an error if cmrGranuleUrlType = "distribution" and no distribution endpoint argument is provided

  • CUMULUS-2011

    • Reconciliation reports are now generated within an AsyncOperation

  • CUMULUS-2016

    • Upgrade TEA to version 79

Fixed

  • CUMULUS-1991

    • Added missing DISTRIBUTION_ENDPOINT environment variable for API lambdas. This environment variable is required for API requests to move granules.

  • CUMULUS-1961

    • Fixed granules and executions query params not getting sent to API in granule list operation in @cumulus/api-client

Deprecated

  • @cumulus/aws-client/S3.calculateS3ObjectChecksum()

  • @cumulus/aws-client/S3.getS3ObjectReadStream()

  • @cumulus/common/log.convertLogLevel()

  • @cumulus/collection-config-store

  • @cumulus/common/util.sleep()

  • CUMULUS-1930

    • @cumulus/common/log.convertLogLevel()
    • @cumulus/common/util.isNull()
    • @cumulus/common/util.isUndefined()
    • @cumulus/common/util.negate()
    • @cumulus/common/util.noop()
    • @cumulus/common/util.isNil()
    • @cumulus/common/util.renameProperty()
    • @cumulus/common/util.lookupMimeType()
    • @cumulus/common/util.thread()
    • @cumulus/common/util.mkdtempSync()

Removed

  • The deprecated @cumulus/common.bucketsConfigJsonObject function has been removed
  • The deprecated @cumulus/common.CollectionConfigStore class has been removed
  • The deprecated @cumulus/common.concurrency module has been removed
  • The deprecated @cumulus/common.constructCollectionId function has been removed
  • The deprecated @cumulus/common.launchpad module has been removed
  • The deprecated @cumulus/common.LaunchpadToken class has been removed
  • The deprecated @cumulus/common.Semaphore class has been removed
  • The deprecated @cumulus/common.stringUtils module has been removed
  • The deprecated @cumulus/common/aws.cloudwatchlogs function has been removed
  • The deprecated @cumulus/common/aws.deleteS3Files function has been removed
  • The deprecated @cumulus/common/aws.deleteS3Object function has been removed
  • The deprecated @cumulus/common/aws.dynamodb function has been removed
  • The deprecated @cumulus/common/aws.dynamodbDocClient function has been removed
  • The deprecated @cumulus/common/aws.getExecutionArn function has been removed
  • The deprecated @cumulus/common/aws.headObject function has been removed
  • The deprecated @cumulus/common/aws.listS3ObjectsV2 function has been removed
  • The deprecated @cumulus/common/aws.parseS3Uri function has been removed
  • The deprecated @cumulus/common/aws.promiseS3Upload function has been removed
  • The deprecated @cumulus/common/aws.recursivelyDeleteS3Bucket function has been removed
  • The deprecated @cumulus/common/aws.s3CopyObject function has been removed
  • The deprecated @cumulus/common/aws.s3ObjectExists function has been removed
  • The deprecated @cumulus/common/aws.s3PutObject function has been removed
  • The deprecated @cumulus/common/bucketsConfigJsonObject function has been removed
  • The deprecated @cumulus/common/CloudWatchLogger class has been removed
  • The deprecated @cumulus/common/collection-config-store.CollectionConfigStore class has been removed
  • The deprecated @cumulus/common/collection-config-store.constructCollectionId function has been removed
  • The deprecated @cumulus/common/concurrency.limit function has been removed
  • The deprecated @cumulus/common/concurrency.mapTolerant function has been removed
  • The deprecated @cumulus/common/concurrency.promiseUrl function has been removed
  • The deprecated @cumulus/common/concurrency.toPromise function has been removed
  • The deprecated @cumulus/common/concurrency.unless function has been removed
  • The deprecated @cumulus/common/config.parseConfig function has been removed
  • The deprecated @cumulus/common/config.resolveResource function has been removed
  • The deprecated @cumulus/common/DynamoDb.get function has been removed
  • The deprecated @cumulus/common/DynamoDb.scan function has been removed
  • The deprecated @cumulus/common/FieldPattern class has been removed
  • The deprecated @cumulus/common/launchpad.getLaunchpadToken function has been removed
  • The deprecated @cumulus/common/launchpad.validateLaunchpadToken function has been removed
  • The deprecated @cumulus/common/LaunchpadToken class has been removed
  • The deprecated @cumulus/common/message.buildCumulusMeta function has been removed
  • The deprecated @cumulus/common/message.buildQueueMessageFromTemplate function has been removed
  • The deprecated @cumulus/common/message.getCollectionIdFromMessage function has been removed
  • The deprecated @cumulus/common/message.getMaximumExecutions function has been removed
  • The deprecated @cumulus/common/message.getMessageExecutionArn function has been removed
  • The deprecated @cumulus/common/message.getMessageExecutionName function has been removed
  • The deprecated @cumulus/common/message.getMessageFromTemplate function has been removed
  • The deprecated @cumulus/common/message.getMessageGranules function has been removed
  • The deprecated @cumulus/common/message.getMessageStateMachineArn function has been removed
  • The deprecated @cumulus/common/message.getQueueName function has been removed
  • The deprecated @cumulus/common/message.getQueueNameByUrl function has been removed
  • The deprecated @cumulus/common/message.hasQueueAndExecutionLimit function has been removed
  • The deprecated @cumulus/common/Semaphore class has been removed
  • The deprecated @cumulus/common/string.globalReplace function has been removed
  • The deprecated @cumulus/common/string.isNonEmptyString function has been removed
  • The deprecated @cumulus/common/string.isValidHostname function has been removed
  • The deprecated @cumulus/common/string.match function has been removed
  • The deprecated @cumulus/common/string.matches function has been removed
  • The deprecated @cumulus/common/string.replace function has been removed
  • The deprecated @cumulus/common/string.toLower function has been removed
  • The deprecated @cumulus/common/string.toUpper function has been removed
  • The deprecated @cumulus/common/testUtils.getLocalstackEndpoint function has been removed
  • The deprecated @cumulus/common/util.setErrorStack function has been removed
  • The @cumulus/common/util.uuid function has been removed
  • The deprecated @cumulus/common/workflows.getWorkflowArn function has been removed
  • The deprecated @cumulus/common/workflows.getWorkflowFile function has been removed
  • The deprecated @cumulus/common/workflows.getWorkflowList function has been removed
  • The deprecated @cumulus/common/workflows.getWorkflowTemplate function has been removed
  • @cumulus/aws-client/StepFunctions.toSfnExecutionName()
  • @cumulus/aws-client/StepFunctions.fromSfnExecutionName()
  • @cumulus/aws-client/StepFunctions.getExecutionArn()
  • @cumulus/aws-client/StepFunctions.getExecutionUrl()
  • @cumulus/aws-client/StepFunctions.getStateMachineArn()
  • @cumulus/aws-client/StepFunctions.pullStepFunctionEvent()
  • @cumulus/common/test-utils/throttleOnce()
  • @cumulus/integration-tests/api/distribution.invokeApiDistributionLambda()
  • @cumulus/integration-tests/api/distribution.getDistributionApiRedirect()
  • @cumulus/integration-tests/api/distribution.getDistributionApiFileStream()

1.24.0

Diff
lfrederick
 published 1.24.0 •

Changelog

Source

[v1.24.0] 2020-06-03

BREAKING CHANGES

  • CUMULUS-1969

    • The DiscoverPdrs task now expects provider_path to be provided at event.config.provider_path, not event.config.collection.provider_path
    • event.config.provider_path is now a required parameter of the DiscoverPdrs task
    • event.config.collection is no longer a parameter to the DiscoverPdrs task
    • Collections no longer support the provider_path property. The tasks that relied on that property are now referencing config.meta.provider_path. Workflows should be updated accordingly.

  • CUMULUS-1997

    • @cumulus/cmr-client/CMRSearchConceptQueue parameters have been changed to take a cmrSettings object containing clientId, provider, and auth information. This can be generated using @cumulus/cmrjs/cmr-utils/getCmrSettings. The cmrEnvironment variable has been removed.

Added

  • CUMULUS-1800
    • Added task configuration setting named syncChecksumFiles to the SyncGranule task. This setting is false by default, but when set to true, all checksum files associated with data files that are downloaded will be downloaded as well.
  • CUMULUS-1952
    • Updated HTTP(S) provider client to accept username/password for Basic authorization. This change adds support for Basic Authorization such as Earthdata login redirects to ingest (i.e. as implemented in SyncGranule), but not to discovery (i.e. as implemented in DiscoverGranules). Discovery still expects the provider's file system to be publicly accessible, but not the individual files and their contents.
    • NOTE: Using this in combination with the HTTP protocol may expose usernames and passwords to intermediary network entities. HTTPS is highly recommended.
  • CUMULUS-1997
    • Added optional launchpad configuration to @cumulus/hyrax-metadata-updates task config schema.

Fixed

  • CUMULUS-1997
    • Updated all CMR operations to use configured authentication scheme
  • CUMULUS-2010
    • Updated @cumulus/api/launchpadSaml to support multiple userGroup attributes from the SAML response

1.23.2

Diff
lfrederick
 published 1.23.2 •

Changelog

Source

[v1.23.2] 2020-05-22

BREAKING CHANGES

  • Updates to the Cumulus archive API:

    • All endpoints now return a 401 response instead of a 403 for any request where the JWT passed as a Bearer token is invalid.
    • POST /refresh and DELETE /token/<token> endpoints now return a 401 response for requests with expired tokens

  • CUMULUS-1894

    • @cumulus/ingest/granule.handleDuplicateFile()
      • The copyOptions parameter has been removed
      • An ACL parameter has been added
    • @cumulus/ingest/granule.renameS3FileWithTimestamp()
      • Now returns undefined

  • CUMULUS-1896 Updated all Cumulus core lambdas to utilize the new message adapter streaming interface via cumulus-message-adapter-js v1.2.0. Users of this version of Cumulus (or later) must utilize version 1.3.0 or greater of the cumulus-message-adapter to support core lambdas.

  • CUMULUS-1912

    • @cumulus/api reconciliationReports list endpoint returns a list of reconciliationReport records instead of S3Uri.

  • CUMULUS-1969

    • The DiscoverGranules task now expects provider_path to be provided at event.config.provider_path, not event.config.collection.provider_path
    • config.provider_path is now a required parameter of the DiscoverGranules task

MIGRATION STEPS

  • To take advantage of the new TTL-based access token expiration implemented in CUMULUS-1777 (see notes below) and clear out existing records in your access tokens table, do the following:

    1. Log out of any active dashboard sessions
    2. Use the AWS console or CLI to delete your <prefix>-AccessTokensTable DynamoDB table
    3. Re-deploy your data-persistence module, which should re-create the <prefix>-AccessTokensTable DynamoDB table
    4. Return to using the Cumulus API/dashboard as normal

  • This release requires the Cumulus Message Adapter layer deployed with Cumulus Core to be at least 1.3.0, as the core lambdas have updated to cumulus-message-adapter-js v1.2.0 and the new CMA interface. As a result, users should:

    1. Follow the Cumulus Message Adapter (CMA) deployment instructions and install a CMA layer version >=1.3.0
    2. If you are using any custom Node.js Lambdas in your workflows and the Cumulus CMA layer/cumulus-message-adapter-js, you must update your lambda to use cumulus-message-adapter-js v1.2.0 and follow the migration instructions in the release notes. Prior versions of cumulus-message-adapter-js are not compatible with CMA >= 1.3.0.

  • Migrate existing s3 reconciliation report records to database (CUMULUS-1911):

    • After update your data persistence module and Cumulus resources, run the command:
    ./node_modules/.bin/cumulus-api migrate --stack `<your-terraform-deployment-prefix>` --migrationVersion migration5

Added

  • Added a limit for concurrent Elasticsearch requests when doing an index from database operation

  • Added the es_request_concurrency parameter to the archive and cumulus Terraform modules

  • CUMULUS-1995

    • Added the es_index_shards parameter to the archive and cumulus Terraform modules to configure the number of shards for the ES index
      • If you have an existing ES index, you will need to reindex and then change index to take advantage of shard updates

  • CUMULUS-1894

    • Added @cumulus/aws-client/S3.moveObject()

  • CUMULUS-1911

    • Added ReconciliationReports table
    • Updated CreateReconciliationReport lambda to save Reconciliation Report records to database
    • Updated dbIndexer and IndexFromDatabase lambdas to index Reconciliation Report records to Elasticsearch
    • Added migration_5 to migrate existing s3 reconciliation report records to database and Elasticsearch
    • Updated @cumulus/api package, tf-modules/archive and tf-modules/data-persistence Terraform modules

  • CUMULUS-1916

    • Added util function for seeding reconciliation reports when running API locally in dashboard

Changed

  • CUMULUS-1777
    • The expirationTime property is now a required field of the access tokens model.
    • Updated the AccessTokens table to set a TTL on the expirationTime field in tf-modules/data-persistence/dynamo.tf. As a result, access token records in this table whose expirationTime has passed should be automatically deleted by DynamoDB.
    • Updated all code creating access token records in the Dynamo AccessTokens table to set the expirationTime field value in seconds from the epoch.
  • CUMULUS-1912
    • Updated reconciliationReports endpoints to query against Elasticsearch, delete report from both database and s3
    • Added @cumulus/api-client/reconciliationReports
  • CUMULUS-1999
    • Updated @cumulus/common/util.deprecate() so that only a single deprecation notice is printed for each name/version combination

Fixed

  • CUMULUS-1894
    • The SyncGranule task can now handle files larger than 5 GB
  • CUMULUS-1987
    • Remove granule from CMR operation in @cumulus/api now passes token to CMR when fetching granule metadata, allowing removal of private granules
  • CUMULUS-1993
    • For a given queue, the sqs-message-consumer Lambda will now only schedule workflows for rules matching the queue and the collection information in each queue message (if any)
      • The consumer also now only reads each queue message once per Lambda invocation, whereas previously each message was read once per queue rule per Lambda invocation
    • Fixed bug preventing the deletion of multiple SNS rules that share the same SNS topic

Deprecated

  • CUMULUS-1894

    • @cumulus/ingest/granule.copyGranuleFile()
    • @cumulus/ingest/granule.moveGranuleFile()

  • CUMULUS-1987 - Deprecated the following functions:

    • @cumulus/cmrjs/getMetadata(cmrLink) -> @cumulus/cmr-client/CMR.getGranuleMetadata(cmrLink)
    • @cumulus/cmrjs/getFullMetadata(cmrLink)

1.23.1

Diff
lfrederick
 published 1.23.1 •
1
…
…
22
SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

About

Packages

npm

Go

Maven

PyPI

Rubygems

Stay in touch

Get open source security insights delivered straight into your inbox.

  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc