@cumulus/common
Advanced tools
Common libraries used in Cumulus.
npm install @cumulus/common
Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.
To make a contribution, please see our contributing guidelines.
[v9.3.0] 2021-07-26
@cumulus/api-client will now throw an error if the status code
does not match the expected response (200 for most requests and 202 for a few requests that
trigger async operations). Previously the helpers in this package would return the response
regardless of the status code, so you may need to update any code using helpers from this
package to catch or to otherwise handle errors that you may encounter.archive_api_reserved_concurrency
terraform variable on the Cumulus module and increased if you are seeing throttling errors.
The default reserved concurrency value is 8.cmr_custom_host variable for cumulus module can now be used to configure Cumulus to
integrate with a custom CMR host name and protocol (e.g.
http://custom-cmr-host.com). Note that you must include a protocol
(http:// or `https://) if specifying a value for this variable.rds_connetion_heartbeat and it's
behavior has been replaced by a more robust database connection 'retry'
solution. Users can remove this value from their configuration, regardless
of value. See the Changed section notes on CUMULUS-2528 for more details.Added user doc describing new features related to the Cumulus dead letter archive.
CUMULUS-2327
CUMULUS-2460
POST /executions/search-by-granules for retrieving executions from a list of granules or granule querysearchExecutionsByGranules to @cumulus/api-client/executionsCUMULUS-2475
GET endpoint to distribution APICUMULUS-2463
PUT /granules reingest action allows a user to override the default execution
to use by providing an optional workflowName or executionArn parameter on
the request body.PUT /granules/bulkReingest action allows a user to override the default
execution/workflow combination to reingest with by providing an optional
workflowName on the request body.Adds workflowName and executionArn params to @cumulus/api-client/reingestGranules
CUMULUS-2476
HEAD Distribution requests replicating current behavior of TEACUMULUS-2478
CUMULUS-2486
CUMULUS-2487
CUMULUS-2569
CUMULUS-2568
deletePdr/PDR deletion functionality to @cumulus/api-client/pdrsremoveCollectionAndAllDependencies to integration test helpersexample/spec/apiUtils.waitForApiStatus to wait for a
record to be returned by the API with a specific value for
statusexample/spec/discoverUtils.uploadS3GranuleDataForDiscovery to upload granule data fixtures
to S3 with a randomized granule ID for discover-granules based
integration testsexample/spec/Collections.removeCollectionAndAllDependencies to remove a collection and
all dependent objects (e.g. PDRs, granules, executions) from the
database via the API@cumulus/api-client:
pdrs.deletePdr - Delete a PDR via the APIreplays.postKinesisReplays - Submit a POST request to the /replays endpoint for replaying Kinesis messages@cumulus/api-client/granules.getGranuleResponse to return the raw endpoint response from the GET /granules/<granuleId> endpoint
@cumulus/integration-tests to example/spec/helpers/workflowUtils:
startWorkflowExecutionstartWorkflowexecuteWorkflowbuildWorkflowtestWorkflowbuildAndExecuteWorkflowbuildAndStartWorkflowexample/spec/helpers/workflowUtils.executeWorkflow now uses
waitForApiStatus to ensure that the execution is completed or
failed before resolvingexample/spec/helpers/testUtils.updateAndUploadTestFileToBucket
now accepts an object of parameters rather than positional
argumentspayload in the input payload test fixture for reconciliation report integration testsexample/spec/parallel/ingest/ingestFromPdrSpec.jsexample/spec/parallel/ingest/ingestFromPdrWithChildWorkflowMetaSpec.jsexample/spec/parallel/ingest/ingestFromPdrWithExecutionNamePrefixSpec.jsexample/spec/parallel/ingest/ingestPdrWithNodeNameSpec.js@cumulus/api-client/CumulusApiClientError error class to include new properties that can be accessed directly on
the error object:
statusCode - The HTTP status code of the API responseapiMessage - The message from the API responseparams.pRetryOptions parameter to
@cumulus/api-client/granules.deleteGranule to control the retry
behaviorcmr_custom_host variable to accept a full protocol and host name
(e.g. http://cmr-custom-host.com), whereas it previously only accepted a host nameexample/cumulus-tf deployment to the new Cumulus Distributionexample/README.mdrds_connection_heartbeat as a configuration option from all
Cumulus terraform modulesdbHeartBeat as an environmental switch from
@cumulus/db.getKnexClient in favor of more comprehensive general db
connect retry solutionrds_connection_timing_configuration string map to allow for
configuration and tuning of Core's internal database retry/connection
timeout behaviors. These values map to connection pool configuration
values for tarn (https://github.com/vincit/tarn.js/) which Core's database
module / knex(https://www.npmjs.com/package/knex) use for this purpose:
@cumulus/db and all terraform mdules to set default retry
configuration values for the database module to cover existing database
heartbeat connection failures as well as all other knex/tarn connection
creation failures.cmr_custom_host variable was not properly forwarded into archive, ingest, and sqs-message-remover modules from cumulus moduleparse-pdr set a granule's provider to the entire provider record when a NODE_NAME
is present. Expected behavior consistent with other tasks is to set the provider name in that field.@cumulus/api-client/pdrs.getPdr to request correct endpoint for returning a PDR from the APIpublished: true and with a CMR link in the Dynamo/PostgreSQL databases. Now,
the CMR deletion and the Dynamo/PostgreSQL record updates will all succeed or fail
together, preventing the database records from being out of sync with CMR.@cumulus/api-client/pdrs.getPdr to request correct
endpoint for returning a PDR from the APIFAQs
Common utilities used across tasks
The npm package @cumulus/common receives a total of 438 weekly downloads. As such, @cumulus/common popularity was classified as not popular.
We found that @cumulus/common demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 0 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
A supply chain attack on Rspack's npm packages injected cryptomining malware, potentially impacting thousands of developers.
Research
Security News
Socket researchers discovered a malware campaign on npm delivering the Skuld infostealer via typosquatted packages, exposing sensitive data.
Security News
Sonar’s acquisition of Tidelift highlights a growing industry shift toward sustainable open source funding, addressing maintainer burnout and critical software dependencies.