@cumulus/cmr-client

@cumulus/cmr-client

A Node.js client to read from, write to, and delete from NASA's Common Metadata Repository (CMR) API.

API

See API.md

CMR Docs

CMR REST API endpoint documentation is here:

• https://cmr.earthdata.nasa.gov/search/site/search_api_docs.html
• https://cmr.earthdata.nasa.gov/ingest/site/ingest_api_docs.html

About Cumulus

Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.

Cumulus Documentation

Test

Test with npm run test.

Contributing

To make a contribution, please see our contributing guidelines.

Changelog

[v7.1.0] 2021-03-12

Notable changes

• sync-granule task will now properly handle syncing 0 byte files to S3
• SQS/Kinesis rules now support scheduling workflows to a custom queue via the rule.queueUrl property. If provided, this value should be the full URL for an SQS queue.

Added

• tf-modules/cumulus module now supports a cmr_custom_host variable that can be used to set to an arbitrary host for making CMR requests (e.g. https://custom-cmr-host.com).
• Added buckets variable to tf-modules/archive
• CUMULUS-2345
  • Deploy ORCA with Cumulus, see example/cumulus-tf/orca.tf and example/cumulus-tf/terraform.tfvars.example
  • Add CopyToGlacier step to example IngestAndPublishGranule workflow
• CUMULUS-2424
  • Added childWorkflowMeta to queue-pdrs config. An object passed to this config value will be merged into a child workflow message's meta object. For an example of how this can be used, see example/cumulus-tf/discover_and_queue_pdrs_with_child_workflow_meta_workflow.asl.json.
• CUMULUS-2427
  • Added support for using a custom queue with SQS and Kinesis rules. Whatever queue URL is set on the rule.queueUrl property will be used to schedule workflows for that rule. This change allows SQS/Kinesis rules to use any throttled queues defined for a deployment.

Fixed

• CUMULUS-2394
  • Updated PDR and Granule writes to check the step function workflow_start_time against the createdAt field for each record to ensure old records do not overwrite newer ones

Changed

• <prefix>-lambda-api-gateway IAM role used by API Gateway Lambda now supports accessing all buckets defined in your buckets variable except "internal" buckets
• Updated the default scroll duration used in ESScrollSearch and part of the reconciliation report functions as a result of testing and seeing timeouts at its current value of 2min.
• CUMULUS-2355
  • Added logic to disable /s3Credentials endpoint based upon value for environment variable DISABLE_S3_CREDENTIALS. If set to "true", the endpoint will not dispense S3 credentials and instead return a message indicating that the endpoint has been disabled.
• CUMULUS-2397
  • Updated /elasticsearch endpoint's reindex function to prevent reindexing when source and destination indices are the same.
• CUMULUS-2420
  • Updated test function waitForAsyncOperationStatus to take a retryObject and use exponential backoff. Increased the total test duration for both AsycOperation specs and the ReconciliationReports tests.
  • Updated the default scroll duration used in ESScrollSearch and part of the reconciliation report functions as a result of testing and seeing timeouts at its current value of 2min.
• CUMULUS-2427
  • Removed queueUrl from the parameters object for @cumulus/message/Build.buildQueueMessageFromTemplate
  • Removed queueUrl from the parameters object for @cumulus/message/Build.buildCumulusMeta

Fixed

• Fixed issue in @cumulus/ingest/S3ProviderClient.sync() preventing 0 byte files from being synced to S3.

Removed

• Removed variables from tf-modules/archive:
  • private_buckets
  • protected_buckets
  • public_buckets