@cumulus/cmrjs

@cumulus/cmrjs

A Node.js client for Cumulus-specific functionality for reading from, writing to, and deleting from NASA's Common Metadata Repository (CMR) API. General CMR functionality should go in @cumulus/cmr-client.

About Cumulus

Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.

Cumulus Documentation

Test

Test with npm run test.

CMR Docs

Documentation on the CMR REST API endpoints is here:

• https://cmr.earthdata.nasa.gov/search/site/search_api_docs.html
• https://cmr.earthdata.nasa.gov/ingest/site/ingest_api_docs.html

Contributing

To make a contribution, please see our contributing guidelines.

Changelog

[v9.4.0] 2021-08-16

Notable changes

• @cumulus/sync-granule task should now properly handle syncing files from HTTP/HTTPS providers where basic auth is required and involves a redirect to a different host (e.g. downloading files protected by Earthdata Login)

Added

• CUMULUS-2591
  • Adds failedExecutionStepName to failed execution's jsonb error records. This is the name of the Step Function step for the last failed event in the execution's event history.
• CUMULUS-2548
  • Added allowed_redirects field to PostgreSQL providers table
  • Added allowedRedirects field to DynamoDB <prefix>-providers table
  • Added @cumulus/aws-client/S3.streamS3Upload to handle uploading the contents of a readable stream to S3 and returning a promise
• CUMULUS-2373
  • Added replaySqsMessages lambda to replay archived incoming SQS messages from S3.
  • Added /replays/sqs endpoint to trigger an async operation for the replaySqsMessages lambda.
  • Added unit tests and integration tests for new endpoint and lambda.
  • Added getS3PrefixForArchivedMessage to ingest/sqs package to get prefix for an archived message.
  • Added new async_operation type SQS Replay.
• CUMULUS-2460
  • Adds POST /executions/workflows-by-granules for retrieving workflow names common to a set of granules
  • Adds workflowsByGranules to @cumulus/api-client/executions
• CUMULUS-2635
  • Added helper functions:
    • @cumulus/db/translate/file/translateApiPdrToPostgresPdr

Fixed

• CUMULUS-2548
  • Fixed @cumulus/ingest/HttpProviderClient.sync to properly handle basic auth when redirecting to a different host and/or host with a different port
• CUMULUS-2626
  • Update PDR migration to correctly find Executions by a Dynamo PDR's execution field
• CUMULUS-2635
  • Update data-migration2 to migrate PDRs before migrating granules.
  • Update data-migration2 unit tests testing granules migration to reference PDR records to better model the DB schema.
  • Update migratePdrRecord to use translateApiPdrToPostgresPdr function.

Changed

• CUMULUS-2373
  • Updated getS3KeyForArchivedMessage in ingest/sqs to store SQS messages by queueName.
• CUMULUS-2630
  • Updates the example/cumulus-tf deployment to change archive_api_reserved_concurrency from 2 to 8 to prevent throttling with the dashboard.