@cumulus/cmrjs

@cumulus/cmrjs

A Node.js client to read from, write to, and delete from NASA's Common Metadata Repository (CMR) API.

What is Cumulus?

Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.

Cumulus Documentation

Test

Test with npm run test.

CMR Docs

Documentation on the CMR REST API endpoints is here:

• https://cmr.earthdata.nasa.gov/search/site/search_api_docs.html
• https://cmr.earthdata.nasa.gov/ingest/site/ingest_api_docs.html

Contributing

See Cumulus README

Changelog

[v1.9.1] - 2018-08-22

Please Note To take advantage of the added granule tracking API functionality, updates are required for the message adapter and its libraries. You should be on the following versions:

• cumulus-message-adapter 1.0.9+
• cumulus-message-adapter-js 1.0.4+
• cumulus-message-adapter-java 1.2.7+
• cumulus-message-adapter-python 1.0.5+

Added

• CUMULUS-687 Added logs endpoint to search for logs from a specific workflow execution in @cumulus/api. Added integration test.
• CUMULUS-836 - @cumulus/deployment supports a configurable docker storage driver for ECS. ECS can be configured with either devicemapper (the default storage driver for AWS ECS-optimized AMIs) or overlay2 (the storage driver used by the NGAP 2.0 AMI). The storage driver can be configured in app/config.yml with ecs.docker.storageDriver: overlay2 | devicemapper. The default is overlay2.
  • To support this configuration, a Handlebars helper ifEquals was added to packages/deployment/lib/kes.js.
• CUMULUS-836 - @cumulus/api added IAM roles required by the NGAP 2.0 AMI. The NGAP 2.0 AMI runs a script register_instances_with_ssm.py which requires the ECS IAM role to include ec2:DescribeInstances and ssm:GetParameter permissions.

Fixed

• CUMULUS-836 - @cumulus/deployment uses overlay2 driver by default and does not attempt to write --storage-opt dm.basesize to fix this error.
• CUMULUS-413 Kinesis processing now captures all errors.
  • Added kinesis fallback mechanism when errors occur during record processing.
  • Adds FallbackTopicArn to @cumulus/api/lambdas.yml
  • Adds fallbackConsumer lambda to @cumulus/api
  • Adds fallbackqueue option to lambda definitions capture lambda failures after three retries.
  • Adds kinesisFallback SNS topic to signal incoming errors from kinesis stream.
  • Adds kinesisFailureSQS to capture fully failed events from all retries.
• CUMULUS-855 Adds integration test for kinesis' error path.
• CUMULUS-686 Added workflow task name and version tracking via @cumulus/api executions endpoint under new tasks property, and under workflow_tasks in step input/output.
  • Depends on cumulus-message-adapter 1.0.9+, cumulus-message-adapter-js 1.0.4+, cumulus-message-adapter-java 1.2.7+ and cumulus-message-adapter-python 1.0.5+
• CUMULUS-771
  • Updated sync-granule to stream the remote file to s3
  • Added integration test for ingesting granules from ftp provider
  • Updated http/https integration tests for ingesting granules from http/https providers
• CUMULUS-862 Updated @cumulus/integration-tests to handle remote lambda output
• CUMULUS-856 Set the rule state to have default value ENABLED

Changed

• In @cumulus/deployment, changed the example app config.yml to have additional IAM roles