@cumulus/discover-granules

@cumulus/discover-granules

Discover Granules in FTP/HTTP/HTTPS/SFTP/S3 endpoints

What is Cumulus?

Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.

Cumulus Documentation

Contributing

See Cumulus README

Changelog

[v1.5.2] - 2018-05-15

Fixed

• CUMULUS-514 - "Unable to Delete the Granules"
  • updated cmrjs.deleteConcept to return success if the record is not found in CMR.

Added

• CUMULUS-547 - The distribution API now includes an "earthdataLoginUsername" query parameter when it returns a signed S3 URL
• CUMULUS-527 - "parse-pdr queues up all granules and ignores regex"
  • Add an optional config property to the ParsePdr task called "granuleIdFilter". This property is a regular expression that is applied against the filename of the first file of each granule contained in the PDR. If the regular expression matches, then the granule is included in the output. Defaults to '.', which will match all granules in the PDR.
• File checksums in PDRs now support MD5
• Deployment support to subscribe to an SNS topic that already exists
• CUMULUS-470, CUMULUS-471 In-region S3 Policy lambda added to API to update bucket policy for in-region access.
• CUMULUS-533 Added fields to granule indexer to support EMS ingest and archive record creation
• CUMULUS-534 Track deleted granules
  • added deletedgranule type to cumulus index.
  • Important Note: Force custom bootstrap to re-run by adding this to app/config.yml es: elasticSearchMapping: 7
• You can now deploy cumulus without ElasticSearch. Just add es: null to your app/config.yml file. This is only useful for debugging purposes. Cumulus still requires ElasticSearch to properly operate.
• @cumulus/integration-tests includes and exports the addRules function, which seeds rules into the DynamoDB table.
• Added capability to support EFS in cloud formation template. Also added optional capability to ssh to your instance and privileged lambda functions.
• Added support to force discovery of PDRs that have already been processed and filtering of selected data types
• @cumulus/cmrjs uses an environment variable USER_IP_ADDRESS or fallback IP address of 10.0.0.0 when a public IP address is not available. This supports lambda functions deployed into a VPC's private subnet, where no public IP address is available.

Changed

• CUMULUS-550 Custom bootstrap automatically adds new types to index on deployment