@cumulus/discover-granules

@cumulus/discover-granules

Discover Granules in FTP/HTTP/HTTPS/SFTP/S3 endpoints

About Cumulus

Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.

Cumulus Documentation

Contributing

To make a contribution, please see our contributing guidelines.

Changelog

[v10.1.0] 2022-02-23

Added

• CUMULUS-2775
  • Added a configurable parameter group for the RDS serverless database cluster deployed by tf-modules/rds-cluster-tf. The allowed parameters for the parameter group can be found in the AWS documentation of allowed parameters for an Aurora PostgreSQL cluster. By default, the following parameters are specified:
    • shared_preload_libraries: pg_stat_statements,auto_explain
    • log_min_duration_statement: 250
    • auto_explain.log_min_duration: 250
• CUMULUS-2781
  • Add api_config secret to hold API/Private API lambda configuration values
• CUMULUS-2840
  • Added an index on granule_cumulus_id to the RDS files table.

Changed

• CUMULUS-2492
  • Modify collectionId logic to accomodate trailing underscores in collection short names. e.g. shortName____
• CUMULUS-2847
  • Move DyanmoDb table name into API keystore and initialize only on lambda cold start
• CUMULUS-2833
  • Updates provider model schema titles to display on the dashboard.
• CUMULUS-2837
  • Update process-s3-dead-letter-archive to unpack SQS events in addition to Cumulus Messages
  • Update process-s3-dead-letter-archive to look up execution status using getCumulusMessageFromExecutionEvent (common method with sfEventSqsToDbRecords)
  • Move methods in api/lib/cwSfExecutionEventUtils to @cumulus/message/StepFunctions
• CUMULUS-2775
  • Changed the timeout_action to ForceApplyCapacityChange by default for the RDS serverless database cluster tf-modules/rds-cluster-tf
• CUMULUS-2781
  • Update API lambda to utilize api_config secret for initial environment variables

Fixed

• CUMULUS-2853
  • Move OAUTH_PROVIDER to lambda env variables to address regression in CUMULUS-2781
  • Add logging output to api app router
• Added Cloudwatch permissions to <prefix>-steprole in tf-modules/ingest/iam.tf to address the Error: error creating Step Function State Machine (xxx): AccessDeniedException: 'arn:aws:iam::XXX:role/xxx-steprole' is not authorized to create managed-rule error in non-NGAP accounts:
  • events:PutTargets
  • events:PutRule
  • events:DescribeRule