@cumulus/common - npm Package Versions

[v1.8.1] - 2018-08-01

Note IAM roles should be re-deployed with this release.

• Cumulus-726
  • Added function to @cumulus/integration-tests: sfnStep includes getStepInput which returns the input to the schedule event of a given step function step.
  • Added IAM policy @cumulus/deployment: Lambda processing IAM role includes kinesis::PutRecord so step function lambdas can write to kinesis streams.
• Cumulus Community Edition
  • Added Google OAuth authentication token logic to @cumulus/api. Refactored token endpoint to use environment variable flag OAUTH_PROVIDER when determining with authentication method to use.
  • Added API Lambda memory configuration variable api_lambda_memory to @cumulus/api and @cumulus/deployment.

Changed

• Cumulus-726
  • Changed function in @cumulus/api: models/rules.js#addKinesisEventSource was modified to call to deleteKinesisEventSource with all required parameters (rule's name, arn and type).
  • Changed function in @cumulus/integration-tests: getStepOutput can now be used to return output of failed steps. If users of this function want the output of a failed event, they can pass a third parameter eventType as 'failure'. This function will work as always for steps which completed successfully.

Removed

• Cumulus-726

  • Configuration change to @cumulus/deployment: Removed default auto scaling configuration for Granules and Files DynamoDB tables.

• CUMULUS-688

  • Add integration test for ExecutionStatus
  • Function addition to @cumulus/integration-tests: api includes getExecutionStatus which returns the execution status from the Cumulus API

[v1.7.1] - 2018-07-27 - [BACKPORT]

Fixed

• CUMULUS-766 - Backport from 1.8.0 - Populate database fileSize field from S3 if value not present in Ingest payload

[v1.8.0] - 2018-07-23

Added

• CUMULUS-718 Adds integration test for Kinesis triggering a workflow.

• GITC-776-3 Added more flexibility for rules. You can now edit all fields on the rule's record We may need to update the api documentation to reflect this.

• CUMULUS-681 - Add ingest-in-place action to granules endpoint

  • new applyWorkflow action at PUT /granules/{granuleid} Applying a workflow starts an execution of the provided workflow and passes the granule record as payload. Parameter(s):
    • workflow - the workflow name

• CUMULUS-685 - Add parent exeuction arn to the execution which is triggered from a parent step function

Changed

• CUMULUS-768 - Integration tests get S3 provider data from shared data folder

Fixed

• CUMULUS-746 - Move granule API correctly updates record in dynamo DB and cmr xml file
• CUMULUS-766 - Populate database fileSize field from S3 if value not present in Ingest payload

[v1.7.0] - 2018-07-02

[v1.6.0] - 2018-06-06

[v1.5.5] - 2018-05-30

Added

• CUMULUS-530 - PDR tracking through Queue-granules
  • Add optional pdr property to the sync-granule task's input config and output payload.
• CUMULUS-548 - Create a Lambda task that generates EMS distribution reports
  • In order to supply EMS Distribution Reports, you must enable S3 Server Access Logging on any S3 buckets used for distribution. See How Do I Enable Server Access Logging for an S3 Bucket? The "Target bucket" setting should point at the Cumulus internal bucket. The "Target prefix" should be "<STACK_NAME>/ems-distribution/s3-server-access-logs/", where "STACK_NAME" is replaced with the name of your Cumulus stack.

Fixed

• CUMULUS-546 - Kinesis Consumer should catch and log invalid JSON
  • Kinesis Consumer lambda catches and logs errors so that consumer doesn't get stuck in a loop re-processing bad json records.
• EMS report filenames are now based on their start time instead of the time instead of the time that the report was generated
• CUMULUS-552 - Cumulus API returns different results for the same collection depending on query
  • The collection, provider and rule records in elasticsearch are now replaced with records from dynamo db when the dynamo db records are updated.

Added

• @cumulus/deployment's default cloudformation template now configures storage for Docker to match the configured ECS Volume. The template defines Docker's devicemapper basesize (dm.basesize) using ecs.volumeSize. This addresses ECS default of limiting Docker containers to 10GB of storage (Read more).

[v1.5.4] - 2018-05-21

Added

• CUMULUS-535 - EMS Ingest, Archive, Archive Delete reports
  • Add lambda EmsReport to create daily EMS Ingest, Archive, Archive Delete reports
  • ems.provider property added to @cumulus/deployment/app/config.yml. To change the provider name, please add ems: provider property to app/config.yml.
• CUMULUS-480 Use DynamoDB to store granules, pdrs and execution records
  • Activate PointInTime feature on DynamoDB tables
  • Increase test coverage on api package
  • Add ability to restore metadata records from json files to DynamoDB
• CUMULUS-459 provide API endpoint for moving granules from one location on s3 to another

[v1.5.3] - 2018-05-18

Fixed

• CUMULUS-557 - "Add dataType to DiscoverGranules output"
  • Granules discovered by the DiscoverGranules task now include dataType
  • dataType is now a required property for granules used as input to the QueueGranules task
• CUMULUS-550 Update deployment app/config.yml to force elasticsearch updates for deleted granules

[v1.5.2] - 2018-05-15

Fixed

• CUMULUS-514 - "Unable to Delete the Granules"
  • updated cmrjs.deleteConcept to return success if the record is not found in CMR.

Added

• CUMULUS-547 - The distribution API now includes an "earthdataLoginUsername" query parameter when it returns a signed S3 URL
• CUMULUS-527 - "parse-pdr queues up all granules and ignores regex"
  • Add an optional config property to the ParsePdr task called "granuleIdFilter". This property is a regular expression that is applied against the filename of the first file of each granule contained in the PDR. If the regular expression matches, then the granule is included in the output. Defaults to '.', which will match all granules in the PDR.
• File checksums in PDRs now support MD5
• Deployment support to subscribe to an SNS topic that already exists
• CUMULUS-470, CUMULUS-471 In-region S3 Policy lambda added to API to update bucket policy for in-region access.
• CUMULUS-533 Added fields to granule indexer to support EMS ingest and archive record creation
• CUMULUS-534 Track deleted granules
  • added deletedgranule type to cumulus index.
  • Important Note: Force custom bootstrap to re-run by adding this to app/config.yml es: elasticSearchMapping: 7
• You can now deploy cumulus without ElasticSearch. Just add es: null to your app/config.yml file. This is only useful for debugging purposes. Cumulus still requires ElasticSearch to properly operate.
• @cumulus/integration-tests includes and exports the addRules function, which seeds rules into the DynamoDB table.
• Added capability to support EFS in cloud formation template. Also added optional capability to ssh to your instance and privileged lambda functions.
• Added support to force discovery of PDRs that have already been processed and filtering of selected data types
• @cumulus/cmrjs uses an environment variable USER_IP_ADDRESS or fallback IP address of 10.0.0.0 when a public IP address is not available. This supports lambda functions deployed into a VPC's private subnet, where no public IP address is available.

Changed

• CUMULUS-550 Custom bootstrap automatically adds new types to index on deployment

[v1.5.1] - 2018-04-23

Fixed

• add the missing dist folder to the hello-world task
• disable uglifyjs on the built version of the pdr-status-check (read: https://github.com/webpack-contrib/uglifyjs-webpack-plugin/issues/264)