@cumulus/test-data

@cumulus/test-data

@cumulus/test-data provides a collection of example data for use in testing Cumulus modules.

About Cumulus

Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.

Cumulus Documentation

Installation

npm install @cumulus/test-data

Using data from this package

Using require or import:

const payload = require('@cumulus/test-data/payloads/new-message-schema/ingest.json');
import payload from '@cumulus/test-data/payloads/new-message-schema/ingest.json';

Contributing

To make a contribution, please see our contributing guidelines.

Changelog

[v9.4.0] 2021-08-16

Notable changes

• @cumulus/sync-granule task should now properly handle syncing files from HTTP/HTTPS providers where basic auth is required and involves a redirect to a different host (e.g. downloading files protected by Earthdata Login)

Added

• CUMULUS-2591
  • Adds failedExecutionStepName to failed execution's jsonb error records. This is the name of the Step Function step for the last failed event in the execution's event history.
• CUMULUS-2548
  • Added allowed_redirects field to PostgreSQL providers table
  • Added allowedRedirects field to DynamoDB <prefix>-providers table
  • Added @cumulus/aws-client/S3.streamS3Upload to handle uploading the contents of a readable stream to S3 and returning a promise
• CUMULUS-2373
  • Added replaySqsMessages lambda to replay archived incoming SQS messages from S3.
  • Added /replays/sqs endpoint to trigger an async operation for the replaySqsMessages lambda.
  • Added unit tests and integration tests for new endpoint and lambda.
  • Added getS3PrefixForArchivedMessage to ingest/sqs package to get prefix for an archived message.
  • Added new async_operation type SQS Replay.
• CUMULUS-2460
  • Adds POST /executions/workflows-by-granules for retrieving workflow names common to a set of granules
  • Adds workflowsByGranules to @cumulus/api-client/executions
• CUMULUS-2635
  • Added helper functions:
    • @cumulus/db/translate/file/translateApiPdrToPostgresPdr

Fixed

• CUMULUS-2548
  • Fixed @cumulus/ingest/HttpProviderClient.sync to properly handle basic auth when redirecting to a different host and/or host with a different port
• CUMULUS-2626
  • Update PDR migration to correctly find Executions by a Dynamo PDR's execution field
• CUMULUS-2635
  • Update data-migration2 to migrate PDRs before migrating granules.
  • Update data-migration2 unit tests testing granules migration to reference PDR records to better model the DB schema.
  • Update migratePdrRecord to use translateApiPdrToPostgresPdr function.

Changed

• CUMULUS-2373
  • Updated getS3KeyForArchivedMessage in ingest/sqs to store SQS messages by queueName.
• CUMULUS-2630
  • Updates the example/cumulus-tf deployment to change archive_api_reserved_concurrency from 2 to 8 to prevent throttling with the dashboard.