@cumulus/test-data

@cumulus/test-data

@cumulus/test-data provides a collection of example data for use in testing Cumulus modules.

About Cumulus

Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.

Cumulus Documentation

Installation

npm install @cumulus/test-data

Using data from this package

Using require or import:

const payload = require('@cumulus/test-data/payloads/new-message-schema/ingest.json');
import payload from '@cumulus/test-data/payloads/new-message-schema/ingest.json';

Contributing

To make a contribution, please see our contributing guidelines.

Changelog

[v5.0.0] 2021-01-12

BREAKING CHANGES

• CUMULUS-2020
  • Elasticsearch data mappings have been updated to improve search and the API has been update to reflect those changes. See Migration notes on how to update the Elasticsearch mappings.

Migration notes

• CUMULUS-2020
  • Elasticsearch data mappings have been updated to improve search. For example, case insensitive searching will now work (e.g. 'MOD' and 'mod' will return the same granule results). To use the improved Elasticsearch queries, reindex to create a new index with the correct types. Then perform a change index operation to use the new index.
• CUMULUS-2258
  • Because the egress_lambda_log_group and egress_lambda_log_subscription_filter resource were removed from the cumulus module, new definitions for these resources must be added to cumulus-tf/main.tf. For reference on how to define these resources, see example/cumulus-tf/thin_egress_app.tf.
  • The tea_stack_name variable being passed into the cumulus module should be removed
• CUMULUS-2344
  • Regarding instructions for CUMULUS-2020, you can now do a change index operation before a reindex operation. This will ensure that new data will end up in the new index while Elasticsearch is reindexing.

BREAKING CHANGES

• CUMULUS-2020
  • Elasticsearch data mappings have been updated to improve search and the API has been updated to reflect those changes. See Migration notes on how to update the Elasticsearch mappings.

Added

• CUMULUS-2318
  • Addedasync_operation_image as cumulus module variable to allow for override of the async_operation container image. Users can optionally specify a non-default docker image for use with Core async operations.
• CUMULUS-2219
  • Added lzards-backup Core task to facilitate making LZARDS backup requests in Cumulus ingest workflows
• CUMULUS-2092
  • Add documentation for Granule Not Found Reports
• HYRAX-320
  • @cumulus/hyrax-metadata-updatesAdd component URI encoding for entry title id and granule ur to allow for values with special characters in them. For example, EntryTitleId 'Sentinel-6A MF/Jason-CS L2 Advanced Microwave Radiometer (AMR-C) NRT Geophysical Parameters' Now, URLs generated from such values will be encoded correctly and parsable by HyraxInTheCloud
• CUMULUS-1370
  • Add documentation for Getting Started section including FAQs
• CUMULUS-2092
  • Add documentation for Granule Not Found Reports
• CUMULUS-2219
  • Added lzards-backup Core task to facilitate making LZARDS backup requests in Cumulus ingest workflows
• CUMULUS-2280
  • In local api, retry to create tables if they fail to ensure localstack has had time to start fully.
• CUMULUS-2290
  • Add queryFields to granule schema, and this allows workflow tasks to add queryable data to granule record. For reference on how to add data to queryFields field, see example/cumulus-tf/kinesis_trigger_test_workflow.tf.
• CUMULUS-2318
  • Addedasync_operation_image as cumulus module variable to allow for override of the async_operation container image. Users can optionally specify a non-default docker image for use with Core async operations.

Changed

• CUMULUS-2020
  • Updated Elasticsearch mappings to support case-insensitive search
• CUMULUS-2124
  • cumulus-rds-tf terraform module now takes engine_version as an input variable.
• CUMULUS-2279
  • Changed the formatting of granule CMR links: instead of a link to the /search/granules.json endpoint, now it is a direct link to /search/concepts/conceptid.format
• CUMULUS-2296
  • Improved PDR spec compliance of parse-pdr by updating @cumulus/pvl to parse fields in a manner more consistent with the PDR ICD, with respect to numbers and dates. Anything not matching the ICD expectations, or incompatible with Javascript parsing, will be parsed as a string instead.
• CUMULUS-2344
  • Elasticsearch API now allows you to reindex to an index that already exists
  • If using the Change Index operation and the new index doesn't exist, it will be created

Removed

• CUMULUS-2258
  • Removed tea_stack_name variable from tf-modules/distribution/variables.tf and tf-modules/cumulus/variables.tf
  • Removed egress_lambda_log_group and egress_lambda_log_subscription_filter resources from tf-modules/distribution/main.tf