@cumulus/integration-tests

@cumulus/integration-tests

This package provides a CLI and functions for testing Cumulus.

⚠️ The documented API of this package will not change without a deprecation warning being provided in earlier releases. Code in this package that is not documented in this README may change without warning, and is not considered part of the package's public API.

About Cumulus

Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.

Cumulus Documentation

Installation

$ npm install @cumulus/integration-tests

API

Modules

Collections

Executions

Granules

Providers

Rules

Collections

Example

const Collections = require('@cumulus/integration-test/Collections');

• Collections
  • createCollection(prefix, [overrides]) ⇒ Promise.<Object> ⏏
    • ~addCustomUrlPathToCollectionFiles(collection, customFilePath) ⇒ Array.<Object>
    • ~buildCollection(params) ⇒ Object
    • ~collectionExists(stackName, collection) ⇒ boolean
    • ~addCollection(stackName, collection) ⇒ Promise.<undefined>
    • ~addCollections(stackName, bucketName, dataDirectory, [postfix], [customFilePath], [duplicateHandling]) ⇒ Promise.<Array.<Object>>

createCollection(prefix, [overrides]) ⇒ Promise.<Object> ⏏

Create a randomized collection using the Cumulus API.

The default collection is very simple. It expects that, for any discovered file, the granule ID is everything in the filename before the extension. For example, a file named gran-1.txt would have a granuleId of gran-1. Filenames can only contain a single . character.

Collection defaults:

• name: random string starting with collection-name-
• version: random string starting with collection-version-
• reportToEms: false
• granuleId: '^[^.]+$'
• granuleIdExtraction: '^([^.]+)\..+$'
• sampleFileName: 'asdf.jpg'
• files:

  [
    {
      bucket: 'protected',
      regex: '^[^.]+\..+$',
      sampleFileName: 'asdf.jpg'
    }
  ]
  

Kind: Exported function
Returns: Promise.<Object> - the generated collection

Param	Type	Description
prefix	string	the Cumulus stack name
[overrides]	Object	properties to set on the collection, overriding the defaults

createCollection~addCustomUrlPathToCollectionFiles(collection, customFilePath) ⇒ Array.<Object>

Given a Cumulus collection configuration, return a list of the filetype configs with their url_paths updated.

Kind: inner method of createCollection
Returns: Array.<Object> - a list of collection filetype configs

Param	Type	Description
collection	Object	a Cumulus collection
customFilePath	string	path to be added to the end of the url_path

createCollection~buildCollection(params) ⇒ Object

Update a collection with a custom file path, duplicate handling, and name updated with the postfix.

Kind: inner method of createCollection
Returns: Object - an updated collection

Param	Type	Description
params	Object
params.collection	Object	a collection configuration
params.customFilePath	string	path to be added to the end of the url_path
params.duplicateHandling	string	duplicate handling setting
params.postfix	string	a string to be appended to the end of the name

createCollection~collectionExists(stackName, collection) ⇒ boolean

Returns true if collection exists. False otherwise.

Kind: inner method of createCollection

Param	Type	Description
stackName	string	the prefix of the Cumulus stack
collection	Object	a Cumulus collection

createCollection~addCollection(stackName, collection) ⇒ Promise.<undefined>

Add a new collection to Cumulus

Kind: inner method of createCollection

Param	Type	Description
stackName	string	the prefix of the Cumulus stack
collection	Object	a Cumulus collection

createCollection~addCollections(stackName, bucketName, dataDirectory, [postfix], [customFilePath], [duplicateHandling]) ⇒ Promise.<Array.<Object>>

Add collections to database

Kind: inner method of createCollection
Returns: Promise.<Array.<Object>> - - collections that were added

Param	Type	Description
stackName	string	Cloud formation stack name
bucketName	string	S3 internal bucket name
dataDirectory	string	the directory of collection json files
[postfix]	string	string to append to collection name
[customFilePath]	string
[duplicateHandling]	string

Executions

Example

const Executions = require('@cumulus/integration-test/Executions');

• Executions
  • findExecutionArn(prefix, matcher, [queryParameters], [options]) ⇒ Promise.<string> ⏏
  • getExecutionWithStatus(params) ⇒ Promise.<Object> ⏏

findExecutionArn(prefix, matcher, [queryParameters], [options]) ⇒ Promise.<string> ⏏

Find the execution ARN matching the matcher function

Kind: Exported function
Returns: Promise.<string> - the ARN of the matching execution

Param	Type	Default	Description
prefix	string		the name of the Cumulus stack
matcher	function		a predicate function that takes an execution and determines if this is the execution that is being searched for
[queryParameters]	Object		Optional query parameters
[options]	Object
[options.timeout]	integer	0	the number of seconds to wait for a matching execution to be found

getExecutionWithStatus(params) ⇒ Promise.<Object> ⏏

Wait for an execution to have an expected status and return the execution

Kind: Exported function
Returns: Promise.<Object> - the execution as returned by the GET /executions/<execution-arn> endpoint

Param	Type	Default	Description
params	Object
params.prefix	string		the name of the Cumulus stack
params.arn	string		the execution ARN to fetch
params.status	string		the status to wait for
[params.callback]	function	cumulusApiClient.invokeApifunction	an async function to invoke the API Lambda that takes a prefix / user payload
[params.timeout]	integer	30	the number of seconds to wait for the execution to reach a terminal state

Granules

Example

const Granules = require('@cumulus/integration-test/Granules');

• Granules
  • getGranuleWithStatus(params) ⇒ Promise.<Object> ⏏
    • ~waitForListGranulesResult(params) ⇒ Promise.<Object>

getGranuleWithStatus(params) ⇒ Promise.<Object> ⏏

Wait for a granule to have an expected status and return the granule

Kind: Exported function
Returns: Promise.<Object> - the granule as returned by the GET /granules/<granule-id> endpoint

Param	Type	Default	Description
params	Object
params.prefix	string		the name of the Cumulus stack
params.granuleId	string		the granuleId of the granule
params.collectionId	string		the collectionId of the granule
params.status	string		the status to wait for
params.updatedAt	string		minimum updatedAt time the granule must have to return
[params.callback]	function	cumulusApiClient.invokeApifunction	an async function to invoke the API Lambda that takes a prefix / user payload
[params.timeout]	integer	30	the number of seconds to wait for the execution to reach a terminal state

getGranuleWithStatus~waitForListGranulesResult(params) ⇒ Promise.<Object>

Wait for listGranules to return at least a single value before returning an empty result

Kind: inner method of getGranuleWithStatus
Returns: Promise.<Object> - - results of a successful listGranules

Param	Type	Description
params	Object	parameters to listGranules function

Providers

Example

const Providers = require('@cumulus/integration-test/Providers');

createProvider(prefix, [overrides]) ⇒ Promise.<Object> ⏏

Create a provider using the Cumulus API

Provider defaults:

• id: random string starting with provider-
• protocol: s3
• globalConnectionLimit: 10

Kind: Exported function
Returns: Promise.<Object> - the generated provider

Param	Type	Description
prefix	string	the Cumulus stack name
[overrides]	Object	properties to set on the provider, overriding the defaults

Rules

Example

const Rules = require('@cumulus/integration-test/Rules');

createOneTimeRule(prefix, [overrides]) ⇒ Promise.<Object> ⏏

Create a onetime rule using the Cumulus API

Rule defaults:

• name: random string starting with rule_
• rule: { type: 'onetime' }

Kind: Exported function
Returns: Promise.<Object> - the generated rule

Param	Type	Description
prefix	string	the name of the Cumulus stack
[overrides]	Object	properties to set on the rule, overriding the defaults

CLI Usage

Usage: cumulus-test TYPE COMMAND [options]


  Options:

    -V, --version                   output the version number
    -s, --stack-name <stackName>    AWS Cloud Formation stack name (default: null)
    -b, --bucket-name <bucketName>  AWS S3 internal bucket name (default: null)
    -w, --workflow <workflow>       Workflow name (default: null)
    -i, --input-file <inputFile>    Workflow input JSON file (default: null)
    -h, --help                      output usage information


  Commands:

    workflow  Execute a workflow and determine if the workflow completes successfully

For example, to test the HelloWorld workflow:

cumulus-test workflow --stack-name helloworld-cumulus --bucket-name cumulus-bucket-internal --workflow HelloWorldWorkflow --input-file ./helloWorldInput.json

Contributing

To make a contribution, please see our contributing guidelines.

Generated automatically using npm run build-docs