@cumulus/aws-client
Advanced tools
Utilities for working with AWS. These utilities can be used for interacting with live AWS services or Localstack. For ease of setup, testing, and credential management, code interacting with AWS services should use the helpers in this module.
⚠️ The documented API of this package will not change without a deprecation warning being provided in earlier releases. Code in this package that is not documented in this README may change without warning, and is not considered part of the package's public API.
npm install @cumulus/aws-client
To use these utilities with Localstack, make sure you have a running instance of Localstack and set this environment variable:
NODE_ENV=test
Class to efficiently search all of the items in a DynamoDB table, without loading them all into memory at once. Handles paging.
Class to efficiently list all of the objects in an S3 bucket, without loading them all into memory at once. Handles paging of listS3ObjectsV2 requests.
Class to use when interacting with S3
Promise.<Object>Create a KMS key
See https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/KMS.html#createKey-property for allowed params and return value.
Promise.<string>Encrypt a string using KMS
stringDecrypt a KMS-encrypted string, Base 64 encoded
Promise.<CloudFormation.Stack>Promise.<CloudFormation.StackResources>Promise.<Object>Promise.<CloudFormation.Stack>Describes a given CloudFormation stack
Kind: inner method of CloudFormation
Returns: Promise.<CloudFormation.Stack> - The resources belonging to the stack
| Param | Type | Description |
|---|---|---|
| StackName | string | The name of the CloudFormation stack to query |
Promise.<CloudFormation.StackResources>Describes the resources belonging to a given CloudFormation stack
See CloudFormation.StackResources
Kind: inner method of CloudFormation
Returns: Promise.<CloudFormation.StackResources> - The resources belonging to the stack
| Param | Type | Description |
|---|---|---|
| StackName | string | The name of the CloudFormation stack to query |
Promise.<Object>Get parameter values for the given CloudFormation stack
Kind: inner method of CloudFormation
Returns: Promise.<Object> - Object keyed by parameter names
| Param | Type | Description |
|---|---|---|
| stackName | string | The name of the CloudFormation stack to query |
| parameterKeys | Array.<string> | Key names for the stack parameters that you want to return |
Promise.<CloudWatchEvents.PutRuleResponse>Create a CloudWatch Events rule
Kind: inner method of CloudwatchEvents
| Param | Type | Description |
|---|---|---|
| name | string | the rule name |
| schedule | string | a ScheduleExpression |
| state | string | the state of the rule |
| [description] | string | |
| [role] | string | a Role ARN |
Promise.<Object>Promise.<Object>PromisePromise.<Object>PromisePromise.<Object>Call DynamoDb client scan
See DocumentClient.scan()
for descriptions of params and the return data.
Kind: static property of DynamoDb
| Param | Type |
|---|---|
| params | Object |
Promise.<Object>Create a DynamoDB table and then wait for the table to exist
Kind: static method of DynamoDb
Returns: Promise.<Object> - the output of the createTable call
| Param | Type | Description |
|---|---|---|
| params | Object | the same params that you would pass to AWS.createTable See https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-dynamodb/classes/dynamodb.html#createtable |
PromiseDelete a DynamoDB table and then wait for the table to not exist
Kind: static method of DynamoDb
| Param | Type | Description |
|---|---|---|
| params | Object | the same params that you would pass to AWS.deleteTable See https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/clients/client-dynamodb/classes/dynamodb.html#deletetable |
Promise.<Object>Call DynamoDb client get
See DocumentClient.get()
for descriptions of params and the return data.
Kind: inner method of DynamoDb
Throws:
RecordDoesNotExist if a record cannot be found| Param | Type | Description |
|---|---|---|
| params | Object | |
| params.tableName | string | Table name to read |
| params.item | GetCommandInput.Key | Key identifying object to get |
| params.client | DynamoDBDocument | Instance of a DynamoDb DocumentClient |
| params.getParams | Object | Additional parameters for DocumentClient.get() |
PromiseDo a parallel scan of DynamoDB table using a document client.
See https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Scan.html#Scan.ParallelScan. See DocumentClient.scan().
Kind: inner method of DynamoDb
| Param | Type | Description |
|---|---|---|
| params | Object | |
| params.totalSegments | number | Total number of segments to divide table into for parallel scanning |
| params.scanParams | ScanInput | Params for the DynamoDB client scan operation See https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Scan.html |
| params.processItemsFunc | function | Function used to process returned items by scan |
| [params.dynamoDbClient] | DynamoDBDocument | Instance of Dynamo DB document client |
| [params.retryOptions] | pRetry.Options | Retry options for scan operations |
Promise.<Object>Describe a Kinesis stream.
Kind: inner method of Kinesis
Returns: Promise.<Object> - The stream description response
| Param | Type | Description |
|---|---|---|
| params | Object | |
| params.StreamName | string | A Kinesis stream name |
| retryOptions | Object | Options passed to p-retry module |
Bare check for EventBridge shape
Kind: inner method of Lambda
Invoke a Lambda function
Kind: inner method of Lambda
Promise.<Array>stringObjectstringstringPromisePromise.<boolean>PromisePromisePromise.<boolean>Array.<string>PromisePromisePromise.<{etag: string}>Promise.<undefined>Deprecated
Gets an object from S3.
Kind: static property of S3
Delete a bucket and all of its objects from S3
Kind: static property of S3
Returns: the promised result of S3.deleteBucket
| Param | Description |
|---|---|
| bucket | name of the bucket |
Promise.<Array>Fetch complete list of S3 objects
listObjectsV2 is limited to 1,000 results per call. This function continues listing objects until there are no more to be fetched.
The passed params must be compatible with the listObjectsV2 call.
https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#listObjectsV2-property
Kind: static method of S3
Returns: Promise.<Array> - resolves to an array of objects corresponding to
the Contents property of the listObjectsV2 response
| Param | Type | Description |
|---|---|---|
| params | Object | params for the s3.listObjectsV2 call |
Fetch lazy list of S3 objects
listObjectsV2 is limited to 1,000 results per call. This function continues listing objects until there are no more to be fetched.
The passed params must be compatible with the listObjectsV2 call.
https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#listObjectsV2-property
Kind: static method of S3
| Param | Description |
|---|---|
| params | params for the s3.listObjectsV2 call |
stringJoin strings into an S3 key without a leading slash
Kind: inner method of S3
Returns: string - the full S3 key
| Param | Type | Description |
|---|---|---|
| ...args | string | Array.<string> | the strings to join |
Objectparse an s3 uri to get the bucket and key
Kind: inner method of S3
Returns: Object - Returns an object with Bucket and Key properties
| Param | Type | Description |
|---|---|---|
| uri | string | must be a uri with the s3:// protocol |
stringGiven a bucket and key, return an S3 URI
Kind: inner method of S3
Returns: string - an S3 URI
| Param | Type | Description |
|---|---|---|
| bucket | string | an S3 bucket name |
| key | string | an S3 key |
stringConvert S3 TagSet Object to query string e.g. [{ Key: 'tag', Value: 'value }] to 'tag=value'
Kind: inner method of S3
Returns: string - tags query string
| Param | Type | Description |
|---|---|---|
| tagset | Array.<Object> | S3 TagSet array |
PromiseDelete an object from S3
Kind: inner method of S3
Returns: Promise - promise of the object being deleted
| Param | Type | Description |
|---|---|---|
| bucket | string | bucket where the object exists |
| key | string | key of the object to be deleted |
Get an object header from S3
Kind: inner method of S3
Returns: returns response from S3.headObject as a promise
| Param | Description |
|---|---|
| Bucket | name of bucket |
| Key | key for object (filepath + filename) |
| retryOptions | options to control retry behavior when an object does not exist. See https://github.com/tim-kos/node-retry#retryoperationoptions By default, retries will not be performed |
Promise.<boolean>Test if an object exists in S3
Kind: inner method of S3
Returns: Promise.<boolean> - a Promise that will resolve to a boolean indicating
if the object exists
| Param | Description |
|---|---|
| params | same params as https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#headObject-property |
Wait for an object to exist in S3
Kind: inner method of S3
| Param | Default | Description |
|---|---|---|
| params.interval | 1000 | interval before retries, in ms |
| params.timeout | 30000 | timeout, in ms |
Put an object on S3
Kind: inner method of S3
| Param | Description |
|---|---|
| params | same params as https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#putObject-property |
Upload a file to S3
Kind: inner method of S3
Copy an object from one location on S3 to another
Kind: inner method of S3
Upload data to S3
see https://github.com/aws/aws-sdk-js-v3/tree/main/lib/lib-storage
Kind: inner method of S3
Upload data to S3 using a stream
Kind: inner method of S3
Get a readable stream for an S3 object
Kind: inner method of S3
Downloads the given s3Obj to the given filename in a streaming manner
Kind: inner method of S3
Get the size of an S3 object
Kind: inner method of S3
Get object Tagging from S3
Kind: inner method of S3
Puts object Tagging in S3 https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#putObjectTagging-property
Kind: inner method of S3
Gets an object from S3.
Kind: inner method of S3
Get an object from S3, waiting for it to exist and, if specified, have the correct ETag.
Kind: inner method of S3
Transform streaming response from S3 object to text content
Kind: inner method of S3
Fetch the contents of an S3 object
Kind: inner method of S3
Fetch JSON stored in an S3 object
Kind: inner method of S3
Check if a file exists in an S3 object
Kind: inner method of S3
Delete files from S3
Kind: inner method of S3
| Param | Description |
|---|---|
| s3Objs | An array of objects containing keys 'Bucket' and 'Key' |
PromiseUpload the file associated with the given stream to an S3 bucket
Kind: inner method of S3
Returns: Promise - A promise
| Param | Type | Description |
|---|---|---|
| fileStream | ReadableStream | The stream for the file's contents |
| bucket | string | The S3 bucket to which the file is to be uploaded |
| key | string | The key to the file in the bucket |
| s3opts | Object | Options to pass to the AWS sdk call (defaults to {}) |
List the objects in an S3 bucket
Kind: inner method of S3
PromiseDelete a list of buckets and all of their objects from S3
Kind: inner method of S3
Returns: Promise - the promised result of S3.deleteBucket
| Param | Type | Description |
|---|---|---|
| buckets | Array | list of bucket names |
Calculate the cryptographic hash of an S3 object
Kind: inner method of S3
| Param | Type | Description |
|---|---|---|
| params | Object | |
| params.s3 | S3 | an S3 instance |
| params.algorithm | string | cksum, or an algorithm listed in openssl list -digest-algorithms |
| params.bucket | string | |
| params.key | string |
Promise.<boolean>Validate S3 object checksum against expected sum
Kind: inner method of S3
Returns: Promise.<boolean> - returns true for success
Throws:
InvalidChecksum - Throws error if validation fails| Param | Type | Description |
|---|---|---|
| params | Object | params |
| params.algorithm | string | checksum algorithm |
| params.bucket | string | S3 bucket |
| params.key | string | S3 key |
| params.expectedSum | number | string | expected checksum |
| [params.options] | Object | crypto.createHash options |
Array.<string>Extract the S3 bucket and key from the URL path parameters
Kind: inner method of S3
Returns: Array.<string> - [Bucket, Key]
| Param | Type | Description |
|---|---|---|
| pathParams | string | path parameters from the URL bucket/key in the form of |
PromiseCreate an S3 bucket
Kind: inner method of S3
| Param | Type | Description |
|---|---|---|
| Bucket | string | the name of the S3 bucket to create |
PromiseCreate multiple S3 buckets
Kind: inner method of S3
| Param | Type | Description |
|---|---|---|
| buckets | Array.<string> | the names of the S3 buckets to create |
Promise.<{etag: string}>Copy an S3 object to another location in S3 using a multipart copy
Kind: inner method of S3
Returns: Promise.<{etag: string}> - object containing the ETag of the
destination object
note: this method may error if used with zero byte files. see CUMULUS-2557 and https://github.com/nasa/cumulus/pull/2117.
| Param | Type | Default | Description |
|---|---|---|---|
| params | Object | ||
| params.sourceBucket | string | ||
| params.sourceKey | string | ||
| params.destinationBucket | string | ||
| params.destinationKey | string | ||
| [params.sourceObject] | S3.HeadObjectOutput | Output from https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#headObject-property | |
| [params.ACL] | string | an S3 Canned ACL | |
| [params.copyTags] | boolean | false | |
| [params.chunkSize] | number | chunk size of the S3 multipart uploads |
Copy an S3 object to another location in S3
Kind: inner method of S3
| Param | Default | Description |
|---|---|---|
| params.ACL | an S3 Canned ACL | |
| params.copyTags | false | |
| params.chunkSize | chunk size of the S3 multipart uploads |
Promise.<undefined>Move an S3 object to another location in S3
Kind: inner method of S3
| Param | Type | Default | Description |
|---|---|---|---|
| [params.ACL] | string | an S3 Canned ACL | |
| [params.copyTags] | boolean | false | |
| [params.chunkSize] | number | chunk size of the S3 multipart uploads |
Promise.<undefined>Promise.<undefined>Publish a message to an SNS topic. Does not catch errors, to allow more specific handling by the caller.
Kind: inner method of SNS
| Param | Description |
|---|---|
| snsTopicArn | SNS topic ARN |
| message | Message object |
| retryOptions | options to control retry behavior when publishing a message fails. See https://github.com/tim-kos/node-retry#retryoperationoptions |
Create an SNS topic with a given name.
Kind: inner method of SNS
Returns: - ARN of the created SNS topic
| Param | Description |
|---|---|
| snsTopicName | Name of the SNS topic |
Create an SQS Queue. Properly handles localstack queue URLs
Kind: inner method of SQS
Send a message to AWS SQS
Kind: inner method of SQS
Receives SQS messages from a given queue. The number of messages received can be set and the timeout is also adjustable.
Kind: inner method of SQS
Bare check for SQS message Shape
Kind: inner method of SQS
Extract SQS message body
Kind: inner method of SQS
Delete a given SQS message from a given queue.
Kind: inner method of SQS
Test if an SQS queue exists
Kind: inner method of SQS
Promise.<DescribeExecutionOutput>Promise.<DescribeStateMachineOutput>Promise.<GetExecutionHistoryOutput>Promise.<Object>Promise.<boolean>Promise.<DescribeExecutionOutput>Call StepFunctions DescribeExecution
See StepFunctions.describeExecution()
for descriptions of params and the return data.
If a ThrottlingException is received, this function will retry using an exponential backoff.
Kind: static method of StepFunctions
| Param | Type |
|---|---|
| params | DescribeExecutionInput |
Promise.<DescribeStateMachineOutput>Call StepFunctions DescribeStateMachine
See StepFunctions.describeStateMachine()
for descriptions of params and the return data.
If a ThrottlingException is received, this function will retry using an exponential backoff.
Kind: static method of StepFunctions
| Param | Type |
|---|---|
| params | DescribeStateMachineInput |
Promise.<GetExecutionHistoryOutput>Call StepFunctions GetExecutionHistory
See StepFunctions.getExecutionHistory()
for descriptions of params and the return data.
If a ThrottlingException is received, this function will retry using an exponential backoff.
Kind: static method of StepFunctions
| Param | Type |
|---|---|
| params | GetExecutionHistoryInput |
Promise.<Object>Call StepFunctions ListExecutions
See StepFunctions.listExecutions()
for descriptions of params and the return data.
If a ThrottlingException is received, this function will retry using an exponential backoff.
Kind: static method of StepFunctions
| Param | Type |
|---|---|
| params | Object |
Promise.<boolean>Check if a Step Function Execution exists
If a ThrottlingException is received, this function will retry using an exponential backoff.
Kind: inner method of StepFunctions
| Param | Type | Description |
|---|---|---|
| executionArn | string | the ARN of the Step Function Execution to check for |
Class to efficiently search all of the items in a DynamoDB table, without loading them all into memory at once. Handles paging.
Kind: global class
Promise.<Array>Drain all values from the searchQueue, and return to the user. Warning: This can be very memory intensive.
Kind: instance method of DynamoDbSearchQueue
Returns: Promise.<Array> - array of search results.
Promise.<Object>View the next item in the queue
This does not remove the object from the queue. When there are no more items in the queue, returns 'null'.
Kind: instance method of DynamoDbSearchQueue
Returns: Promise.<Object> - an item from the DynamoDB table
Promise.<Object>Remove the next item from the queue
When there are no more items in the queue, returns 'null'.
Kind: instance method of DynamoDbSearchQueue
Returns: Promise.<Object> - an item from the DynamoDB table
Class to efficiently list all of the objects in an S3 bucket, without loading them all into memory at once. Handles paging of listS3ObjectsV2 requests.
Kind: global class
Promise.<Object>View the next item in the queue
This does not remove the object from the queue. When there are no more items in the queue, returns 'null'.
Kind: instance method of S3ListObjectsV2Queue
Returns: Promise.<Object> - an S3 object description
Promise.<Object>Remove the next item from the queue
When there are no more items in the queue, returns 'null'.
Kind: instance method of S3ListObjectsV2Queue
Returns: Promise.<Object> - an S3 object description
Class to use when interacting with S3
Kind: global class
Promise.<string>Promise.<string>Promise.<string>Returns an HTTPS URL that can be used to perform a GET on the given object store URL
Kind: instance method of S3ObjectStore
Returns: Promise.<string> - a signed URL
Throws:
| Param | Type | Description |
|---|---|---|
| objectUrl | string | the URL of the object to sign |
| [options] | string | options to pass to S3.getObject |
| [queryParams] | string | a mapping of parameter key/values to put in the URL |
| presignOptions | RequestPresigningArguments | presignOptions |
Promise.<string>Returns an HTTPS URL that can be used to perform a HEAD on the given object store URL
Kind: instance method of S3ObjectStore
Returns: Promise.<string> - a signed URL
Throws:
| Param | Type | Description |
|---|---|---|
| objectUrl | string | the URL of the object to sign |
| [options] | string | options to pass to S3.getObject |
| [queryParams] | string | a mapping of parameter key/values to put in the URL |
| presignOptions | RequestPresigningArguments | presignOptions |
Promise.<Object>Create a KMS key
See https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/KMS.html#createKey-property for allowed params and return value.
Kind: global function
| Param | Type |
|---|---|
| params | Object |
Promise.<string>Encrypt a string using KMS
Kind: global function
Returns: Promise.<string> - the Base 64 encoding of the encrypted value
| Param | Type | Description |
|---|---|---|
| KeyId | string | the KMS key to use for encryption |
| Plaintext | string | the string to be encrypted |
stringDecrypt a KMS-encrypted string, Base 64 encoded
Kind: global function
Returns: string - the plaintext
| Param | Type | Description |
|---|---|---|
| ciphertext | string | a KMS-encrypted value, Base 64 encoded |
Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.
To make a contribution, please see our contributing guidelines.
Generated automatically using npm run build-docs
[v20.3.0] 2025-08-18
engine_version default value to 17.4 in example/rds-cluster-tf/variables.tf.tf-modules/cumulus-rds-tf module to take additional parameter enable_upgrade in support of
migration from Aurora PostgreSQl v13 to v17.>=1.12.25.100.xFAQs
Utilities for working with AWS
The npm package @cumulus/aws-client receives a total of 468 weekly downloads. As such, @cumulus/aws-client popularity was classified as not popular.
We found that @cumulus/aws-client demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 6 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Ruby maintainers from Bundler and rbenv teams are building rv to bring Python uv's speed and unified tooling approach to Ruby development.
Security News
Following last week’s supply chain attack, Nx published findings on the GitHub Actions exploit and moved npm publishing to Trusted Publishers.
Security News
AGENTS.md is a fast-growing open format giving AI coding agents a shared, predictable way to understand project setup, style, and workflows.