@cumulus/checksum
Checksum
The @cumulus/checksum
library provides checksum functionality used by Cumulus
packages and tasks. Currently the supported input includes file streams, and
supported checksum algorithms include cksum
and the algorithms available to
the crypto
package, as documented
here.
Usage
const fs = require('fs');
const { generateChecksumFromStream } = require('@cumulus/checksum');
const stream = fs.createReadStream('myDataFile.hdf');
const myCksum = generateChecksumFromStream('cksum', stream);
API
checksum
checksum.generateChecksumFromStream(algorithm, stream, [options]) ⇒ Promise.<(number|string)>
Create file checksum from readable stream
Kind: static method of checksum
Returns: Promise.<(number|string)>
- the file checksum
Param | Type | Description |
---|
algorithm | string | Checksum algorithm type |
stream | stream.Readable | A readable file stream |
[options] | Object | Checksum options, see crypto.createHash() |
checksum.validateChecksumFromStream(algorithm, stream, expectedSum, [options]) ⇒ Promise.<boolean>
Validate expected checksum against calculated checksum
Kind: static method of checksum
Returns: Promise.<boolean>
- whether expectedSum === calculatedSum
Param | Type | Description |
---|
algorithm | string | Checksum algorithm |
stream | stream.Readable | A readable file stream |
expectedSum | number | string | expected checksum |
[options] | Object | Checksum options |
About Cumulus
Cumulus is a cloud-based data ingest, archive, distribution and management
prototype for NASA's future Earth science data streams.
Cumulus Documentation
Generated automatically using npm run build-docs
[v18.2.0] 2024-02-02
Migration Notes
From this release forward, Cumulus Core will be tested against PostgreSQL v13. Users
should migrate their datastores to Aurora PostgreSQL 13.9+ compatible data
stores as soon as possible after upgrading to this release.
Database Upgrade
Users utilizing the cumulus-rds-tf
module should reference cumulus-rds-tf
upgrade
instructions.
Breaking Changes
- CUMULUS-2889
- Removed unused CloudWatch Logs AWS SDK client. This change removes the CloudWatch Logs
client from the
@cumulus/aws-client
package.
- CUMULUS-2890
- Removed unused CloudWatch AWS SDK client. This change removes the CloudWatch client
from the
@cumulus/aws-client
package.
Changed
- CUMULUS-3492
- add teclark to select-stack.js
- CUMULUS-3444
- Update
cumulus-rds-tf
module to take additional parameters in support of
migration from Aurora PostgreSQl v11 to v13. See Migration Notes for more details
- CUMULUS-3564
- Update webpack configuration to explicitly disable chunking
- CUMULUS-2895
- Updated KMS code to aws sdk v3
- CUMULUS-2888
- Update CloudWatch Events code to AWS SDK v3
- CUMULUS-2893
- Updated Kinesis code to AWS SDK v3
- CUMULUS-3555
- Revert 3540, un-stubbing cmr facing tests
- Raise memory_size of ftpPopulateTestLambda to 512MB
- CUMULUS-2887
- Updated CloudFormation code to aws sdk v3
- CUMULUS-2899
- Updated SNS code to aws sdk v3
- CUMULUS_3499
- Update AWS-SDK dependency pin to "2.1490" to prevent SQS issue. Dependency
pin expected to be changed with the resolution to CUMULUS-2900
- CUMULUS-2894
- Update Lambda code to AWS SDK v3
- CUMULUS-3432
- Update
cumulus-rds-tf
engine_version
to 13.9
- Update
cumulus-rds-tf
parameter_group_family
to aurora-postgresql13
- Update development/local stack postgres image version to postgres:13.9-alpine
- CUMULUS-2900
- Update SQS code to AWS SDK v3
- CUMULUS-3352
- Update example project to use CMA v2.0.3 for integration testing
- Update example deployment to deploy cnmResponse lambda version
2.1.1-aplha.2-SNAPSHOT
- Update example deployment to deploy cnmToGranule lambda
version 1.7.0-alpha.2-SNAPSHOT
- CUMULUS-3501
- Updated CreateReconciliationReport lambda to save report record to Elasticsearch.
- Created docker image cumuluss/async-operation:48 from v16.1.2, and used it as default async_operation_image.
- CUMULUS-3502
- Upgraded localstack to v3.0.0 to support recent aws-sdk releases and update unit tests.
- CUMULUS-3540
- stubbed cmr interfaces in integration tests allow integration tests to pass
- needed while cmr is failing to continue needed releases and progress
- this change should be reverted ASAP when cmr is working as needed again
Fixed
- CUMULUS-3177
- changed
_removeGranuleFromCmr
function for granule bulkDelete
to not throw an error and instead catch the error when the granule is not found in CMR
- CUMULUS-3293
- Process Dead Letter Archive is fixed to properly copy objects from
/sqs/
to /failed-sqs/
location
- CUMULUS-3467
- Added
childWorkflowMeta
to QueueWorkflow
task configuration
- CUMULUS-3474
- Fixed overridden changes to
rules.buildPayload' to restore changes from ticket
CUMULUS-2969which limited the definition object to
nameand
arn` to
account for AWS character limits.
- CUMULUS-3479
- Fixed typo in s3-replicator resource declaration where
var.lambda_memory_size
is supposed to be var.lambda_memory_sizes
- CUMULUS-3510
- Fixed
@cumulus/api
validateAndUpdateSqsRule
method to allow 0 retries and 0 visibilityTimeout
in rule's meta. This fix from CUMULUS-2863 was not in release 16 and later.
- CUMULUS-3562
- updated crypto-js to 4.2.0
- updated aws-sdk/client-api-gateway to 3.499 to avoid older crypto-js dependency