@cumulus/post-to-cmr
Advanced tools
This lambda function posts granule metadata to CMR (Common Metadata Repository).
It will use the information contained in a metadata file on S3 and post that information to the CMR service.
The S3 metadata file can be either ECHO10 xml metadata with extension .cmr.xml or UMM-G JSON with extension .cmr.json.
The move-granules task and the (api)granules.move function will both update the metadata files on S3. For UMM-G JSON metadata, RelatedUrls are updated, while ECHO10 XML metadata will maintain the metadata in OnlineAccessURLs.
A granule's files URL, and Type/Description for UMMG and URLDescription for ECHO10, are modified for each file in the granule based on its bucket location. The metadata URLs are based on the file bucket storage type. Files placed in protected buckets will get a url to the distribution endpoint. Files in public buckets will get direct https links. URLs not directly related to the granule's files are unmodified and preserved as they exist.
For more information on configuring a Cumulus Message Adapter task, see the Cumulus workflow input/output documentation.
Config object fields:
| field name | type | default | description |
|---|---|---|---|
| bucket | string | (required) | Name of S3 bucket containing public/private key pair to decrypt CMR credentials |
| process | string | (required) | Process the granules went through |
| stack | string | (required) | Name of deployment stack |
| cmr | object | (required) | CMR credentials object |
| concurrency | number | 20 | Maximum concurrency of requests to CMR |
| republish | boolean | false | Whether to remove published granules from CMR and republish them again |
Input object fields:
| field name | type | default | description |
|---|---|---|---|
| granules | array<object> | (required) | List of granule objects |
Output object fields:
| field name | type | default | values | description |
|---|---|---|---|---|
| granules | array<object> | N/A | List of granule objects published to CMR | |
| process | string | N/A | Process the granules went through |
Cumulus is a cloud-based data ingest, archive, distribution and management prototype for NASA's future Earth science data streams.
To make a contribution, please see our contributing guidelines.
[v21.0.0] 2025-09-09
Please follow the instructions before upgrading Cumulus
checkCrossCollectionCollisions may be set to false to
disable the behavior on a per-workflow, per-collection or other config
driven criteria.parse-pdr task component to throw an error if multiple
granules within the same PDR have the same granuleId after applying the
granuleIdFilter, unless the uniquifyGranuleId configuration parameter is
explicitly set to true.updateGranulesCmrMetadataFileLinks to always ensure
producerGranuleId identifier is set in updated CMR metadatacnm_response_task to use newest version v3.2.0, which supports
producerGranuleId.cumulus-tf includes cnm_response_version = "3.2.0" or greater.producer_granule_id to Postgres granules
table.producerGranuleId property to granule record schema.@cumulus api/db/message packages to handle producer_granule_id
and producerGranuleId.@cumulus/api/lib/writeGranulesFromMessage to set producerGranuleId
= granuleId if not set.queue-granules task to set producerGranuleId = granuleId if not
set.producerGranuleId to LzardsBackup task component and lambda input/output schemaLzardsBackup task component to submit producerGranuleId for storage in the lzards record as a key in the metadata object.parse-pdr task component to have the following behaviors:
uniquifyGranuleId configuration value is set to true, parse-PDR will
update the granuleId for all found granules to have a unique granule hash
appended to the existing IDparse-pdr such that if the uniquifyGranuleId configuration
parameter is not set to true , and a duplicate granuleId is created as
part of the output after passing the granuleIdFilter, the task will
throw with an error.ingestFromPdrWithUniqueGranuleIdsSpec.js to the spec tests to
demonstrate the ingest workflow works as expected with unique granuleIds and
producerGranuleIds set.ingest terraform module for deployment
with Core. This task will update a payload of existing granules to have
'uniquified' IDs and preserve the original identifier in the
producerGranuleId fieldIngestGranuleSuccessSpec/IngestUMMGSuccessSpec to validate
producerGranuleId is populated in CMR post ingestproducerGranuleId in the default test case@cumulus/cmrjs/cmr-utilsupdateCMRMetadata to take updateGranuleIdentifiers configuration
flag/producerGranuleId such that that routine now will modify the CMR
metadata object with the correct GranuleUR/ProducerGranuleId values in
the CMR metadata.cmr-utilsgetCmrMetadata helper to @cumulus/integration-tests to allow
access to the full CMR metadata object for verification of record metadata
fieldsApiFileGranuleIdOptional to @cumulus/types/api for cases where an
ApiFile is being generated and refactored existing code to use this type
instead of custom relaxed typingupdate-granules-cmr-metadata-file-links to use the updated cmrjs
logic to set producerGranuleId identifiers in the CMR metadata, either equal
to granuleId or the producerGranuleID set on the granule.@cumulus/tasks/sync-granule/GranuleFetcher to allow and pass through an
incoming granule.producerGranuleId@cumulus/api/lib/ingest.reingestGranule to only update the original granule
to 'queued' if the original payload contains the granule. This avoids a situation
where the original granule is updated to 'queued', but the reingest workflow
creates a new granule, leaving the original granule stuck in 'queued'.getGranuleIdAndCollectionIdFromFile query method to @cumulus/db to
retrieve granule and collection metadata from a file's S3 location.GET /granules/files/get_collection_and_granule_id/:bucket/:key in @cumulus/api to
return the granule ID and collection ID associated with a file.getFileGranuleAndCollectionByBucketAndKey method to
@cumulus/api-client/granules to allow use of new endpoint.move-granules task to validate cross-collection file collisions
using the new lookup logic when checkCrossCollectionCollisions is enabled.@cumulus/db to add getGranuleIdAndCollectionIdFromFile query methodupdate-granules-cmr-metadata-file-links task READMEurlPathTemplate to allow falling back from one null/undefined interpolated value to a second argumentcnmResponse lambda version
3.1.0-alpha.2-SNAPSHOT which utilizes producerGranuleId.cnmToGranule lambda version 2.1.0.FakeProcessing task configuration matchFilesWithProducerGranuleId
to determine if the generated cmr file names should match
granuleId or producerGranuleIdAddUniqueGranuleId task configuration hashLength to accept
additional types and removed the use of hashDepth.FilesToGranules task configuration
matchFilesWithProducerGranuleId to accept additional types.ParsePdr task configuration hashLength to accept additional
types.tf-modules/cumulus AddUniqueGranuleId task output.CNMExampleWorkflow to uniquify
granuleIds based on collection configurationKinesisTestTriggerWithUniqueGranuleIdsSpec.js to the spec test to
demonstrate that the CNM ingest workflow ingests granules with unique
granuleIds and producerGranuleIds set, and that CnmResponse sends responses
using producerGranuleIdsproducerGranuleId
when mapping files to their granules.ingestGranule, discoverGranules,
lzardsBackup, cnmWorkflow, and orca specs.workflow_configurations variable to the tf-modules/ingest and
tf-modules/cumulus modules.
The property sf_event_sqs_to_db_records_types has been added to
workflow_template.json under the cumulus_meta field to control which record
types should be written to the database during different workflow execution statuses.
Currently, both "execution" and "pdr" must be written to the database, so the
record type list must include both.SfSqsReport task to set meta.reportMessageSource in the Cumulus message.@cumulus/api/sfEventSqsToDbRecords lambda to determine which
record types ("execution", "granule", "pdr") should be written to the database based on the
cumulus_meta.sf_event_sqs_to_db_records_types and meta.reportMessageSource fields.
By default, all record types will be written to the database.@cumulus/api/lib.writeRecords.writeGranuleExecutionAssociationsFromMessage
to write granule-execution associations from message.@cumulus/integration-tests cmr.generateAndStoreCmrXml to
apply matchFilesWithProducerGranuleId when generaing OnlineAccessURL.KinesisTestTriggerWithUniqueGranuleIdsSpec to cover "duplicate"
Granules in separate Collections.includeTimestampHashKey parameter to the generateUniqueGranuleId function in the @cumulus/ingest/granule, with a default value of false.includeTimestampHashKey configuration to the add-unique-granuleId and parse-pdr tasks, also with a default value of false."Generate Unique GranuleId" to explain the algorithm for generating unique granuleIds.producer_granule_id migration script to disable autovacuum before the
migration and re-enable it afterward to improve performance.cumuluss/async-operation:54. Users should update their references to async-operation with the new version.FAQs
Post a given granule to CMR
The npm package @cumulus/post-to-cmr receives a total of 58 weekly downloads. As such, @cumulus/post-to-cmr popularity was classified as not popular.
We found that @cumulus/post-to-cmr demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 6 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Socket CEO Feross Aboukhadijeh discusses the recent npm supply chain attacks on PodRocket, covering novel attack vectors and how developers can protect themselves.
Security News
Maintainers back GitHub’s npm security overhaul but raise concerns about CI/CD workflows, enterprise support, and token management.
Product
Socket Firewall is a free tool that blocks malicious packages at install time, giving developers proactive protection against rising supply chain attacks.