@sanity/import

What is @sanity/import?

@sanity/import is an npm package designed to facilitate the import of data into a Sanity.io dataset. It allows users to import documents and assets from various sources, making it easier to manage and migrate content within the Sanity content platform.

What are @sanity/import's main functionalities?

Import JSON Data

This feature allows you to import JSON data into a specified Sanity dataset. The code sample demonstrates how to read a JSON file and import its contents into Sanity using the @sanity/import package.

const sanityImport = require('@sanity/import');
const fs = require('fs');

const inputStream = fs.createReadStream('path/to/your/data.json');
const options = {
  dataset: 'your-dataset-name',
  projectId: 'your-project-id',
  token: 'your-auth-token',
  assetConcurrency: 3,
  operation: 'createOrReplace'
};

sanityImport(inputStream, options).then(() => {
  console.log('Data import completed!');
}).catch((err) => {
  console.error('Import failed: ', err);
});

Import Assets

This feature allows you to import assets (e.g., images, files) into a specified Sanity dataset. The code sample demonstrates how to read a ZIP file containing assets and import them into Sanity using the @sanity/import package.

const sanityImport = require('@sanity/import');
const fs = require('fs');

const inputStream = fs.createReadStream('path/to/your/assets.zip');
const options = {
  dataset: 'your-dataset-name',
  projectId: 'your-project-id',
  token: 'your-auth-token',
  assetConcurrency: 3,
  operation: 'createOrReplace'
};

sanityImport(inputStream, options).then(() => {
  console.log('Assets import completed!');
}).catch((err) => {
  console.error('Import failed: ', err);
});

Other packages similar to @sanity/import

contentful-import

The contentful-import package is used to import data into Contentful, a content management system similar to Sanity. It allows users to import content types, entries, and assets into a Contentful space. Compared to @sanity/import, contentful-import is specific to the Contentful platform and offers similar functionalities for data migration and content management.

README

@sanity/import

Imports documents from an ndjson-stream to a Sanity dataset

Installing

npm install --save @sanity/import

Usage

const fs = require('fs')
const sanityClient = require('@sanity/client')
const sanityImport = require('@sanity/import')

const client = sanityClient({
  projectId: '<your project id>',
  dataset: '<your target dataset>',
  token: '<token-with-write-perms>',
  useCdn: false
})

// Input can either be a stream or an array of documents
const input = fs.createReadStream('my-documents.ndjson')
sanityImport(input, {
  client: client,
  operation: 'create', // `create`, `createOrReplace` or `createIfNotExists`
}).then(numDocs => {
  console.log('Imported %d documents', numDocs)
}).catch(err => {
  console.error('Import failed: %s', err.message)
})

CLI-tool

This functionality is built in to the @sanity/cli package as well as a standalone @sanity/import-cli package.

Future improvements

• When documents are imported, record which IDs are actually touched
  • Only upload assets for documents that are still within that window
  • Only strengthen references for documents that are within that window
  • Only count number of imported documents from within that window
• Asset uploads and strengthening can be done in parallel, but we need a way to cancel the operations if one of the operations fail
• Introduce retrying of asset uploads based on hash + indexing delay
• Validate that dataset exists upon start
• Reference verification
  • Create a set of all document IDs in import file
  • Create a set of all document IDs in references
  • Create a set of referenced ID that do not exist locally
  • Batch-wise, check if documents with missing IDs exist remotely
  • When all missing IDs have been cross-checked with the remote API (or a max of say 100 items have been found missing), reject with useful error message.

License

MIT-licensed. See LICENSE.