@bluecadet/launchpad-content

Launchpad Content

The content package downloads and locally caches content from various common web APIs.

To download content, all you need to do is define content sources and provide credentials as needed.

The following launchpad.json would download jsons and images from the Flickr API to .downloads/flickr-images (a combination of the default .downloads/ directory and the id field of the content source):

{
  "content": {
    "sources": [
      {
        "id": "flickr-images",
        "type": "json",
        "files": {
            "spaceships.json": "https://api.flickr.com/services/feeds/photos_public.gne?format=json&nojsoncallback=1&tags=spaceship",
            "rockets.json": "https://api.flickr.com/services/feeds/photos_public.gne?format=json&nojsoncallback=1&tags=rocket"
        }
      }
    ]
  }
}

See ContentOptions Parameters for a full list of content settings.

Source Settings

Every source needs:

• id: ID of the individual source (required)
• type: Source type (default: json)
• Additional source-specific settings

Currently supported sources are:

• json (via HTTP)
• airtable
• contentful
• sanity
• strapi

Authentication

Some content sources require credentials to access their APIs.

These can all be stored in a .env or .env.local file which will be automatically loaded by launchpad.

.env.local

AIRTABLE_API_KEY=<YOUR_AIRTABLE_API_KEY>

CONTENTFUL_PREVIEW_TOKEN=<YOUR_CONTENTFUL_PREVIEW_TOKEN>
CONTENTFUL_DELIVERY_TOKEN=<YOUR_CONTENTFUL_DELIVERY_TOKEN>
CONTENTFUL_USE_PREVIEW_API=false

SANITY_API_TOKEN=<YOUR_API_TOKEN>

STRAPI_IDENTIFIER=<YOUR_API_USER>
STRAPI_PASSWORD=<YOUR_API_PASS>

launchpad.config.js

export default defineConfig({
	content: {
		sources: [
			{
				id: "airtable-cms",
				type: "airtable",
				apiKey: process.env.AIRTABLE_API_KEY,
			},
			{
				id: "contentful-cms",
				type: "contentful",
				previewToken: process.env.CONTENTFUL_PREVIEW_TOKEN,
				deliveryToken: process.env.CONTENTFUL_DELIVERY_TOKEN,
				usePreviewApi: false,
			},
			{
				id: "sanity-cms",
				type: "sanity",
				apiToken: process.env.SANITY_API_TOKEN,
			},
			{
				id: "strapi-cms",
				type: "strapi",
				identifier: process.env.STRAPI_IDENTIFIER,
			},
		],
	},
});

Post Processing

Once content is downloaded it can be processed to transform text and images. For example, launchpad can convert markdown to html and create scaled derivatives of each image.

• Content Transforms Documetation
• Image Transforms Documetation

ContentOptions

Options for all content and media downloads. Each of these settings can also be configured per ContentSource.

Property	Type	Default	Description
sources	Array.<SourceOptions>	[]	A list of content source options. This defines which content is downloaded from where.
imageTransforms	Array.<Object.<string, number>>	[]	A list of image transforms to apply to a copy of each downloaded image.
contentTransforms	Object.<string, string>	{}	A list of content transforms to apply to all donwloaded content.
downloadPath	string	'.downloads/'	The path at which to store all downloaded files.
credentialsPath	string	'.credentials.json'	The path to the json containing credentials for all content sources.
tempPath	boolean	'%DOWNLOAD_PATH%/.tmp/'	Temp file directory path.
backupPath	boolean	'%DOWNLOAD_PATH%/.backups/'	Temp directory path where all downloaded content will be backed up before removal.
keep	boolean	''	Which files to keep in dest if clearOldFilesOnSuccess or clearOldFilesOnStart are true. E.g. '\*.json|\*.csv|\*.xml|\*.git\*'
strip	string	''	Strips this string from all media file paths when saving them locally
backupAndRestore	boolean	true	Back up files before downloading and restore originals for all sources on failure of any single source.
maxConcurrent	number	4	Max concurrent downloads.
maxTimeout	number	30000	Max request timeout in ms.
clearOldFilesOnSuccess	boolean	true	Remove all existing files in dest dir when downloads succeed. Ignores files that match keep
clearOldFilesOnStart	boolean	false	Will remove all existing files _before_ downloads starts. false will ensure that existing files are only deleted after a download succeeds.
ignoreCache	boolean	false	Will always download files regardless of whether they've been cached
enableIfModifiedSinceCheck	boolean	true	Enables the HTTP if-modified-since check. Disabling this will assume that the local file is the same as the remote file if it already exists.
enableContentLengthCheck	boolean	true	Compares the HTTP header content-length with the local file size. Disabling this will assume that the local file is the same as the remote file if it already exists.
abortOnError	boolean	true	If set to true, errors will cause syncing to abort all remaining tasks immediately
ignoreImageTransformCache	boolean	false	Set to true to always re-generate transformed images, even if cached versions of the original and transformed image already exist.
ignoreImageTransformErrors	boolean	true	Set to false if you want to abort a content source from downloading if any of the image transforms fail. Leaving this to true will allow for non-image files to fail quietly.
forceClearTempFiles	boolean	true	Set to false if you want to keep all contents of the tempPath dir before downloading