@sanity/assist
Advanced tools
This is a Sanity Studio v3 plugin.
Free your team to do more of what they’re great at (and less busy work) with the AI assistant that works with structured content. Attach reusable AI instructions to fields and documents to supercharge your editorial workflow.
You create the instructions; Sanity AI Assist does the rest. Learn more about writing instructions in the Sanity documentation.
Read the release announcement here.
Using this feature requires Sanity to send data to OpenAI.com for processing. It uses generative AI; you should verify the data before using it.
In your Studio project folder, install the following plugin dependency:
npm install @sanity/assist sanity@latest
This plugin requires sanity version 3.16.0 or greater.
Note: Before using the plugin, your project must have Sanity AI Assist enabled at the API level.
Contact your Sanity enterprise representative to get started, or contact the sales team.
In sanity.config.ts, add assist to the plugins array:
import { assist } from '@sanity/assist'
export default defineConfig({
/* other config */
plugins: [
/* other plugins */
assist()
]
})
After installing and adding the plugin and having the AI Assist feature enabled for your project and its datasets, you need to create a token for the plugin to access the AI Assist API. This needs to be done by a member of the project with token creation permissions (typically someone with an admin or developer role).
You will find a new API token entry for your project named “Sanity AI” in your project's API settings on sanity.io/manage.
The plugin will now work for any dataset in your project.
Note: You can revoke this token at any time to disable Sanity AI Assist service. A new token has to be generated via the plugin UI for it to work again.
If your project is using custom roles (Enterprise), there are some additional considerations.
To see AI Assist presence when running instructions, users will need read access to
documents of _type=="sanity.assist.task.status".
To edit instructions, users will need read and write access to documents of _type=="sanity.assist.schemaType.annotations".
Note that instructions run using the permissions of the user invoking it, so only fields that the user themselves can edit can be changed by the instruction instance.
By default, most object, array, and string field types have AI writing assistance enabled. Your assistant can write to all compatible fields that it detects.
The assistant can also create array items, including Portable Text blocks, when the type has been imported to the Studio's schema as a custom type (learn more about strict schemas).
// this will disable AI assistance wherever it is used,
// ie: as field, document, array types
defineType({
name: 'policy',
type: 'document',
options: {
aiWritingAssistance: {exclude: true}
},
fields: [
// ...
]
})
// this disables AI assistance only for the specific field
defineType({
name: 'product',
type: 'object',
fields: [
defineField({
name: 'sku',
type: 'string',
options: {
aiWritingAssistance: {exclude: true}
}
})
]
})
// this disables AI assistance for the specific array member
// if all types in the `of` array are excluded, the array type is also considered excluded
defineType({
name: 'product',
type: 'array',
of: [
defineArrayMember({
type: 'customProduct',
options: {
aiWritingAssistance: {exclude: true}
}
})
]
})
The following types are not supported, and behave as excluded types:
Types and fields with hidden or readonly with a truthy value (true or function) are not supported.
(Hidden and readOnly fields can be referenced in instructions still)
Fields with these types will not be changed by the assistant, do not have AI Assist actions, and cannot be referenced in instructions.
Objects where all fields are excluded or unsupported and arrays where all member types are excluded or unsupported will also be excluded.
To enable AI assist for references, first, your project must have an existing embeddings-index with the documents it should be able to reference.
You can manage your indexes directly in the studio using the Embeddings Index Dashboard plugin.
Set options.aiWritingAssistance.embeddingsIndex for reference fields/types you want to enable reference instructions for.
Reference fields with this options set can have instructions attached to them, and will be visited when running instructions for object fields and arrays.
AI assist will use the embeddings-index, filtered by the types allowed by the reference to look up contextually relevant references. For arrays or portable text fields with references, one more references can be added. Use the instruction text to control this.
Example:
import {defineArrayMember} from 'sanity'
defineField({
type: 'reference',
name: 'articleReference',
title: 'Article referene',
to: [ { type: 'article'} ],
options: {
aiWritingAssistance: {
embeddingsIndex: 'article-index'
},
},
})
An example instruction attached to this field could be:
Given <Document field: Title> suggest a related article
Running it would use the article-index to find an related article based on the current document title.
There are limits to how much text the AI can process when processing an instruction. Under the hood, the AI Assist will add information about your schema, which adds to what's commonly referred to as “the context window.”
If you have a very large schema (that is, many document and field types), it can be necessary to exclude types to limit how much of the context window is used for the schema itself.
We recommend excluding any and all types which rarely would benefit from automated workflows. A quick win is typically to exclude array types. It can be a good idea to exclude most non-block types from Portable Text arrays. This will ensure that the Sanity Assist outputs mostly formatted text.
It is also possible to exclude fields/types when creating an instruction. See Field and type filters for more.
This plugin adds an AI Context document type.
If your Studio uses Structure Builder to configure the studio structure, you might have to add this document type to your structure.
The document type name can be imported from the plugin:
import {contextDocumentTypeName} from '@sanity/assist'
// put into structure in structure
S.documentTypeListItem(contextDocumentTypeName)
When creating instructions for a document, objects fields, array fields or portable text fields, you can explicitly control what will be visited by AI Assist. By default, all fields and types configured for assist will be included.
Opting out fields/types per instruction is done using the respective field/type filter checkboxes under the instruction. When using these filters, it is not necessary to tell Assist what to include in the instruction text itself.
Note that if the schema targeted by the instruction changes, the following behavior applies:
AI Assist can optionally generate captions for images. This has to be enabled on an image-type/field,
by setting the options.captionField on the image type, where captionField is the field name of a
custom string-field on the image object:
AI assist offers full document translations, which is ideal for pairing with @sanity/document-internationalization.
Given a document written in one language, AI assist can translate the document in place to a language specified by a language field in the document.
When the document translation feature is enabled, AI Assist will go through the document field by field, translating all string and portable text fields into the language specified in the document's language field.
This works well with @sanity/document-internationalization, where documents are duplicated from a source language and set a hidden language field.
AI assist allows editors to translate these documents into the desired language immediately.
To enable full document translations, set translate.document.languageField to the path of the language field in your documents.
All documents with a language field will get a "Translate document" instruction added to the assist drop-down for the document.
To limit which document types get "Translate document" further, provide translate.document.documentTypes with an array of document type names.
If the studio is using @sanity/document-internationalization, these options should be the same as those used for that plugin.
Example configs
// This will add a "Translate document" instruction to all documents with a language field
assist({
translate: {
document: {
languageField: 'language'
}
}
})
// This will add a "Translate document" instruction only to the 'article' document type
assist({
translate: {
document: {
languageField: 'language',
documentTypes: ['article']
}
}
})
All configuration params
assist({
translate: {
/** Config for document types with a single language field that determines the language for the whole document. */
document: {
/**
* Required config, enable document tranlations.
*
* Path to language field in documents. Can be a hidden field.
* For instance: 'config.language'
*
* For projects that use the `@sanity/document-internationalization` plugin,
* this should be the same as `languageField` config for that plugin (which defaults to 'language')
*/
languageField: string,
/**
* `documentTypes` should be an array of strings where each entry must match a name from your document schemas.
*
* this property will add a translate instruction to these document types if defined.
* If undefined, the instruction will be added to all documents with aiAssistance enabled and a field matching `languageField` config.
*
* Documents with translation support will get a "Translate document>" instruction added.
**/
documentTypes: string[]
}
}
})
AI assist offers field-level translations, which is ideal for using alongside withsanity-plugin-internationalized-array and (@sanity/language-filter)[https://github.com/sanity-io/language-filter]
Given a document with field values in different languages, AI assist can transfer and translate from one language to the others.
The typical use case would be for documents that use internationalized wrapper types to hold values for multiple languages.
AI Assist supports complex values, so language fields that hold nested objects, portable text, or arrays will also be translated.
When initiating translations, editors select a language to translate from and which languages to translate to. This means that AI Assist supports partial translations in cases where editors are responsible for only some languages in the document.
To enable field-level translations, set translate.field.documentTypes to an array with which document types should get field translations, and translate.field.languages
assist({
translate: {
field: {
documentTypes: ['article'],
languages: [
{id: 'en', title: 'English'},
{id: 'de', title: 'German'}
]
},
},
})
These documents will get a "Translate fields" instruction added to the document AI Assist dropdown.
Out of the box, this is sufficient config for document types using internationalizedArray* types for localization sanity-plugin-internationalized-array.
It will also work without further config for object types named "locale*", with one field per language:
Example locale object supported by default
// Object type with name starting with 'locale', and one field per language language
defineType({
type: 'object',
name: 'localeString',
fields: [
defineField({
// these do not have to be string, could be any type
type: 'string',
name: 'en',
title: 'English'
}),
defineField({
type: 'string',
name: 'de',
title: 'German'
})
]
})
If your schema is not using either of these structures, confer Custom language fields.
Languages must be an array of objects with an id and title.
assist({
translate: {
field: {
languages: [
{id: 'en', title: 'English'},
{id: 'de', title: 'German'}
]
},
},
})
Or an asynchronous function that returns an array of objects with an id and title.
assist({
translate: {
field: {
languages: async () => {
const response = await fetch('https://example.com/languages')
return response.json()
}
},
},
})
The async function contains a configured Sanity Client in the first parameter, allowing you to store Language options as documents. Your query should return an array of objects with an id and title.
assist({
translate: {
field: {
languages: async () => {
const response = await client.fetch(`*[_type == "language"]{ id, title }`)
return response
}
},
},
})
Additionally, you can "pick" fields from a document, to pass into the query. For example, if you have a concept of "Markets" where only certain language fields are required in certain markets.
In this example, each language document has an array of strings called markets to declare where that language can be used. And the document being authored has a single market field.
assist({
translate: {
field: {
selectLanguageParams: {
market: 'market'
},
languages: async (client, {market = ``}) => {
const response = await client.fetch(
`*[_type == "language" && $market in markets]{ id, title }`,
{market}
)
return response
},
},
},
})
By providing a function to translate.field.translationOutputs, complete control over which fields belong to which language is given.
translationOutputs is used when an editor uses the "Translate fields" instruction.
It determines the relationships between document paths: Given a document path and a language, it should return into which sibling paths translations are output.
translationOutputs is invoked once per path in the document (limited to a depth of 6), with the following:
documentMember - the field or array item for a given path; contains the path and its schemaType,enclosingType - the schema type of the parent holding the membertranslateFromLanguageId - the languageId for the language the users want to to translate fromtranslateToLanguageIds - all languageIds the user can translate toThe function should return a TranslationOutput[] array that contains all the paths where translations from documentMember (in the language given by translateFromLanguageId) should be output.
The function should return undefined for all documentMembers that should not be directly translated, or are nested fields under a translated path.
The default translationOutputs is available using import {defaultTranslationOutputs} from '@sanity/assist.
Given the following document:
{
titles: {
_type: 'languageObject',
en: {
_type: 'titleObject',
title: 'Some title',
subtitle: 'Some subtitle'
},
de: {
_type: 'titleObject',
}
}
}
When translating from English to German, translationOutputs will be
invoked multiple times.
The following parameters will be the same every invocation:
translateFromLanguageId will be 'en'translateToLanguageIds will be ['de']documentMember and enclosingType will change between each invocation, and take the following values:
{path: 'titles', name: 'titles', schemaType: ObjectSchemaType}, ObjectSchemaType{path: 'titles.en', name: 'en', schemaType: ObjectSchemaType}, ObjectSchemaType{path: 'titles.en.title', name: 'title', schemaType: StringSchemaType}, ObjectSchemaType{path: 'titles.en.subtitle', name: 'subtitle', schemaType: StringSchemaType}, ObjectSchemaType{path: 'titles.de', name: 'de', schemaType: ObjectSchemaType}, ObjectSchemaTypeTo indicate that you want everything under title.en to be translated into title.de, translationOutputs needs to return [id: 'de', outputPath: 'titles.de'] when invoked with documentMember.path: 'titles.en'.
The following function enables this:
function translationOutputs(member, enclosingType, translateFromLanguageId, translateToLanguageIds) {
const parentIsLanguageWrapper = enclosingType.jsonType === 'object' && enclosingType.name.startsWith('language')
if (parentIsLanguageWrapper && translateFromLanguageId === member.name) {
// [id: 'de', ]
return translateToLanguageIds.map((translateToId) => ({
id: translateToId,
// in this example, member.path is 'titles.en'
// so this changes titles.en -> titles.de
outputPath: [...member.path.slice(0, -1), translateToId],
}))
}
// ignore other members
return undefined
}
assist({
translate: {
field: {
documentTypes: ['article'],
selectLanguageParams: {market: 'market'},
apiVersion: '2023-01-01',
languages: (client, {market}) => {
return client.fetch(
`*[_type == "language" && $market in markets]{ id, title }`,
{market}
)
},
translationOutputs: (member, enclosingType, fromLanguageId, toLanguageIds) => {
// When the document member is named the same as fromLangagueId
// and it is a field in a object with a name starting with "language"
// then we return the paths to all other sibling language fields (and their langauge id)
// It is ok that the member is an object, then all child fields will be translated
if (translateFromLanguageId === member.name && enclosingType.jsonType === 'object' && enclosingType.name.startsWith('locale')) {
return translateToLanguageIds.map((translateToId) => ({
id: translateToId,
//changes path.to.en -> path.to.de (for instance)
outputPath: [...member.path.slice(0, -1), translateToId],
}))
}
// all other member paths are skipped
return undefined
}
},
},
})
Large Language Models (LLMs) are a new technology. Constraints and limitations are still being explored, but some common caveats to the field that you may run into using AI Assist are:
defineField({
type: 'image',
name: 'inlineImage',
title: 'Image',
fields: [
defineField({
type: 'string',
name: 'caption',
title: 'Caption',
}),
],
options: {
captionField: 'caption',
},
})
This will add a "Generate caption" action to the configured field. "Generate caption" action will automatically run whenever the image changes.
captionField can be a nested field, if the image has object field, ie captionField: 'wrapper.caption'.
Fields within array items are not supported.
This version of the feature uses OpenAI.com as a third-party sub-processor. Their security posture has been vetted by Sanity's security team, and approved for use.
MIT © Sanity
This plugin uses @sanity/plugin-kit with default configuration for build & watch scripts.
See Testing a plugin in Sanity Studio on how to run this plugin with hotreload in the studio.
Run "CI & Release" workflow. Make sure to select the main branch and check "Release new version".
Semantic release will only release on configured branches, so it is safe to run release on any branch.
FAQs
You create the instructions; Sanity AI Assist does the rest.
The npm package @sanity/assist receives a total of 3,131 weekly downloads. As such, @sanity/assist popularity was classified as popular.
We found that @sanity/assist demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 61 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Node.js will be enforcing stricter semver-major PR policies a month before major releases to enhance stability and ensure reliable release candidates.
Security News
Research
Socket's threat research team has detected five malicious npm packages targeting Roblox developers, deploying malware to steal credentials and personal data.
Security News
vlt introduced its new package manager and a serverless registry this week, innovating in a space where npm has stagnated.