@sanity/assist

@sanity/assist

This is a Sanity Studio v3 plugin.

Table of contents

• Table of contents
• About Sanity AI Assist
• Installation
• Setup
  • Add the plugin
  • Enabling the AI Assist API
• Schema configuration
  • Disable AI Assist for a schema type
  • Disable for a field
  • Disable for an array type
  • Unsupported types
  • Troubleshooting
• Included document types
• License
• Develop & test
  • Release new version

About Sanity AI Assist

Free your team to do more of what they’re great at (and less busy work) with the AI assistant that works with structured content. Attach reusable AI instructions to fields and documents to supercharge your editorial workflow.

You create the instructions; Sanity AI Assist does the rest. Learn more about writing instructions in the Sanity documentation.

Read the release announcement here.

Using this feature requires Sanity to send data to OpenAI.com for processing. It uses generative AI; you should verify the data before using it.

Installation

In your Studio project folder, install the following plugin dependency:

npm install @sanity/assist sanity@latest

This plugin requires sanity version 3.16.0 or greater.

Setup

Note: Before using the plugin, your project must have Sanity AI Assist enabled at the API level.

Contact your Sanity enterprise representative to get started, or contact the sales team.

Add the plugin

In sanity.config.ts, add assist to the plugins array:

import { assist } from '@sanity/assist'

export default defineConfig({
  /* other config */
  plugins: [
    /* other plugins */
    assist()
  ]
})

Enabling the AI Assist API

After installing and adding the plugin and having the AI Assist feature enabled for your project and its datasets, you need to create a token for the plugin to access the AI Assist API. This needs to be done by a member of the project with token creation permissions (typically someone with an admin or developer role).

• Start the studio and open any document
• Click the sparkle icon* (✨) in the document header near the close document X-button
• Then select Manage instructions

• Selecting Manage instructions will open an inspector panel
• Click the Enable AI assistance button to create a token and enable AI Assist for everyone with access to the project

You will find a new API token entry for your project named “Sanity AI” in your project's API settings on sanity.io/manage.

The plugin will now work for any dataset in your project.

Note: You can revoke this token at any time to disable Sanity AI Assist service. A new token has to be generated via the plugin UI for it to work again.

Schema configuration

By default, most object, array, and string field types have AI writing assistance enabled. Your assistant can write to all compatible fields that it detects.

The assistant can also create array items, including Portable Text blocks, when the type has been imported to the Studio's schema as a custom type (learn more about strict schemas).

Disable AI Assist for a schema type

// this will disable AI assistance wherever it is used,
// ie: as field, document, array types
defineType({
 name: 'policy',
  type: 'document',
  options: {
    aiWritingAssistance: {exclude: true}
 },
  fields: [
    // ...
  ]
})

Disable for a field

// this disables AI assistance only for the specific field
defineType({
 name: 'product',
  type: 'object',
  fields: [
    defineField({
      name: 'sku',
      type: 'string',
      options: {
        aiWritingAssistance: {exclude: true}
     }
  })
  ]
})

Disable for an array type

// this disables AI assistance for the specific array member
// if all types in the `of` array are excluded, the array type is also considered excluded
defineType({
 name: 'product',
  type: 'array',
  of: [
    defineArrayMember({
      type: 'customProduct',
      options: {
        aiWritingAssistance: {exclude: true}
     }
  })
  ]
})

Unsupported types

The following types are not supported, and behave as excluded types:

• Number
• Reference
• Slug
• Url
• Date
• Datetime
• Image (supported when image has custom fields)
• File (never supported, even when file has custom fields)

Types and fields with hidden or readonly with a truthy value (true or function) are not supported. (Hidden and readOnly fields can be referenced in instructions still)

Fields with these types will not be changed by the assistant, do not have AI Assist actions, and cannot be referenced in instructions.

Objects where all fields are excluded or unsupported and arrays where all member types are excluded or unsupported will also be excluded.

Troubleshooting

There are limits to how much text the AI can process when processing an instruction. Under the hood, the AI Assist will add information about your schema, which adds to what's commonly referred to as “the context window.”

If you have a very large schema (that is, many document and field types), it can be necessary to exclude types to limit how much of the context window is used for the schema itself.

We recommend excluding any and all types which rarely would benefit from automated workflows. A quick win is typically to exclude array types. It can be a good idea to exclude most non-block types from Portable Text arrays. This will ensure that the Sanity Assist outputs mostly formatted text.

Included document types

This plugin adds an AI Context document type.

If your Studio uses Structure Builder to configure the studio structure, you might have to add this document type to your structure.

The document type name can be imported from the plugin:

import {contextDocumentTypeName} from '@sanity/assist'

// put into structure in structure  
S.documentTypeListItem(contextDocumentTypeName)

Caveats

Large Language Models (LLMs) are a new technology. Constraints and limitations are still being explored, but some common caveats to the field that you may run into using AI Assist are:

• Limits to instruction length: Long instructions on deep content structures may exhaust model context
• Timeouts: To be able to write structured content, we're using the largest language models - long-running results may time out or intermittently fail
• Limited capacity: The underlying LLM APIs used by AI Assist are resource constrained

Other features

Caption generation

AI Assist can optionally generate captions for images. This has to be enabled on an image-type/field, by setting the options.captionField on the image type, where captionField is the field name of a custom string-field on the image object:

defineField({
    type: 'image',
    name: 'inlineImage',
    title: 'Image',
    fields: [
      defineField({
        type: 'string',
        name: 'caption',
        title: 'Caption',
      }),
    ],
    options: {
      captionField: 'caption', 
    },
}),

This will add a "Generate caption" action to the configured field. "Generate caption" action will automatically run whenever the image changes.

Third party sub-processors

This version of the feature uses OpenAI.com as a third-party sub-processor. Their security posture has been vetted by Sanity's security team, and approved for use.

License

MIT © Sanity

Develop & test

This plugin uses @sanity/plugin-kit with default configuration for build & watch scripts.

See Testing a plugin in Sanity Studio on how to run this plugin with hotreload in the studio.

Release new version

Run "CI & Release" workflow. Make sure to select the main branch and check "Release new version".

Semantic release will only release on configured branches, so it is safe to run release on any branch.