@annotorious/plugin-segment-anything

Annotorious SegmentAnything Plugin

A fully browser-based smart polygon selection tool for Annotorious based on the sam2-hiera-tiny SegmentAnything model. The basic approach is inspired by this blog post and demo code.

Important: this plugin only supports @annotorious/openseadragon at this time. Support for plain (JPEG, PNG,...) images is not yet implemented. Get in touch via the forum if you are interested in using this with the @annotorious/annotorious or @annotorious/react packages.

Using with OpenSeadragon

npm install @annotorious/plugin-segment-anything

import OpenSeadragon from 'openseadragon';
import { createOSDAnnotator } from '@annotorious/openseadragon';
import { mountOpenSeadragonPlugin } from '@annotorious/plugin-segment-anything/openseadragon';

import '@annotorious/openseadragon/annotorious-openseadragon.css';

const viewer = OpenSeadragon({
  /** init your viewer **/
});

const anno = createOSDAnnotator(viewer, { /* options */ });

// Initialize the plugin
const plugin = mountOpenSeadragonPlugin(anno);

// This will start initializing the plugin, incl.
// download of the model (this may take a while).
plugin.init();

plugin.on('downloadStart', () => {
  console.log('downloading the model - this may take a while');
});

plugin.on('downloadProgress', progress => {
  if (progress.complete)
    console.log('downloading complete');
});

plugin.on('initialized', () => {
  console.log('plugin ready');
});

plugin.on('encodingStart', () => {
  console.log('busy - encoding viewport');
});

plugin.on('encodingFinished', () => {
  console.log('ready - click to create annotation!');
});