@stability/stablestudio-plugin

🔌 Plugins

🗺 Contents – ℹ️ About · 📦 Installation · 🤝 Contract · 🔮 Future

⬆️ Top-Level README

🚧 👷 PLUGINS ARE EXPERIMENTAL AND NOT FINAL 🛠 🚧

The contract and implementation are likely to change...

ℹ️ About

In order to make StableStudio easier to extend, we've ripped out the "back-end" into a plugin system.

This means you can implement an entirely different inference stack, StableStudio doesn't care if it's local or a hosted API.

⭐️ Features

We're hoping this list expands over time, but here's what's available right now...

• 🏞 Image Generation

  All aspects of image generation are handled by plugins including the default input, how styles are implemented, how to fetch existing images, etc.

• 🪪 Plugin Manifest

  Plugins can provide a manifest with a markdown description, author details, version information, etc.

• ⚙️ Plugin Settings

  Plugins can declare settings which are available to the user in the settings menu.

🧪 Development

If you want to play around with an example plugin which has nice developer tooling, check out the example plugin.

You can load the UI using example plugin instead of the default Stability plugin by running...

yarn dev:use-example-plugin

Any changes you make within the example plugin will be hot-reloaded into StableStudio.

📦 Installation

This is mainly a feature intended for developers, we're likely to change how plugins work for end-users.

Never install plugins from untrusted sources!

Although they are sand-boxed to the browser, a malicious actor could read through your image history or steal your API key if you're not careful.

To install a plugin, first enable "developer mode" in the settings menu...

Now you can point to any fetch-able JavaScript file which exports a plugin...

If loaded successfully, you'll see it in the settings menu...

✅ That's it!

As of now, you can only have one active plugin at a time.

The main purpose of plugins in their current form is to allow developers to experiment with different inference stacks.

🤝 Contract

Check out the TypeScript source to see full documentation of the API.

Plugins are effectively just a bag of JavaScript functions.

StableStudio calls these functions for core functionality, such as createStableDiffusionImages, getStableDiffusionStyles, getStableDiffusionDefaultInput, etc.

Functionality degrades gracefully, for example, if you don't implement getStableDiffusionStyles, no styles will be shown in the UI.

🔮 Future

Plugins were mostly created to rip out calls to Stability's gRPC API, so their functionality is quite limited.

Given the importance of a robust plugin system, we're really hoping to gather feedback on how we can improve the API.

Here's what's on our mind when it comes to the future of StableStudio plugins...

• 🧠 More Inference Stacks

  The default Stability plugin enables inference relying on Stability's API, but we're hoping to expand support for local inference.

  We're particularly interested in creating a plugin for using stable-diffusion-webui's REST API.

  Though it's not fully-ready, we're excited about the future of WebGPU, and would love to support it via a plugin.

  We've stubbed out two empty packages to house these potential plugins...

  stablestudio-plugin-webui

  stablestudio-plugin-webgpu

• 🎨 Deeper UI Extensibility

  Currently plugins can only change "back-end" functionality. We'd love to allow plugins to create entire new features or modify existing ones.

• 🧱 Multiple Plugins

  Only one plugin can be loaded at a time with our current implementation. Allowing multiple plugins to play nicely with each other would be a huge win.

• 🛍 Plugin Library

  If there is enough interest and community momentum, we'd love to create a plugin library where users can easily install plugins from a curated list.