scepter

🪄SCEPTER

🪄SCEPTER is an open-source code repository dedicated to generative training, fine-tuning, and inference, encompassing a suite of downstream tasks such as image generation, transfer, editing. SCEPTER integrates popular community-driven implementations as well as proprietary methods by Tongyi Lab of Alibaba Group, offering a comprehensive toolkit for researchers and practitioners in the field of AIGC. This versatile library is designed to facilitate innovation and accelerate development in the rapidly evolving domain of generative models.

SCEPTER offers 3 core components:

• Generative training and inference framework
• Easy implementation of popular approaches
• Interactive user interface: SCEPTER Studio

🎉 News

• [2024.04]: New StyleBooth demo on SCEPTER Studio forText-Based Style Editing.
• [2024.03]: We optimize the training UI and checkpoint management. New LAR-Gen model has been added on SCEPTER Studio, supporting zoom-out, virtual try on, inpainting.
• [2024.02]: We release new SCEdit controllable image synthesis models for SD v2.1 and SD XL. Multiple strategies applied to accelerate inference time for SCEPTER Studio.
• [2024.01]: We release SCEPTER Studio, an integrated toolkit for data management, model training and inference based on Gradio.
• [2024.01]: SCEdit support controllable image synthesis for training and inference.
• [2023.12]: We propose SCEdit, an efficient and controllable generation framework.
• [2023.12]: We release 🪄SCEPTER library.

🖼 Gallery for Recent Works

StyleBooth

Origin Image Gold Dragon Tuner	Graffiti Art	Adorable Kawaii	game-retro game	Vincent van Gogh

Origin Image	Lowpoly	Colored Pencil Art	Watercolor	misc-disco

🛠️ Installation

• Create new environment

conda env create -f environment.yaml
conda activate scepter

• We recommend installing the specific version of PyTorch and accelerate toolbox xFormers. You can install these recommended version by pip:

pip install -r requirements/recommended.txt

• Install SCEPTER by the pip command:

pip install scepter

🧩 Generative Framework

Tutorials

Documentation	Key Features
Train	DDP / FSDP / FairScale / Xformers
Inference	Dynamic load/unload
Dataset Management	Local / Http / OSS / Modelscope

📝 Popular Approaches

Currently supported approaches

Tasks	Methods	Links
Text-to-image generation	SD v1.5
Text-to-image generation	SD v2.1
Text-to-image generation	SD-XL
Efficient Tuning	LoRA
Efficient Tuning	Res-Tuning(NeurIPS23)
Controllable image synthesis	🌟SCEdit(CVPR24)
Image editing	🌟LAR-Gen
Image editing	🌟StyleBooth

🖥️ SCEPTER Studio

Launch

To fully experience SCEPTER Studio, you can launch the following command line:

pip install scepter
python -m scepter.tools.webui

or run after clone repo code

git clone https://github.com/modelscope/scepter.git
PYTHONPATH=. python scepter/tools/webui.py --cfg scepter/methods/studio/scepter_ui.yaml

The startup of SCEPTER Studio eliminates the need for manual downloading and organizing of models; it will automatically load the corresponding models and store them in a local directory. Depending on the network and hardware situation, the initial startup usually requires 15-60 minutes, primarily involving the download and processing of SDv1.5, SDv2.1, and SDXL models. Therefore, subsequent startups will become much faster (about one minute) as downloading is no longer required.

To support the sharing and downloading of models, please make sure that you have installed zip and Git Large File Storage (git lfs).

Usage Demo

Image Editing	Training	Model Sharing	Model Inference	Data Management

Modelscope Studio & Huggingface Space

We deploy a work studio on Modelscope that includes only the inference tab, please refer to ms_scepter_studio and hf_scepter_studio

🔍 Learn More

• Alibaba TongYi Vision Intelligence Lab

  Discover more about open-source projects on image generation, video generation, and editing tasks.

• ModelScope library

  ModelScope Library is the model library of ModelScope project, which contains a large number of popular models.

• SWIFT library

  SWIFT (Scalable lightWeight Infrastructure for Fine-Tuning) is an extensible framwork designed to faciliate lightweight model fine-tuning and inference.

BibTeX

If our work is useful for your research, please consider citing:

@misc{scepter,
    title = {SCEPTER, https://github.com/modelscope/scepter},
    author = {SCEPTER},
    year = {2023}
}

License

This project is licensed under the Apache License (Version 2.0).

Acknowledgement

Thanks to Stability-AI, SWIFT library and Fooocus for their awesome work.