🪄SCEPTER is an open-source code repository dedicated to generative training, fine-tuning, and inference, encompassing a suite of downstream tasks such as image generation, transfer, editing. SCEPTER integrates popular community-driven implementations as well as proprietary methods by Tongyi Lab of Alibaba Group, offering a comprehensive toolkit for researchers and practitioners in the field of AIGC. This versatile library is designed to facilitate innovation and accelerate development in the rapidly evolving domain of generative models.
SCEPTER offers 3 core components:
Clay Style, De-Text, Segmentation, etc.Text-Based Style Editing.zoom-out, virtual try on, inpainting.ACE is a unified foundational model framework that supports a wide range of visual generation tasks. By defining CU for unifying multi-modal inputs across different tasks and incorporating long-context CU, we introduce historical contextual information into visual generation tasks, paving the way for ChatGPT-like dialog systems in visual generation.
| Model | Status |
|---|---|
| ACE-0.6B-512px |    |
| ACE-0.6B-1024px |  |
| ACE-12B-FLUX-dev | Coming Soon |
We offer a demonstration training YAML that enables the end-to-end training of ACE using a toy dataset. For a comprehensive overview of the hyperparameter configurations, please consult scepter/methods/edit/dit_ace_0.6b_512.yaml.
Please find the dataset class located in scepter/modules/data/dataset/ms_dataset.py,
designed to facilitate end-to-end training using an open-source toy dataset.
Download a dataset zip file from modelscope, and then extract its contents into the cache/datasets/ directory.
Should you wish to prepare your own datasets, we recommend consulting scepter/modules/data/dataset/ms_dataset.py for detailed guidance on the required data format.
The ACE checkpoint has been uploaded to both ModelScope and HuggingFace platforms:
In the provided training YAML configuration, we have designated the Modelscope URL as the default checkpoint URL. Should you wish to transition to Hugging Face, you can effortlessly achieve this by modifying the PRETRAINED_MODEL value within the YAML file (replace the prefix "ms://iic" to "hf://scepter-studio").
You can easily start training procedure by executing the following command:
# ACE-0.6B-512px
PYTHONPATH=. python scepter/tools/run_train.py --cfg scepter/methods/edit/dit_ace_0.6b_512.yaml
# ACE-0.6B-1024px
PYTHONPATH=. python scepter/tools/run_train.py --cfg scepter/methods/edit/dit_ace_0.6b_1024.yaml
We have developed a chatbot interface utilizing Gradio, designed to convert user input in natural language into visually captivating images that align semantically with the specified instructions. You can easily access this functionality by launching Scepter Studio with the following command:
PYTHONPATH=. python scepter/tools/webui.py --cfg scepter/methods/studio/scepter_ui.yaml --language zh --tab chatbot
Upon starting, you will find a "ChatBot" tab within the Gradio application, which serves as a chat-based interface to handle any requests related to image editing or generation.
| ACE Workflow Examples | ||
|---|---|---|
| Control | Semantic | Element |
|
|
|
| Yarn Style | Soft Watercolor Style | ||||
|---|---|---|---|---|---|
 |  |  |  |  |  |
| Travel Style | WuKong Style | ||||
 |  |  |  |  |  |
| Example Workflow Case | |||
|---|---|---|---|
| Base | +Mantra | +Tuner | +Control |
|
|
|
|
conda command:conda env create -f environment.yaml
conda activate scepter
pip command:We recommend installing the specific version of PyTorch and accelerate toolbox xFormers. You can install these recommended version by pip:
pip install -r requirements/recommended.txt
pip install scepter
| Documentation | Key Features |
|---|---|
| Train | DDP / FSDP / FairScale / Xformers |
| Inference | Dynamic load/unload |
| Dataset Management | Local / Http / OSS / Modelscope |
| Tasks | Methods | Links |
|---|---|---|
| Text-to-image Generation | SD v1.5 |  |
| Text-to-image Generation | SD v2.1 | |
| Text-to-image Generation | SD-XL | |
| Text-to-image Generation | FLUX | |
| Efficient Tuning | LoRA |  |
| Efficient Tuning | Res-Tuning(NeurIPS23) |   |
| Controllable Image Synthesis | 🌟SCEdit(CVPR24) |   |
| Image Editing | 🌟LAR-Gen |   |
| Image Editing | 🌟StyleBooth |   |
| Image Generation and Editing | 🌟ACE |    |
To fully experience SCEPTER Studio, you can launch the following command line:
pip install scepter
python -m scepter.tools.webui
or run after clone repo code
git clone https://github.com/modelscope/scepter.git
PYTHONPATH=. python scepter/tools/webui.py --cfg scepter/methods/studio/scepter_ui.yaml
The startup of SCEPTER Studio eliminates the need for manual downloading and organizing of models; it will automatically load the corresponding models and store them in a local directory. Depending on the network and hardware situation, the initial startup usually requires 15-60 minutes, primarily involving the download and processing of SDv1.5, SDv2.1, and SDXL models. Therefore, subsequent startups will become much faster (about one minute) as downloading is no longer required.
| Image Editing | Training | Model Sharing | Model Inference | Data Management |
|---|---|---|---|---|
We deploy a work studio on Modelscope that includes only the inference tab, please refer to ms_scepter_studio and hf_scepter_studio
We support the use of all models in the ComfyUI Workflow through the following methods:
git clone https://github.com/modelscope/scepter.git
cd path/to/scepter
pip install -e .
cp -r path/to/scepter/workflow/ path/to/ComfyUI/custom_nodes/ComfyUI-Scepter
cd path/to/ComfyUI
python main.py
Note: You can use the nodes by dragging the sample images into ComfyUI. Additionally, our nodes can automatically pull models from ModelScope or HuggingFace by selecting the model_source field, or you can place the already downloaded models in a local path.
Alibaba TongYi Vision Intelligence Lab
Discover more about open-source projects on image generation, video generation, and editing tasks.
ModelScope Library is the model library of ModelScope project, which contains a large number of popular models.
SWIFT (Scalable lightWeight Infrastructure for Fine-Tuning) is an extensible framwork designed to faciliate lightweight model fine-tuning and inference.
If our work is useful for your research, please consider citing:
@misc{scepter,
title = {SCEPTER, https://github.com/modelscope/scepter},
author = {SCEPTER},
year = {2023}
}
This project is licensed under the Apache License (Version 2.0).
Thanks to Stability-AI, SWIFT library, Fooocus and ComfyUI for their awesome work.
FAQs
Unknown package
We found that scepter demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 5 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Fluent Assertions is facing backlash after dropping the Apache license for a commercial model, leaving users blindsided and questioning contributor rights.
Research
Security News
Socket researchers uncover the risks of a malicious Python package targeting Discord developers.
Security News
The UK is proposing a bold ban on ransomware payments by public entities to disrupt cybercrime, protect critical services, and lead global cybersecurity efforts.