github.com/instill-ai/visual-data-pipeline

🔮 Instill Core

✨ Make your unstructured data AI-ready FAST & build unique versatile AI-first applications EASILY. ✨

Quick start

There are a few ways to start exploring:

1. Instill Cloud: Explore our public cloud powered by Instill Core with ready-to-use APIs, automations, AI Assistants and more. See the quickstart on how to get started.
2. Self-hosting Instill Core: Follow the installation steps below or documentation for more details to build versatile AI applications locally.

What is Instill Core?

Instill Core is your end-to-end AI platform for data, pipeline and model orchestration.

Instill Core simplifies infrastructure hassle and encompasses these core features:

• DATA - Instill Artifact: Transforms unstructured data (documents, images, audio, video) into AI-ready formats.
• PIPELINE - Instill VDP: Quickly build any versatile AI-first APIs or automated workflows.
• MODEL - Instill Model: MLOps/LLMOps platform for model serving and monitoring.

What can you build?

• 📖 Parsing PDF Files to Markdown: Cookbook
• 🧱 Generating Structured Outputs from LLMs: Cookbook & Tutorial
• 🕸️ Web scraping & Google Search with Structured Insights: Tutorial & YouTube
• 🌱 Instance segmentation on microscopic plant stomata images: Cookbook

See Examples for more!

Installation

Prerequisites

Operating System	Requirements and Instructions
macOS or Linux	Instill Core works natively
Windows	• Use Windows Subsystem for Linux (WSL2) • Install latest yq from GitHub Repository • Install latest Docker Desktop and enable WSL2 integration (tutorial) • (Optional) Install cuda-toolkit on WSL2 (NVIDIA tutorial)
All Systems	• Docker Engine v25 or later • Docker Compose v2 or later • Install latest stable Docker and Docker Compose

Steps

Use stable release version

Execute the following commands to pull pre-built images with all the dependencies to launch:

$ git clone -b v0.48.0-beta https://github.com/instill-ai/instill-core.git && cd instill-core

# Launch all services
$ make all

[!NOTE] We have restructured our project repositories. If you need to access 🔮 Instill Core projects up to version v0.13.0-beta, please refer to the instill-ai/deprecated-core repository.

Use the latest version for local development

Execute the following commands to build images with all the dependencies to launch:

$ git clone https://github.com/instill-ai/instill-core.git && cd instill-core

# Launch all services
$ make latest PROFILE=all

[!IMPORTANT] Code in the main branch tracks under-development progress towards the next release and may not work as expected. If you are looking for a stable alpha version, please use latest release.

🚀 That's it! Once all the services are up with health status, the UI is ready to go at http://localhost:3000. Please find the default login credentials in the documentation.

To shut down all running services:

$ make down

Explore the documentation to discover all available deployment options.

Deployment

You can either self-host 🔮 Instill Core or choose our fully managed ☁️ Instill Cloud to get access to all the latest features. Visit the Deployment Overview for more details.

Client Access

• 📺 Instill Console
• ⌨️ Instill CLI
• 📦 Instill SDK:
  • Python SDK
  • TypeScript SDK
  • Stay tuned, as more SDKs are on the way!

Documentation

Please visit our official documentation for more.

Additional resources:

• API Reference
• Cookbooks
• Tutorials
• Examples

Contributing

We welcome contributions from our community! Checkout the methods below:

1. Cookbooks: Help us create helpful pipelines and guides for the community. Visit our Cookbook repository to get started.

2. Issues: Contribute to improvements by raising tickets using templates here or discuss in existing ones you think you can help with.

Community Standards

We are committed to maintaining a respectful and welcoming atmosphere for all contributors. Before contributing, please read:

• Contributing Guidelines
• Code of Conduct

Support

Get help by joining our Discord community where you can post any questions on our #ask-for-help channel.

Contributors ✨

Thank you to all these wonderful people (emoji key):

Vibhor Bhatt	Miguel Ortiz	Sajda Kabir	Henry Chen	Hari Bhandari	Shiva Gaire
Zubeen	ShihChun-H	Ikko Eltociear Ashimine	Farookh Zaheer Siddiqui	Brian Gallagher	hairyputtar
David Marx	Deniz Parlak	Po-Yu Chen	Po Chun Chiu	Sarthak	HR Wu
phelan	Chang, Hui-Tang	Xiaofei Du	Ping-Lin Chang	Tony Wang	Pratik date
Juan Vallés	Naman Anand	totuslink	Praharsh Jain	Utsav Paul	CaCaBlocker
Rafael Melo	Jeremy Shih	Romit Mohane	ChunHao	Amelia C	楊竣凱
andre.liang	Zoodane	George Strong	Anni	Mubeen Kodvavi	RCKT
Wojciech Bandzerewicz	Gary	Leo	felixcorleone	Zoe	Daniel
Manul Thanura	Akash Jana	Anish0203	Prathamesh Tugaonkar	Shubham

This project follows the all-contributors specification. Contributions of any kind welcome!

License

See the LICENSE file for licensing information.