You're Invited:Meet the Socket Team at BlackHat and DEF CON in Las Vegas, Aug 4-6.RSVP
Socket
Book a DemoInstallSign in
Socket

mage-ai

Package Overview
Dependencies
Maintainers
2
Alerts
File Explorer

Advanced tools

Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

mage-ai

Mage is a tool for building and deploying data pipelines.

0.9.76
pipPyPI
Maintainers
2

Mage AI

Give your data team magical powers.

Mage AI GitHub repo stars Mage AI Docker downloads Mage AI license Join the Mage AI community


Mage AI hero

Mage is a hybrid framework for transforming and integrating data. It combines the best of both worlds: the flexibility of notebooks with the rigor of modular code.


  • Extract and synchronize data from 3rd party sources.
  • Transform data with real-time and batch pipelines using Python, SQL, and R.
  • Load data into your data warehouse or data lake using our pre-built connectors.
  • Run, monitor, and orchestrate thousands of pipelines without losing sleep.

Plus hundreds of enterprise-class features, infrastructure innovations, and magical surprises.

Available in two spellbinding versions


Mage Pro For teams. Fully managed platform for integrating and transforming data. Mage OSS Self-hosted. System to build, run, and manage data pipelines.

Try out Mage Pro

It’s magic.

For documentation on getting started, how to develop, and how to deploy to production check out the live
Developer documentation portal.


🏃‍♀️ Install

The recommended way to install the latest version of Mage is through Docker with the following command:

docker pull mageai/mageai:latest

You can also install Mage using pip or conda, though this may cause dependency issues without the proper environment.

pip install mage-ai
conda install -c conda-forge mage-ai

Looking for help? The fastest way to get started is by checking out our documentation here.

Looking for quick examples? Open a demo project right in your browser or check out our guides.

🎮 Demo

Live demo

Build and run a data pipeline with our demo app.

WARNING

The live demo is public to everyone, please don’t save anything sensitive (e.g. passwords, secrets, etc).

Demo video (5 min)

Mage quick start demo

Click the image to play video


🔮 Features

🎶OrchestrationSchedule and manage data pipelines with observability.
📓NotebookInteractive Python, SQL, & R editor for coding data pipelines.
🏗️Data integrationsSynchronize data from 3rd party sources to your internal destinations.
🚰Streaming pipelinesIngest and transform real-time data.
dbtBuild, run, and manage your dbt models with Mage.

A sample data pipeline defined across 3 files ➝


  • Load data ➝
    @data_loader
    def load_csv_from_file() -> pl.DataFrame:
        return pl.read_csv('default_repo/titanic.csv')
    
  • Transform data ➝
    @transformer
    def select_columns_from_df(df: pl.DataFrame, *args) -> pl.DataFrame:
        return df[['Age', 'Fare', 'Survived']]
    
  • Export data ➝
    @data_exporter
    def export_titanic_data_to_disk(df: pl.DataFrame) -> None:
        df.to_csv('default_repo/titanic_transformed.csv')
    

Water mage casting spell

FAQs

Did you know?

Socket

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

Related posts