Multi-Agent Resource Optimization (MARO) platform is an instance of Reinforcement learning as a Service (RaaS) for real-world resource optimization. It can be applied to many important industrial domains, such as container inventory management <https://maro.readthedocs.io/en/latest/scenarios/container_inventory_management.html>_ in logistics, bike repositioning <https://maro.readthedocs.io/en/latest/scenarios/citi_bike.html>_ in transportation, virtual machine <https://maro.readthedocs.io/en/latest/scenarios/vm_scheduling.html>_ provisioning in data centers, and asset management in finance. Besides Reinforcement Learning <https://www.andrew.cmu.edu/course/10-703/textbook/BartoSutton.pdf>_ (RL), it also supports other planning/decision mechanisms, such as Operations Research <https://en.wikipedia.org/wiki/Operations_research>_.

Key Components of MARO:

Simulation toolkit: it provides some predefined scenarios, and the reusable wheels for building new scenarios.
RL toolkit: it provides a full-stack abstraction for RL, such as agent manager, agent, RL algorithms, learner, actor, and various shapers.
Distributed toolkit: it provides distributed communication components, interface of user-defined functions for message auto-handling, cluster provision, and job orchestration.

.. image:: https://github.com/microsoft/maro/blob/master/docs/source/images/maro_overview.svg :target: https://github.com/microsoft/maro/blob/master/docs/source/images/maro_overview.svg :alt: MARO Key Components

.. list-table:: :header-rows: 1

- File/folder
- Description
- maro
- MARO source code.
- docs
- MARO docs, it is host on readthedocs <https://maro.readthedocs.io/en/latest/>_.
- examples
- Showcase of MARO.
- notebooks
- MARO quick-start notebooks.

Try MARO playground <#run-playground>_ to have a quick experience.

Install MARO from `PyPI <https://pypi.org/project/pymaro/#files>`_

Notes: The CLI commands (including the visualization tool) are not included in pymaro package. To enable these support, you need to install from source.

Mac OS / Linux

.. code-block:: sh

pip install pymaro
Windows

.. code-block:: powershell

Install torch first, if you don't have one.

pip install torch===1.6.0 torchvision===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

pip install pymaro

Install MARO from Source

Notes: Install from source if you want to use the CLI commands (including the visualization tool).

Prerequisites
- C++ Compiler
  - Linux or Mac OS X: gcc
  - Windows: Build Tools for Visual Studio 2017 <https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=BuildTools&rel=15>_
Enable Virtual Environment
- Mac OS / Linux
  
  .. code-block:: sh
  
  If your environment is not clean, create a virtual environment firstly.
  
  python -m venv maro_venv source ./maro_venv/bin/activate
- Windows
  
  .. code-block:: powershell
  
  If your environment is not clean, create a virtual environment firstly.
  
  python -m venv maro_venv
  
  You may need this for SecurityError in PowerShell.
  
  Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy Unrestricted
  
  Activate the virtual environment.
  
  .\maro_venv\Scripts\activate
Install MARO

.. code-block:: sh

Git Clone the whole source code.

git clone https://github.com/microsoft/maro.git
- Mac OS / Linux
  
  .. code-block:: sh
  
  Install MARO from source.
  
  bash scripts/install_maro.sh
- Windows
  
  .. code-block:: powershell
  
  Install MARO from source.
  
  .\scripts\install_maro.bat
Notes: If your package is not found, remember to set your PYTHONPATH
- Mac OS / Linux
.. code-block:: sh

export PYTHONPATH=PATH-TO-MARO
- Windows
.. code-block:: powershell

$Env:PYTHONPATH=PATH-TO-MARO

Quick Example

.. code-block:: python

from maro.simulator import Env

env = Env(scenario="cim", topology="toy.5p_ssddd_l0.0", start_tick=0, durations=100)

metrics, decision_event, is_done = env.step(None)

while not is_done: metrics, decision_event, is_done = env.step(None)

print(f"environment metrics: {env.metrics}")

`Environment Visualization <https://maro.readthedocs.io/en/latest/>`_

.. code-block:: sh

Enable environment dump feature, when initializing the environment instance

env = Env(scenario="cim", topology="toy.5p_ssddd_l0.0", start_tick=0, durations=100, options={"enable-dump-snapshot": "./dump_data"})

Inspect environment with the dump data

maro inspector dashboard --source_path ./dump_data/YOUR_SNAPSHOT_DUMP_FOLDER

Show Cases ^^^^^^^^^^

Case I - Container Inventory Management

.. image:: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/cim_inter_epoch.gif :target: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/cim_inter_epoch.gif :alt: CIM Inter Epoch

.. image:: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/cim_intra_epoch_by_ports.gif :target: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/cim_intra_epoch_by_ports.gif :alt: CIM Intra Epoch
Case II - Citi Bike

.. image:: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/citi_bike_inter_epoch.gif :target: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/citi_bike_inter_epoch.gif :alt: Citi Bike Inter Epoch

.. image:: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/citi_bike_intra_epoch_by_station.gif :target: https://github.com/microsoft/maro/blob/master/docs/source/images/visualization/dashboard/citi_bike_intra_epoch_by_station.gif :alt: Citi Bike Intra Epoch

Run Playground

Pull from Docker Hub <https://hub.docker.com/r/maro2020/playground>_

.. code-block:: sh

Pull the docker image from docker hub

docker pull maro2020/playground

Run playground container.

Redis commander (GUI for redis) -> http://127.0.0.1:40009

Jupyter lab with maro -> http://127.0.0.1:40010

docker run -p 40009:40009 -p 40010:40010 maro2020/playground
Build from source
- Mac OS / Linux
  
  .. code-block:: sh
  
  Build playground image.
  
  bash ./scripts/build_playground.sh
  
  Run playground container.
  
  Redis commander (GUI for redis) -> http://127.0.0.1:40009
  
  Jupyter lab with maro -> http://127.0.0.1:40010
  
  docker run -p 40009:40009 -p 40010:40010 maro2020/playground
- Windows
  
  .. code-block:: powershell
  
  Build playground image.
  
  .\scripts\build_playground.bat
  
  Run playground container.
  
  Redis commander (GUI for redis) -> http://127.0.0.1:40009
  
  Jupyter lab with maro -> http://127.0.0.1:40010
  
  docker run -p 40009:40009 -p 40010:40010 maro2020/playground

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct <https://opensource.microsoft.com/codeofconduct/>. For more information see the Code of Conduct FAQ <https://opensource.microsoft.com/codeofconduct/faq/> or contact opencode@microsoft.com <mailto:opencode@microsoft.com>_ with any additional questions or comments.

Container Inventory Management <https://maro.readthedocs.io/en/latest/scenarios/container_inventory_management.html>_ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. image:: https://github.com/microsoft/maro/blob/master/docs/source/images/scenario/cim_vis.gif :target: https://github.com/microsoft/maro/blob/master/docs/source/images/scenario/cim_vis.gif :alt: CIM Vis

Wenlei Shi, Xinran Wei, Jia Zhang, Xiaoyuan Ni, Arthur Jiang, Jiang Bian, Tie-Yan Liu. "\ Cooperative Policy Learning with Pre-trained Heterogeneous Observation Representations <https://arxiv.org/abs/2012.13099>_\ ". AAMAS 2021

Xihan Li, Jia Zhang, Jiang Bian, Yunhai Tong, Tie-Yan Liu. "\ A Cooperative Multi-Agent Reinforcement Learning Framework for Resource Balancing in Complex Logistics Network <https://arxiv.org/abs/1903.00714>_\ ". AAMAS 2019

MSRA Top-10 Hack-Techs in 2021 <https://mp.weixin.qq.com/s/Y4kjQq8gKcsEsVadjdwnEQ>_

Open Source Platform MARO: Anywhere Door for Resource Optimization <https://mp.weixin.qq.com/s/zXIpgzomLhDWS_YUFmRlEQ>_

AI from "Point" to "Surface" <https://mp.weixin.qq.com/s/PggO49wwmS7grTO0nEMgVQ>_

`Cite Us <https://github.com/microsoft/maro/blob/master/CITATION>`_

License

Licensed under the MIT <https://github.com/microsoft/maro/blob/master/LICENSE>_ License.

Keywords

FAQs

What is pymaro?

Is pymaro well maintained?

Did you know?

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

pymaro

Contents

Install MARO from `PyPI <https://pypi.org/project/pymaro/#files>`_