Collective Mind (CM) is a very lightweight Python-based framework featuring a unified CLI, Python API, and minimal dependencies. It is available through PYPI.
CM is designed for creating and managing portable and technology-agnostic automations for MLOps, DevOps and ResearchOps. It aims to assist researchers and engineers in automating their repetitive, tedious and time-consuming tasks to build, run, benchmark and optimize various applications across diverse and continuously changing models, data, software and hardware.
Collective Mind is a part of Collective Knowledge (CK) - an educational community project to learn how to run AI, ML and other emerging workloads in the most efficient and cost-effective way across diverse and ever-evolving systems using the MLPerf benchmarking methodology.
The diagram below illustrates the primary classes, functions, and internal automations within the Collective Mind framework:
The CM API documentation is available here.
Collective Mind is continuously enhanced through public and private CM4* Git repositories, which serve as the unified interface for various collections of reusable automations and artifacts.
The most notable projects and repositories powered by CM are:
CM4MLOPS repository powered by CM - a collection of portable, extensible and technology-agnostic automation recipes with a common CLI and Python API (CM scripts) to unify and automate all the manual steps required to compose, run, benchmark and optimize complex ML/AI applications on diverse platforms with any software and hardware.
The two key automations are *script" and cache: see online catalog at CK playground, online MLCommons catalog.
CM scripts extend the concept of cmake with simple Python automations, native scripts
and JSON/YAML meta descriptions. They require Python 3.7+ with minimal dependencies and are
continuously extended by the community and MLCommons members
to run natively on Ubuntu, MacOS, Windows, RHEL, Debian, Amazon Linux
and any other operating system, in a cloud or inside automatically generated containers
while keeping backward compatibility.
CM scripts were originally developed based on the following requirements from the MLCommons members to help them automatically compose and optimize complex MLPerf benchmarks, applications and systems across diverse and continuously changing models, data sets, software and hardware from Nvidia, Intel, AMD, Google, Qualcomm, Amazon and other vendors:
See the online documentation at MLCommons to run MLPerf inference benchmarks across diverse systems using CM.
CM4ABTF repository powered by CM - a collection of portable automations and CM scripts to run the upcoming automotive MLPerf benchmark across different models, data sets, software and hardware from different vendors.
CM4MLPerf-results powered by CM - a simplified and unified representation of the past MLPerf results in the CM format for further visualization and analysis using CK graphs.
CM4Research repository powered by CM - a unified interface designed to streamline the preparation, execution, and reproduction of experiments in research projects.
Collective Knowledge Playground - a unified and open-source platform designed to index all CM scripts similar to PYPI, assist users in preparing CM commands to:
These initiatives aim to help academia and industry collaboratively enhance the efficiency and cost-effectiveness of AI systems.
Artifact Evaluation automation - a community-driven initiative leveraging the Collective Mind framework to automate artifact evaluation and support reproducibility efforts at ML and systems conferences.
CM scripts extend the concept of cmake with simple Python automations, native scripts
and JSON/YAML meta descriptions. They require Python 3.7+ with minimal dependencies and are
continuously extended by the community and MLCommons members
to run natively on Ubuntu, MacOS, Windows, RHEL, Debian, Amazon Linux
and any other operating system, in a cloud or inside automatically generated containers
while keeping backward compatibility.
CM scripts were originally developed based on the following requirements from the MLCommons members to help them automatically compose and optimize complex MLPerf benchmarks, applications and systems across diverse and continuously changing models, data sets, software and hardware from Nvidia, Intel, AMD, Google, Qualcomm, Amazon and other vendors:
CM4MLOPS / CM4MLPerf - a collection of portable, extensible and technology-agnostic automation recipes with a common CLI and Python API (CM scripts) to unify and automate all the manual steps required to compose, run, benchmark and optimize complex ML/AI applications on diverse platforms with any software and hardware: see online catalog at CK playground, online MLCommons catalog
CM4ABTF - a unified CM interface and automation recipes to run automotive benchmark across different models, data sets, software and hardware from different vendors.
CM4Research - a unified CM interface an automation recipes to make it easier to reproduce results from published research papers.
If you found CM automations, please cite this article: [ ArXiv ], [ BibTex ].
You can learn more about the motivation behind these projects from the following presentations:
Collective Mind (CM) was originally developed by Grigori Fursin, as a part of the Collective Knowledge educational initiative, sponsored by cTuning.org and cKnowledge.org, and contributed to MLCommons for the benefit of all.
This open-source technology, including CM4MLOps/CM4MLPerf, CM4ABTF, CM4Research, and more, is a collaborative project supported by MLCommons, FlexAI, cTuning and our amazing volunteers, collaborators, and contributors!
FAQs
Collective Mind automation framework (CM)
We found that cmind demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
The Socket Research Team breaks down a malicious wrapper package that uses obfuscation to harvest credentials and exfiltrate sensitive data.
Research
Security News
Attackers used a malicious npm package typosquatting a popular ESLint plugin to steal sensitive data, execute commands, and exploit developer systems.
Security News
The Ultralytics' PyPI Package was compromised four times in one weekend through GitHub Actions cache poisoning and failure to rotate previously compromised API tokens.
