/badge.svg){kind=icon}
/badge.svg){kind=icon}
iden is a simple Python library to manage a dataset of shards when training a machine learning
model.
iden uses a lazy loading approach to load the shard's data, so it is easy to manage shards without
loading their data.
iden supports different format to store shards on disk.
We highly recommend installing
a virtual environment.
iden can be installed from pip using the following command:
pip install iden
To make the package as slim as possible, only the minimal packages required to use iden are
installed.
To include all the dependencies, you can use the following command:
pip install iden[all]
Please check the get started page to see how to
install only some specific dependencies or other alternatives to install the library.
The following is the corresponding iden versions and tested dependencies.
iden | coola | objectory | numpy* | pyyaml* | safetensors* | torch* | python |
|---|---|---|---|---|---|---|---|
main | >=0.8.4,<1.0 | >=0.2,<1.0 | >=1.22,<2.0 | >=6.0,<7.0 | >=0.4,<1.0 | >=2.0,<3.0 | >=3.9,<3.14 |
0.2.0 | >=0.8.4,<1.0 | >=0.2,<1.0 | >=1.22,<2.0 | >=6.0,<7.0 | >=0.4,<1.0 | >=2.0,<3.0 | >=3.9,<3.14 |
0.1.0 | >=0.8.4,<1.0 | >=0.2,<1.0 | >=1.22,<2.0 | >=6.0,<7.0 | >=0.4,<1.0 | >=2.0,<3.0 | >=3.9,<3.14 |
0.0.4 | >=0.3,<1.0 | >=0.1,<1.0 | >=1.22,<2.0 | >=6.0,<7.0 | >=0.4,<1.0 | >=2.0,<3.0 | >=3.9,<3.13 |
0.0.3 | >=0.3,<1.0 | >=0.1,<1.0 | >=1.22,<2.0 | >=6.0,<7.0 | >=0.4,<1.0 | >=2.0,<3.0 | >=3.9,<3.12 |
0.0.2 | >=0.4,<1.0 | >=0.1,<1.0 | >=1.22,<2.0 | >=6.0,<7.0 | >=0.4,<1.0 | >=2.0,<2.1 | >=3.9,<3.12 |
0.0.1 | >=0.4,<1.0 | >=0.1,<1.0 | >=1.22,<2.0 | >=6.0,<7.0 | >=0.4,<1.0 | >=2.0,<2.1 | >=3.9,<3.12 |
iden | cloudpickle* | joblib* |
|---|---|---|
main | >=3.0,<4.0 | >=1.3,<2.0 |
* indicates an optional dependency
Please check the instructions in CONTRIBUTING.md.
Everyone is welcome to contribute to the community. If you have any questions or suggestions, you can submit Github Issues. We will reply to you as soon as possible. Thank you very much.
:warning: While iden is in development stage, no API is guaranteed to be stable from one
release to the next.
In fact, it is very likely that the API will change multiple times before a stable 1.0.0 release.
In practice, this means that upgrading iden to a new version will possibly break any code that
was using the old version of iden.
iden is licensed under BSD 3-Clause "New" or "Revised" license available in LICENSE
file.
FAQs
simple library to manage a dataset of shards to train machine learning models
We found that iden demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
npm now supports Trusted Publishing with OIDC, enabling secure package publishing directly from CI/CD workflows without relying on long-lived tokens.
Research
/Security News
A RubyGems malware campaign used 60 malicious packages posing as automation tools to steal credentials from social media and marketing tool users.
Security News
The CNA Scorecard ranks CVE issuers by data completeness, revealing major gaps in patch info and software identifiers across thousands of vulnerabilities.