Product
Introducing SSO
Streamline your login process and enhance security by enabling Single Sign-On (SSO) on the Socket platform, now available for all customers on the Enterprise plan, supporting 20+ identity providers.
Readme
This repository contains an experimental torrent-style download for model weights in Hugging Face.
The core implementation is main_hub.py
, which downloads files from Hugging Face and converts them into seedable torrents.
pip install hf-torrent
Before launching downloads, make sure you have aria2c installed and
aria2c --enable-rpc --rpc-listen-all \
--max-upload-limit=4M \
--optimize-concurrent-downloads=true \
--bt-detach-seed-only=true \
--seed-ratio=10.0
Then you can start using HF-Torrent with the following command:
hf-torrent <your target repo>
Or if you have your preferred torrent clients, you can pull the torrent file only
hf-torrent --get-torrent <your target repo>
To install, run the following command:
pip install -r requirements.txt
It is easy to contribute to seeding! First, change --seed-ratio=0
(from 10 to 0) when launching the aria2c
RPC. Setting 0
means the daemon will seed infinitely.
Next, change the upload bandwidth limit to a value that suits you, for example: --max-upload-limit=20M
(4M by default).
After making these changes, launching seeding is as simple as downloading models. You can start seeding by running the following script and help distribute your favorite models!
python download.py <your target repo>
To create a popular torrent, run the following command:
python generate_popular_torrents.py -r efederici/retnet-120m
This script will (1) download model files from Hugging Face, (2) iterate through the folder, (3) generate torrents for each file, and then put them under hf-torrent-store/{repo}
.
We are looking for a wanted model list. Please open an issue if you have any favorite models to share.
FAQs
A tool to download HuggingFace models via BitTorrent.
We found that hf-torrent demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Product
Streamline your login process and enhance security by enabling Single Sign-On (SSO) on the Socket platform, now available for all customers on the Enterprise plan, supporting 20+ identity providers.
Security News
Tea.xyz, a crypto project aimed at rewarding open source contributions, is once again facing backlash due to an influx of spam packages flooding public package registries.
Security News
As cyber threats become more autonomous, AI-powered defenses are crucial for businesses to stay ahead of attackers who can exploit software vulnerabilities at scale.