bert4torch
Advanced tools
Documentation | Torch4keras | Examples | build_MiniLLM_from_scratch | bert4vector
安装稳定版
pip install bert4torch
安装最新版
pip install git+https://github.com/Tongjilibo/bert4torch
git clone https://github.com/Tongjilibo/bert4torch，修改example中的预训练模型文件路径和数据路径即可启动脚本torch==1.10版本进行开发，现已切换到 torch2.0开发，如其他版本遇到不适配，欢迎反馈LLM模型: 加载chatglm、llama、 baichuan、ziya、bloom等开源大模型权重进行推理和微调，命令行一行部署大模型
核心功能：加载bert、roberta、albert、xlnet、nezha、bart、RoFormer、RoFormer_V2、ELECTRA、GPT、GPT2、T5、GAU-alpha、ERNIE等预训练权重继续进行finetune、并支持在bert基础上灵活定义自己模型
丰富示例：包含llm、pretrain、sentence_classfication、sentence_embedding、sequence_labeling、relation_extraction、seq2seq、serving等多种解决方案
实验验证：已在公开数据集实验验证，使用如下examples数据集和实验指标
易用trick：集成了常见的trick，即插即用
其他特性：加载transformers库模型一起使用；调用方式简洁高效；有训练进度条动态展示；配合torchinfo打印参数量；默认Logger和Tensorboard简便记录训练过程；自定义fit过程，满足高阶需求
训练过程：
| 功能 | bert4torch | transformers | 备注 |
|---|---|---|---|
| 训练进度条 | ✅ | ✅ | 进度条打印loss和定义的metrics |
| 分布式训练dp/ddp | ✅ | ✅ | torch自带dp/ddp |
| 各类callbacks | ✅ | ✅ | 日志/tensorboard/earlystop/wandb等 |
| 大模型推理，stream/batch输出 | ✅ | ✅ | 各个模型是通用的，无需单独维护脚本 |
| 大模型微调 | ✅ | ✅ | lora依赖peft库，pv2自带 |
| 丰富tricks | ✅ | ❌ | 对抗训练等tricks即插即用 |
| 代码简洁易懂，自定义空间大 | ✅ | ❌ | 代码复用度高, keras代码训练风格 |
| 仓库的维护能力/影响力/使用量/兼容性 | ❌ | ✅ | 目前仓库个人维护 |
| 一键部署大模型 |
# 联网下载全部文件
bert4torch serve --checkpoint_path Qwen2-0.5B-Instruct
# 加载本地大模型，联网下载bert4torch_config.json
bert4torch serve --checkpoint_path /data/pretrain_ckpt/Qwen/Qwen2-0.5B-Instruct --config_path Qwen/Qwen2-0.5B-Instruct
# 加载本地大模型，且bert4torch_config.json已经下载并放于同名目录下
bert4torch serve --checkpoint_path /data/pretrain_ckpt/Qwen/Qwen2-0.5B-Instruct
# 命令行
bert4torch serve --checkpoint_path /data/pretrain_ckpt/Qwen/Qwen2-0.5B-Instruct --mode cli
# gradio网页
bert4torch serve --checkpoint_path /data/pretrain_ckpt/Qwen/Qwen2-0.5B-Instruct --mode gradio
# openai_api
bert4torch serve --checkpoint_path /data/pretrain_ckpt/Qwen/Qwen2-0.5B-Instruct --mode openai
| 更新日期 | bert4torch | torch4keras | 版本说明 |
|---|---|---|---|
| 20250925 | 0.6.0 | 0.3.2 | 增加 Qwen3-moe, 支持 gptq、awq等主流量化方式，其他代码优化 |
| 20250721 | 0.5.9.post2 | 0.3.1 | 增加 Ernie4_5, 修复hub下载bug, 拆分出 openai_client |
| 20250621 | 0.5.8 | 0.3.0 | 增加 Qwen3-Embedding和 Qwen3-Reranker, 支持 temperature设置为0, 修复 sdpa和 global_point的bug, 拆分 attention_utils |
| 20250511 | 0.5.7 | 0.2.9.post2 | 命令行参数修改为 bert4torch serve, 增加 Qwen3 |
| 20250401 | 0.5.6 | 0.2.9 | 命令行支持图片输入, 修复rope在batch推理和超长时候的bug |
| 20250215 | 0.5.5 | 0.2.8 | 增加 deepseek-r1, internvl, internlm3, glm4v, modernbert, mllama, qwen2vl, qwenvl |
| 20240928 | 0.5.4 | 0.2.7 | 【新功能】增加 deepseek、MiniCPM、MiniCPMV、llama3.2、Qwen2.5；支持device_map=auto;【修复】修复batch_generate和n>1的bug |
| 20240814 | 0.5.3 | 0.2.6 | 【新功能】增加 llama3.1/Yi1.5；自动选择从hfmirror下载；支持命令行参数 bert4torch serve |
预训练模型支持多种代码加载方式
from bert4torch.models import build_transformer_model
# 1. 仅指定config_path: 从头初始化模型结构, 不加载预训练模型
model = build_transformer_model('./model/bert4torch_config.json')
# 2. 仅指定checkpoint_path:
## 2.1 文件夹路径: 自动寻找路径下的*.bin/*.safetensors权重文件 + 需把bert4torch_config.json下载并放于该目录下
model = build_transformer_model(checkpoint_path='./model')
## 2.2 文件路径/列表: 文件路径即权重路径/列表, bert4torch_config.json会从同级目录下寻找
model = build_transformer_model(checkpoint_path='./pytorch_model.bin')
## 2.3 model_name: hf上预训练权重名称, 会自动下载hf权重以及bert4torch_config.json文件
model = build_transformer_model(checkpoint_path='google-bert/bert-base-chinese')
# 3. 同时指定config_path和checkpoint_path(本地路径名或model_name排列组合):
# 本地路径从本地加载，pretrained_model_name会联网下载
config_path = './model/bert4torch_config.json' # 或'google-bert/bert-base-chinese'
checkpoint_path = './model/pytorch_model.bin' # 或'google-bert/bert-base-chinese'
model = build_transformer_model(config_path, checkpoint_path)
预训练权重链接和bert4torch_config.json
*注：
高亮格式(如 bert-base-chinese)的表示可直接 build_transformer_model()联网下载
国内镜像网站加速下载
HF_ENDPOINT=https://hf-mirror.com python your_script.pyexport HF_ENDPOINT=https://hf-mirror.com后再执行python代码import os
os.environ['HF_ENDPOINT'] = "https://hf-mirror.com"
@misc{bert4torch,
title={bert4torch},
author={Bo Li},
year={2022},
howpublished={\url{https://github.com/Tongjilibo/bert4torch}},
}
 微信号 |
 微信群 |
Star History Chart |
FAQs
an elegant bert4torch
We found that bert4torch demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Research
/Security News
Socket researchers discovered nine malicious NuGet packages that use time-delayed payloads to crash applications and corrupt industrial control systems.
Security News
Socket CTO Ahmad Nassri discusses why supply chain attacks now target developer machines and what AI means for the future of enterprise security.
Security News
Learn the essential steps every developer should take to stay secure on npm and reduce exposure to supply chain attacks.