ncnn is a high-performance neural network inference framework optimized for the mobile platform
This fork aims to provide vulkan functionality to the ncnn python bindings, which is absent in the official package. All that is needed to use vulkan is the addition of net.opt.use_vulkan_compute = True.
Also in this fork, failed vram allocation will result in a runtime error that is catchable, allowing you to clear the allocated vram on this error by using ncnn.destroy_gpu_instance().
import cv2
from ncnn_vulkan import ncnn
import numpy as np
net = ncnn.Net()
# Use vulkan compute
net.opt.use_vulkan_compute = True
# Load model param and bin
net.load_param("./x4.param")
net.load_model("./x4.bin")
ex = net.create_extractor()
# Load image using opencv
img = cv2.imread("./example.jpg")
# Convert image to ncnn Mat
mat_in = ncnn.Mat.from_pixels(
img,
ncnn.Mat.PixelType.PIXEL_BGR,
img.shape[1],
img.shape[0]
)
# Normalize image (required)
# Note that passing in a normalized numpy array will not work.
mean_vals = []
norm_vals = [1 / 255.0, 1 / 255.0, 1 / 255.0]
mat_in.substract_mean_normalize(mean_vals, norm_vals)
# Try/except block to catch out-of-memory error
try:
# Make sure the input and output names match the param file
ex.input("data", mat_in)
ret, mat_out = ex.extract("output")
out = np.array(mat_out)
# Transpose the output from `c, h, w` to `h, w, c` and put it back in 0-255 range
output = out.transpose(1, 2, 0) * 255
# Save image using opencv
cv2.imwrite('./out.png', output)
except:
ncnn.destroy_gpu_instance()
ncnn is a high-performance neural network inference computing framework optimized for mobile platforms. ncnn is deeply considerate about deployment and uses on mobile phones from the beginning of design. ncnn does not have third party dependencies. It is cross-platform, and runs faster than all known open source frameworks on mobile phone cpu. Developers can easily deploy deep learning algorithm models to the mobile platform by using efficient ncnn implementation, create intelligent APPs, and bring the artificial intelligence to your fingertips. ncnn is currently being used in many Tencent applications, such as QQ, Qzone, WeChat, Pitu and so on.
ncnn 是一个为手机端极致优化的高性能神经网络前向计算框架。ncnn 从设计之初深刻考虑手机端的部署和使用。无第三方依赖，跨平台，手机端 cpu 的速度快于目前所有已知的开源框架。基于 ncnn，开发者能够将深度学习算法轻松移植到手机端高效执行，开发出人工智能 APP，将 AI 带到你的指尖。ncnn 目前已在腾讯多款应用中使用，如 QQ，Qzone，微信，天天P图等。
| System | CPU (32bit) | CPU (64bit) | GPU (32bit) | GPU (64bit) |
|---|---|---|---|---|
| Linux (GCC) |  |  | — |  |
| Linux (Clang) |  |  | — |  |
| Linux (ARM) |  |  | — | — |
| Linux (MIPS) |  |  | — | — |
| Linux (RISC-V) | — |  | — | — |
| Linux (LoongArch) | — |  | — | — |
| Windows |  |  | — |  |
| Windows (ARM) |  |  | — | — |
| macOS | — |  | — |  |
| macOS (ARM) | — |  | — |  |
| Android |  |  |  |  |
| Android-x86 |  |  |  |  |
| iOS |  | | — |  |
| iOS Simulator |  | | — | — |
| WebAssembly |  | — | — | — |
| RISC-V GCC/Newlib |  |  | — | — |
how to build ncnn library on Linux / Windows / macOS / Raspberry Pi3 / Android / NVIDIA Jetson / iOS / WebAssembly / AllWinner D1 / Loongson 2K1000
download prebuild binary package for android and ios
use ncnn with alexnet with detailed steps, recommended for beginners :)
ncnn 组件使用指北 alexnet 附带详细步骤，新人强烈推荐 :)
use netron for ncnn model visualization
out-of-the-box web model conversion
ncnn param and model file spec
ncnn operation param weight table
how to implement custom layer step by step
| Windows | Linux | Android | macOS | iOS | |
|---|---|---|---|---|---|
| intel-cpu | ✔️ | ✔️ | ❔ | ✔️ | / |
| intel-gpu | ✔️ | ✔️ | ❔ | ❔ | / |
| amd-cpu | ✔️ | ✔️ | ❔ | ✔️ | / |
| amd-gpu | ✔️ | ✔️ | ❔ | ❔ | / |
| nvidia-gpu | ✔️ | ✔️ | ❔ | ❔ | / |
| qcom-cpu | ❔ | ✔️ | ✅ | / | / |
| qcom-gpu | ❔ | ✔️ | ✔️ | / | / |
| arm-cpu | ❔ | ❔ | ✅ | / | / |
| arm-gpu | ❔ | ❔ | ✔️ | / | / |
| apple-cpu | / | / | / | ✔️ | ✅ |
| apple-gpu | / | / | / | ✔️ | ✔️ |
FAQs
ncnn is a high-performance neural network inference framework optimized for the mobile platform
We found that ncnn-vulkan demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
In this segment of the Risky Business podcast, Feross Aboukhadijeh and Patrick Gray discuss the challenges of tracking malware discovered in open source softare.
Research
Security News
A threat actor's playbook for exploiting the npm ecosystem was exposed on the dark web, detailing how to build a blockchain-powered botnet.
Security News
NVD’s backlog surpasses 20,000 CVEs as analysis slows and NIST announces new system updates to address ongoing delays.