
Security News
Deno 2.2 Improves Dependency Management and Expands Node.js Compatibility
Deno 2.2 enhances Node.js compatibility, improves dependency management, adds OpenTelemetry support, and expands linting and task automation for developers.
ai.natml.vision.movenet-3d
Advanced tools
Realtime 3D pose tracking with MoveNet in augmented reality with ARFoundation.
Add the following items to your Unity project's Packages/manifest.json
:
{
"scopedRegistries": [
{
"name": "NatML",
"url": "https://registry.npmjs.com",
"scopes": ["ai.fxn", "ai.natml"]
}
],
"dependencies": {
"ai.natml.vision.movenet-3d": "1.0.2"
}
}
These steps assume that you are starting with an AR scene in Unity with an ARSession
and ARSessionOrigin
. In your pose detection script, first create the MoveNet 3D predictor:
MoveNet3DPredictor predictor;
async void Start () {
// Create the MoveNet 3D predictor
predictor = await MoveNet3DPredictor.Create();
}
Then in Update
, acquire the latest CPU camera image and depth image from ARFoundation, then predict the pose:
// Assign these in the inspector
public Camera arCamera;
public ARCameraManager cameraManager;
public AROcclusionManager occlusionManager;
void Update () {
// Get the camera image
if (cameraManager.TryAcquireLatestCpuImage(out var image))
// Get the depth image
if (occlusionManager.TryAcquireEnvironmentDepthCpuImage(out var depth)) {
// Create an ML feature for the camera image
var imageType = image.GetFeatureType();
var imageFeature = new MLImageFeature(imageType.width, imageType.height);
imageFeature.CopyFrom(image);
// Create an ML feature for the depth image
var depthFeature = new MLXRCpuDepthFeature(depth, arCamera);
// Predict
MoveNet3DPredictor.Pose pose = predictor.Predict(imageFeature, depthFeature);
}
}
The pose contains 3D world positions for each detected keypoint.
Note that on older iOS devices that don't support environment depth, you can use the human depth image instead which is supported by iPhone XS/XR or newer.
Thank you very much!
FAQs
Realtime 3D pose detection with MoveNet.
The npm package ai.natml.vision.movenet-3d receives a total of 55 weekly downloads. As such, ai.natml.vision.movenet-3d popularity was classified as not popular.
We found that ai.natml.vision.movenet-3d demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Deno 2.2 enhances Node.js compatibility, improves dependency management, adds OpenTelemetry support, and expands linting and task automation for developers.
Security News
React's CRA deprecation announcement sparked community criticism over framework recommendations, leading to quick updates acknowledging build tools like Vite as valid alternatives.
Security News
Ransomware payment rates hit an all-time low in 2024 as law enforcement crackdowns, stronger defenses, and shifting policies make attacks riskier and less profitable.