node-red-contrib-max-audio-classifier

node-red-contrib-max-audio-classifier

This repo contains a Node-RED node that wraps the functionality of the MAX Audio Classifier from the Model Asset eXchange (MAX) in a form that can be used within a Node-RED flow.

The process of creating this node with the Node Generator tool has been documented in a blog post here.

Following the steps outlined in the post, you should be able to create your own nodes to wrap any microservice that is defined with an OpenAPI specification. This is a common format used by tools like Swagger to document many API's, and this process should allow you to create functional nodes for them with little to no code required.

About the MAX Audio Classifier

This model recognizes a signed 16-bit PCM wav file as an input, generates embeddings, applies PCA transformation/quantization, uses the embeddings as an input to a multi-attention classifier and outputs top 5 class predictions and probabilities as output. The model currently supports 527 classes which are part of the Audioset Ontology.

Notes:

To use this node effectively, you'll need a running instance of the MAX Audio Classifier model from the Model Asset eXchange.

If you'd like more information on how to get started with this and other free, open-source deep learning models from the Model Asset eXchange, visit this tutorial.

Install:

Run the following command in your Node-RED user directory - typically ~/.node-red

$ npm install node-red-contrib-max-audio-classifier

Usage:

Since this node acts essentially as a wrapper around the MAX Audio Classifier model, it provides the same methods defined in that model's OpenAPI specification, documented with Swagger here.

get_metadata

Inputs:

none

Outputs:

payload object

Returns the metadata associated with the model.

predict

Inputs:

payload Buffer

Buffer data of 16-bit PCM wav file.

Outputs:

payload object

Prediction.

Visit this link at the Node-RED Library for an example flow that uses this node in a typical way. In it, you can identify sounds in audio WAV files or through captured audio from your computer's microphone. The clip below shows a file (included in the assets/ directory) being sent to the model, followed by the prediction appearing in the Node-RED debug console on the right-hand side of the screen.

NOTE: To use the Microphone and File Inject nodes pictured in this example flow, you'll need to add them to your Node-RED palette by installing the node-red-contrib-browser-utils package. You can accomplish this from within Node-RED by using the Palette Manager or by running the following command in your Node-RED directory as described above:

$ npm install node-red-contrib-browser-utils

For more information, visit the package's page in the Library.