@picovoice/cobra-web-factory

cobra-web

The Picovoice Cobra library for web browsers, powered by WebAssembly. Intended (but not required) to be used with the @picovoice/web-voice-processor package.

This library processes audio for voice detection in-browser, offline. All processing is done via WebAssembly and Workers in a separate thread.

Compatibility

• Chrome / Edge
• Firefox
• Safari

This library requires several modern browser features: WebAssembly, Web Workers, and promises. Internet Explorer will not work.

If you are using this library with the @picovoice/web-voice-processor to access the microphone, that requires some additional browser features like Web Audio API. Its overall browser support is approximately the same.

AccessKey

Cobra requires a valid Picovoice AccessKey at initialization. AccessKey acts as your credentials when using Cobra SDKs. You can get your AccessKey for free. Make sure to keep your AccessKey secret. Signup or Login to Picovoice Console to get your AccessKey.

Packages

The Cobra SDK for Web is split into separate worker and factory packages; import each as required.

Workers

For typical cases, use the worker package. The worker package creates complete CobraWorker instances that can be immediately used with @picovoice/web-voice-processor.

• @picovoice/cobra-web-worker

Factories

Factory packages allow you to create instances of Cobra directly. Useful for building your own custom Worker/Worklet, or some other bespoke purpose.

• @picovoice/cobra-web-factory

Installation & Usage

Worker

To obtain a CobraWorker, we can use the static create factory method from the CobraWorkerFactory. Here is a complete example that:

1. Obtains a CobraWorker from the CobraWorkerFactory
2. Responds to voice activity detection event by defining and passing a callback function
3. Creates a WebVoiceProcessor to obtain microphone permission and forward microphone audio to the CobraWorker

E.g.:

yarn add @picovoice/web-voice-processor @picovoice/cobra-web-worker

import { WebVoiceProcessor } from "@picovoice/web-voice-processor"
import { CobraWorkerFactory } from "@picovoice/cobra-web-worker";
  
// The worker will call the callback function upon a detection event with
// the probability of the voice activity as the input argument
function cobraCallback(voiceProbability) {
  // voiceProbability: Probability of voice activity. It is a floating-point number within [0, 1].
  const threshold = // .. detection threshold
  if voiceProbability >= threshold {
    // .. voice detected!
  }
}

async function startCobra() {
  // Create a Cobra Worker
  // Note: you receive a Worker object, _not_ an individual Cobra instance
  const accessKey = // .. AccessKey string provided by Picovoice Console (https://console.picovoice.ai/)
  const cobraWorker = await CobraWorkerFactory.create(
      accessKey,
      cobraCallback
  );

  // Start up the web voice processor. It will request microphone permission 
  // and immediately (start: true) start listening.
  // It downsamples the audio to voice recognition standard format (16-bit 16kHz linear PCM, single-channel)
  // The incoming microphone audio frames will then be forwarded to the Cobra Worker
  // n.b. This promise will reject if the user refuses permission! Make sure you handle that possibility.
  const webVp =
      await WebVoiceProcessor.init({
        engines: [cobraWorker],
        start: true,
      });
}

startCobra()

...

// Finished with Cobra? Release the WebVoiceProcessor and the worker.
if (done) {
  webVp.release()
  cobraWorker.sendMessage({
    command: "release"
  })
}

Factory

If you wish to build your own worker, or perhaps not use workers at all, use the factory packages. This will let you instantiate Cobra engine instances directly.

The audio passed to the worker in the process function must be of the correct format. The WebVoiceProcessor handles downsampling in the examples above to standard voice recognition format (16-bit, 16kHz linear PCM, single-channel). Use an Int16Array typed array. If you are not using WebVoiceProcessor, you must ensure the audio passed to Cobra is of that format. The Cobra instance provides the length of the array required via .frameLength.

E.g.:

import { Cobra } from "@picovoice/cobra-web-factory";

async function startCobra() {
  const accessKey = // .. AccessKey string provided by Picovoice Console (https://console.picovoice.ai/)
  const handle = await Cobra.create(accessKey);
  return handle;
}

const cobraHandle = startCobra()

// Send Cobra frames of audio (check handle.frameLength for size of array)
const audioFrames =  new Int16Array( /* Provide data with correct format and size*/ )
const cobraResult = cobraHandle.process(audioFrames)
// cobraResult: Probability of voice activity. It is a floating-point number within [0, 1].

...

Build from source (IIFE + ESM outputs)

This library uses Rollup and TypeScript along with Babel and other popular rollup plugins. There are two outputs: an IIFE version intended for script tags / CDN usage, and a JavaScript module version intended for use with modern JavaScript/TypeScript development (e.g. Angular, Create React App, Webpack).

yarn
yarn build

The output will appear in the ./dist/ folder.

For example usage refer to the web demo