Huge News!Announcing our $40M Series B led by Abstract Ventures.Learn More →
Socket
Sign inDemoInstall
Socket
Sign inDemoInstall

@speechly/browser-client

Package Overview
Dependencies
Maintainers
1
Versions
79
Alerts
File Explorer

Advanced tools

License
Socket logo

Install Socket

Detect and block malicious and high-risk dependencies

Install

@speechly/browser-client - npm Package Compare versions

Comparing version 0.5.1 to 1.0.0-beta.1

112

index.d.ts

@@ -25,6 +25,11 @@

*
* @param deviceID - device ID to use when connecting to the API.
* @param cb - the callback to invoke when initialisation is completed (either successfully or with an error).
* @param appId - app ID to use when connecting to the API.
* @param deviceId - device ID to use when connecting to the API.
* @param token - login token in JWT format, which was e.g. cached from previous session.
* If the token is not provided or is invalid, a new token will be fetched instead.
*
* @returns - the token that was used to establish connection to the API, so that it can be cached for later.
* If the provided token was used, it will be returned instead.
*/
initialize(deviceID: string, cb: ErrorCallback): void;
initialize(appId: string, deviceId: string, token?: string): Promise<string>;
/**

@@ -35,21 +40,14 @@ * Closes the client.

* Calling `initialize` again after calling `close` should be possible.
*
* @param closeCode - WebSocket close code to send to the API.
* @param closeReason - WebSocket close reason to send to the API.
*/
close(closeCode: number, closeReason: string): Error | void;
close(): Promise<void>;
/**
* Starts a new audio context by sending the start event to the API.
* The callback must be invoked after the API has responded with confirmation or an error has occured.
*
* @param cb - the callback to invoke after the starting has finished.
* The promise returned should resolve or reject after the API has responded with confirmation or an error has occured.
*/
startContext(cb: ContextCallback): void;
startContext(): Promise<string>;
/**
* Stops an audio context by sending the stop event to the API.
* The callback must be invoked after the API has responded with confirmation or an error has occured.
*
* @param cb - the callback to invoke after the stopping has finished.
* The promise returned should resolve or reject after the API has responded with confirmation or an error has occured.
*/
stopContext(cb: ContextCallback): void;
stopContext(): Promise<string>;
/**

@@ -78,2 +76,3 @@ * Sends audio to the API.

private readonly debug;
private readonly appId;
private readonly storage;

@@ -83,6 +82,7 @@ private readonly microphone;

private readonly activeContexts;
private readonly reconnectAttemptCount;
private readonly reconnectMinDelay;
private deviceId?;
private authToken?;
private state;
private reconnectAttemptCount;
private nextReconnectDelay;
private stateChangeCb;

@@ -105,11 +105,8 @@ private segmentChangeCb;

* the microphone functionality will not work due to security restrictions by the browser.
*
* @param cb - the callback which is invoked when the initialization is complete.
*/
initialize(cb?: ErrorCallback): void;
initialize(): Promise<void>;
/**
* Closes the client by closing the API connection and disabling the microphone.
* @param cb - the callback which is invoked when closure is complete.
*/
close(cb?: ErrorCallback): void;
close(): Promise<void>;
/**

@@ -119,8 +116,7 @@ * Starts a new SLU context by sending a start context event to the API and unmuting the microphone.

*/
startContext(cb?: ContextCallback): void;
startContext(): Promise<string>;
/**
* Stops current SLU context by sending a stop context event to the API and muting the microphone.
* @param cb - the callback which is invoked when the context stop was acknowledged by the API.
*/
stopContext(cb?: ContextCallback): void;
stopContext(): Promise<string>;
/**

@@ -169,2 +165,3 @@ * Adds a listener for client state change events.

private reconnectWebsocket;
private initializeWebsocket;
private readonly handleMicrophoneAudio;

@@ -188,6 +185,10 @@ private setState;

/**
* The URL of Speechly API endpoint.
* The URL of Speechly login endpoint.
*/
url?: string;
loginUrl?: string;
/**
* The URL of Speechly SLU API endpoint.
*/
apiUrl?: string;
/**
* The sample rate of the audio to use.

@@ -214,3 +215,3 @@ */

*/
storage?: Storage;
storage?: Storage_2;
}

@@ -246,8 +247,2 @@

/**
* A callback that receives either an error or a contextId.
* @public
*/
export declare type ContextCallback = (error?: Error, contextId?: string) => void;
/**
* Default sample rate for microphone streams.

@@ -353,8 +348,2 @@ * @public

/**
* A callback that receives an optional error.
* @public
*/
export declare type ErrorCallback = (error?: Error) => void;
/**
* The intent detected by the SLU API.

@@ -408,6 +397,4 @@ * @public

* This method will be called by the Client as part of client initialisation process.
*
* @param cb - the callback that is invoked after initialisation is completed (either successfully or with an error).
*/
initialize(cb: ErrorCallback): void;
initialize(): Promise<void>;
/**

@@ -419,7 +406,4 @@ * Closes the microphone, tearing down all the infrastructure.

* This method will be called by the Client as part of client closure process.
*
* @param cb - the callback that should be invoked after the closure process is completed
* (either successfully or with an error).
*/
close(cb: ErrorCallback): void;
close(): Promise<void>;
/**

@@ -495,3 +479,3 @@ * Mutes the microphone. If the microphone is muted, the `onAudio` callbacks should not be called.

*/
export declare interface Storage {
declare interface Storage_2 {
/**

@@ -503,6 +487,4 @@ * Initialises the storage.

* This method will be called by the Client as part of client initialisation process.
*
* @param cb - the callback that is invoked after initialisation is completed (either successfully or with an error).
*/
initialize(cb: ErrorCallback): void;
initialize(): Promise<void>;
/**

@@ -513,7 +495,4 @@ * Closes the storage.

* This method will be called by the Client as part of client closure process.
*
* @param cb - the callback that should be invoked after the closure process is completed
* (either successfully or with an error).
*/
close(cb: ErrorCallback): void;
close(): Promise<void>;
/**

@@ -523,6 +502,4 @@ * Retrieves a key from the storage.

* @param key - the key to retrieve
* @param cb - the callback that should be invoked after retrieval operation is done,
* either with the value or with an error.
*/
get(key: string, cb: StorageGetCallback): void;
get(key: string): Promise<string>;
/**

@@ -533,15 +510,16 @@ * Adds a key to the storage, possibly overwriting existing value.

* @param val - the value to write
* @param cb - the callback that should be invoked after retrieval operation is done,
* either with the value or with an error.
*/
set(key: string, val: string, cb: ErrorCallback): void;
set(key: string, val: string): Promise<void>;
/**
* Adds a key to the storage, possibly overwriting existing value.
*
* @param key - the key to write
* @param genFn - generator function that will be invoked if the key cannot be found in the storage.
* The return value of the function will be used as the value that will be stored under the given key.
*/
getOrSet(key: string, genFn: () => string): Promise<string>;
}
export { Storage_2 as Storage }
/**
* A callback that receives either an error or the value retrieved from the storage.
* @public
*/
export declare type StorageGetCallback = (error?: Error, val?: string) => void;
/**
* A callback that is invoked whenever new tentative entities are received from the API.

@@ -548,0 +526,0 @@ * @public

4

microphone/browser_audio_processor.d.ts

@@ -5,3 +5,3 @@ import { AudioFilter } from './sampler';

initialize(): Promise<void>;
close(): void;
close(): Promise<void>;
mute(): void;

@@ -24,3 +24,3 @@ unmute(): void;

initialize(): Promise<void>;
close(): void;
close(): Promise<void>;
mute(): void;

@@ -27,0 +27,0 @@ unmute(): void;

36

microphone/browser_audio_processor.js

@@ -71,7 +71,7 @@ "use strict";

const opts = {
video: false
video: false,
};
if (this.nativeResamplingSupported) {
opts.audio = {
sampleRate: this.sampleRate
sampleRate: this.sampleRate,
};

@@ -115,17 +115,19 @@ }

close() {
if (!this.initialized) {
throw types_1.ErrNotInitialized;
}
// Stop all media tracks
const stream = this.mediaStream;
stream.getTracks().forEach(t => t.stop());
// Disconnect and stop ScriptProcessorNode
const proc = this.audioProcessor;
proc.disconnect();
proc.removeEventListener(audioProcessEvent, this.handleAudio);
// Unset all audio infrastructure
this.mediaStream = undefined;
this.audioTrack = undefined;
this.audioProcessor = undefined;
this.initialized = false;
return __awaiter(this, void 0, void 0, function* () {
if (!this.initialized) {
throw types_1.ErrNotInitialized;
}
// Stop all media tracks
const stream = this.mediaStream;
stream.getTracks().forEach(t => t.stop());
// Disconnect and stop ScriptProcessorNode
const proc = this.audioProcessor;
proc.disconnect();
proc.removeEventListener(audioProcessEvent, this.handleAudio);
// Unset all audio infrastructure
this.mediaStream = undefined;
this.audioTrack = undefined;
this.audioProcessor = undefined;
this.initialized = false;
});
}

@@ -132,0 +134,0 @@ mute() {

5

microphone/browser_microphone.d.ts

@@ -1,2 +0,1 @@

import { ErrorCallback } from '../types';
import { AudioCallback, Microphone } from './types';

@@ -9,4 +8,4 @@ import { AudioProcessor } from './browser_audio_processor';

onAudio(cb: AudioCallback): void;
initialize(cb: ErrorCallback): void;
close(cb: ErrorCallback): void;
initialize(): Promise<void>;
close(): Promise<void>;
mute(): void;

@@ -13,0 +12,0 @@ unmute(): void;

29

microphone/browser_microphone.js
"use strict";
var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {
function adopt(value) { return value instanceof P ? value : new P(function (resolve) { resolve(value); }); }
return new (P || (P = Promise))(function (resolve, reject) {
function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }
function rejected(value) { try { step(generator["throw"](value)); } catch (e) { reject(e); } }
function step(result) { result.done ? resolve(result.value) : adopt(result.value).then(fulfilled, rejected); }
step((generator = generator.apply(thisArg, _arguments || [])).next());
});
};
Object.defineProperty(exports, "__esModule", { value: true });

@@ -15,15 +24,13 @@ const browser_audio_processor_1 = require("./browser_audio_processor");

}
initialize(cb) {
this.audioProcessor
.initialize()
.then(() => {
initialize() {
return __awaiter(this, void 0, void 0, function* () {
yield this.audioProcessor.initialize();
this.mute();
cb();
})
.catch(cb);
});
}
close(cb) {
this.mute();
this.audioProcessor.close();
cb();
close() {
return __awaiter(this, void 0, void 0, function* () {
this.mute();
return this.audioProcessor.close();
});
}

@@ -30,0 +37,0 @@ mute() {

10

microphone/types.d.ts

@@ -1,2 +0,1 @@

import { ErrorCallback } from '../types';
/**

@@ -49,6 +48,4 @@ * Default sample rate for microphone streams.

* This method will be called by the Client as part of client initialisation process.
*
* @param cb - the callback that is invoked after initialisation is completed (either successfully or with an error).
*/
initialize(cb: ErrorCallback): void;
initialize(): Promise<void>;
/**

@@ -60,7 +57,4 @@ * Closes the microphone, tearing down all the infrastructure.

* This method will be called by the Client as part of client closure process.
*
* @param cb - the callback that should be invoked after the closure process is completed
* (either successfully or with an error).
*/
close(cb: ErrorCallback): void;
close(): Promise<void>;
/**

@@ -67,0 +61,0 @@ * Mutes the microphone. If the microphone is muted, the `onAudio` callbacks should not be called.

42

package.json
{
"name": "@speechly/browser-client",
"version": "0.5.1",
"version": "1.0.0-beta.1",
"description": "Browser client for Speechly API",

@@ -37,27 +37,31 @@ "private": false,

"dependencies": {
"async-retry": "^1.3.1",
"jsonwebtoken": "^8.5.1",
"locale-code": "^2.0.2",
"uuid": "^7.0.2"
"uuid": "^8.0.0"
},
"devDependencies": {
"@microsoft/api-extractor": "^7.7.8",
"@types/jest": "^25.1.2",
"@types/uuid": "^7.0.2",
"@typescript-eslint/eslint-plugin": "^2.19.2",
"@typescript-eslint/parser": "^2.0.0",
"eslint": "^6.8.0",
"eslint-config-standard-with-typescript": "^13.0.0",
"eslint-plugin-import": "^2.20.1",
"eslint-plugin-jest": "^23.7.0",
"eslint-plugin-node": "^9.2.0",
"@microsoft/api-extractor": "^7.8.0",
"@types/async-retry": "^1.4.1",
"@types/jest": "^25.2.1",
"@types/jsonwebtoken": "^8.3.9",
"@types/uuid": "^7.0.3",
"@typescript-eslint/eslint-plugin": "^2.31.0",
"@typescript-eslint/parser": "^2.31.0",
"eslint": "^7.0.0",
"eslint-config-standard-with-typescript": "^16.0.0",
"eslint-plugin-import": "^2.20.2",
"eslint-plugin-jest": "^23.10.0",
"eslint-plugin-node": "^11.1.0",
"eslint-plugin-promise": "^4.2.1",
"eslint-plugin-standard": "^4.0.1",
"eslint-plugin-tsdoc": "^0.2.1",
"jest": "^25.1.0",
"minimist": "0.2.1",
"prettier": "^1.19.1",
"eslint-plugin-tsdoc": "^0.2.4",
"jest": "^26.0.1",
"minimist": "^1.2.5",
"prettier": "^2.0.5",
"rimraf": "^3.0.2",
"ts-jest": "^25.2.0",
"ts-loader": "^6.2.1",
"ts-jest": "^25.5.1",
"ts-loader": "^7.0.3",
"tsc-watch": "^4.2.3",
"typedoc": "^0.16.10",
"typedoc": "^0.17.6",
"typedoc-plugin-markdown": "^2.2.16",

@@ -64,0 +68,0 @@ "typescript": "^3.8.3"

21

speechly/client.d.ts

@@ -1,3 +0,1 @@

import { ErrorCallback } from '../types';
import { ContextCallback } from '../websocket';
import { ClientOptions, StateChangeCallback, SegmentChangeCallback, TentativeTranscriptCallback, TranscriptCallback, TentativeEntitiesCallback, EntityCallback, IntentCallback } from './types';

@@ -12,2 +10,3 @@ /**

private readonly debug;
private readonly appId;
private readonly storage;

@@ -17,6 +16,7 @@ private readonly microphone;

private readonly activeContexts;
private readonly reconnectAttemptCount;
private readonly reconnectMinDelay;
private deviceId?;
private authToken?;
private state;
private reconnectAttemptCount;
private nextReconnectDelay;
private stateChangeCb;

@@ -39,11 +39,8 @@ private segmentChangeCb;

* the microphone functionality will not work due to security restrictions by the browser.
*
* @param cb - the callback which is invoked when the initialization is complete.
*/
initialize(cb?: ErrorCallback): void;
initialize(): Promise<void>;
/**
* Closes the client by closing the API connection and disabling the microphone.
* @param cb - the callback which is invoked when closure is complete.
*/
close(cb?: ErrorCallback): void;
close(): Promise<void>;
/**

@@ -53,8 +50,7 @@ * Starts a new SLU context by sending a start context event to the API and unmuting the microphone.

*/
startContext(cb?: ContextCallback): void;
startContext(): Promise<string>;
/**
* Stops current SLU context by sending a stop context event to the API and muting the microphone.
* @param cb - the callback which is invoked when the context stop was acknowledged by the API.
*/
stopContext(cb?: ContextCallback): void;
stopContext(): Promise<string>;
/**

@@ -103,4 +99,5 @@ * Adds a listener for client state change events.

private reconnectWebsocket;
private initializeWebsocket;
private readonly handleMicrophoneAudio;
private setState;
}

264

speechly/client.js
"use strict";
var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {
function adopt(value) { return value instanceof P ? value : new P(function (resolve) { resolve(value); }); }
return new (P || (P = Promise))(function (resolve, reject) {
function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }
function rejected(value) { try { step(generator["throw"](value)); } catch (e) { reject(e); } }
function step(result) { result.done ? resolve(result.value) : adopt(result.value).then(fulfilled, rejected); }
step((generator = generator.apply(thisArg, _arguments || [])).next());
});
};
var __importDefault = (this && this.__importDefault) || function (mod) {

@@ -15,6 +24,7 @@ return (mod && mod.__esModule) ? mod : { "default": mod };

const parsers_1 = require("./parsers");
const async_retry_1 = __importDefault(require("async-retry"));
const deviceIdStorageKey = 'speechly-device-id';
const defaultSpeechlyURL = 'wss://api.speechly.com/ws';
const initialReconnectDelay = 1000;
const initialReconnectCount = 5;
const authTokenKey = 'speechly-auth-token';
const defaultApiUrl = 'wss://api.speechly.com/ws/v1';
const defaultLoginUrl = 'https://api.speechly.com/login';
/**

@@ -28,7 +38,7 @@ * A client for Speechly Spoken Language Understanding (SLU) API. The client handles initializing the microphone

constructor(options) {
var _a, _b, _c, _d, _e, _f, _g;
var _a, _b, _c, _d, _e, _f, _g, _h;
this.activeContexts = new Map();
this.reconnectAttemptCount = 5;
this.reconnectMinDelay = 1000;
this.state = types_1.ClientState.Disconnected;
this.reconnectAttemptCount = initialReconnectCount;
this.nextReconnectDelay = initialReconnectDelay;
this.stateChangeCb = () => { };

@@ -110,3 +120,15 @@ this.segmentChangeCb = () => { };

}
this.reconnectWebsocket();
// If for some reason deviceId is missing, there's nothing else we can do but fail completely.
if (this.deviceId === undefined) {
this.setState(types_1.ClientState.Failed);
return;
}
// Make sure we don't have concurrent reconnection procedures or attempt to reconnect from a failed state.
if (this.state === types_1.ClientState.Connecting || this.state === types_1.ClientState.Failed) {
return;
}
this.setState(types_1.ClientState.Connecting);
this.reconnectWebsocket(this.deviceId)
.then(() => this.setState(types_1.ClientState.Connected))
.catch(() => this.setState(types_1.ClientState.Failed));
};

@@ -123,5 +145,6 @@ this.handleMicrophoneAudio = (audioChunk) => {

this.debug = (_a = options.debug) !== null && _a !== void 0 ? _a : false;
this.appId = options.appId;
this.microphone = (_b = options.microphone) !== null && _b !== void 0 ? _b : new microphone_1.BrowserMicrophone((_c = options.sampleRate) !== null && _c !== void 0 ? _c : microphone_1.DefaultSampleRate);
this.websocket = (_d = options.apiClient) !== null && _d !== void 0 ? _d : new websocket_1.WebsocketClient((_e = options.url) !== null && _e !== void 0 ? _e : defaultSpeechlyURL, options.appId, options.language, (_f = options.sampleRate) !== null && _f !== void 0 ? _f : microphone_1.DefaultSampleRate);
this.storage = (_g = options.storage) !== null && _g !== void 0 ? _g : new storage_1.LocalStorage();
this.websocket = (_d = options.apiClient) !== null && _d !== void 0 ? _d : new websocket_1.WebsocketClient((_e = options.loginUrl) !== null && _e !== void 0 ? _e : defaultLoginUrl, (_f = options.apiUrl) !== null && _f !== void 0 ? _f : defaultApiUrl, options.language, (_g = options.sampleRate) !== null && _g !== void 0 ? _g : microphone_1.DefaultSampleRate);
this.storage = (_h = options.storage) !== null && _h !== void 0 ? _h : new storage_1.LocalStorage();
this.microphone.onAudio(this.handleMicrophoneAudio);

@@ -139,12 +162,28 @@ this.websocket.onResponse(this.handleWebsocketResponse);

* the microphone functionality will not work due to security restrictions by the browser.
*
* @param cb - the callback which is invoked when the initialization is complete.
*/
initialize(cb = () => { }) {
if (this.state !== types_1.ClientState.Disconnected) {
return cb(new Error('Cannot initialize client - client is not in Disconnected state'));
}
this.setState(types_1.ClientState.Connecting);
this.microphone.initialize((err) => {
if (err !== undefined) {
initialize() {
return __awaiter(this, void 0, void 0, function* () {
if (this.state !== types_1.ClientState.Disconnected) {
throw Error('Cannot initialize client - client is not in Disconnected state');
}
this.setState(types_1.ClientState.Connecting);
try {
// 1. Initialise the storage and fetch deviceId (or generate new one and store it).
yield this.storage.initialize();
this.deviceId = yield this.storage.getOrSet(deviceIdStorageKey, uuid_1.v4);
// 2. Fetch auth token. It doesn't matter if it's not present.
try {
this.authToken = yield this.storage.get(authTokenKey);
}
catch (err) {
if (this.debug) {
console.warn('[SpeechlyClient]', 'Error fetching auth token from storage:', err);
}
}
// 2. Initialise the microphone stack.
yield this.microphone.initialize();
// 3. Initialise websocket.
yield this.initializeWebsocket(this.deviceId);
}
catch (err) {
switch (err) {

@@ -160,42 +199,5 @@ case microphone_1.ErrDeviceNotSupported:

}
return cb(err);
throw err;
}
const initializeWebsocket = (deviceId, cb) => {
this.websocket.initialize(deviceId, (err) => {
if (err !== undefined) {
this.reconnectWebsocket();
// TODO: I think this can be confusing for the end user.
// We should only invoke callback when initialization is finished, either successfully or with an error.
// We can instead pass the callback to reconnect and let that invoke it.
return cb();
}
this.setState(types_1.ClientState.Connected);
return cb();
});
};
this.storage.initialize((err) => {
if (err !== undefined) {
return cb(err);
}
this.storage.get(deviceIdStorageKey, (err, val) => {
if (err !== undefined) {
// Device ID was not found in the storage, generate new ID and store it.
const deviceId = uuid_1.v4();
return this.storage.set(deviceIdStorageKey, deviceId, (err) => {
if (err !== undefined) {
// At this point we couldn't load device ID from storage, nor we could store a new one there.
// Give up initialisation and return an error.
return cb(err);
}
// Newly generated ID was stored, proceed with initialization.
this.deviceId = deviceId;
return initializeWebsocket(deviceId, cb);
});
}
// Device ID was found in the storage, proceed with initialization.
const deviceId = val;
this.deviceId = deviceId;
return initializeWebsocket(deviceId, cb);
});
});
this.setState(types_1.ClientState.Connected);
});

@@ -205,22 +207,29 @@ }

* Closes the client by closing the API connection and disabling the microphone.
* @param cb - the callback which is invoked when closure is complete.
*/
close(cb = () => { }) {
this.storage.close((err) => {
close() {
return __awaiter(this, void 0, void 0, function* () {
const errs = [];
if (err !== undefined) {
try {
yield this.storage.close();
}
catch (err) {
errs.push(err.message);
}
this.microphone.close((err) => {
if (err !== undefined) {
errs.push(err.message);
}
const wsErr = this.websocket.close(1000, 'client disconnecting');
if (wsErr !== undefined) {
errs.push(wsErr.message);
}
this.activeContexts.clear();
this.setState(types_1.ClientState.Disconnected);
return errs.length > 0 ? cb(Error(errs.join(','))) : cb();
});
try {
yield this.microphone.close();
}
catch (err) {
errs.push(err.message);
}
try {
yield this.websocket.close();
}
catch (err) {
errs.push(err.message);
}
this.activeContexts.clear();
this.setState(types_1.ClientState.Disconnected);
if (errs.length > 0) {
throw Error(errs.join(','));
}
});

@@ -232,20 +241,20 @@ }

*/
startContext(cb = () => { }) {
if (this.state !== types_1.ClientState.Connected) {
return cb(Error('Cannot start context - client is not connected'));
}
// Re-set reconnection settings here, so that we avoid flip-flopping in `_reconnectWebsocket`.
this.reconnectAttemptCount = initialReconnectCount;
this.nextReconnectDelay = initialReconnectDelay;
this.setState(types_1.ClientState.Starting);
this.websocket.startContext((err, contextId) => {
if (err !== undefined) {
startContext() {
return __awaiter(this, void 0, void 0, function* () {
if (this.state !== types_1.ClientState.Connected) {
throw Error('Cannot start context - client is not connected');
}
this.setState(types_1.ClientState.Starting);
let contextId;
try {
contextId = yield this.websocket.startContext();
}
catch (err) {
this.setState(types_1.ClientState.Connected);
return cb(err);
throw err;
}
this.setState(types_1.ClientState.Recording);
this.microphone.unmute();
const ctxId = contextId;
this.activeContexts.set(ctxId, new Map());
return cb(undefined, ctxId);
this.activeContexts.set(contextId, new Map());
return contextId;
});

@@ -255,23 +264,21 @@ }

* Stops current SLU context by sending a stop context event to the API and muting the microphone.
* @param cb - the callback which is invoked when the context stop was acknowledged by the API.
*/
stopContext(cb = () => { }) {
if (this.state !== types_1.ClientState.Recording) {
return cb(new Error('Cannot stop context - client is not recording'));
}
this.setState(types_1.ClientState.Stopping);
this.microphone.mute();
this.websocket.stopContext((err, contextId) => {
if (err !== undefined) {
// Sending stop event failed, which means recovering from this isn't viable.
// Developers should react to this state by reloading the app.
stopContext() {
return __awaiter(this, void 0, void 0, function* () {
if (this.state !== types_1.ClientState.Recording) {
throw Error('Cannot stop context - client is not recording');
}
this.setState(types_1.ClientState.Stopping);
this.microphone.mute();
let contextId;
try {
contextId = yield this.websocket.stopContext();
}
catch (err) {
this.setState(types_1.ClientState.Failed);
return cb(err);
throw err;
}
const ctxId = contextId;
if (!this.activeContexts.delete(ctxId)) {
console.warn('[SpeechlyClient]', 'Attempted to remove non-existent context', ctxId);
}
this.setState(types_1.ClientState.Connected);
return cb(undefined, ctxId);
this.activeContexts.delete(contextId);
return contextId;
});

@@ -335,28 +342,31 @@ }

}
reconnectWebsocket() {
if (this.deviceId === undefined) {
return this.setState(types_1.ClientState.Disconnected);
}
const deviceId = this.deviceId;
if (this.reconnectAttemptCount < 1) {
return this.setState(types_1.ClientState.Disconnected);
}
if (this.state !== types_1.ClientState.Connecting) {
this.setState(types_1.ClientState.Connecting);
}
if (this.debug) {
console.log('[SpeechlyClient]', `Attempting to re-connect to the server in ${this.nextReconnectDelay.toString()}ms`);
}
// TODO: extract this as a "re-trier" (check existing libraries first).
setTimeout(() => {
this.reconnectAttemptCount = this.reconnectAttemptCount - 1;
this.nextReconnectDelay = this.nextReconnectDelay * 2;
this.websocket.initialize(deviceId, (err) => {
if (err !== undefined) {
return this.reconnectWebsocket();
reconnectWebsocket(deviceId) {
return __awaiter(this, void 0, void 0, function* () {
return async_retry_1.default((_, attempt) => __awaiter(this, void 0, void 0, function* () {
if (this.debug) {
console.log('[SpeechlyClient]', 'WebSocket reconnection attempt number:', attempt);
}
this.setState(types_1.ClientState.Connected);
yield this.initializeWebsocket(deviceId);
}), {
retries: this.reconnectAttemptCount,
minTimeout: this.reconnectMinDelay,
});
}, this.nextReconnectDelay);
});
}
initializeWebsocket(deviceId) {
return __awaiter(this, void 0, void 0, function* () {
// Initialise websocket and save the auth token.
this.authToken = yield this.websocket.initialize(this.appId, deviceId, this.authToken);
// Cache the auth token in local storage for future use.
try {
yield this.storage.set(authTokenKey, this.authToken);
}
catch (err) {
// No need to fail if the token caching failed, we will just re-fetch it next time.
if (this.debug) {
console.warn('[SpeechlyClient]', 'Error caching auth token in storage:', err);
}
}
});
}
setState(newState) {

@@ -363,0 +373,0 @@ if (this.state === newState) {

10

speechly/parsers.js

@@ -11,3 +11,3 @@ "use strict";

endTimestamp: end_timestamp,
isFinal: false
isFinal: false,
};

@@ -24,3 +24,3 @@ });

endTimestamp: data.end_timestamp,
isFinal: true
isFinal: true,
};

@@ -37,3 +37,3 @@ }

endPosition: end_position,
isFinal: false
isFinal: false,
};

@@ -50,3 +50,3 @@ });

endPosition: data.end_position,
isFinal: true
isFinal: true,
};

@@ -59,3 +59,3 @@ }

intent: data.intent,
isFinal: isFinal
isFinal: isFinal,
};

@@ -62,0 +62,0 @@ }

10

speechly/segment.js

@@ -15,3 +15,3 @@ "use strict";

const entities = new Array(this.entities.size);
this.entities.forEach(v => {
this.entities.forEach((v) => {
entities[i] = v;

@@ -26,7 +26,7 @@ i++;

entities: entities,
intent: this.intent
intent: this.intent,
};
}
updateTranscript(words) {
words.forEach(w => {
words.forEach((w) => {
// Only accept tentative words if the segment is tentative.

@@ -40,3 +40,3 @@ if (!this.isFinalized || w.isFinal) {

updateEntities(entities) {
entities.forEach(e => {
entities.forEach((e) => {
// Only accept tentative entities if the segment is tentative.

@@ -64,3 +64,3 @@ if (!this.isFinalized || e.isFinal) {

// Filter away any transcripts which were not finalized.
this.words = this.words.filter(w => w.isFinal);
this.words = this.words.filter((w) => w.isFinal);
if (!this.intent.isFinal) {

@@ -67,0 +67,0 @@ this.intent.intent = '';

2

speechly/state.js

@@ -26,4 +26,4 @@ "use strict";

[types_1.ClientState.Stopping, 'Stopping'],
[types_1.ClientState.Recording, 'Recording']
[types_1.ClientState.Recording, 'Recording'],
]);
//# sourceMappingURL=state.js.map

8

speechly/types.d.ts

@@ -18,6 +18,10 @@ import { Microphone } from '../microphone';

/**
* The URL of Speechly API endpoint.
* The URL of Speechly login endpoint.
*/
url?: string;
loginUrl?: string;
/**
* The URL of Speechly SLU API endpoint.
*/
apiUrl?: string;
/**
* The sample rate of the audio to use.

@@ -24,0 +28,0 @@ */

12

storage/storage.d.ts

@@ -1,10 +0,10 @@

import { ErrorCallback } from '../types';
import { Storage as IStorage, StorageGetCallback } from './types';
import { Storage as IStorage } from './types';
export declare class LocalStorage implements IStorage {
private readonly storage;
constructor();
initialize(cb: ErrorCallback): void;
close(cb: ErrorCallback): void;
get(key: string, cb: StorageGetCallback): void;
set(key: string, val: string, cb: ErrorCallback): void;
initialize(): Promise<void>;
close(): Promise<void>;
get(key: string): Promise<string>;
set(key: string, val: string): Promise<void>;
getOrSet(key: string, genFn: () => string): Promise<string>;
}

48

storage/storage.js
"use strict";
var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {
function adopt(value) { return value instanceof P ? value : new P(function (resolve) { resolve(value); }); }
return new (P || (P = Promise))(function (resolve, reject) {
function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }
function rejected(value) { try { step(generator["throw"](value)); } catch (e) { reject(e); } }
function step(result) { result.done ? resolve(result.value) : adopt(result.value).then(fulfilled, rejected); }
step((generator = generator.apply(thisArg, _arguments || [])).next());
});
};
Object.defineProperty(exports, "__esModule", { value: true });

@@ -8,21 +17,34 @@ const types_1 = require("./types");

}
initialize(cb) {
cb();
initialize() {
return __awaiter(this, void 0, void 0, function* () { });
}
close(cb) {
cb();
close() {
return __awaiter(this, void 0, void 0, function* () { });
}
get(key, cb) {
const val = this.storage.getItem(key);
if (val === null) {
return cb(types_1.ErrKeyNotFound);
}
return cb(undefined, val);
get(key) {
return __awaiter(this, void 0, void 0, function* () {
const val = this.storage.getItem(key);
if (val === null) {
throw types_1.ErrKeyNotFound;
}
return val;
});
}
set(key, val, cb) {
this.storage.setItem(key, val);
return cb();
set(key, val) {
return __awaiter(this, void 0, void 0, function* () {
this.storage.setItem(key, val);
});
}
getOrSet(key, genFn) {
return __awaiter(this, void 0, void 0, function* () {
let val = this.storage.getItem(key);
if (val === null) {
val = genFn();
this.storage.setItem(key, val);
}
return val;
});
}
}
exports.LocalStorage = LocalStorage;
//# sourceMappingURL=storage.js.map

31

storage/types.d.ts

@@ -1,2 +0,1 @@

import { ErrorCallback } from '../types';
/**

@@ -13,7 +12,2 @@ * Error to be thrown if storage API is not supported by the device.

/**
* A callback that receives either an error or the value retrieved from the storage.
* @public
*/
export declare type StorageGetCallback = (error?: Error, val?: string) => void;
/**
* The interface for local key-value storage.

@@ -29,6 +23,4 @@ * @public

* This method will be called by the Client as part of client initialisation process.
*
* @param cb - the callback that is invoked after initialisation is completed (either successfully or with an error).
*/
initialize(cb: ErrorCallback): void;
initialize(): Promise<void>;
/**

@@ -39,7 +31,4 @@ * Closes the storage.

* This method will be called by the Client as part of client closure process.
*
* @param cb - the callback that should be invoked after the closure process is completed
* (either successfully or with an error).
*/
close(cb: ErrorCallback): void;
close(): Promise<void>;
/**

@@ -49,6 +38,4 @@ * Retrieves a key from the storage.

* @param key - the key to retrieve
* @param cb - the callback that should be invoked after retrieval operation is done,
* either with the value or with an error.
*/
get(key: string, cb: StorageGetCallback): void;
get(key: string): Promise<string>;
/**

@@ -59,6 +46,12 @@ * Adds a key to the storage, possibly overwriting existing value.

* @param val - the value to write
* @param cb - the callback that should be invoked after retrieval operation is done,
* either with the value or with an error.
*/
set(key: string, val: string, cb: ErrorCallback): void;
set(key: string, val: string): Promise<void>;
/**
* Adds a key to the storage, possibly overwriting existing value.
*
* @param key - the key to write
* @param genFn - generator function that will be invoked if the key cannot be found in the storage.
* The return value of the function will be used as the value that will be stored under the given key.
*/
getOrSet(key: string, genFn: () => string): Promise<string>;
}

2

tsdoc-metadata.json

@@ -8,5 +8,5 @@ // This file is read by tools that parse documentation comments conforming to the TSDoc standard.

"packageName": "@microsoft/api-extractor",
"packageVersion": "7.7.10"
"packageVersion": "7.8.0"
}
]
}

34

websocket/types.d.ts

@@ -1,2 +0,1 @@

import { ErrorCallback } from '../types';
/**

@@ -134,7 +133,2 @@ * The interface for response returned by WebSocket client.

/**
* A callback that receives either an error or a contextId.
* @public
*/
export declare type ContextCallback = (error?: Error, contextId?: string) => void;
/**
* The interface for a client for Speechly SLU WebSocket API.

@@ -162,6 +156,11 @@ * @public

*
* @param deviceID - device ID to use when connecting to the API.
* @param cb - the callback to invoke when initialisation is completed (either successfully or with an error).
* @param appId - app ID to use when connecting to the API.
* @param deviceId - device ID to use when connecting to the API.
* @param token - login token in JWT format, which was e.g. cached from previous session.
* If the token is not provided or is invalid, a new token will be fetched instead.
*
* @returns - the token that was used to establish connection to the API, so that it can be cached for later.
* If the provided token was used, it will be returned instead.
*/
initialize(deviceID: string, cb: ErrorCallback): void;
initialize(appId: string, deviceId: string, token?: string): Promise<string>;
/**

@@ -172,21 +171,14 @@ * Closes the client.

* Calling `initialize` again after calling `close` should be possible.
*
* @param closeCode - WebSocket close code to send to the API.
* @param closeReason - WebSocket close reason to send to the API.
*/
close(closeCode: number, closeReason: string): Error | void;
close(): Promise<void>;
/**
* Starts a new audio context by sending the start event to the API.
* The callback must be invoked after the API has responded with confirmation or an error has occured.
*
* @param cb - the callback to invoke after the starting has finished.
* The promise returned should resolve or reject after the API has responded with confirmation or an error has occured.
*/
startContext(cb: ContextCallback): void;
startContext(): Promise<string>;
/**
* Stops an audio context by sending the stop event to the API.
* The callback must be invoked after the API has responded with confirmation or an error has occured.
*
* @param cb - the callback to invoke after the stopping has finished.
* The promise returned should resolve or reject after the API has responded with confirmation or an error has occured.
*/
stopContext(cb: ContextCallback): void;
stopContext(): Promise<string>;
/**

@@ -193,0 +185,0 @@ * Sends audio to the API.

21

websocket/websocket_client.d.ts

@@ -1,8 +0,6 @@

import { ErrorCallback } from '../types';
import { APIClient, ResponseCallback, CloseCallback, ContextCallback } from './types';
import { APIClient, ResponseCallback, CloseCallback } from './types';
export declare class WebsocketClient implements APIClient {
private readonly baseUrl;
private readonly languageCode;
private readonly sampleRate;
private readonly appId;
private readonly loginUrl;
private readonly apiUrl;
private authToken?;
private websocket?;

@@ -15,9 +13,10 @@ private startCbs;

onClose(cb: CloseCallback): void;
constructor(baseUrl: string, appId: string, language: string, sampleRate: number);
initialize(deviceId: string, cb: ErrorCallback): void;
close(closeCode: number, closeReason: string): Error | void;
startContext(cb: ContextCallback): void;
stopContext(cb: ContextCallback): void;
constructor(loginUrl: string, apiUrl: string, languageCode: string, sampleRate: number);
initialize(appId: string, deviceId: string, token?: string): Promise<string>;
close(): Promise<void>;
startContext(): Promise<string>;
stopContext(): Promise<string>;
sendAudio(audioChunk: Int16Array): Error | void;
private isOpen;
private closeWebsocket;
private readonly onWebsocketMessage;

@@ -24,0 +23,0 @@ private readonly onWebsocketClose;

211

websocket/websocket_client.js
"use strict";
var __awaiter = (this && this.__awaiter) || function (thisArg, _arguments, P, generator) {
function adopt(value) { return value instanceof P ? value : new P(function (resolve) { resolve(value); }); }
return new (P || (P = Promise))(function (resolve, reject) {
function fulfilled(value) { try { step(generator.next(value)); } catch (e) { reject(e); } }
function rejected(value) { try { step(generator["throw"](value)); } catch (e) { reject(e); } }
function step(result) { result.done ? resolve(result.value) : adopt(result.value).then(fulfilled, rejected); }
step((generator = generator.apply(thisArg, _arguments || [])).next());
});
};
Object.defineProperty(exports, "__esModule", { value: true });
const jsonwebtoken_1 = require("jsonwebtoken");
const types_1 = require("./types");
class WebsocketClient {
constructor(baseUrl, appId, language, sampleRate) {
constructor(loginUrl, apiUrl, languageCode, sampleRate) {
this.startCbs = [];

@@ -21,3 +31,3 @@ this.stopCbs = [];

case types_1.WebsocketResponseType.Started:
this.startCbs.forEach(cb => {
this.startCbs.forEach((cb) => {
try {

@@ -33,3 +43,3 @@ cb(undefined, response.audio_context);

case types_1.WebsocketResponseType.Stopped:
this.stopCbs.forEach(cb => {
this.stopCbs.forEach((cb) => {
try {

@@ -49,12 +59,11 @@ cb(undefined, response.audio_context);

this.onWebsocketClose = (event) => {
this.onCloseCb(Error(`Websocket was closed: ${event.reason}`));
this.websocket = undefined;
this.onCloseCb(Error(`Websocket was closed with code "${event.code}" and reason "${event.reason}"`));
};
this.onWebsocketError = (event) => {
this.close(1000, 'Client disconnecting due to an error');
this.onWebsocketError = (_event) => {
this.closeWebsocket(1000, 'Client disconnecting due to an error');
this.onCloseCb(Error('Websocket was closed because of error'));
};
this.baseUrl = baseUrl;
this.languageCode = language;
this.sampleRate = sampleRate;
this.appId = appId;
this.loginUrl = loginUrl;
this.apiUrl = generateWsUrl(apiUrl, languageCode, sampleRate);
}

@@ -67,43 +76,63 @@ onResponse(cb) {

}
initialize(deviceId, cb) {
if (this.websocket !== undefined) {
return cb(Error('Cannot initialize an already initialized websocket client'));
}
const url = generateWsUrl(this.baseUrl, deviceId, this.languageCode, this.sampleRate);
initializeWebsocket(url, this.appId, (err, ws) => {
if (err !== undefined) {
return cb(err);
initialize(appId, deviceId, token) {
return __awaiter(this, void 0, void 0, function* () {
if (this.websocket !== undefined) {
throw Error('Cannot initialize an already initialized websocket client');
}
this.websocket = ws;
if (token !== undefined && isValidToken(token, appId, deviceId)) {
// If the token is still valid, don't refresh it.
this.authToken = token;
}
else {
this.authToken = yield login(this.loginUrl, appId, deviceId);
}
this.websocket = yield initializeWebsocket(this.apiUrl, this.authToken);
this.websocket.addEventListener('message', this.onWebsocketMessage);
this.websocket.addEventListener('error', this.onWebsocketError);
this.websocket.addEventListener('close', this.onWebsocketClose);
return cb();
return this.authToken;
});
}
close(closeCode, closeReason) {
if (this.websocket === undefined) {
return Error('Websocket is not open');
}
this.websocket.removeEventListener('message', this.onWebsocketMessage);
this.websocket.removeEventListener('error', this.onWebsocketError);
this.websocket.removeEventListener('close', this.onWebsocketClose);
this.websocket.close(closeCode, closeReason);
this.websocket = undefined;
close() {
return __awaiter(this, void 0, void 0, function* () {
return this.closeWebsocket(1000, 'Client has ended the session');
});
}
startContext(cb) {
if (!this.isOpen()) {
return cb(Error('Websocket is not ready'));
}
this.startCbs.push(cb);
const ws = this.websocket;
ws.send(StartEventJSON);
startContext() {
return __awaiter(this, void 0, void 0, function* () {
if (!this.isOpen()) {
throw Error('Websocket is not ready');
}
const ws = this.websocket;
return new Promise((resolve, reject) => {
this.startCbs.push((err, id) => {
if (err !== undefined) {
reject(err);
}
else {
resolve(id);
}
});
ws.send(StartEventJSON);
});
});
}
stopContext(cb) {
if (!this.isOpen()) {
return cb(new Error('websocket is not ready'));
}
this.stopCbs.push(cb);
const ws = this.websocket;
ws.send(StopEventJSON);
stopContext() {
return __awaiter(this, void 0, void 0, function* () {
if (!this.isOpen()) {
throw Error('Websocket is not ready');
}
const ws = this.websocket;
return new Promise((resolve, reject) => {
this.stopCbs.push((err, id) => {
if (err !== undefined) {
reject(err);
}
else {
resolve(id);
}
});
ws.send(StopEventJSON);
});
});
}

@@ -120,2 +149,12 @@ sendAudio(audioChunk) {

}
closeWebsocket(code, message) {
if (this.websocket === undefined) {
throw Error('Websocket is not open');
}
this.websocket.removeEventListener('message', this.onWebsocketMessage);
this.websocket.removeEventListener('error', this.onWebsocketError);
this.websocket.removeEventListener('close', this.onWebsocketClose);
this.websocket.close(code, message);
this.websocket = undefined;
}
}

@@ -125,5 +164,5 @@ exports.WebsocketClient = WebsocketClient;

const StopEventJSON = JSON.stringify({ event: 'stop' });
function generateWsUrl(baseUrl, deviceId, languageCode, sampleRate) {
const secondsInHour = 60 * 60;
function generateWsUrl(baseUrl, languageCode, sampleRate) {
const params = new URLSearchParams();
params.append('deviceId', deviceId);
params.append('languageCode', languageCode);

@@ -133,20 +172,68 @@ params.append('sampleRate', sampleRate.toString());

}
function initializeWebsocket(url, protocol, cb) {
const ws = new WebSocket(url, protocol);
const errhandler = () => {
ws.removeEventListener('close', errhandler);
ws.removeEventListener('error', errhandler);
ws.removeEventListener('open', openhandler);
cb(Error('Connection failed'));
};
const openhandler = () => {
ws.removeEventListener('close', errhandler);
ws.removeEventListener('error', errhandler);
ws.removeEventListener('open', openhandler);
cb(undefined, ws);
};
ws.addEventListener('close', errhandler);
ws.addEventListener('error', errhandler);
ws.addEventListener('open', openhandler);
function isValidToken(token, appId, deviceId) {
const decoded = jsonwebtoken_1.decode(token);
if (decoded === null || typeof decoded !== 'object') {
return false;
}
if (decoded.exp === undefined || typeof decoded.exp !== 'number') {
return false;
}
// If the token will expire in an hour or less, mark it as invalid.
if (decoded.exp - Date.now() / 1000 < secondsInHour) {
return false;
}
if (decoded.appId !== appId) {
return false;
}
if (decoded.deviceId !== deviceId) {
return false;
}
return true;
}
function login(baseUrl, appId, deviceId) {
var _a;
return __awaiter(this, void 0, void 0, function* () {
const body = { appId, deviceId };
const response = yield fetch(baseUrl, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(body),
});
const json = yield response.json();
if (response.status !== 200) {
throw Error((_a = json.error) !== null && _a !== void 0 ? _a : `Speechly API login request failed with ${response.status}`);
}
if (json.access_token === undefined) {
throw Error('Invalid login response from Speechly API');
}
if (!isValidToken(json.access_token, appId, deviceId)) {
throw Error('Invalid token received from Speechly API');
}
return json.access_token;
});
}
function initializeWebsocket(url, protocol) {
return __awaiter(this, void 0, void 0, function* () {
const ws = new WebSocket(url, protocol);
return new Promise((resolve, reject) => {
const errhandler = () => {
ws.removeEventListener('close', errhandler);
ws.removeEventListener('error', errhandler);
ws.removeEventListener('open', openhandler);
reject(Error('Connection failed'));
};
const openhandler = () => {
ws.removeEventListener('close', errhandler);
ws.removeEventListener('error', errhandler);
ws.removeEventListener('open', openhandler);
resolve(ws);
};
ws.addEventListener('close', errhandler);
ws.addEventListener('error', errhandler);
ws.addEventListener('open', openhandler);
});
});
}
//# sourceMappingURL=websocket_client.js.map
types.d.ts
types.js
types.js.map
index.js.map

Sorry, the diff of this file is not supported yet

microphone/browser_audio_processor.js.map

Sorry, the diff of this file is not supported yet

microphone/browser_microphone.js.map

Sorry, the diff of this file is not supported yet

microphone/types.js.map

Sorry, the diff of this file is not supported yet

speechly/client.js.map

Sorry, the diff of this file is not supported yet

speechly/segment.js.map

Sorry, the diff of this file is not supported yet

speechly/types.js.map

Sorry, the diff of this file is not supported yet

storage/storage.js.map

Sorry, the diff of this file is not supported yet

storage/types.js.map

Sorry, the diff of this file is not supported yet

websocket/types.js.map

Sorry, the diff of this file is not supported yet

websocket/websocket_client.js.map

Sorry, the diff of this file is not supported yet

SocketSocket SOC 2 Logo

Product

  • Package Alerts
  • Integrations
  • Docs
  • Pricing
  • FAQ
  • Roadmap
  • Changelog

About

Packages

npm

Go

Maven

PyPI

Rubygems

Stay in touch

Get open source security insights delivered straight into your inbox.

  • Terms
  • Privacy
  • Security

Made with ⚡️ by Socket Inc