symbl-real-time-nebula
Advanced tools
A real time copilot assistant for sales agent to help make quick suggestions whiles they are on a call with a customer.
A real time copilot assistant for sales agent to help make quick suggestions whiles they are on a call with a customer.
To initialise the sdk(CopilotAssistantAgent) you need to pass an object with the following parameters.
symblAccessToken - Symbl Token. You can get it from Platform (required)nebulaApiKey - Nebula API Key. You can get it from Nebula (required)conversationId - Conversation ID from symbl session. (required)connectionId - Connection ID from symbl session. (required)trackers - An array of trackers to be used for the conversation. (optional)enableAllTrackers - A boolean to enable all trackers. (optional) Which is true by default.enableInterimResults - A boolean to enable interim results. (optional) Which is true by default.NOTE: You can't pass both trackers and enableAllTrackers to be true at the same time. If you pass both, trackers will be used.
The trackers array is a collection of tracker objects. Each tracker is used to identify specific phrases or words within a given context. Trackers are primarily characterized by a name and a vocabulary list. An optional id can also be included for each tracker, typically sourced from a management API.
Each element in the trackers array is an object with the following fields:
name (string): The name of the tracker, representing the category or type of phrases it tracks.vocabulary (array of strings): A list of phrases or words that this tracker is responsible for identifying.id (string, optional): A unique identifier for the tracker, usually obtained from a management API.When the sdk is initialised, it will automatically connect to the nebula websocket and start listening for suggestions based on the first stage and the meta data provided.
const sdk = CopilotAssistantAgent({
symblAccessToken, // required
nebulaApiKey, // required
conversationId, // required
connectionId, // required
trackers, // optional
enableAllTrackers, // optional
enableInterimResults,
});
The onSuggestions method accepts a callback function that is invoked when suggestions are available.
Syntax
sdk.onSuggestions(callback);
The callback function takes an object as a parameter. This object contains a suggestion property which is of type string
{
suggestion: "Hi ...";
}
suggestion - Is of type string and contains the suggestion from nebula.sdk.onSuggestions(({ suggestion }) => {
console.log(suggestion);
});
This method is called when the sdk receives a transcription from symbl with the isFinal as true. It returns the following parameters.
Syntax:
sdk.onCompletedTranscriptions(callback);
The callback function takes an array of objects as a parameter. This array contains the transcriptions from symbl of format below and is called messages
{
messages: [
{
from: {
id: string,
name: string,
userId: string,
},
payload: {
content: string,
contentType: "text/plain",
},
id: string,
channel: {
id: "realtime-api",
},
metadata: {
disablePunctuation: true,
originalContent: string,
words: string,
originalMessageId: string,
},
dismissed: false,
duration: {
startTime: string,
endTime: string,
timeOffset: number,
duration: number,
},
entities: [],
},
];
}
messages - Is of type array and contains the transcriptions from symbl of format; messages
from - Is of type object and contains the user or speaker information.
id - Is of type string and contains the id of the user or speaker.name - Is of type string and contains the name of the user or speaker.userId - Is of type string and contains the userId of the user or speaker.payload - Is of type object and contains the payload of the transcription.
content - Is of type string and contains the content of the transcription.contentType - Is of type string and contains the contentType of the transcription.id - Is of type string and contains the id of the transcription.channel - Is of type object and contains the channel of the transcription.
id - Is of type string and contains the id of the channel.metadata - Is of type object and contains the metadata of the transcription.
disablePunctuation - Is of type boolean and contains the disablePunctuation of the transcription.originalContent - Is of type string and contains the originalContent of the transcription.words - Is of type string and contains the words of the transcription.originalMessageId - Is of type string and contains the originalMessageId of the transcription.dismissed - Is of type boolean and contains the dismissed of the transcription.duration - Is of type object and contains the duration of the transcription.
startTime - Is of type string and contains the startTime of the transcription.endTime - Is of type string and contains the endTime of the transcription.timeOffset - Is of type number and contains the timeOffset of the transcription.duration - Is of type number and contains the duration of the transcription.entities - Is of type array and contains the entities of the transcription.sdk.onCompletedTranscriptions(({ messages }) => {
console.log(messages);
});
This method is called when the sdk receives a transcription from symbl in realtime as the users speak. It returns the following parameters.
Syntax:
sdk.onRealtimeTranscript(callback);
The callback function takes an object as a parameter. This object contains a message property which is of type object
{
message: {
type: "recognition_result",
isFinal: false,
payload: {
raw: {
alternatives: [
{
words: [],
transcript: "Hello",
confidence: 0,
},
],
},
},
punctuated: {
transcript: "Hello",
},
user: {
name: "agent",
userId: "agent@123.com",
"id": "1a726dac-1c95-484d-a99e-69e36950bf32"
}
}}
message - Is of type object and contains the transcription from symbl.
type - Is of type string and contains the type of the transcription.isFinal - Is of type boolean and contains the isFinal of the transcription.payload - Is of type object and contains the payload of the transcription.
raw - Is of type object and contains the raw of the transcription.
alternatives - Is of type array and contains the alternatives of the transcription.
words - Is of type array and contains the words of the transcription.transcript - Is of type string and contains the transcript of the transcription.confidence - Is of type number and contains the confidence of the transcription.punctuated - Is of type object and contains the punctuated of the transcription.
transcript - Is of type string and contains the transcript of the transcription.user - Is of type object and contains the user or speaker information.
name - Is of type string and contains the name of the user or speaker.userId - Is of type string and contains the userId of the user or speaker.id - Is of type string and contains the id of the transcription.To access the actual transcript, you can use the following code snippet.
sdk.onRealtimeTranscript(({ message }) => {
const { transcript } = message.puntuated;
console.log(transcript);
});
You can also access the user or speaker name using the following code snippet.
sdk.onRealtimeTranscript(({ message }) => {
const { name } = message.user;
console.log(name);
});
This method is called when the sdk receives any response from symbl. It returns the following parameters.
Syntax:
sdk.onSymblResponse(callback);
// or
sdk.onSymblResponse(({ response }) => {
console.log(response);
});
The callback function takes an object as a parameter. This object contains a response property.
To check the possible details of the response visit this url Syml Streaming API Docs
Also note to look of for the following in the docs after visiting the url;
Response reference
This method is called when the sdk receives a tracker response from symbl. It returns the following parameters.
Syntax:
sdk.onTrackerResponse(callback);
// or
sdk.onTrackerResponse(({ trackers }) => {
console.log(trackers);
});
The callback function takes an array of objects. This object contains a trackers property which is of type array
{
trackers: [
{
name: "test",
matches: [
{
type: "vocabulary",
value: "test",
messageRefs: [
{
id: "7c4b04c6-cc86-4ef8-8a97-a26c1eb9a21f",
text: "It was a test.",
offset: 9,
},
],
insightRefs: [],
},
],
},
];
}
trackers - Is of type array and contains the trackers from symbl.
name - Is of type string and contains the name of the tracker.matches - Is of type array and contains the matches of the tracker.
type - Is of type string and contains the type of the match.value - Is of type string and contains the value of the match.messageRefs - Is of type array and contains the messageRefs of the match.
id - Is of type string and contains the id of the messageRefs.text - Is of type string and contains the text of the messageRefs.offset - Is of type number and contains the offset of the messageRefs.insightRefs - Is of type array and contains the insightRefs of the match.This method returns the transcriptions received from symbl. It accepted an option parameter called fromMemory which is false by default. When set to true, it returns the transcriptions from memory. When set to false, it returns the transcriptions making request to symbl api. It returns as an array of objects, with the format;
[
{
"from": {
"id": string,
"name": string,
"userId": string
},
"payload": {
"content": string,
"contentType": "text/plain"
},
"id": string,
"channel": {
"id": "realtime-api"
},
"metadata": {
"disablePunctuation": true,
"originalContent": string,
"words": string,
"originalMessageId": string
},
"dismissed": false,
"duration": {
"startTime": string,
"endTime": string,
"timeOffset": number,
"duration": number
},
"entities": []
}
]
Syntax:
const transcriptions = await sdk.getTranscriptions(fromMemory);
// or
const transcriptions = await sdk.getTranscriptions(true);
This method stops the sdk from listening to the symbl websocket and closes the connection.
Syntax:
sdk.stop();
This function calls the nebula api for suggestions. The suggestions are received from the onSuggestions callback function. It returns the following parameters.
Parameters:
options (Object): The options for calling Nebula.
systemPrompt (string): The system prompt for Nebula which is optional.messages (Array): The messages to send to Nebula.
messages[].role (string): The role of the message (either "human" or "assistant").messages[].text (string): The text content of the message.You can get more information about Nebula Chat model Returns:
Promise<void>.Example:
await sdk.callNebulaAPI({
systemPrompt: "Help answer question asked politely",
messages: [
{
role: "human",
text: "Agent: Hello how are you \n User: This is Samson",
},
],
});
One important requirement for the solution is to keep latency to the minimum while keeping some reasonable response from Nebula LLM.
We broke it down in different blocks:
Average latency (excluding the initial audio communication): ~1865ms
Note: External factors such as network variability, server load, and the performance of the client's device significantly impact end-to-end latency.
FAQs
A real time copilot assistant for sales agent to help make quick suggestions whiles they are on a call with a customer.
The npm package symbl-real-time-nebula receives a total of 0 weekly downloads. As such, symbl-real-time-nebula popularity was classified as not popular.
We found that symbl-real-time-nebula demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
A malicious npm package disguised as a WhatsApp client is exploiting authentication flows with a remote kill switch to exfiltrate data and destroy files.
Security News
PyPI now supports digital attestations, enhancing security and trust by allowing package maintainers to verify the authenticity of Python packages.
Security News
GitHub removed 27 malicious pull requests attempting to inject harmful code across multiple open source repositories, in another round of low-effort attacks.