@voicify/react-native-voicify-assistant

Introduction

This project includes the models, API methods and classes for interacting with the Voicify Custom Assistant API for JavaScript and TypeScript, as well as a customizable Voicify Assistant component for React Native.

Getting Started

You can install the package from npm:

npm i -s "@voicify/react-native-voicify-assistant"

If you're using TypeScript, all the types are included, so you don't need to install any additional packages.

This SDK offers several out-of-the-box solutions, including:

• A customizable Voicify Custom Assistant component
• A speech to text (STT) class utilizing React Native Voice
• A text to speech class (TTS) that utilizes Expo AV to play the audio file generated from Voicify's API

This SDK also offers a variety of tools for creating your own Voice Assistant, including:

• A Voicify Assistant class for making requests to your Voicify Custom Assistant
• A speech to text class (STT) for providing your own STT
• A text to speech class (TTS) for providing your own TTS

Whether you are looking for a quick and easy way to add a Voicify Assistant to your project, or you are building your own assistant from scratch, this SDK has all the details you need as long as you have a Voicify app to make requests against and the severRootUrl, applicationId and applicationSecret required to make requests. Voicify users can find these resources in the deployments page of the application that they are using with the SDK.

Using the Assistant Drawer UI

The Assistant Drawer UI component is prebuilt to allow for an easy solution for implementing a Voicify Assistant in your project. The component requires some settings that get passed in as props and then its ready to go. All of the props that are not part of the styling, (the header, body and toolbar objects), are required. Contrastly, none of the styling props are required.

For example, the component can be initialized with the required settings and a few styling options.

import { AssistantDrawerUi } from "@voicify/react-native-voicify-assistant";
    const [isAssistantOpen, setIsAssistantOpen] = useState(false)

    const onMicPressed = () => {
        setIsAssistantOpen(true)
    }
    const onModalClosed = () => {
        setIsAssistantOpen(false)
    }

    const onEffect = async (effect: string, data: any) => {
        if (effect == 'Play') {
           console.log(data)
        }
    }
    return (
        <AssistantDrawerUI
            onModalClosed={onModalClosed.bind(this)}
            serverRootUrl='ex: https://assistant.voicify.com'
            appId='your-voicify-appId'
            appKey='your-voicify-appSecret'
            locale='en-US'
            channel="My App"
            device="My device"
            autoRunConversation={true}
            initializeWithWelcomeMessage={false}
            useOutputSpeech={true}
            useVoiceInput={true}
            isModalOpen={isAssistantOpen}
            initializeWithText={false}
            clearConversationOnClose={false}
            effects={['Play']}
            onEffect={onEffect.bind(this)}
            textToSpeechProvider = 'Google'
            voice=""
            header={{
                backgroundColor: "#FFFFFF",
            }}
            body={{
                messageSentFontSize: 16,,
            }}
            toolBar={{
                drawerWelcomeText: "How can i help?",
            }}>
        </AssistantDrawerUI>
    )

The component can also be configured with many more styling options:

    <AssistantDrawerUI
        onModalClosed={onModalClosed.bind(this)}
        serverRootUrl='ex: https://assistant.voicify.com'
        appId='your-voicify-appId'
        appKey='your-voicify-appSecret'
        locale='en-US'
        channel="My App"
        device="My device"
        autoRunConversation={true}
        initializeWithWelcomeMessage={false}
        useOutputSpeech={true}
        useVoiceInput={true}
        isModalOpen={isAssistantOpen}
        initializeWithText={false}
        clearConversationOnClose={false}
        effects={['Play']}
        onEffect={onEffect.bind(this)}
        textToSpeechProvider = 'Google'
        voice=""
        header={{
            backgroundColor: "#FFFFFF",
            assistantImage: "your-header-assistant-image-url",
            assistantImageBorderColor: "#CBCCD2",
            assistantName: "Your Assistant Name",
            assistantNameTextColor: "#000000",
            assistantNameFontSize: 18,
            assistantImageBorderWidth: 1,
            assistantImageBorderRadius: 30,
            assistantImageBorderStyle: 'solid',
            closeAssistantButtonImage: "your-close-assistant-image-url",
            paddingLeft: 12,
            paddingRight: 20,
            paddingTop: 20,
            paddingBottom: 0
        }}
        body={{
            backgroundColor: "#F4F4F6",
            assistantImageBorderRadius: 30,
            assistantImageBorderStyle: 'solid',
            assistantImageBorderWidth: 1,
            aassistantImageBorderColor: "#CBCCD2",
            assistantImageBackgroundColor: "#FFFFFF",
            assistantImage: "your-body-assistant-image-url",
            messageSentTextColor: "#FFFFFF",
            messageReceivedTextColor: "#000000",
            messageSentBackgroundColor: "rgba(0, 0, 0, 0.5)",
            messageReceivedBackgroundColor: "rgba(0, 0, 0, 0.05)",
            messageSentBorderColor: "transparent",
            messageReceivedBorderColor: "#CCCCCC",
            messageSentBorderStyle: 'solid',
            messageSentFontSize: 16,
            messageReceivedFontSize : 16,
            mesasgeSentBorderWidth: 0,
            messageSentBorderTopRightRadius: 0,
            messageSentBorderTopLeftRadius: 8,
            messageSentBorderBottomRightRadius: 8,
            messageSentBorderBottomLeftRadius: 8,
            messageReceivedBorderStyle: 'solid',
            messageReceivedBorderWidth: 1,
            messageReceivedBorderTopRightRadius: 8,
            messageReceivedBorderTopLeftRadius: 0,
            messageReceivedBorderBottomRightRadius: 8,
            messageReceivedBorderBottomLeftRadius: 8,
            borderTopColor: "#CBCCD2",
            borderBottomColor: "#CBCCD2",
            borderTopWidth: 1,
            borderBottomWidth: 1,
            borderStyle: "solid",
            paddingTop: 20,
            paddingLeft: 16,
            paddingRight: 16,
            paddingBottom: 12,
            hintsBackgroundColor: "#FFFFFF",
            hintsBorderColor: "#CCCCCC",
            hintsBorderStyle: "solid",
            hintsBorderWidth: 1,
            hintsPaddingTop: 12,
            hintsPaddingLeft: 12,
            hintsPaddingRight: 12,
            hintsPaddingBottom: 12,
            hintsBorderRadius: 30,
            hintsFontSize: 16,
            hintsTextColor: "#8F97A1",
        }}
        toolBar={{
            drawerWelcomeText: "How can i help?",
            drawerWelcomeTextColor: "#8F97A1",
            drawerWelcomeTextFontStyle: "italic",
            drawerWelcomeFontSize: 18,
            equalizerColor: '#8F97A1',
            micActiveImage: "your-mic-active-image-url",
            micInactiveImage: "your-mic-inactive-image-url",
            sendActiveImage: "your-send-message-button-active-image-url",
            sendInactiveImage: "your-send-message-button-inactive-image-url",
            speakActiveTitleColor: "#3E77A5",
            speakInactiveTitleColor: "#8F97A1",
            typeActiveTitleColor: "#3E77A5",
            typeInactiveTitleColor: "#8F97A1",
            micActiveHighlightColor: "rgba(30, 126, 185, 0.12)",
            micInactiveHighlightColor: "transparent",
            textboxActiveHighlightColor: "rgba(30, 126, 185, 0.12)",
            textboxInactiveHighlightColor: "transparent",
            partialSpeechResultTextColor: "rgba(255 , 255, 255, .2)",
            fullSpeechResultTextColor: "#FFFFFF",
            speechResultBoxBackgroundColor: "rgba(0, 0, 0, 0.5)",
            speechResultTextFontSize: 16,
            assistantStateTextColor: "#8F97A1",
            assistantStateFontSize: 16,
            assistantStateFontStyle: "italic",
            textInputLineColor: "#8F97A1",
            textInputCursorColor: "#707174",
            textInputTextColor: "#8F97A1",
            backgroundColor: "#FFFFFF",
            paddingLeft: 20,
            paddingRight: 20,
            paddingTop: 12,
            paddingBottom: 30,
            speakFontSize: 12,
            typeFontSize: 12,
        }}>
    </AssistantDrawerUI>

Create Your Own Assistant

While the Assistant Drawer UI offers a quick and easy way to integrate a Voicify Assistant, some cases may require more customization. For those cases, this SDK also provides a Voicify Assistant class that can be initialized with your serverRootUrl, applicationId, and applicationSecret from Voicify. Once it's configured, it becomes easy to make requests to your Voicify Custom Assistant. Additionally, the Assistant class can be configured with a TTS provider and a STT provider. In the case that you would like to use your own, you can utilize the VoicifyTextToSpeechProvider and VoicifySpeechToTextProvider classes. If you would like to utilize the providers that come with the SDK, then you can pass in the VoicifyExpoTTSProvider and ReactNativeVoiceSTTProvider into the assistant class.

For example, the assistant class can be initialized with the required settings and provided TTS and STT classes:

import { VoicifyAssistant, VoicifyExpoTTSProvider, ReactNativeVoiceSTTProvider } from "@voicify/react-native-voicify-assistant";

const TTS = new VoicifyExpoTTSProvider({
    appId: 'your-voicify-appId',
    appKey: 'your-voicify-appSecret',
    locale: 'en-US',
    provider: "Google",
    serverRootUrl: "ex: https://assistant.voicify.com",
    voice: ""
});

const STT = new ReactNativeVoiceSTTProvider("en-US")

const Assistant = new VoicifyAssistant({
    serverRootUrl='ex: https://assistant.voicify.com',
    appId: 'your-voicify-appId',
    appKey: 'your-voicify-appSecret',
    locale: 'en-US',
    channel: "My App",
    device: "My device",
    autoRunConversation: false,
    initializeWithWelcomeMessage: false,
    useOutputSpeech: true,
    useVoiceInput: true,a
}, TTS, STT);

Now the assistant can be used to make requests to Voicify:

useEffect(() => {
    Assistant.clearHandlers();
    Assistant.initializeAndStart();
    Assistant.onRequestStarted(req => {
        console.log(req)
    })
    Assistant.onResponseReceived(res => {
        console.log(res)
    })
    Assistant.startNewSession()
    Assistant.onEffect(effect, async (data) => {
        if (effect == "play") {
            console.log(data)
        }
    })
}, [])

Assistant.makeTextRequest("message-to-voicify-assistant")

Additional Notes

There are a few things to note when using the SDK.

Markdown

The Assistant Drawer UI component supports the ability to use markdown when displaying messages received from Voicify. In order to utilize this feature, markdown must be provided in the display text of the conversation item that is being hit. Additionally, if the markdown contains a link, the react-native-inappbrowser-reborn package must be installed into the project in order to open it. For more details about the package, please visit https://www.npmjs.com/package/react-native-inappbrowser-reborn.

React Native Voice Package

The provided speech to text class utilizes the React Native Voice STT package. For more details, please visit https://github.com/react-native-voice/voice.

Expo AV Package

The provided text to speech class utilizes the Expo AV package in order to play the TTS audio file that gets generated from Voicify's API. For more details, please visit https://docs.expo.dev/versions/latest/sdk/av/.