agora-rte-extension

Agora-RTE-Extension

Introduction

An extension system for Agora Web SDK NG.

Installation

npm install --save agora-rte-extension

when using Yarn:

yarn add agora-rte-extension

Usage Guide

1. Extending Processor

a. Choose between VideoProcessor or AudioProcessor

Depend on your media type, which is either video or audio, you should extend VideoProcessor or AudioProcessor accordingly.

import {VideoProcessor, AudioProcessor} from "agora-rte-extension";

//for video processor
class YourVideoProcessor extends VideoProcessor {
}

//for audio processor
class YourAudioProcessor extends AudioProcessor {
}

b. Extending VideoProcessor

For Video Processor, there are 1 property, 4 methods could be implemented:

import {VideoProcessor, AudioProcessor} from "agora-rte-extension";
import type {IProcessorContext} from "agora-rte-extension";

class YourVideoProcessor extends VideoProcessor {
  /**
   * processor name 
   * */
  public name = "YourVideoProcessor"

  /**
   * will be called when AgoraRTC.LocalTrack was connected to the pipedline
   * */
  onPiped(context: IProcessorContext) {
  }

  /**
   * will be called when AgoraRTC.LocalTrack was disconnected from the pipedline,
   * or the current/previous processor was disconnected from the pipeline.
   * */
  onUnpiped() {
  }

  /**
   * will be called when previous processor generated output MediaStreamTrack
   * */
  onTrack(track: MediaStreamTrack, context: IProcessorContext) {
  }

  /**
   * will be called when enable/disable methods on Processor was called
   * */
  onEnableChange(enabled: boolean) {
  }
}

When the video is being processed and is about to generate output, you can call output method on Processor:

import {IProcessorContext} from "agora-rte-extension";

class YourVideoProcessor extends VideoProcessor {
  onTrack(track: MediaStreamTrack, context: IProcessorContext) {
    //...processing
    const processedTrack = this.process(track);

    //output processed MediaStreamTrack
    this.output(processedTrack, context);
  }
}

c. Extending AudioProcessor

For Audio Processor, there are 1 property, 5 methods could be implemented:

import {VideoProcessor, AudioProcessor} from "agora-rte-extension";
import type {IAudioProcessorContext} from "agora-rte-extension";

class YourVideoProcessor extends VideoProcessor {
  /**
   * processor name
   * */
  public name = "YourVideoProcessor"

  /**
   * will be called when AgoraRTC.LocalTrack was connected to the pipedline
   * */
  onPiped(context: IAudioProcessorContext) {
  }

  /**
   * will be called when AgoraRTC.LocalTrack was disconnected from the pipedline,
   * or the current/previous processor was disconnected from the pipeline.
   * */
  onUnpiped() {
  }

  /**
   * will be called when previous processor generated output MediaStreamTrack
   * */
  onTrack(track: MediaStreamTrack, context: IAudioProcessorContext) {
  }
  
  /**
   * will be called when previous processor generated output AudioNode
   * */
  onNode(node: AudioNode, context: IAudioProcessorContext) {
  }

  /**
   * will be called when enable/disable methods on Processor was called
   * */
  onEnableChange(enabled: boolean) {
  }
}

When the audio is being processed and is about to generate output, you can call output method on Processor:

import {IProcessorContext} from "./index";

class YourAudioProcessor extends AudioProcessor {
  onNode(node: AudioNode, context: IAudioProcessorContext) {
    //...processing
    const processedAudioNode = this.process(node);

    this.output(processedAudioNode, context);
  }
}

2. Extending Extension

a.Choose betweenExtension orAudioExtension

Depend on your media type, which is either video or audio, you should extendExtension orAudioExtension accordingly.

import {Extension, AudioExtension} from "agora-rte-extension";

//for video extension
class YourVideoExtension extends Extension<YourVideoProcessor> {
}


//for audio extension
class YourAudioExtension extends AudioExtension<YourAudioProcessor> {
}

b. implement _createProcessor method

for Extension to create VideoProcessor or AudioProcessor, you should implement _createProcessor based on Extension or AudioExtension:

import {Extension, AudioExtension} from 'agora-rte-extension'

class YourVideoExtension extends Extension<YourVideoProcessor> {
  protected _createprocessor(){
    return new YourVideoProcessor();
  }
}

class YourAudioExtension extends AudioExtension<YourAudioProcessor> {
  protected _createprocessor(){
    return new YourAudioProcessor();
  }
}