mse-audio-wrapper

MSE (Media Source Extensions) Audio Wrapper

mse-audio-wrapper is a library to enable Media Source Extensions API playback for unsupported audio containers and raw codecs. MSE Audio Wrapper uses both WEBM and the ISO Base Media File Format (MPEG-4 Part 12) (commonly referred to as Fragmented MP4 or fmp4).

• API
  • MSEAudioWrapper
    • Takes in audio (MP3, AAC, FLAC, Ogg FLAC, Ogg Opus, Ogg Vorbis)
    • Outputs
      • ISOBMFF (fMP4) (MP3, AAC, Flac, Opus)
      • WEBM (Opus, Vorbis)
• Demo
  • React application that demonstrates MSEAudioWrapper being used to support icecast-metadata-js
  • Checkout the demo here!

---

Installing

Install via NPM

• npm i mse-audio-wrapper

Usage

1. To use MSEAudioWrapper, create a new instance of the class by passing in the mimetype of your audio data.

   Note: For directly converting from a HTTP response, use the mimetype contained in the Content-Type header

   import MSEAudioWrapper from "mse-audio-wrapper";
   
   const headers = myHTTPResponse.headers;
   const mimeType = headers.get('Content-Type');
   
   const audioWrapper = new MSEAudioWrapper(mimeType);
   

2. To begin processing audio data, pass in the audio data into the instance's .iterator(). This method returns an iterator that can be consumed using a for ...of or for await...of loop. Repeat this step until all audio data has been read.

   const audioData = response.body;
   
   for (const wrappedAudio of audioWrapper.iterator(audioData)) {
     // Do something with the wrapped data
   }
   

• MSEAudioWrapper will store any partial data until at least one full audio frame can be processed.

  Note: Any data that does not conform to the instance's mimetype will be discarded.

• Once enough data has been received to form at least 4 complete audio frames, and 1022 bytes of audio data, the initial segment will be returned along with a media segment containing the audio data. These values are user configurable using the options parameter in the constructor.

  • 1st Iteration

    "initial segment"
    --ftyp [file type]
    --moov [movie]
    "media segment"
    --moof [movie fragment]
    --mdat [audio data]
    

• Subsequent iterations will only return media segments.

  • nth Iteration

    "media segment"
    --moof [movie fragment]
    --mdat [audio data]
    

Methods

const wrapper = new MSEAudioWrapper("audio/mpeg", {minFramesPerSegment: 2, minBytesPerSegment: 576, preferredContainer: "webm"});

• constructor
  • Creates a new instance of MSEAudioWrapper that can be used to wrap audio for a given mimetype.
  • Parameters:
    • mimetype required Incoming audio codec or container
      • MP3 - audio/mpeg
      • AAC - audio/aac, audio/aacp
      • FLAC - audio/flac
      • Ogg FLAC - application/ogg, audio/ogg
      • Ogg Opus - application/ogg, audio/ogg
      • Ogg Vorbis - application/ogg, audio/ogg
    • options optional

      Options

      • options.minFramesPerSegment optional Minimum audio frames to store before returning a segment
        • Accepts an integer greater than 0
        • Defaults to 4
      • options.minBytesPerSegment optional Minimum audio bytes to store before returning a segment
        • Accepts an integer greater than 0
        • Defaults to 1022
      • options.maxFramesPerSegment optional Maximum audio frames to group in a single segment
        • Accepts an integer greater than 0
        • Defaults to 50
      • options.preferredContainer optional Preferred output container when there are multiple supported containers
        • Accepts "webm", "fmp4"
        • Defaults to "webm"
      • options.codec optional Specifies the type of codec frames that will be passed into iterator(Array<CodecFrame>)
        • Do not use this option when passing in raw Uint8Array data to iterator()
        • Accepts aac, flac, mpeg, opus, vorbis
        • Defaults to undefined
      • options.enableLogging optional Set to true to enable debug logging
        • Defaults to false

      Callbacks

      • options.onMimeType(mimeType) optional Called when the output mimeType is determined.
        • See wrapper.mimeType for a list of the possible output mimetypes
      • options.onCodecUpdate(codecInfo, updateTimestamp) optional Called when there is a change in the codec header.
        • codecInfo is an object containing information about the codec such as bitrate, sampleRate, etc.
        • updateTimestamp is the time in milliseconds within the audio stream when the codec information was updated
• wrapper.iterator(Uint8Array | Array<CodecFrame>)
  • Returns an Iterator that can be used in a for ...of loop to return wrapped audio
  • Parameters:
    • data (Uint8Array | Array<CodecFrame>)
      • Note: CodecFrame must follow the api noted in codec-parser and options.codec must be set with the correct codec.
• wrapper.inputMimeType
  • Getter that returns the mime-type of the incoming audio data
  • Examples:
    • MP3 - audio/mpeg
    • AAC - audio/aac
    • FLAC - audio/flac
    • Ogg FLAC - application/ogg, audio/ogg
    • Ogg Opus - application/ogg, audio/ogg
    • Ogg Vorbis - application/ogg, audio/ogg
• wrapper.mimeType
  • Getter that returns the mime-type of the wrapped audio data
    • Note: For Ogg streams, the mime-type will only be available after the first media segment is returned.
  • Examples:
    • MP3 - audio/mp4;codecs="mp3"
    • AAC - audio/mp4;codecs="mp4a.40.2"
    • FLAC - audio/mp4;codecs="flac"
    • OPUS (ISOBMFF) - audio/mp4;codecs="opus"
    • OPUS (WEBM) - audio/webm;codecs="opus"
    • Vorbis - audio/webm;codecs="vorbis"

---

Demo

mse-audio-wrapper is used in the demo for icecast-metadata-js to allow for Icecast metadata support in Firefox (mp3, aac, flac) and Chrome (flac) by wrapping the streaming audio in ISOBMFF so it can be used with the MediaSource API.

https://github.com/eshaz/icecast-metadata-js/tree/master/src/demo

View the live demo here: https://eshaz.github.io/icecast-metadata-js/