streamsearch

What is streamsearch?

The streamsearch npm package is primarily used for searching for a specific sequence of bytes in a streaming data source. It is particularly useful for parsing streams to find delimiters or separators, such as in multipart/form-data streams where boundaries need to be identified. This package provides a fast and efficient way to scan through data without buffering the entire stream, making it ideal for handling large volumes of data in real-time.

What are streamsearch's main functionalities?

Byte sequence searching in streams

This feature allows the user to search for a specific byte sequence within a stream. The example code initializes a StreamSearch instance with a needle ('example') and pushes a buffer to it. It listens for 'info' events to detect matches and outputs whether the needle was found in the provided data.

const StreamSearch = require('streamsearch');

let needle = Buffer.from('example');
let s = new StreamSearch(needle);

s.on('info', (isMatch, data, start, end) => {
  console.log('Data:', data);
  console.log('Match found:', isMatch);
});

// Simulate streaming data
s.push(Buffer.from('some example data containing the needle'));

Other packages similar to streamsearch

buffer-indexof

Similar to streamsearch, buffer-indexof provides functionality to find the index of a buffer within another buffer. However, it does not support streaming data inherently and is used for buffer objects in memory, which might not be as efficient for large or streaming datasets compared to streamsearch.

README

Description

streamsearch is a module for node.js that allows searching a stream using the Boyer-Moore-Horspool algorithm.

This module is based heavily on the Streaming Boyer-Moore-Horspool C++ implementation by Hongli Lai here.

Requirements

• node.js -- v0.8.0 or newer

Installation

npm install streamsearch

Example

  var StreamSearch = require('streamsearch'),
      inspect = require('util').inspect;

  var needle = new Buffer([13, 10]), // CRLF
      s = new StreamSearch(needle),
      chunks = [
        new Buffer('foo'),
        new Buffer(' bar'),
        new Buffer('\r'),
        new Buffer('\n'),
        new Buffer('baz, hello\r'),
        new Buffer('\n world.'),
        new Buffer('\r\n Node.JS rules!!\r\n\r\n')
      ];
  s.on('data', function(data, start, end) {
    console.log('data: ' + inspect(data.toString('ascii', start, end)));
  });
  s.on('match', function() {
    console.log('match!');
  });
  for (var i = 0, len = chunks.length; i < len; ++i)
    s.push(chunks[i]);

  // output:
  //
  // data: 'foo'
  // data: ' bar'
  // match!
  // data: 'baz, hello'
  // match!
  // data: ' world.'
  // match!
  // data: ' Node.JS rules!!'
  // match!
  // data: ''
  // match!

API

Events

• data(< Buffer >chunk, < integer >start, < integer >end) - Emitted when non-needle data is available. This data is in chunk between start (inclusive) and end (exclusive).

• match() - Emitted when the needle has been found in the stream.

Properties

• maxMatches - < integer > - The maximum number of matches. Defaults to Infinity.

• matches - < integer > - The current match count.

Functions

• (constructor)(< Buffer >needle) - Creates and returns a new instance for searching for needle.

• push(< Buffer >chunk) - integer - Processes chunk. The return value is the last processed index in chunk + 1.

• reset() - (void) - Resets internal state. Useful for when you wish to start searching a new/different stream for example.