streamsearch
Advanced tools
Package description
The streamsearch npm package is primarily used for searching for a specific sequence of bytes in a streaming data source. It is particularly useful for parsing streams to find delimiters or separators, such as in multipart/form-data streams where boundaries need to be identified. This package provides a fast and efficient way to scan through data without buffering the entire stream, making it ideal for handling large volumes of data in real-time.
Byte sequence searching in streams
This feature allows the user to search for a specific byte sequence within a stream. The example code initializes a StreamSearch instance with a needle ('example') and pushes a buffer to it. It listens for 'info' events to detect matches and outputs whether the needle was found in the provided data.
const StreamSearch = require('streamsearch');
let needle = Buffer.from('example');
let s = new StreamSearch(needle);
s.on('info', (isMatch, data, start, end) => {
console.log('Data:', data);
console.log('Match found:', isMatch);
});
// Simulate streaming data
s.push(Buffer.from('some example data containing the needle'));
This package implements the Boyer-Moore-Horspool algorithm for substring searching. While it is efficient for static data, it lacks the streaming capabilities of streamsearch, making it less suitable for real-time data processing where data is received in chunks.
Similar to streamsearch, buffer-indexof provides functionality to find the index of a buffer within another buffer. However, it does not support streaming data inherently and is used for buffer objects in memory, which might not be as efficient for large or streaming datasets compared to streamsearch.
Readme
streamsearch is a module for node.js that allows searching a stream using the Boyer-Moore-Horspool algorithm.
This module is based heavily on the Streaming Boyer-Moore-Horspool C++ implementation by Hongli Lai here.
npm install streamsearch
const { inspect } = require('util');
const StreamSearch = require('streamsearch');
const needle = Buffer.from('\r\n');
const ss = new StreamSearch(needle, (isMatch, data, start, end) => {
if (data)
console.log('data: ' + inspect(data.toString('latin1', start, end)));
if (isMatch)
console.log('match!');
});
const chunks = [
'foo',
' bar',
'\r',
'\n',
'baz, hello\r',
'\n world.',
'\r\n Node.JS rules!!\r\n\r\n',
];
for (const chunk of chunks)
ss.push(Buffer.from(chunk));
// output:
//
// data: 'foo'
// data: ' bar'
// match!
// data: 'baz, hello'
// match!
// data: ' world.'
// match!
// data: ' Node.JS rules!!'
// match!
// data: ''
// match!
maxMatches - < integer > - The maximum number of matches. Defaults to Infinity.
matches - < integer > - The current match count.
(constructor)(< mixed >needle, < function >callback) - Creates and returns a new instance for searching for a Buffer or string needle. callback is called any time there is non-matching data and/or there is a needle match. callback will be called with the following arguments:
isMatch - boolean - Indicates whether a match has been found
data - mixed - If set, this contains data that did not match the needle.
start - integer - The index in data where the non-matching data begins (inclusive).
end - integer - The index in data where the non-matching data ends (exclusive).
isSafeData - boolean - Indicates if it is safe to store a reference to data (e.g. as-is or via data.slice()) or not, as in some cases data may point to a Buffer whose contents change over time.
destroy() - (void) - Emits any last remaining unmatched data that may still be buffered and then resets internal state.
push(< Buffer >chunk) - integer - Processes chunk, searching for a match. The return value is the last processed index in chunk + 1.
reset() - (void) - Resets internal state. Useful for when you wish to start searching a new/different stream for example.
FAQs
Streaming Boyer-Moore-Horspool searching for node.js
The npm package streamsearch receives a total of 8,092,547 weekly downloads. As such, streamsearch popularity was classified as popular.
We found that streamsearch demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Git dependencies in open source packages can introduce significant risks, including lack of version control, stability issues, dependency drift, and difficulty in auditing, making them potential targets for supply chain attacks.
Security News
Node.js has added experimental support for TypeScript, a move that highlights the growing importance of TypeScript in modern development.
Product
Check out what's new at Socket with our Product Changelog. It tracks all public-facing updates, improvements, and fixes so you can take full advantage of our features.