Huge News!Announcing our $40M Series B led by Abstract Ventures.Learn More →

bulksearch

Package Overview

Dependencies

Advanced tools

Install Socket

Detect and block malicious and high-risk dependencies

Install

bulksearch

Superfast lightweight full text search engine.

0.1.202
Source
npm

Version published: 7 years ago

Maintainers: 1

Created: 7 years ago

Source

BulkSearch

Superfast lightweight full text search engine.

Searching full text with BulkSearch is up to 1,000 times faster than ElasticSearch implementation.

Benchmark Comparison: https://jsperf.com/bulksearch

All Features:

Partial Words
Multiple Words
Flexible Word Order
Phonetic Search
Limit Results
Caching
Asynchronous Mode
Custom Matchers
Custom Encoders

Plugins In Progress:

Common Phonetic Encoders:
- Soundex
- Cologne
- Metaphone
- Caverphone
- Levinshtein
- Hamming
- Matchrating
- NGram
Dedicated Memory (Worker)

Installation

Node.js

npm install bulksearch

In your code include as follows:

var BulkSearch = require("BulkSearch");

HTML / Javascript

<html>
<head>
    <script src="https://cdn.rawgit.com/nextapps-de/bulksearch/master/bulksearch.min.js"></script>
</head>
...

AMD

var BulkSearch = require("BulkSearch");

Usage (API)

Create a new index

var index = new BulkSearch();

alternatively you can also use:

var index = BulkSearch.create();

Create a new index with custom options

BulkSearch.create(options)

var index = new BulkSearch({

    // default values:

    type: "integer",
    encode: "icase",
    boolean: "and",
    strict: false,
    ordered: false,
    multi: false,
    cache: false
});

Add items to an index

Index.add_(id, string)

index.add(10025, "John Doe");

Search items

Index.search(string, limit, callback)

index.search("John");

Limit the result

index.search("John", 10);

Perform queries asynchronously

index.search("John", function(result){
    
    // array of results
});

Update item to the index

Index.update(id, string)

index.update(10025, "Road Runner");

Remove item to the index

Index.remove(id)

index.remove(10025);

Destroy the index

index.destroy();

Initialize the index

Index.init(options)

index.init();

Add custom matcher

Index.addMatcher(KEY_VALUE_PAIRS)

index.addMatcher({

    'ä': 'a', // replaces all 'ä' to 'a'
    'ö': 'o',
    'Ü': 'u'
});

Add custom encoder

var index = new BulkSearch({

    encode: function(str){
    
        // do something with str ...
        
        return str;
    }
});

Get info

index.info();

Returns information about the index, e.g.:

{
    "bytes": 3600356288,
    "chunks": 9,
    "fragmentation": 0,
    "fragments": 0,
    "id": 0,
    "length": 7798,
    "matchers": 0,
    "size": 10000,
    "status": false
}

Note: When the fragmentation value is about 50% or higher, your should consider using cleanup() to free all fragmented available memory.

Optimize / Cleanup index

index.cleanup();

Phonetic Encoding

Option	Description	Example
false	Turn off encoding	Reference: "Björn-Phillipp Mayer" Matches: "Phil"
icase	Case in-sensitive encoding	Reference: "Björn-Phillipp Mayer" Matches: "phil"
simple	Phonetic normalizations	Reference: "Björn-Phillipp Mayer" Matches: "bjoern fillip"
advanced	Phonetic normalizations + transformations	Reference: "Björn-Phillipp Mayer" Matches: "filip meier"

Compare Phonetic Search Results

Reference String: "Björn-Phillipp Mayer"

Query	ElasticSearch	BulkSearch (iCase)	BulkSearch (Simple)	BulkSearch (Adv.)
björn	yes	yes	yes	yes
björ	no	yes	yes	yes
bjorn	no	no	yes	yes
bjoern	no	no	no	yes
philipp	no	no	no	yes
filip	no	no	no	yes
björnphillip	no	no	yes	yes
meier	no	no	no	yes
björn meier	no	no	no	yes
meier filip	no	no	no	yes

Improve Memory Usage

Note: The data type of passed IDs has to be specified on creation. It is recommended to uses to most lowest possible data range here, e.g. use "short" when IDs are not higher than 65,535.

ID Type	Range of Values	Memory Usage of every ~ 100,000 Words
Byte	0 - 255	683 kb + 1.3 Mb
Short	0 - 65,535	1.3 Mb + 1.3 Mb
Integer	0 - 4,294,967,295	2.7 Mb + 1.3 Mb
Float	0 - * (16 digits)	5.3 Mb + 1.3 Mb
String	* (unlimited)	1.3 Mb * char count of IDs + 1.3 Mb

The required RAM per instance can be calculated as follow:

BYTES = CONTENT_CHAR_COUNT * (BYTES_OF_ID + 2)

Character count may be less related to the phonetic settings (e.g. when using soundex).

Author BulkSearch: Thomas Wilkerling
License: Apache 2.0 License

FAQs

What is bulksearch?

Is bulksearch well maintained?

Package last updated on 22 Feb 2018

Did you know?

Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.

Install

bulksearch

BulkSearch

Superfast lightweight full text search engine.

Installation

Node.js

HTML / Javascript

Usage (API)

Create a new index

Create a new index with custom options

Add items to an index

Search items

Limit the result

Perform queries asynchronously

Update item to the index

Remove item to the index

Destroy the index

Initialize the index

Add custom matcher

Add custom encoder

Get info

Optimize / Cleanup index

Phonetic Encoding

Compare Phonetic Search Results

Improve Memory Usage

Related posts

Outgoing Biden Administration Issues Sweeping Executive Order on AI-Driven Cybersecurity and Supply Chain Transparency

Fluent Assertions Faces Backlash After Abandoning Open Source Licensing