flexsearch
Advanced tools
FlexSearch is a high-performance, full-text search library for JavaScript. It is designed to be fast and efficient, providing a variety of indexing and search capabilities. FlexSearch can be used in both Node.js and browser environments, making it versatile for different types of applications.
Indexing
FlexSearch allows you to create an index and add documents to it. Each document is associated with a unique ID.
const FlexSearch = require('flexsearch');
const index = new FlexSearch();
index.add(1, 'Hello World');
index.add(2, 'Hi there');Searching
You can perform searches on the indexed documents. The search results will return the IDs of the matching documents.
index.search('Hello', function(results) {
console.log(results); // [1]
});Asynchronous Search
FlexSearch supports asynchronous search operations, which can be useful for handling large datasets or integrating with other asynchronous workflows.
index.search('Hello').then(function(results) {
console.log(results); // [1]
});Custom Configuration
FlexSearch allows for extensive customization of the indexing and search process. You can configure encoding, tokenization, threshold, and resolution to suit your specific needs.
const customIndex = new FlexSearch({
encode: 'icase',
tokenize: 'forward',
threshold: 0,
resolution: 3
});
customIndex.add(1, 'Custom configuration example');Lunr.js is a small, full-text search library for use in the browser and Node.js. It provides a simple and easy-to-use API for indexing and searching documents. Compared to FlexSearch, Lunr.js is more lightweight but may not offer the same level of performance and customization.
Elasticlunr.js is a lightweight full-text search library that is inspired by Lunr.js and Elasticsearch. It offers more features than Lunr.js, such as support for multiple languages and more advanced search capabilities. However, it may still not match the performance and flexibility of FlexSearch.
Search-Index is a powerful, full-text search library for Node.js and the browser. It provides a wide range of features, including real-time indexing, faceted search, and more. While it offers more advanced features than FlexSearch, it may be more complex to set up and use.
Searching full text with FlexSearch is up to 1,000 times faster than with ElasticSearch. Benchmark: https://jsperf.com/flexsearch
All Features:
npm install flexsearch
In your code include as follows:
var FlexSearch = require("FlexSearch");
<html>
<head>
<script src="https://cdn.rawgit.com/nextapps-de/flexsearch/master/flexsearch.min.js"></script>
</head>
...
AMD
var FlexSearch = require("FlexSearch");
var index = new FlexSearch();
alternatively you can also use:
var index = FlexSearch.create();
FlexSearch.create(options)
var index = new FlexSearch({
// default values:
type: "integer",
encode: "icase",
boolean: "and",
size: 4000,
depth: 3,
multi: false,
strict: false,
ordered: false,
async: false,
cache: false
});
Read more: Phonetic Search, Phonetic Comparison, Improve Memory Usage
Index.add_(id, string)
index.add(10025, "John Doe");
Index.search(string, limit, callback)
index.search("John");
index.search("John", 10);
index.search("John", function(result){
// array of results
});
Index.update(id, string)
index.update(10025, "Road Runner");
Index.remove(id)
index.remove(10025);
index.destroy();
Index.init(options)
index.init();
Index.addMatcher(KEY_VALUE_PAIRS)
index.addMatcher({
'ä': 'a', // replaces all 'ä' to 'a'
'ö': 'o',
'Ü': 'u'
});
var index = new FlexSearch({
encode: function(str){
// do something with str ...
return str;
}
});
index.info();
Returns information about the index, e.g.:
{
"bytes": 3600356288,
"chunks": 9,
"fragmentation": 0,
"fragments": 0,
"id": 0,
"length": 7798,
"matchers": 0,
"size": 10000,
"status": false
}
Note: When the fragmentation value is about 50% or higher, your should consider using cleanup() to free all fragmented available memory.
index.cleanup();
| Option | Values | Description |
|---|---|---|
| type |
"byte" "short" "integer" "float" "string" | The data type of passed IDs has to be specified on creation. It is recommended to uses to most lowest possible data range here, e.g. use "short" when IDs are not higher than 65,535. |
| encode |
false "icase" "simple" "advanced" "extra" {function} | The encoding type. Choose one of the built-ins or pass a custom encoding function. |
| boolean |
"and" "or" | The applied boolean model when comparing multiple words. Note: When using "or" the first word is also compared with "and". Example: a query with 3 words, results has either: matched word 1 & 2 and matched word 1 & 3. |
| size | 2500 - 10000 | The size of chunks. It depends on content length which value fits best. Short content length (e.g. User names) are faster with a chunk size of 2,500. Bigger text runs faster with a chunk size of 10,000. Note: It is recommended to use a minimum chunk size of the maximum content length which has to be indexed to prevent fragmentation. |
| depth | 0 - 6 | Set the depth of register. It is recommended to use a value in relation to the number of stored index and content length for an optimum performance-memory value. Note: Increase this options carefully! |
| multi |
true false | Enable multi word processing. |
| ordered |
true false | Multiple words has to be the same order as the matched entry. |
| strict |
true false | Matches exactly needs to be started with the query. |
| cache |
true false | Enable caching. |
| Option | Description | Example | False Positives | Compression Level |
|---|---|---|---|---|
| false | Turn off encoding |
Reference: "Björn-Phillipp Mayer" Matches: "Phil" | no | no |
| icase | Case in-sensitive encoding |
Reference: "Björn-Phillipp Mayer" Matches: "phil" | no | no |
| simple | Phonetic normalizations |
Reference: "Björn-Phillipp Mayer" Matches: "bjoern fillip" | no | ~ 3% |
| advanced | Phonetic normalizations + Literal transformations |
Reference: "Björn-Phillipp Mayer" Matches: "filip meier" | no | ~ 25% |
| extra | Phonetic normalizations + Soundex transformations |
Reference: "Björn-Phillipp Mayer" Matches: "byorn mair" | yes | ~ 50% |
Reference String: "Björn-Phillipp Mayer"
| Query | ElasticSearch | FlexSearch (iCase) | FlexSearch (Simple) | FlexSearch (Adv.) | FlexSearch (Extra) |
|---|---|---|---|---|---|
| björn | yes | yes | yes | yes | yes |
| björ | no | yes | yes | yes | yes |
| bjorn | no | no | yes | yes | yes |
| bjoern | no | no | no | yes | yes |
| philipp | no | no | no | yes | yes |
| filip | no | no | no | yes | yes |
| björnphillip | no | no | yes | yes | yes |
| meier | no | no | no | yes | yes |
| björn meier | no | no | no | yes | yes |
| meier fhilip | no | no | no | yes | yes |
| byorn mair | no | no | no | no | yes |
| (false positives) | yes | no | no | no | yes |
Note: The data type of passed IDs has to be specified on creation. It is recommended to uses to most lowest possible data range here, e.g. use "short" when IDs are not higher than 65,535.
| ID Type | Range of Values | Memory usage of every ~ 100,000 indexed words (Index + Content) |
|---|---|---|
| Byte | 0 - 255 | 683 kb + 2.4 Mb |
| Short | 0 - 65,535 | 1.3 Mb + 2.4 Mb |
| Integer | 0 - 4,294,967,295 | 2.7 Mb + 2.4 Mb |
| Float | 0 - * (16 digits) | 5.3 Mb + 2.4 Mb |
| String | * (unlimited) | 1.3 Mb * char length of IDs + 2.4 Mb |
The required RAM per instance can be calculated as follow:
BYTES = CONTENT_CHAR_COUNT * (BYTES_OF_ID + 2) + CONTENT_WORD_COUNT * 8
Note: Character count may be less related to the phonetic settings, e.g. when using "extra" the count of chars shrinks to ~ 50%.
Author FlexSearch: Thomas Wilkerling
License: Apache 2.0 License
FAQs
Next-Generation full text search library with zero dependencies.
The npm package flexsearch receives a total of 316,818 weekly downloads. As such, flexsearch popularity was classified as popular.
We found that flexsearch demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Fluent Assertions is facing backlash after dropping the Apache license for a commercial model, leaving users blindsided and questioning contributor rights.
Research
Security News
Socket researchers uncover the risks of a malicious Python package targeting Discord developers.
Security News
The UK is proposing a bold ban on ransomware payments by public entities to disrupt cybercrime, protect critical services, and lead global cybersecurity efforts.