elasticsearch-odm-5
Advanced tools
Like Mongoose but for Elasticsearch 5.x. Define models, preform CRUD operations, and build advanced search queries.
Like Mongoose but for Elasticsearch. Define models, preform CRUD operations, and build advanced search queries. Most commands and functionality that exist in Mongoose exist in this library. All asynchronous functions use Bluebird Promises instead of callbacks.
This is currently the only ODM/ORM library that exists for Elasticsearch on Node.js. Waterline has a plugin for Elasticsearch but it is incomplete and doesn't exactly harness it's searching power. Loopback has a storage plugin, but it also doesn't focus on important parts of Elasticsearch, such as mappings and efficient queries. This library automatically handles merging and updating Elasticsearch mappings based on your schema definition.
If you currently have npm elasticsearch installed, you can remove it and access it from client in this library if you still need it.
$ npm install elasticsearch-odm-5
You'll find the API is intuitive if you've used Mongoose or Waterline.
Example (no schema):
let esodm = require('elasticsearch-odm-5');
let Car = esodm.model('Car');
let car = new Car({
type: 'Ford', color: 'Black'
});
esodm.connect('my-index').then(function(){
// be sure to call connect before bootstrapping your app.
car.save().then(function(document){
console.log(document);
});
});
Example (using a schema):
let esodm = require('elasticsearch-odm-5');
let carSchema = new esodm.Schema({
type: String,
color: {type: String, required: true}
});
let Car = esodm.model('Car', carSchema);
.count().create(Object data).update(String id, Object data).remove(String id).removeByIds(Array ids).set(String id).find(Object/String match, Object queryOptions).findById(String id, Object queryOptions).findByIds(Array ids, Object queryOptions).findOne(Object/String match, Object queryOptions).findAndRemove(Object/String match, Object queryOptions).findOneAndRemove(Object/String match, Object queryOptions).makeInstance(Object data).toMapping()Core methods can be called directly on the Elasticsearch ODM instance. These include methods to configure, connect, and get information from your Elasticsearch database. Most methods act upon the official Elasticsearch client.
.connect(String/Object options) -> PromiseReturns a promise that is resolved when the connection is complete. Can be passed a single index name, or a full configuration object. The default host is localhost:9200 when no host is provided, or just an index name is used. This method should be called at the start of your application.
If the index name does not exist, it is automatically created for you.
You can also add any of the Elasticsearch specific options, like SSL configs.
Example:
// when bootstrapping your application
let esodm = require('elasticsearch-odm-5');
esodm.connect({
host: 'localhost:9200',
index: 'my-index',
logging: false, // true by default when NODE_ENV=development
trace: true, // true for elasticsearch default trace logs
ssl: {
ca: fs.readFileSync('./cacert.pem'),
rejectUnauthorized: true
},
options : { // optional index settings & mappings. Override default mappings
settings: {
index: {
number_of_shards: 1,
number_of_replicas: 0
}
}
}
});
// OR
esodm.connect('my-index'); // default host localhost:9200
.disconnect() -> PromiseReturns a promise that is resolved when the disconnection is complete. This method should be called to close elasticsearch connection.
Example:
// when bootstrapping your application
let esodm = require('elasticsearch-odm-5');
esodm.connect('my-index')
.then(function(){
// ... code here
})
.then(esodm.disconnect)
.then(function(){
console.log('disconnected');
});
new Schema(Object options) -> SchemaReturns a new schema definition to be used for models.
.model(String modelName, Optional/Schema schema) -> ModelCreates and returns a new Model, like calling Mongoose.model(). Takes a type name, in mongodb this is also known as the collection name. This is global function and adds the model to Elasticsearch ODM instance.
.client -> ElasticsearchThe raw instance to the underlying Elasticsearch client. Not really needed, but it's there if you need it, for example to run queries that aren't provided by this library.
.stats()Returns a promise that is resolved with index stats for the current Elasticsearch connections.
.removeIndex(String index)Takes an index name, and complete destroys the index. Resolves the promise when it's complete.
.createIndex(String index, Object mappings)Takes an index name, and a json string or object representing your mapping. Resolves the promise when it's complete.
Like Mongoose, instances of models are considered documents, and are returned from calls like find() & create(). Documents include the following functions to make working with them easier.
.save() -> DocumentSaves or updates the document. If it doesn't exist it is created. Like Mongoose, Elasticsearches internal '_id' is copied to 'id' for you. If you'd like to force a custom id, you can set the id property to something before calling save(). Every document gets a createdOn and updatedOn property set with ISO-8601 formatted time.
Note : In order to access document just after insertion you must add {refresh: true} as save() parameter. See index.refresh_interval. Force refresh has a negative impact on elasticsearch, but depending on your use it can be mandatory.
Example :
let esodm = require('elasticsearch-odm-5');
let Car = esodm.model('Car');
let car = new Car({
type: 'Ford', color: 'Black'
});
esodm.connect('my-index').then(function(){
// be sure to call connect before bootstrapping your app.
car
.save({refresh: true}) // Save document and request Elasticsearch to refresh shard
.then(function(document){
console.log(document);
});
});
.remove()Removes the document and destroys the cuurrent document instance. No value is resolved, and missing documents are ignored.
.update(Object data) -> DocumentPartially updates the document. Data passed will be merged with the document, and the updated version will be returned. This also sets the current model instance with the new document.
.set(Object data) -> DocumentCompletely overwrites the document with the data passed, and returns the new document. This also sets the current model instance with the new document.
Will remove any fields in the document that aren't passed.
.toObject()Like Mongoose, strips all non-document properties from the instance and returns a raw object.
Model definitions returned from .model() in core include several static functions to help query and manage documents. Most functions are similar to Mongoose, but due to the differences in Elasticsearch, querying includes some extra advanced features.
.count() -> ObjectObject returned includes a 'count' property with the number of documents for this Model (also known as _type in Elasticsearch). See Elasticsearch count.
.create(Object data) -> DocumentA helper function. Similar to calling new Model(data).save(). Takes an object, and returns the new document.
.update(String id, Object data) -> DocumentA helper function. Similar to calling new Model().update(data). Takes an id and a partial object to update the document with.
.remove(String id)Removes the document by it's id. No value is resolved, and missing documents are ignored.
.removeByIds(Array ids)Help function, see remove. Takes an array of ids.
.set(String id, Object data) -> DocumentCompletely overwrites the document matching the id with the data passed, and returns the new document.
Will remove any fields in the document that aren't passed.
.find(Object/String match, Object queryOptions) -> DocumentThere are four ways to call .find() and it's siblings. You can mix and match styles.
.find({name:'Joe'}).find('some string').find({}, {must: {active: true, sort: 'createdOn'}}}.find({}).must({active: true}).sort('createdOn').then(..)note : current version support up to 10000 results due to elasticsearch default limitations.
Unlike mongoose, finding exact matches requires the fields in your mapping to be set to 'not_analyzed'. By default {index: not_analyzed} is added to all string fields in your Schema unless you override it.
Depending on the analyzer in your mapping, find queries like must, not, and matches may not find any results.
match => Optional. An alias for the 'must' Query Option. Like Mongoose this matches name/value in documents. Also, instead of an object, just a string can be passed which will match against all document fields using the power of an Elasticsearch QueryStringQuery.
queryOptions => Optional (can also use chaining instead). An object with Query Options. Here you can specifiy paging, filtering, sorting and other advanced options. See here for more details. You can set the first argument to null, and only use filters from the query options if you wanted.
returns => Found documents, or null if nothing was found.
Example:
let Car = esodm.model('Car');
// Simple query.
Car.find({color: 'blue'}).then(function(results){
console.log(results);
});
// Nested query (for nested documents/properties).
Car.find({'location.city': 'New York'})
// Find all by passing null or empty object to first argument
Car.find(null, {sort: 'createdOn'})
// Search all fields using a QueryStringQuery.
Car.find('some text')
// Chained query without using Query Options.
// Instead of Mongoose .exec(), we call .then()
Car.find()
.must({color: 'blue'})
.exists('owner')
.sort('createdOn')
.then(...)
.findById(String id, Object queryOptions) -> DocumentFinds a document by id. 'fields' argument is optional and specifies the fields of the document you'd like to include.
.findByIds(Array ids, Object queryOptions) -> DocumentSame as .findById() but for multiple documents.
.findOne(Object/String match, Object queryOptions) -> DocumentSame arguments as .find(). Returns the first matching document.
.findAndRemove(Object/String match, Object queryOptions) -> 'Object'Same arguments as .find(). Removes all matching documents and returns their raw objects.
.findOneAndRemove(Object/String match, Object queryOptions) -> 'Object'Same arguments as .findAndRemove(). Removes the first found document.
.makeInstance(Object data) -> DocumentHelper function. Takes a raw object and creates a document instance out of it. The object would need at least an id property. The document returned can be used normally as if it were returned from other calls like .find().
.toMapping()Returns a complete Elasticsearch mapping for this model based off it's schema. If no schema was used, it returns nothing. Used internally, but it's there if you'd like it.
The query options object includes several options that are normally included in mongoose chained queries, like sort, and paging (skip/limit), and also some advanced features from Elasticsearch. The Elasticsearch Query and Filter DSL is generated using best practices.
Type: Integer
For most use cases, paging is better suited than skip/limit, so this library includes thhis instead. Page 0/1 are the same thing, so either can be used. Page and per_page both use default when the other is set, page defaults to the first, and per_page defaults to 10.
Including page or per_page will result in the response being wrapped in a meta data object like the following. You can call toJSON and toObject on this response and it'll call that method on all document instances under the hits property.
// A paged response that is returned when page or per_page is set.
{
total: 0, // total documents found for the query.
hits: [], // a collection of document instances.
page: 0, // current page requested.
pages: 0 // total number of pages.
}
Type: Array or String
A list of fields to include in the documents returned. For example, you could pass 'id' to only return the matching document id's. See Elasticsearch Fields.
// Query Options.
{
fields: ['name', 'age']
}
// Chained Query.
.find()
.fields(['name', 'age'])
.then(...)
Type: Array or String
A list of fields to sort on. If multiple fields are passed then they are executed in order. Adding a '-' sign to the start of the field name makes it sort descending. Default is ascending. See Elasticsearch Sort.
Example:
// Query Options.
{
sort: ['name', 'createdOn']
}
// Chained Query.
.find()
.sort(['name', 'createdOn'])
.then(...)
Type: String
A string to search all document fields with using Elasticsearch QueryStringQuery. This can be expensive, so use it sparingly.
Example:
// Query Options.
{
q: 'Red dog run'
}
// Chained Query.
.find('Red dog run')
.then(...)
Type: Object
Key value pairs to match documents against. Essentially it's the same as first argument passed to Mongoose .find(). This is also an alias to the first argument passed to .find() in this library. This is a 'must' Bool Filter.
Elasticsearches internal Tokenizers are used, and fields are analyzed.
You can query nested fields using dot notation.
Example:
// Query Options.
{
must: {
name: 'Jim',
'location.country': 'Canada'
}
}
// Chained Query.
.find()
.must({name: 'Jim', 'location.country': 'Canada'})
.then(...)
Type: Object
The same as must, but matches documents where the key value pairs DON'T match. This is a 'must_not' Bool Filter query.
You can query nested fields using dot notation.
Example:
// Query Options.
{
not: {
name: 'Jim',
'location.country': 'Canada'
}
}
// Chained Query.
.find()
.not({name: 'Jim', 'location.country': 'Canada'})
.then(...)
Type: Array or String
A single field name, or array of field names. Matches documents where these field names are missing. A field is considered mising, when it is null, empty, or does not exist. See [MissingFilter] (https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-missing-filter.html).
Example:
// Query Options.
{
missing: ['description', 'name']
}
// Chained Query.
.find()
.missing(['description', 'name'])
.then(...)
Type: Array or String
A single field name, or array of field names. Matches documents where these field names exists. The opposite of missing.
Example:
// Query Options.
{
exists: ['description', 'name']
}
// Chained Query.
.find()
.exists(['description', 'name'])
.then(...)
Models don't require schemas, but it's best to use them - especially if you'll be making search queries. Elasticsearch-odm will generate and update Elasticsearch with the proper mappings based off your schema definition. The schemas are similar to Mongoose, but several new field types have been added which Elasticsearch supports. These are; float, double, long, short, byte, binary, geo_point. Generally for numbers, only the Number type is needed (which converts to Elasticsearch integer). You can read more about Elasticsearch types here.
NOTE
"index": "not_analyzed". See Custom Field Mappings. This is so the .find() call acts like it does in Mongoose by only fidning exact matches, however, this prevents the ability to do full text search on this field. Simply set {"index":"analyzed"} if you'd like full text search instead.Example:
// Before saving a document with this schema, your Elasticsearch
// mappings will automatically be updated.
// Note the various ways you can define a schema field type.
let carSchema = new esodm.Schema({
// native type without options
available: Boolean,
// Elasticsearch type without options
safteyRating: 'float',
// native array type
parts: [String],
// Elasticsearch array type
oldPrices: {type: ['double']},
// with options
color: {type: String, required: true},
// a field named 'type' must be defined like the following.
type: {type: String},
// nested document
owner: {
name: String,
age: Number,
// force a required field
location: {type: 'geo_point', required: true}
},
// nested document array
inspections: [{
date: Date,
grade: Number
}],
// Enable full-text search of this field.
// NOTE: it's better to than use the 'q' paramater in queryOptions
// during searches instead of must/not or match when using 'analyzed'
description: {type:String, index: 'analyzed'}
// Ignore_malformed is an Elasticsearch Core Type field option for numbers
price: {type: 'double', ignore_malformed: true}
});
Schemas include pre and post hooks that function similar to Mongoose. Currently, there are pre/post hooks for 'save' and 'remove'.
Pre Hooks
Same conventions as Mongoose. Function takes a done() callback that must be called when your function is finished. this is scoped to the current document. assing an Error to done() will cancel the current operation. For example, in a pre 'save' hook, passing an error to done() will cause the document not to be saved and will return your error to the save() callers rejection handler.
let schema = new esodm.Schema(...);
schema.pre('save', function(done){
console.log(this); // this = the current document
done(); // OR done(new Error('bad document'));
});
Post Hooks
Same conventions as Mongoose. Does not have a done() callback. Executed after the hooked method. The first argument is the current document which may or may not be a document instance (eg. post remove only receives the raw object as the document no longer exists).
let schema = new esodm.Schema(...);
schema.post('remove', function(document){
console.log(document);
});
Add methods to your schema with the same convention as Mongoose.
// Instance method.
let schema = new esodm.Schema(...);
schema.methods.getFullName = function(){
return this.firstName + ' ' + this.lastName;
});
// Static method.
schema.statics.findByColor = function(color){
return this.find({color: color});
});
This is a library Elasticsearch desperately needed for Node.js. Currently the official npm elasticsearch client has about 23,000 downloads per week, many of them would benefit from this library instead. Pull requests are welcome. There are Mocha and benchmark tests in the root directory.
Elasticsearch 2.x :
FAQs
Like Mongoose but for Elasticsearch 5.x. Define models, preform CRUD operations, and build advanced search queries.
We found that elasticsearch-odm-5 demonstrated a not healthy version release cadence and project activity because the last version was released a year ago. It has 1 open source maintainer collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
Research
An advanced npm supply chain attack is leveraging Ethereum smart contracts for decentralized, persistent malware control, evading traditional defenses.
Security News
Research
Attackers are impersonating Sindre Sorhus on npm with a fake 'chalk-node' package containing a malicious backdoor to compromise developers' projects.
Security News
The npm package for the LottieFiles Player web component was hit with a supply chain attack after a software engineer's npmjs credentials were compromised.