owc-elasticsearch

owc-elasticsearch

An ORM of elasticsearch indexes with built in query body builder to allow you manipulate data in elasticsearch like a relational db.

1. Installation

npm i --save owc-elasticsearch

2. Usage

2.1 require package:

const elasticsearch = require("owc-elasticsearch");

2.2 Initialize ElasticClient

let config = {
    hosts: ["127.0.0.1:9200"],
    apiVersion: "5.6",
    requestTimeout: 30000
};
elasticsearch.ElasticClient.init(config);

• For available config options, please refer to: https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/16.x/configuration.html#config-options

2.3 Create a new model inherited from elasticsearch.ElasticSearch

const es = require("owc-elasticsearch");

class User extends es.ElasticSearch {
    constructor() {
        super();
    }

    /**
     * type
     * @return {string}
     */
    static get type() {
        return "user";
    }

    /**
     * index
     * @return {string}
     */
    static get index() {
        return "share";
    }

    /**
     * Mapping information of the document share-user
     */
    static get mapping()
    {
        let properties = {
            id : {
                type : "integer",
            },
            tenant_id : {
                type : "integer",
            },
            firstname : {
                type : 'completion',
                analyzer : 'english'
            },
            lastname : {
                type : "text",
            },
            midname : {
                type : "text",
            },
            email : {
                type : "text",
            },
            status : {
                type : "integer",
            },
            advertiser_id : {
                type : "integer",
            },
            agency_id : {
                type : "integer",
            },
            avatar_id : {
                type : "text",
            },
            title : {
                type : "text",
            },
            last_login : {
                type : "long",
            },
            default : {
                type : "integer",
            },
            createdAt : {
                type : "long",
            },
            updatedAt : {
                type : "long",
            },
            token : {
                type : "text",
            },
            token_generated_time : {
                type : "long",
            },
            username : {
                type : "text",
                analyzer : 'english',
                search_analyzer : 'english',
                fields : {
                    raw : {
                        type : "keyword"
                    },
                    suggest : {
                        type : "completion",
                        analyzer : "simple",
                        search_analyzer: "simple"
                    }
                }
            }
        };
        let ret = {};
        ret[this.type] = { properties : properties};
        return ret;
    }

    /**
     * returns fields of primary keys
     * @returns {string[]}
     */
    static get primaryKeys() {
        return ["id"];
    }

    /**
     * returns elastic client
     * @return {es.Client|Client}
     */
    static get client() {
        return ElasticClient.client;
    }

    /**
     * Save user into elasticsearch
     * @static
     * @param data json data
     * @param refresh whether the change should be visible to search
     * @return {Promise<any>}
     * @example
     * let tmp = {
     *   id: 1,
     *   tenant_id : 1,
     *   firstname: 'first',
     *   lastname: 'last',
     *   email: "[e-mail]",
     *   status : 1,
     *   default : 1,
     *   username : "owc"
     * }
     * User.saveUser(tmp, true);
     */
    static saveUser(data, refresh = false) {
        let obj = this._setAttributes(data);
        return obj.save(refresh);
    }
}
module.exports = User;

• all the methods in this model are mandatory and must be implemented in order to use this package

3. create index

await User.createIndex(); // return {acknowledged : true}

4. Check whether index exists

await User.indexExists(); // return true or false

5. Save document to elasticsearch

• Please refer to method saveUser for details.
• saveUser automatically add records to elasticsearch or update existing data.

6. Check whether the user exists

await User.exists(1)

7. Get user by id

await User.find().filter({id:1}).one();

8. get all users

await User.find().all();

9. Get all users page by page

await User.find().offset(0).limit(5).all();

10. Filter users that have status 0 and 1

await User.find().filter({ status : [0, 1] }).all();

11. Filter users that have status 0 and 1 and default equals to 2

await User.find().filter({status : [0,1], default : 2}).all();

12. Filter users that have status in (0, 1) or default equals to 2

await User.find().filter({default: 2}).orFilter({status: [0,1]}).all();

13. Find users that have (status in (0, 1) or default equals to 2) and username has keyword 'test'

await User.find().filter({default: 2}).orFilter({status: [0,1]}).query({match : {"username" : 'test'}}).all()

14. Find users that have status in (0, 1), default equals to 2 and username has keyword 'test'

await User.find().filter({default: 2}).andFilter({status: [0,1]}).query({match : {"username" : 'test'}}).all();

15. Find users whose status between 0 and 2 and default equals 2

await User.find().range("status", {gte : 0, lte : 2}).andFilter({default : 2}).all()    

16. Find users whose username has keyword 'test' or 'owc'

await User.find().query([{match : {"username" : 'test'}} ]).orQuery([{match : {"username" : 'owc'}}]).all();

17. Find all users whose username has keyword 'test' and 'search'

await User.find().query([{match : {"username" : 'test'}}, {match : {"username" : 'search'}} ]).all();

18. Find users whose username contains 'sear'

await User.suggest(".*sear.*", "username.suggest", true);

19. Find users whose username contains 'tast' with fuzziness 2 and only select id, username

let options = {
    "fuzzy" : {
        "fuzziness" : 2
    }
}
await User.suggest("tast", "username.suggest", false, ["id", "username"], 5, options);

• "tast" is the keyword to search
• "username.suggest" is the field to search
• "false" indicates it's not a regex search
• ["id", "username"] means only return these two fields.
• "5" means return top 5 suggestions
• options means fuzziness is two

20. Find users whose username has keyword 'test' or 'owc' and doesn't have keyword 'search'

await User.find().query([{match : {"username" : 'test'}} ]).orQuery([{match : {"username" : 'owc'}}]).mustNotQuery([{match : {"username" : 'search'}}]).all();

21. Find users whose username has keyword 'test' or 'owc', lastname has keyword 'second' and username doesn't have keyword 'search'

await User.find().query([{match : {"username" : 'test'}} ]).orQuery([{match : {"username" : 'owc'}}]).andQuery({match: {"lastname": "second"}}).mustNotQuery([{match : {"username" : 'search'}}]).all();

22. Find users whose username has keyword 'test' or 'owc', lastname has keyword 'second' and username doesn't have keyword 'search'

await User.find().query([{match : {"username" : 'test'}} ]).orQuery([{match : {"username" : 'owc'}}]).andQuery({match: {"lastname": "second"}}).mustNotQuery([{match : {"username" : 'search'}}]).all();

23. Find users whose username contains "search fu"

await User.find().query([{match_phrase_prefix : {"username" : 'search fu'}}]).all();

24. Delete user whose id is 1

await User.delete(1);

25. Delete user whose username contains "search"

await User.find().query({match:{username: 'search'}}).deleteAll();

26. Delete index

await User.deleteIndex();

Your own methods

If you can't find the methods you need, you can simply create a new model that inherits from elasticsearch.ElasticSearch. And then you create all rest models that inherit from the model you created.

Test

npm test

Creating models from mysql tables

You can try to install package "owc" and "owc-cli".

• owc-cli can create elasticsearch models from mysql tables automacitally
• owc is node.js MVC framework that can build aws lambda web services much easier.
• Please refer to below websites for details: https://www.npmjs.com/package/owc https://www.npmjs.com/package/owc-cli