@backstage/plugin-search-backend-node - npm Package Compare versions

Comparing version 0.0.0-nightly-20216821837 to 0.0.0-nightly-20217421937

CHANGELOG.md

 # @backstage/plugin-search-backend-node
 
-## 0.0.0-nightly-20216821837
+## 0.0.0-nightly-20217421937
 
 ### Patch Changes
 
+- d9c13d535: Implements configuration and indexing functionality for ElasticSearch search engine. Adds indexing, searching and default translator for ElasticSearch and modifies default backend example-app to use ES if it is configured.
+
+  ## Example configurations:
+
+  ### AWS
+
+  Using AWS hosted ElasticSearch the only configuration options needed is the URL to the ElasticSearch service. The implementation assumes
+  that environment variables for AWS access key id and secret access key are defined in accordance to the [default AWS credential chain.](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-credentials-node.html).
+
+  ```yaml
+  search:
+    elasticsearch:
+      provider: aws
+      node: https://my-backstage-search-asdfqwerty.eu-west-1.es.amazonaws.com
+  ```
+
+  ### Elastic.co
+
+  Elastic Cloud hosted ElasticSearch uses a Cloud ID to determine the instance of hosted ElasticSearch to connect to. Additionally, username and password needs to be provided either directly or using environment variables like defined in [Backstage documentation.](https://backstage.io/docs/conf/writing#includes-and-dynamic-data)
+
+  ```yaml
+  search:
+    elasticsearch:
+      provider: elastic
+      cloudId: backstage-elastic:asdfqwertyasdfqwertyasdfqwertyasdfqwerty==
+      auth:
+        username: elastic
+        password: changeme
+  ```
+
+  ### Others
+
+  Other ElasticSearch instances can be connected to by using standard ElasticSearch authentication methods and exposed URL, provided that the cluster supports that. The configuration options needed are the URL to the node and authentication information. Authentication can be handled by either providing username/password or and API key or a bearer token. In case both username/password combination and one of the tokens are provided, token takes precedence. For more information how to create an API key, see [Elastic documentation on API keys](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-api-key.html) and how to create a bearer token, see [Elastic documentation on tokens.](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-create-service-token.html)
+
+  #### Configuration examples
+
+  ##### With username and password
+
+  ```yaml
+  search:
+    elasticsearch:
+      node: http://localhost:9200
+      auth:
+        username: elastic
+        password: changeme
+  ```
+
+  ##### With bearer token
+
+  ```yaml
+  search:
+    elasticsearch:
+      node: http://localhost:9200
+      auth:
+        bearer: token
+  ```
+
+  ##### With API key
+
+  ```yaml
+  search:
+    elasticsearch:
+      node: http://localhost:9200
+      auth:
+        apiKey: base64EncodedKey
+  ```
+
+- Updated dependencies
+  - @backstage/search-common@0.0.0-nightly-20217421937
+
+## 0.4.0
+
+### Minor Changes
+
+- 97b2eb37b: Change return value of `SearchEngine.index` to `Promise<void>` to support
+  implementation of external search engines.
+
+## 0.3.0
+
+### Minor Changes
+
+- 9f3ecb555: Build search queries using the query builder in `LunrSearchEngine`. This removes
+  the support for specifying custom queries with the lunr query syntax, but makes
+  sure that inputs are properly escaped. Supporting the full lunr syntax is still
+  possible by setting a custom query translator.
+  The interface of `LunrSearchEngine.setTranslator()` is changed to support
+  building lunr queries.
+
+### Patch Changes
+
+- 9f3ecb555: Enhance the search results of `LunrSearchEngine` to support a more natural
+  search experience. This is done by allowing typos (by using fuzzy search) and
+  supporting typeahead search (using wildcard queries to match incomplete words).
 - 4176a60e5: Change search scheduler from starting indexing in a fixed interval (for example
@@ -8,0 +101,0 @@ every 60 seconds), to wait a fixed time between index runs.

dist/index.cjs.js

@@ -68,3 +68,3 @@ 'use strict';
         }
-        this.searchEngine.index(type, documents);
+        await this.searchEngine.index(type, documents);
       }, this.collators[type].refreshInterval * 1e3);
@@ -138,25 +138,42 @@ });
     }) => {
-      let lunrQueryFilters;
-      const lunrTerm = term ? `+${term}` : "";
-      if (filters) {
-        lunrQueryFilters = Object.entries(filters).map(([field, value]) => {
-          if (["string", "number", "boolean"].includes(typeof value)) {
-            if (typeof value === "string") {
-              return ` +${field}:${value.replace(":", "\\:")}`;
-            }
-            return ` +${field}:${value}`;
+      return {
+        lunrQueryBuilder: (q) => {
+          const termToken = lunr__default['default'].tokenizer(term);
+          q.term(termToken, {
+            usePipeline: true,
+            boost: 100
+          });
+          q.term(termToken, {
+            usePipeline: false,
+            boost: 10,
+            wildcard: lunr__default['default'].Query.wildcard.TRAILING
+          });
+          q.term(termToken, {
+            usePipeline: false,
+            editDistance: 2,
+            boost: 1
+          });
+          if (filters) {
+            Object.entries(filters).forEach(([field, value]) => {
+              if (!q.allFields.includes(field)) {
+                throw new Error(`unrecognised field ${field}`);
+              }
+              if (["string", "number", "boolean"].includes(typeof value)) {
+                q.term(lunr__default['default'].tokenizer(value == null ? void 0 : value.toString()), {
+                  presence: lunr__default['default'].Query.presence.REQUIRED,
+                  fields: [field]
+                });
+              } else if (Array.isArray(value)) {
+                this.logger.warn(`Non-scalar filter value used for field ${field}. Consider using a different Search Engine for better results.`);
+                q.term(lunr__default['default'].tokenizer(value), {
+                  presence: lunr__default['default'].Query.presence.OPTIONAL,
+                  fields: [field]
+                });
+              } else {
+                this.logger.warn(`Unknown filter type used on field ${field}`);
+              }
+            });
           }
-          if (Array.isArray(value)) {
-            this.logger.warn(`Non-scalar filter value used for field ${field}. Consider using a different Search Engine for better results.`);
-            return ` ${value.map((v) => {
-              return `${field}:${v}`;
-            })}`;
-          }
-          this.logger.warn(`Unknown filter type used on field ${field}`);
-          return "";
-        }).join("");
-      }
-      return {
-        lunrQueryString: `${lunrTerm}${lunrQueryFilters || ""}`,
-        documentTypes: types || ["*"]
+        },
+        documentTypes: types
       };
@@ -170,3 +187,3 @@ };
   }
-  index(type, documents) {
+  async index(type, documents) {
     const lunrBuilder = new lunr__default['default'].Builder();
@@ -185,34 +202,20 @@ lunrBuilder.pipeline.add(lunr__default['default'].trimmer, lunr__default['default'].stopWordFilter, lunr__default['default'].stemmer);
   }
-  query(query) {
-    const {lunrQueryString, documentTypes} = this.translator(query);
+  async query(query) {
+    const {lunrQueryBuilder, documentTypes} = this.translator(query);
     const results = [];
-    if (documentTypes.length === 1 && documentTypes[0] === "*") {
-      Object.keys(this.lunrIndices).forEach((type) => {
-        try {
-          results.push(...this.lunrIndices[type].search(lunrQueryString).map((result) => {
-            return {
-              result,
-              type
-            };
-          }));
-        } catch (err) {
-          if (err instanceof lunr__default['default'].QueryParseError && err.message.startsWith("unrecognised field"))
-            return;
+    Object.keys(this.lunrIndices).filter((type) => !documentTypes || documentTypes.includes(type)).forEach((type) => {
+      try {
+        results.push(...this.lunrIndices[type].query(lunrQueryBuilder).map((result) => {
+          return {
+            result,
+            type
+          };
+        }));
+      } catch (err) {
+        if (err instanceof Error && err.message.startsWith("unrecognised field")) {
+          return;
         }
-      });
-    } else {
-      Object.keys(this.lunrIndices).filter((type) => documentTypes.includes(type)).forEach((type) => {
-        try {
-          results.push(...this.lunrIndices[type].search(lunrQueryString).map((result) => {
-            return {
-              result,
-              type
-            };
-          }));
-        } catch (err) {
-          if (err instanceof lunr__default['default'].QueryParseError && err.message.startsWith("unrecognised field"))
-            return;
-        }
-      });
-    }
+        throw err;
+      }
+    });
     results.sort((doc1, doc2) => {
@@ -226,3 +229,3 @@ return doc2.result.score - doc1.result.score;
     };
-    return Promise.resolve(realResultSet);
+    return realResultSet;
   }
@@ -229,0 +232,0 @@ }

dist/index.d.ts

@@ -0,3 +1,4 @@
+import { DocumentCollator, DocumentDecorator, SearchEngine, IndexableDocument, QueryTranslator, SearchQuery, SearchResultSet } from '@backstage/search-common';
+export { SearchEngine } from '@backstage/search-common';
 import { Logger } from 'winston';
-import { IndexableDocument, SearchQuery, SearchResultSet, DocumentCollator, DocumentDecorator } from '@backstage/search-common';
 import lunr from 'lunr';
@@ -27,26 +28,2 @@
 }
-/**
- * A type of function responsible for translating an abstract search query into
- * a concrete query relevant to a particular search engine.
- */
-declare type QueryTranslator = (query: SearchQuery) => unknown;
-/**
- * Interface that must be implemented by specific search engines, responsible
- * for performing indexing and querying and translating abstract queries into
- * concrete, search engine-specific queries.
- */
-interface SearchEngine {
-    /**
-     * Override the default translator provided by the SearchEngine.
-     */
-    setTranslator(translator: QueryTranslator): void;
-    /**
-     * Add the given documents to the SearchEngine index of the given type.
-     */
-    index(type: string, documents: IndexableDocument[]): void;
-    /**
-     * Perform a search query against the SearchEngine.
-     */
-    query(query: SearchQuery): Promise<SearchResultSet>;
-}
 
@@ -111,4 +88,4 @@ declare type IndexBuilderOptions = {
 declare type ConcreteLunrQuery = {
-    lunrQueryString: string;
-    documentTypes: string[];
+    lunrQueryBuilder: lunr.Index.QueryBuilder;
+    documentTypes?: string[];
 };
@@ -125,6 +102,6 @@ declare type LunrQueryTranslator = (query: SearchQuery) => ConcreteLunrQuery;
     setTranslator(translator: LunrQueryTranslator): void;
-    index(type: string, documents: IndexableDocument[]): void;
+    index(type: string, documents: IndexableDocument[]): Promise<void>;
     query(query: SearchQuery): Promise<SearchResultSet>;
 }
 
-export { IndexBuilder, LunrSearchEngine, Scheduler, SearchEngine };
+export { IndexBuilder, LunrSearchEngine, Scheduler };

package.json

 {
   "name": "@backstage/plugin-search-backend-node",
-  "version": "0.0.0-nightly-20216821837",
+  "version": "0.0.0-nightly-20217421937",
   "main": "dist/index.cjs.js",
@@ -22,3 +22,3 @@ "types": "dist/index.d.ts",
   "dependencies": {
-    "@backstage/search-common": "^0.1.2",
+    "@backstage/search-common": "^0.0.0-nightly-20217421937",
     "winston": "^3.2.1",
@@ -29,4 +29,4 @@ "lunr": "^2.3.9",
   "devDependencies": {
-    "@backstage/backend-common": "^0.0.0-nightly-20216821837",
-    "@backstage/cli": "^0.7.2"
+    "@backstage/backend-common": "^0.0.0-nightly-20217421937",
+    "@backstage/cli": "^0.0.0-nightly-20217421937"
   },
@@ -33,0 +33,0 @@ "files": [

New alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

Fixed alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

Improved metrics

Total package byte prevSize

39615

increased by9.07%

Worsened metrics

Lines of code

322

decreased by-5.85%

Dependency changes

• + Added@backstage/search-common@0.0.0-nightly-20220923026(transitive)

• + Added@backstage/types@0.0.0-nightly-20230919021144(transitive)

• - Removed@backstage/config@0.1.15(transitive)

• - Removed@backstage/search-common@0.1.3(transitive)

• - Removed@backstage/types@0.1.3(transitive)

• - Removedlodash@4.17.21(transitive)

• Updated@backstage/search-common@^0.0.0-nightly-20217421937