@backstage/plugin-search-backend-node - npm Package Compare versions

Comparing version 0.0.0-nightly-20215923438 to 0.0.0-nightly-20216821837

CHANGELOG.md

 # @backstage/plugin-search-backend-node
 
-## 0.0.0-nightly-20215923438
+## 0.0.0-nightly-20216821837
 
 ### Patch Changes
 
+- 4176a60e5: Change search scheduler from starting indexing in a fixed interval (for example
+  every 60 seconds), to wait a fixed time between index runs.
+  This makes sure that no second index process for the same document type is
+  started when the previous one is still running.
+
+## 0.2.2
+
+### Patch Changes
+
+- 9c8ea7e24: Handle errors in collators and decorators and log them.
+- 7e7cec86a: Fixed bug preventing searches with filter values containing `:` from returning results.
+
+## 0.2.1
+
+### Patch Changes
+
+- 14aad6113: Improved the quality of free text searches in LunrSearchEngine.
+
+## 0.2.0
+
+### Minor Changes
+
+- 5aff84759: This release represents a move out of a pre-alpha phase of the Backstage Search
+  plugin, into an alpha phase. With this release, you gain more control over the
+  layout of your search page on the frontend, as well as the ability to extend
+  search on the backend to encompass everything Backstage users may want to find.
+
+  If you are updating to version `v0.4.0` of `@backstage/plugin-search` from a
+  prior release, you will need to make modifications to your app backend.
+
+  First, navigate to your backend package and install the two related search
+  backend packages:
+
+  ```sh
+  cd packages/backend
+  yarn add @backstage/plugin-search-backend @backstage/plugin-search-backend-node
+  ```
+
+  Wire up these new packages into your app backend by first creating a new
+  `search.ts` file at `src/plugins/search.ts` with contents like the following:
+
+  ```typescript
+  import { useHotCleanup } from '@backstage/backend-common';
+  import { createRouter } from '@backstage/plugin-search-backend';
+  import {
+    IndexBuilder,
+    LunrSearchEngine,
+  } from '@backstage/plugin-search-backend-node';
+  import { PluginEnvironment } from '../types';
+  import { DefaultCatalogCollator } from '@backstage/plugin-catalog-backend';
+
+  export default async function createPlugin({
+    logger,
+    discovery,
+  }: PluginEnvironment) {
+    // Initialize a connection to a search engine.
+    const searchEngine = new LunrSearchEngine({ logger });
+    const indexBuilder = new IndexBuilder({ logger, searchEngine });
+
+    // Collators are responsible for gathering documents known to plugins. This
+    // particular collator gathers entities from the software catalog.
+    indexBuilder.addCollator({
+      defaultRefreshIntervalSeconds: 600,
+      collator: new DefaultCatalogCollator({ discovery }),
+    });
+
+    // The scheduler controls when documents are gathered from collators and sent
+    // to the search engine for indexing.
+    const { scheduler } = await indexBuilder.build();
+
+    // A 3 second delay gives the backend server a chance to initialize before
+    // any collators are executed, which may attempt requests against the API.
+    setTimeout(() => scheduler.start(), 3000);
+    useHotCleanup(module, () => scheduler.stop());
+
+    return await createRouter({
+      engine: indexBuilder.getSearchEngine(),
+      logger,
+    });
+  }
+  ```
+
+  Then, ensure the search plugin you configured above is initialized by modifying
+  your backend's `index.ts` file in the following ways:
+
+  ```diff
+  +import search from './plugins/search';
+  // ...
+  +const searchEnv = useHotMemoize(module, () => createEnv('search'));
+  // ...
+  +apiRouter.use('/search', await search(searchEnv));
+  // ...
+  ```
+
+### Patch Changes
+
 - db1c8f93b: The `<Search...Next /> set of components exported by the Search Plugin are now updated to use the Search Backend API. These will be made available as the default non-"next" versions in a follow-up release.
@@ -12,3 +108,3 @@
 - Updated dependencies [db1c8f93b]
-  - @backstage/search-common@0.0.0-nightly-20215923438
+  - @backstage/search-common@0.1.2
 
@@ -15,0 +111,0 @@ ## 0.1.4

dist/index.cjs.js

@@ -48,6 +48,17 @@ 'use strict';
         this.logger.debug(`Collating documents for ${type} via ${this.collators[type].collate.constructor.name}`);
-        let documents = await this.collators[type].collate.execute();
+        let documents;
+        try {
+          documents = await this.collators[type].collate.execute();
+        } catch (e) {
+          this.logger.error(`Collating documents for ${type} via ${this.collators[type].collate.constructor.name} failed: ${e}`);
+          return;
+        }
         for (let i = 0; i < decorators.length; i++) {
           this.logger.debug(`Decorating ${type} documents via ${decorators[i].constructor.name}`);
-          documents = await decorators[i].execute(documents);
+          try {
+            documents = await decorators[i].execute(documents);
+          } catch (e) {
+            this.logger.error(`Decorating ${type} documents via ${decorators[i].constructor.name} failed: ${e}`);
+            return;
+          }
         }
@@ -67,5 +78,30 @@ if (!documents || documents.length === 0) {
 
+function runPeriodically(fn, delayMs) {
+  let cancel;
+  let cancelled = false;
+  const cancellationPromise = new Promise((resolve) => {
+    cancel = () => {
+      resolve();
+      cancelled = true;
+    };
+  });
+  const startRefresh = async () => {
+    while (!cancelled) {
+      try {
+        await fn();
+      } catch {
+      }
+      await Promise.race([
+        new Promise((resolve) => setTimeout(resolve, delayMs)),
+        cancellationPromise
+      ]);
+    }
+  };
+  startRefresh();
+  return cancel;
+}
+
 class Scheduler {
   constructor({logger}) {
-    this.intervalTimeouts = [];
+    this.runningTasks = [];
     this.logger = logger;
@@ -75,3 +111,3 @@ this.schedule = [];
   addToSchedule(task, interval) {
-    if (this.intervalTimeouts.length) {
+    if (this.runningTasks.length) {
       throw new Error("Cannot add task to schedule that has already been started.");
@@ -84,5 +120,3 @@ }
     this.schedule.forEach(({task, interval}) => {
-      this.intervalTimeouts.push(setInterval(() => {
-        task();
-      }, interval));
+      this.runningTasks.push(runPeriodically(() => task(), interval));
     });
@@ -92,6 +126,6 @@ }
     this.logger.info("Stopping all scheduled search tasks.");
-    this.intervalTimeouts.forEach((timeout) => {
-      clearInterval(timeout);
+    this.runningTasks.forEach((cancel) => {
+      cancel();
     });
-    this.intervalTimeouts = [];
+    this.runningTasks = [];
   }
@@ -113,2 +147,5 @@ }
           if (["string", "number", "boolean"].includes(typeof value)) {
+            if (typeof value === "string") {
+              return ` +${field}:${value.replace(":", "\\:")}`;
+            }
             return ` +${field}:${value}`;
@@ -139,2 +176,4 @@ }
     const lunrBuilder = new lunr__default['default'].Builder();
+    lunrBuilder.pipeline.add(lunr__default['default'].trimmer, lunr__default['default'].stopWordFilter, lunr__default['default'].stemmer);
+    lunrBuilder.searchPipeline.add(lunr__default['default'].stemmer);
     Object.keys(documents[0]).forEach((field) => {
@@ -141,0 +180,0 @@ lunrBuilder.field(field);

dist/index.d.ts

@@ -89,3 +89,3 @@ import { Logger } from 'winston';
     private schedule;
-    private intervalTimeouts;
+    private runningTasks;
     constructor({ logger }: {
@@ -95,4 +95,5 @@ logger: Logger;
     /**
-     * Adds each task and interval to the schedule
-     *
+     * Adds each task and interval to the schedule.
+     * When running the tasks, the scheduler waits at least for the time specified
+     * in the interval once the task was completed, before running it again.
      */
@@ -99,0 +100,0 @@ addToSchedule(task: Function, interval: number): void;

package.json

 {
   "name": "@backstage/plugin-search-backend-node",
-  "version": "0.0.0-nightly-20215923438",
+  "version": "0.0.0-nightly-20216821837",
   "main": "dist/index.cjs.js",
@@ -22,3 +22,3 @@ "types": "dist/index.d.ts",
   "dependencies": {
-    "@backstage/search-common": "^0.0.0-nightly-20215923438",
+    "@backstage/search-common": "^0.1.2",
     "winston": "^3.2.1",
@@ -29,4 +29,4 @@ "lunr": "^2.3.9",
   "devDependencies": {
-    "@backstage/backend-common": "^0.0.0-nightly-20215923438",
-    "@backstage/cli": "^0.0.0-nightly-20215923438"
+    "@backstage/backend-common": "^0.0.0-nightly-20216821837",
+    "@backstage/cli": "^0.7.2"
   },
@@ -33,0 +33,0 @@ "files": [

README.md

 # search-backend-node
 
 This plugin is part of a suite of plugins that comprise the Backstage search
-platform, which is still very much under development. This plugin specifically
-is responsible for:
+platform. This particular plugin is responsible for all aspects of the search
+indexing process, including:
 
-- Allowing other backend plugins to register the fact that they have documents
-  that they'd like to be indexed by a search engine (known as `collators`)
-- Allowing other backend plugins to register the fact that they have metadata
-  that they'd like to augment existing documents in the search index with
-  (known as `decorators`)
+- Providing connections to search engines where actual document indices live
+  and queries can be made.
+- Defining a mechanism for plugins to expose documents that they'd like to be
+  indexed (called `collators`).
+- Defining a mechanism for plugins to add extra metadata to documents that the
+  source plugin may not be aware of (known as `decorators`).
 - A scheduler that, at configurable intervals, compiles documents to be indexed
-  and passes them to a search engine for indexing
-- Types for all of the above
+  and passes them to a search engine for indexing.
+- A builder class to wire up all of the above.
+- Naturally, types for all of the above.
 
@@ -16,0 +18,0 @@ Documentation on how to develop and improve the search platform is currently

New alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

Fixed alerts

License Policy Violation

License

This package is not allowed per your license policy. Review the package's license to ensure compliance.

Found 1 instance in 1 package

Improved metrics

Total package byte prevSize

36322

increased by28.1%

Lines of code

342

increased by12.87%

Number of lines in readme file

24

increased by9.09%

Dependency changes

• + Added@backstage/config@0.1.15(transitive)

• + Added@backstage/search-common@0.1.3(transitive)

• + Added@backstage/types@0.1.3(transitive)

• + Addedlodash@4.17.21(transitive)

• - Removed@backstage/search-common@0.0.0-nightly-20220923026(transitive)

• - Removed@backstage/types@0.0.0-nightly-20230919021144(transitive)

• Updated@backstage/search-common@^0.1.2