@spotify/backstage-plugin-insights-backend

Spotify Plugins for Backstage: Insights - Backend

Backstage Insights helps you measure Backstage adoption, identify anomalous trends, and understand user behavior.

Prerequisites

1. Complete Spotify plugin bundle setup

This package is a Backstage plugin and is part of the "Spotify Plugins for Backstage" bundle. For you to use this plugin, you must purchase a license key at https://backstage.spotify.com/. After purchasing a license key, follow the instructions on https://backstage.spotify.com/account before continuing.

2. Check your Backstage version

Please ensure your Backstage version matches the supported version shown under the installation instructions on https://backstage.spotify.com/account, and always check compatibility before updating Backstage.

3. Ensure the sign-in resolver provides the user entity associated with the current identity

In order for Insights to properly identify the user entity associated with the currently signed-in user, the SignInResolver must issue an identity token with a sub (subject) claim that is an entity ref to the user entity in the catalog corresponding to the signed-in user. Many authentication providers already provide this behavior, but if using a custom provider, you may need to add this behavior yourself. For examples, see the Google auth sign-in resolver or the GitHub auth sign-in resolver. For information on configuring a sign-in resolver, see the identity resolver docs on backstage.io.

Installation

1. Getting the plugin

Add the Insights backend as dependency to your Backstage backend app:

yarn workspace backend add @spotify/backstage-plugin-insights-backend

2. Integrate the plugin with your backend app

Create a new file, packages/backend/src/plugins/backstageInsights.ts with the following content:

import { Router } from 'express';
import { PluginEnvironment } from '../types';
import { createRouter } from '@spotify/backstage-plugin-insights-backend';

export default async function createPlugin(
  env: PluginEnvironment,
): Promise<Router> {
  return await createRouter({
    ...env,
  });
}

Wire up the Insights backend in packages/backend/src/index.ts. You'll need to import the module from the previous step, create a plugin environment, and add the router to the express app:

 // packages/backend/src/index.ts

 import proxy from './plugins/proxy';
 import techdocs from './plugins/techdocs';
 import search from './plugins/search';
 import permission from './plugins/permission';
+ import backstageInsights from './plugins/backstageInsights';

 /* ... */

 const techdocsEnv = useHotMemoize(module, () => createEnv('techdocs'));
 const searchEnv = useHotMemoize(module, () => createEnv('search'));
 const appEnv = useHotMemoize(module, () => createEnv('app'));
 const permissionEnv = useHotMemoize(module, () => createEnv('permission'));
+ const backstageInsightsEnv = useHotMemoize(module, () => createEnv('backstage-insights'));

 /* ... */

 apiRouter.use('/techdocs', await techdocs(techdocsEnv));
 apiRouter.use('/proxy', await proxy(proxyEnv));
 apiRouter.use('/search', await search(searchEnv));
 apiRouter.use('/permission', await permission(permissionEnv));
+ apiRouter.use('/backstage-insights', await backstageInsights(backstageInsightsEnv));

3. Create an API token for your organization

Backstage Insights is a hosted service where data collection, hosting, and aggregation are handled on your behalf by Spotify. In order to protect your data, an API token is required to securely communicate between your Backstage backend and Spotify's servers.

To create a token, navigate to backstage.spotify.com/account/tokens, signing in as necessary. There, create a token using a name that clearly indicates its purpose and use, like backstage-insights-production-token.

Upon creation, you'll be shown the token value only once. Make note of this value, as you'll need it in the next step.

4. Configure the Spotify Backstage API token

Add the following entry in your app-config.yaml, which will enable communication with Spotify's servers.

spotify:
  apiToken: ${SPOTIFY_BACKSTAGE_API_TOKEN}

Using whatever means you normally use to set environment variables in your Backstage backend process, you'll need to set SPOTIFY_BACKSTAGE_API_TOKEN to the API token you created above. This value should be treated as a secret, so don't check it in!

Additional configuration

Extra entity metadata capture

The Insights backend integrates with the Software Catalog in order to hydrate analytics events with extra entity metadata associated with the event. This makes it possible, for example, to filter or group data based not only on entity identifiers, but also on data like the entity's type.

By default, the following fields are captured on analytics events associated with catalog entities: kind, metadata.namespace, spec.type, spec.lifecycle, and spec.owner.

If you wish to capture extra fields, you can provide extra fields via config. Common fields can be found on the Catalog's YAML File Format docs, though you can also reference any custom metadata you store on entities too.

backstageInsights:
  captureEntityFields:
    - spec.someCustomField

Similarly, the following fields are captured on all analytics events in relation to the signed-in user who performed the action: metadata.namespace, spec.memberOf. Extra metadata about the signed-in user can similarly be configured for capture via config:

backstageInsights:
  captureIdentityFields:
    - spec.profile.someCustomField

Note: in order to avoid collecting personal data, the Insights backend applies redaction logic to hydrated metadata, however the logic is not perfect. It's your responsibility to understand what types of data would be collected by configuring these additional fields.

Identity groups filter

By default, membership groups for users are captured with analytic events, which are then used in the frontend to filter and segment the data on dashboards. However, some groups may not be logical representations of groups that you'd want to segment by. As an example, you may populate these groups from an external source that is used for application management, and therefore have no relation to Backstage usage.

For this reason, we provide a captureIdentityGroupsFilter config option to specify which types of groups you want to capture.

Some examples:

• Only capture specific group types

  backstageInsights:
    captureIdentityGroupsFilter:
      'spec.type':
        - department
        - guild
        - team
  

• Only capture groups annotated with an insights flag

  backstageInsights:
    captureIdentityGroupsFilter:
      'metadata.includeInInsights': true
  

Total users count filter

The Insights frontend will use the Software Catalog to determine the total number of users who can access Backstage. This number is used in combination with usage data in a variety of dashboards and views to give a sense of overall adoption within your organization.

If this number is incorrect (for example, because you retain deactivated users, because your Catalog contains non-technical staff, or because you're rolling out Backstage to parts of your organization gradually in stages), you can supply additional filters to correct the number. These filters take the form of an EntityFilterQuery and can be supplied on the backstageInsights.totalUsersEntityQueryFilter configuration key.

Some examples:

• Only count members of the rnd group:

  backstageInsights:
    totalUsersEntityQueryFilter:
      'relations.memberOf': 'group:default/rnd'
  

• Only count users under the corporate namespace:

  backstageInsights:
    totalUsersEntityQueryFilter:
      'metadata.namespace': corporate
  

For more details, see the EntityFilterQuery documentation on backstage.io.