playwright-slack-report

playwright-slack-report

Publish your Playwright test results to your favorite Slack channel(s).

🚀 Features

• 💌 Send results your Playwright test results to one or more Slack channels
• 📊 Conditionally send results to Slack channels based on test results
• 📄 Include additional meta information into your test summary e.g. Branch, BuildId etc
• 🧑‍🎨 Define your own custom Slack message layout!

📦 Installation

Run following commands:

yarn

yarn add playwright-slack-report -D

npm

npm install playwright-slack-report -D

Modify your playwright.config.ts file to include the following:

  reporter: [
    [
      "./node_modules/playwright-slack-report/dist/src/SlackReporter.js",
      {
        channels: ["pw-tests", "ci"], // provide one or more Slack channels
        sendResults: "always", // "always" , "on-failure", "off"
      },
    ],
    ["dot"], // other reporters
  ],

Run your tests by providing your SLACK_BOT_USER_OAUTH_TOKEN as an environment variable:

SLACK_BOT_USER_OAUTH_TOKEN=[your Slack bot user OAUTH token] npx playwright test

ℹ️ How do I find my Slack bot oauth token?

You will need to have Slack administrator rights to perform the steps below.

1. Navigate to https://api.slack.com/apps
2. Click the Create New App button and select "From scratch"

1. Input a name for your app and select the target workspace, then click on the Create App button

1. Under the Features menu, select OAuth & Permissions and scroll down to Scopes section

1. Click the Add an OAuth Scope button and select the following scopes:

• chat:write
• chat:write.public
• chat:write.customize

1. Scroll up to the OAuth Tokens for Your Workspace and click the Install to Workspace button

You will be prompted with the message below, click the Allow button

The final step will be to copy the generated Bot User OAuth Token aka SLACK_BOT_USER_OAUTH_TOKEN.

Treat this token as a secret.

⚙️ Configuration

An example advanced configuration is shown below:

  import { generateCustomLayout } from "./my_custom_layout";

  ...

  reporter: [
    [
      "./node_modules/playwright-slack-report/dist/src/SlackReporter.js",
      {
        channels: ["pw-tests", "ci"], // provide one or more Slack channels
        sendResults: "always", // "always" , "on-failure", "off"
      },
      layout: generateCustomLayout,
      maxNumberOfFailuresToShow: 4,
      meta: [
        {
          key: 'BUILD_NUMBER',
          value: '323332-2341',
        },
        {
          key: 'WHATEVER_ENV_VARIABLE',
          value: process.env.SOME_ENV_VARIABLE, // depending on your CI environment, this can be the branch name, build id, etc
        },
        {
          key: 'HTML Results',
          value: '<https://your-build-artifacts.my.company.dev/pw/23887/playwright-report/index.html|📊>',
        },
      ],      
    ],
  ],

channels

An array of Slack channels to post to, atleast one channel is required

sendResults

Can either be "always", "on-failure" or "off", this configuration is required:

• always - will send the results to Slack at completion of the test run
• on-failure - will send the results to Slack only if a test failures are encountered
• off - turns off the reporter, it will not send the results to Slack

layout

A function that returns a layout object, this configuration is optional. See section below for more details.

• meta - an array of meta data to be sent to Slack, this configuration is optional.

maxNumberOfFailuresToShow

Limits the number of failures shown in the Slack message, defaults to 10.

Examples:

...
meta: [
  {
    key: 'Suite',
    value: 'Nightly full regression',
  },
  {
    key: 'GITHUB_REPOSITORY',
    value: 'octocat/telsa-ui',
  },
  {
    key: 'GITHUB_REF',
    value: process.env.GITHUB_REF,
  },
], 
...

🎨 Define your own Slack message custom layout

You can define your own Slack message layout to suit your needs.

Firstly, install the necessary type definitions:

yarn add @slack/types -D

Next, define your layout function. The signature of this function should adhere to example below:

import { Block, KnownBlock } from "@slack/types";
import { SummaryResults } from "playwright-slack-report/dist/src";

const generateCustomLayout = (summaryResults: SummaryResults):Array<KnownBlock | Block> => {
  // your implementation goes here
}

export default generateCustomLayout;

In your, playwright.confing.ts file, add your function into the config.

  import { generateCustomLayout } from "./my_custom_layout";

  ...

  reporter: [
    [
      "./node_modules/playwright-slack-report/dist/src/SlackReporter.js",
      {
        channels: ["pw-tests", "ci"], // provide one or more Slack channels
        sendResults: "always", // "always" , "on-failure", "off"
      },
      layout: generateCustomLayout,
      ...   
    ],
  ],

Pro Tip: You can use the block-kit provided by Slack when creating your layout.

Examples:

Example 1: - very simple summary

import { Block, KnownBlock } from '@slack/types';
import { SummaryResults } from '..';

export default function generateCustomLayoutSimpleExample(
  summaryResults: SummaryResults,
): Array<Block | KnownBlock> {
  return [
    {
      type: 'section',
      text: {
        type: 'mrkdwn',
        text:
          summaryResults.failed === 0
            ? ':tada: All tests passed!'
            : `😭${summaryResults.failed} failure(s) out of ${summaryResults.tests.length} tests`,
      },
    },
  ];
}

Generates the following message in Slack:

Example 2: - very simple summary (with Meta information)

Add the meta block in your config:

  reporter: [
    [
      "./node_modules/playwright-slack-report/dist/src/SlackReporter.js",
      {
        channels: ["demo"],
        sendResults: "always", // "always" , "on-failure", "off",
        layout: generateCustomLayout,
        meta: [
          {
            key: 'EXAMPLE_META_node_env',
            value: process.env.HOME ,
          },
        ],
      },
      
    ],
  ],

Create the function to generate the layout:

import { Block, KnownBlock } from '@slack/types';
import { SummaryResults } from '..';

export default function generateCustomLayoutSimpleMeta(
  summaryResults: SummaryResults,
): Array<Block | KnownBlock> {
  const meta: { type: string; text: { type: string; text: string; }; }[] = [];
  if (summaryResults.meta) {
    for (let i = 0; i < summaryResults.meta.length; i += 1) {
      const { key, value } = summaryResults.meta[i];
      meta.push({
        type: 'section',
        text: {
          type: 'mrkdwn',
          text: `\n*${key}* :\t${value}`,
        },
      });
    }
  }
  return [
    {
      type: 'section',
      text: {
        type: 'mrkdwn',
        text:
          summaryResults.failed === 0
            ? ':tada: All tests passed!'
            : `😭${summaryResults.failed} failure(s) out of ${summaryResults.tests.length} tests`,
      },
    },
    ...meta,
  ];
}

Generates the following message in Slack:

🔑 License

MIT

✨ Contributing

Clone the project and run npm install

Make your changes Run the tests using npm run pw

To execute and test the entire package:

Run npm pack

Create a new playwright project using yarn create playwright Modify the package.json and a local dependancy to the generated tgz file

e.g.

  "dependencies": {
    "playwright-slack-report": "/home/ry/_repo/playwright-slack-report/playwright-slack-report-1.0.3.tgz"
  }

• Execute npm install
• Set your SLACK_BOT_USER_OAUTH_TOKEN environment variable
• Modify the playwright.config.ts as above
• Run the tests using npx playwright text

🐛 Something not working for you?

Feel free to raise a github issue for any bugs or feature requests.