slack-ctrf

Slack Test Results Notification

Send Slack message with test results from popular testing frameworks

A Slack test reporting tool that supports all major testing frameworks. Generate, publish and alert your team with detailed test results, including summaries, in-depth reports, failed test analyses, flaky test detection and AI analyses directly to your chosen Slack channel.

💚

CTRF tooling is open source and free to use

You can support the project with a follow and a star

Maintained by Matthew Thomas
Contributions are very welcome!
Explore more integrations

Features

• Send Test Results to Slack: Automatically send test results to a Slack channel.
• Send Flaky Test Details to Slack: Automatically send flaky test details to a Slack channel.
• Send AI Test Summary to Slack: Automatically send AI test summary to a Slack channel.
• Send Failed Test Details to Slack: Automatically send failed test details to a Slack channel.
• Build your own Slack message: Create and customize your own Slack test reports with our flexible templating system.
• Tagging: Tag users, channels and groups in the message.
• Conditional Notifications: Use the --onFailOnly option to send notifications only if tests fail.

Setup

You'll need a CTRF report generated by your testing framework. CTRF reporters are available for most testing frameworks and easy to install.

No CTRF reporter? No problem!

Use junit-to-ctrf to convert a JUnit report to CTRF.

Use an Incoming Webhook

• Go to the Slack API: Incoming Webhooks page.
• Click on "Create a Slack App" or use an existing app.
• Add the "Incoming Webhooks" feature to your app.
• Activate the Incoming Webhook and add a new webhook to your workspace.
• Copy the webhook URL provided.

Set the webhook URL as an environment variable in your shell or CI environment:

export SLACK_WEBHOOK_URL='https://hooks.slack.com/services/your/webhook/url'

Make sure to replace 'https://hooks.slack.com/services/your/webhook/url' with your actual webhook URL.

You might want to store the webhook URL as a secret.

You can also use the --webhook-url option to set the webhook URL.

Use an OAuth Token

• Go to the Slack API: Slack Apps page.
• Follow the instructions to create a Slack App.
• Add the 'chat:write' scope to your app.
• Generate a Bot OAuth Token.
• Invite the bot to your channel.
• Copy the Bot User OAuth Token.

Set the OAuth Token as an environment variable in your shell or CI environment:

export SLACK_OAUTH_TOKEN='xoxb-your/bot/user/oauth/token'

You must also set the channel ID as an environment variable when using the OAuth Token:

export SLACK_CHANNEL_ID='C0123456789'

You can also use the --oauth-token and --channel-id options to set the OAuth Token or channel ID respectively.

Usage

You can use a glob pattern or a single file path to send the test results summary to Slack.

Results

To send the test results summary to Slack:

npx slack-ctrf results /path/to/ctrf-report.json

You can use a glob pattern with multiple files which will be merged together:

npx slack-ctrf results "ctrf/*.json"

AI Summary

To send AI failed test summary to Slack:

npx slack-ctrf ai /path/to/ctrf-report.json

Use --consolidated argumement to send a single message with all AI summaries

npx slack-ctrf ai /path/to/ctrf-report.json --consolidated

See the AI Test Reporter to add AI summaries to your CTRF report

Flaky

To send flaky test report to Slack:

npx slack-ctrf flaky /path/to/ctrf-report.json

Failed

To send failed test report to Slack:

npx slack-ctrf failed /path/to/ctrf-report.json

Use --consolidated argumement to send a single message with all AI summaries

npx slack-ctrf failed /path/to/ctrf-report.json --consolidated

Build your own Slack message

The custom method lets you build your own report using a Handlebars template allowing you to leverage data from your CTRF report to build a dynamic Slack message. The template can be a Block Kit JSON or Slack flavored markdown context block.

To send a message to Slack using a custom Handlebars template:

npx slack-ctrf custom /path/to/ctrf-report.json /path/to/template.hbs --blockkit

npx slack-ctrf custom /path/to/ctrf-report.json /path/to/template.hbs --markdown

The template has access to the entire CTRF report data through the ctrf variable. For example:

Tests: {{ctrf.summary.tests}}

Tests: 100

Block Kit JSON

Provide a Block Kit JSON template to the --blockkit option. See the Slack Block Kit Documentation and Slack Block Kit Builder for help creating a template.

The Block Kit JSON template can include Handlebars syntax to access test report data. For example:

{
  "blocks": [
    {
      "type": "header",
      "text": {
        "type": "plain_text",
        "text": "Test Results: {{ctrf.summary.total}} tests run"
      }
    },
    {
      "type": "section",
      "text": {
        "type": "mrkdwn",
        "text": "*Passed:* {{ctrf.summary.passed}}\n*Failed:* {{ctrf.summary.failed}}"
      }
    }
  ]
}

See the templates directory for examples.

Slack Flavored Markdown

Provide a Slack flavored markdown template to the --markdown option. See the Slack Flavored Markdown Documentation for help creating a template.

See the templates directory for examples.

Handlebars

Handlebars is a simple templating language that lets you insert data into your message in a declarative way. You can use placeholders, conditionals, and loops to dynamically generate content based on your test results.

CTRF Properties

The ctrf object provides access to your test results data. Here are the common properties:

Summary (ctrf.summary)

• tests: Total number of tests
• passed: Number of passed tests
• failed: Number of failed tests
• skipped: Number of skipped tests
• start: Test suite start time
• stop: Test suite end time

Individual Tests (ctrf.tests)

An array of test results, each containing:

• name: Test name
• status: Test status ("passed", "failed", "skipped")
• message: Test output/error message
• duration: Test duration in milliseconds
• retries: Number of retries (for flaky tests)

Example accessing test data:

{{#each ctrf.tests}}
  Test: {{this.name}} - Status: {{this.status}}
{{/each}}

Helpers

The template can include helpers to format the data. There is a wide range of helpers available from custom helpers handlebars-helpers-ctrf, handlebars built-in helpers and slack-ctrf helpers

Send Only on Failures

To send the test results summary to Slack only if there are failed tests, use the --onFailOnly option:

npx slack-ctrf results /path/to/ctrf-file.json --onFailOnly

or using the alias:

npx slack-ctrf results /path/to/ctrf-file.json -f

Custom Notification Title

You can choose a custom title for your notification, use the --title option:

npx slack-ctrf results /path/to/ctrf-file.json --title "Custom Title"

or using the alias:

npx slack-ctrf results /path/to/ctrf-file.json -t "Custom Title"

Add Prefix and Suffix

Add custom markdown text to as a prefix, or suffix to the message:

npx slack-ctrf results /path/to/ctrf-file.json -p "prefix" -s "suffix"

Read about compatible markdown

Tagging a user, channel or group

You can tag users, channels or groups in the message by using the --prefix or --suffix options.

You need to a Slack user ID, channel ID or group ID.

You can find your user ID by:

• Clicking on your profile picture in Slack
• Clicking the three dots (...)
• Selecting "Copy member ID"

Once you have your user ID (which will look something like U01ABC123DE), use that in your command instead of your display name:

npx slack-ctrf results /path/to/ctrf-file.json -s "<@U0123456789> please review the results"

You can also tag a channel by using the # symbol with the channel ID:

npx slack-ctrf results /path/to/ctrf-file.json -s "<#C0123456789> please review the results"

And finally, you can tag a group by using the \!subteam^ symbol with the group ID:

npx slack-ctrf results /path/to/ctrf-file.json -s "<\!subteam^0123456789> please review the results"

Options

• --onFailOnly, -f: Send notification only if there are failed tests.
• --title, -t: Title of the notification.
• --prefix, -p: Custom markdown text to add as a prefix to the message
• --suffix, -s: Custom markdown text to add as a suffix to the message
• --webhook-url, -w: Incoming webhook URL
• --oauth-token, -o: OAuth token
• --channel-id, -ch: Channel ID

Merge reports

You can merge reports if your chosen reporter generates multiple reports through design, parallelisation or otherwise.

If you use a glob pattern, the reports will be merged automatically.

Otherwise, the ctrf-cli package provides a method to merge multiple ctrf json files into a single file.

After executing your tests, use the following command:

npx ctrf merge <directory>

Replace directory with the path to the directory containing the CTRF reports you want to merge.

Programmatic Usage

You can use the package programmatically to send notifications to Slack. To install the package, run:

npm install slack-ctrf

The package exports the following functions:

• sendTestResultsToSlack
• sendFlakyResultsToSlack
• sendFailedResultsToSlack
• sendAISummaryToSlack
• sendCustomMarkdownTemplateToSlack
• sendCustomBlockKitTemplateToSlack