github.com/ctrf-io/github-test-reporter

Publish and View Test Results Reports in Github Actions

Integrate Test Reporting into Your GitHub Actions Workflow

A GitHub test reporting tool that supports all major testing frameworks. Generate, publish and alert your team with detailed test results, including summaries, in-depth reports, failed test analyses, flaky test detection and AI analyses directly within your GitHub Actions CI/CD workflow and Pull Requests.

Choose from a variety of pre-built reports or create custom reports tailored to your project's needs, ensuring that test results are always where you need them.

💚

CTRF tooling is open source and free to use

You can support the project with a follow and a star

Maintained by Matthew Thomas
Contributions are very welcome!
Explore more integrations

Key Features

📊 Comprehensive Test Reports

Access powerful built-in reports including:

• 📈 Historical Test Trends
• 📝 Detailed Test Results
• ❌ Failed Tests Overview
• 🔁 Flaky Tests Analysis

🎨 Custom Report Builder

Create and customize your own test reports with our flexible templating system. Perfect for teams with specific reporting needs or unique project requirements.

🤖 AI-Powered Test Analysis

Get intelligent insights on test failures with our AI-generated reports. Quickly understand why tests failed and how to fix them using leading AI models.

🔌 Universal Framework Support

Compatible with all major testing frameworks through standardized CTRF reports

Report Showcase

Checkout all the built-in reports here

Visual Overview

Table of Contents

• Usage
• Available Inputs
• Pull Requests
• Status Checks
• Build Your Own Report
• Customizing Report Order
• Community Reports
• GitHub Token
• Storing Artifacts
• Filtering
• Integrations
• Generating an AI Report
• Run With NPX
• Report Showcase
• What is CTRF?

Usage

To get started add the following to your workflow file:

- name: Publish Test Report
  uses: ctrf-io/github-test-reporter@v1
  with:
    report-path: './ctrf/*.json'
  if: always()

This will publish the default reports summary-report, failed-report, flaky-report, skipped-report, and test-report to the job summary.

Generate a CTRF report

CTRF reporters are available for most testing frameworks and easy to install.

No CTRF reporter? No problem!

Use the JUnit to CTRF integration or junit-to-ctrf cli to convert a JUnit report to CTRF.

Available Inputs

There are several inputs available

- name: Publish Test Report
  uses: ctrf-io/github-test-reporter@v1
  with:
    # Core Configuration
    report-path: './ctrf/*.json' # Path or glob pattern to the CTRF report JSON file.
    template-path: './templates/custom-summary.hbs' # Path to the Handlebars template for customizing markdown output.

    # Reports - Choose as many as you like. Default is false
    summary-report: true
    github-report: false
    test-report: false
    test-list-report: false
    failed-report: false
    fail-rate-report: false
    flaky-report: false
    flaky-rate-report: false
    failed-folded-report: false
    previous-results-report: false
    insights-report: false
    slowest-report: false
    ai-report: false
    skipped-report: false
    suite-folded-report: false
    suite-list-report: false
    pull-request-report: false
    commit-report: false
    custom-report: false
    community-report: false

    # Behavior Options
    summary: true # Post report to the job summary. Default is true
    pull-request: false # Comment on pull request with report. Default is false
    issue: '' # Issue number to comment on. Works with standard issues and pull-request. Default is no issue
    status-check: false # Create a status check for the workflow. Default is false
    status-check-name: 'Test Reporter Results' # Name of the status check. Default is GitHub Test Reporter Results 
    community-report-name: 'summary-short' # Name of the community report to use. Default is summary-short
    title: '' # Set a custom title to display on the report.
    annotate: true # Add failed test annotations. Default is true
    on-fail-only: false # Add a pull request comment only if tests fail. Default is false
    exit-on-fail: false # Exit the workflow with a failure status if any tests fail. Default is false
    use-suite-name: false # Prefix test names with the suite name for better grouping. Default is false
    update-comment: false # Update existing Pull Request comment. Default is false
    overwrite-comment: false # Overwrite existing Pull Request comment. Default is false
    comment-tag: false # Tag to match Pull Request comment
    write-ctrf-to-file: 'ctrf/ctrf-report.json' # Path to write the processed CTRF report for future processing. Default no write
    upload-artifact: true # Upload to workflow artifact the processed CTRF report for future processing. Default false
    comment-tag: '' # Tag to match Pull Request comment

    # Advanced Options
    artifact-name: 'ctrf-report' # Name of the artifact containing test reports. Default is ctrf-report
    previous-results-max: 10 # Maximum number of previous test results to display in the report. Default is 10
    fetch-previous-results: false # Always fetch previous workflow runs when using custom templates. Default is false
    max-workflow-runs-to-check: 400 # Maximum number of workflow runs to check for previous results. Default is 400
    group-by: 'filePath' # Specify grouping for applicable reports (e.g., suite or file path). Default is filePath
    always-group-by: false # Force grouping by suite or file path for all reports. Default is false
    report-order: 'summary-report,failed-report,flaky-report,skipped-report,test-report' # Comma-separated list of report types to specify the order in which reports should be displayed
    integrations-config: '{}' # JSON configuration for integrations with other developer tools
  if: always()

Only report-path is required.

Pull Requests

You can add a pull request comment by using the pull-request-report input:

- name: Publish Test Report
  uses: ctrf-io/github-test-reporter@v1
  with:
    report-path: './ctrf/*.json'
    pull-request-report: true
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  if: always()

This uses the built-in pull request comment report.

Additionally, you can add any report to a pull request comment by adding the pull-request input:

- name: Publish Test Report
  uses: ctrf-io/github-test-reporter@v1
  with:
    report-path: './ctrf/*.json'
    flaky-rate-report: true
    pull-request: true
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  if: always()

The pull-request input works with all reports, including custom.

requires a GITHUB_TOKEN with pull request write permission.

Note: Special considerations apply to pull requests from forks. See Fork Pull Requests for details.

You can also comment on a specific issue or pull request by using the issue input and providing the issue number ():

- name: Publish Test Report
  uses: ctrf-io/github-test-reporter@v1
  with:
    report-path: './ctrf/*.json'
    issue: '123'
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  if: always()

Requires a GITHUB_TOKEN with issue or pull request write permission.

Comment Management Inputs

--update-comment An existing tagged comment is found, the new report is appended to it. Otherwise, a new comment is created.

--overwrite-comment An existing tagged comment is found, that comment's entire content is replaced with the new report. Otherwise, a new comment is created.

--comment-tag A unique identifier for comments posted. Used to find and update/overwrite existing comments.

For example, the following command creates or updates a comment tagged with the current workflow and job names:

- name: Publish Test Report
  uses: ctrf-io/github-test-reporter@v1
  with:
    report-path: './ctrf/*.json'
    flaky-rate-report: true
    pull-request: true
    update-comment: true
    comment-tag: '${{ github.workflow }}-${{ github.job }}'
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  if: always()

Status Checks

The status-check input creates a status check for the workflow.

- name: Publish Test Report
  uses: ctrf-io/github-test-reporter@v1
  with:
    report-path: './ctrf/*.json'
    status-check: true
    status-check-name: 'GitHub Test Reporter Results'
  if: always()

Requires a GITHUB_TOKEN with status check write permission.

Build Your Own Report

The custom-report input lets you build your own report using a Handlebars template. The template can include any markdown content and leverage data from your CTRF report and GitHub properties, allowing for dynamic and customizable report.

Add the following to your workflow file:

- name: Publish Test Report
  uses: ctrf-io/github-test-reporter@v1
  with:
    report-path: './ctrf/*.json'
    template-path: './templates/custom-report.hbs'
    custom-report: true
  if: always()

For detailed instructions on building your own report, see the documentation.

For inspiration, check out the built-in reports and community reports

Community Reports

We welcome and encourage contributions of community-built reports. Community reports allow users to share custom reports designed for specific use cases.

To submit a community-built report create a Pull Request.

You can see available community built reports

Add the following to your workflow file:

- name: Publish Test Report
  uses: ctrf-io/github-test-reporter@v1
  with:
    report-path: './ctrf/*.json'
    community-report: true
    communty-report-name: summary-short
  if: always()

Customizing Report Order

The GitHub Test Reporter allows you to customize the order in which reports appear in your job summary or pull request comments. This feature gives you complete control over how your test results are presented. To customize the order of reports, use the report-order parameter with a comma-separated list of report types:

- name: Publish Test Report
  uses: ctrf-io/github-test-reporter@v1
  with:
    report-path: './ctrf/*.json'
    summary-report: true
    failed-report: true
    flaky-report: true
    insights-report: true
    test-report: true
    # Order reports with the most important information first
    report-order: 'summary-report,failed-report,flaky-report,insights-report,test-report'
  if: always()

If report-order is not provided, a default order is used.

GitHub Token

previous-results-report, insights-report, flaky-rate-report, fail-rate-report, and slowest-report need a GITHUB_TOKEN:

- name: Publish Test Report
  uses: ctrf-io/github-test-reporter@v1
  with:
    report-path: './ctrf/*.json'
    flaky-rate-report: true
  env:
    GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  if: always()

Storing Artifacts

Some reports require you to store CTRF reports as artifacts, use the upload-artifact input or the actions/upload-artifact@v4 action:

- name: Upload test results
  uses: actions/upload-artifact@v4
  with:
    name: ctrf-report
    path: path-to-your-ctrf-report.json
  if: always()

Filtering

previous-results-report, flaky-rate-report and fail-rate-report filter previous results as follows:

• Runs from the same branch for events of type push, schedule and workflow_dispatch from the same workflow id
• Runs from the same pull request for events of type pull_request from the same workflow id

This ensures that you only see workflow runs that are related to your current branch or pull request

Integrations

CTRF tooling offers seamless developer tool integration, allowing you to combine the GitHub Test Reporter with the following tools:

Integration	Description	Repository
Slack Test Reporter	Send test results and notifications to Slack channels	ctrf-io/slack-test-reporter
Microsoft Teams Test Reporter	Post test results and alerts to Teams channels	ctrf-io/teams-test-reporter
AI Test Reporter	Intelligent test analysis using leading AI models	ctrf-io/ai-test-reporter

For detailed information about configuring and using these integrations, see our Integrations Documentation.

Integrations are currently in beta. Please report any issues to the GitHub Test Reporter repository.

Generating an AI Report

You can generate human-readable AI report for your failed tests using models from the leading AI providers by using the AI Test Reporter integration or the AI Test Reporter directly.

Run With NPX

You can run using npx, see full instructions here

What is CTRF?

A test results report schema that provides a standardized format for JSON test reports.

Consistency Across Tools: Different testing tools and frameworks often produce reports in varied formats. CTRF ensures a uniform structure, making it easier to understand and compare reports, regardless of the testing tool used.

Language and Framework Agnostic: It provides a universal reporting schema that works seamlessly with any programming language and testing framework.

Facilitates Better Analysis: With a standardized format, programatically analyzing test outcomes across multiple platforms becomes more straightforward.

Support Us

If you find this project useful, consider giving it a GitHub star ⭐ It means a lot to us.