buildkite-test-collector

Buildkite Collectors for JavaScript

Official Buildkite Test Engine collectors for JavaScript test frameworks ✨

⚒ Supported test frameworks: Jest, Jasmine, Mocha, Cypress, Playwright, and more coming soon.

📦 Supported CI systems: Buildkite, GitHub Actions, CircleCI, and others via the BUILDKITE_ANALYTICS_* environment variables.

👉 Installing

• Create a test suite, and copy the API token that it gives you.

• Add the buildkite-test-collector package:

  # If you use npm:
  npm install --save-dev buildkite-test-collector
  
  # or, if you use yarn:
  yarn add --dev buildkite-test-collector
  

• Add the Buildkite test collector to your testing framework:

  Jest

  Update your Jest configuration:
  

  // jest.config.js
  
  // Send results to Test Engine
  reporters: [
    'default',
    'buildkite-test-collector/jest/reporter'
  ],
  
  // Enable column + line capture for Test Engine
  testLocationInResults: true
  

  If you would like to pass in the API token using a custom environment variable, you can do so using the report options.

  // jest.config.js
  
  // Send results to Test Engine
  reporters: [
    "default",
    [
      "buildkite-test-collector/jest/reporter",
      { token: process.env.CUSTOM_ENV_VAR },
    ],
  ];
  

  Vitest

  Update your Vitest configuration:
  

  // vitest.config.js OR vite.config.js
  
  
  test: {
  // Send results to Test Engine
    reporters: [
      'default',
      'buildkite-test-collector/vitest/reporter'
    ],
  
    // Enable column + line capture for Test Engine
    includeTaskLocation: true,
  }
  

  If you would like to pass in the API token using a custom environment variable, you can do so using the report options.

  // vitest.config.js OR vite.config.js
  
  
  test: {
  // Send results to Test Engine
    reporters: [
      'default',
      [
        "buildkite-test-collector/vitest/reporter",
        { token: process.env.CUSTOM_ENV_VAR },
      ],
    ],
  }
  

  Jasmine

  Add the Buildkite reporter to Jasmine:
  

  // SpecHelper.js
  var BuildkiteReporter = require("buildkite-test-collector/jasmine/reporter");
  var buildkiteReporter = new BuildkiteReporter();
  
  jasmine.getEnv().addReporter(buildkiteReporter);
  

  If you would like to pass in the API token using a custom environment variable, you can do so using the report options.

  // SpecHelper.js
  var buildkiteReporter = new BuildkiteReporter(undefined, {
    token: process.env.CUSTOM_ENV_VAR,
  });
  

  Mocha

  Install mocha-multi-reporters in your project:
  

  npm install mocha-multi-reporters --save-dev
  

  and configure it to run your desired reporter and the Buildkite reporter

  // config.json
  {
    "reporterEnabled": "spec, buildkite-test-collector/mocha/reporter"
  }
  

  Now update your test script to use the buildkite reporter via mocha-multi-reporters:

    // package.json
    "scripts": {
      "test": "mocha --reporter mocha-multi-reporters --reporter-options configFile=config.json"
    },
  

  If you would like to pass in the API token using a custom environment variable, you can do so using the report options.

  Since the reporter options are passed in as a json file, we ask you to put the environment variable name as a string value in the config.json, which will be retrieved using dotenv in the mocha reporter.

  // config.json
  {
    "reporterEnabled": "spec, buildkite-test-collector/mocha/reporter",
    "buildkiteTestCollectorMochaReporterReporterOptions": {
      "token_name": "CUSTOM_ENV_VAR_NAME"
    }
  }
  

  Playwright

  Update your Playwright configuration:
  

  // playwright.config.js
  
  // Send results to Test Engine
  reporter: [
    ['line'],
    ['buildkite-test-collector/playwright/reporter']
  ],
  

  If you would like to pass in the API token using a custom environment variable, you can do so using the report options.

  // jest.config.js
  
  // Send results to Test Engine
  reporter: [
    ['line'],
    ['buildkite-test-collector/playwright/reporter', { token: process.env.CUSTOM_ENV_VAR },]
  ],
  

  If you would like to pass execution tags through to Test Engine, then you can use Playwright's tagging syntax as follows:

  test('has tags', { tag: ['@type:feature'] }, ...)
  

  This will be threaded through to Test Engine as an execution tag with key set to type and value set to feature.

  Cypress

  Update your Cypress configuration:
  

   // cypress.config.js
  
   // Send results to Test Engine
  reporter: "buildkite-test-collector/cypress/reporter",
  

  If you would like to pass in the API token using a custom environment variable, you can do so using the reporterOptions.

  // cypress.config.js
  
  // Send results to Test Engine
  reporterOptions: {
   token_name: "CUSTOM_ENV_VAR_NAME"
  }
  

• Run your tests locally:
  

  env BUILDKITE_ANALYTICS_TOKEN=xyz npm test
  

• Add the BUILDKITE_ANALYTICS_TOKEN secret to your CI, push your changes to a branch, and open a pull request 🎉

  git checkout -b add-bk-test-engine
  git commit -am "Add Buildkite Test Engine"
  git push origin add-bk-test-engine
  

📓 Notes

This jest collector uses the onRunComplete hook to report test results to the Buildkite API. This interferes with the --forceExit CLI option and interrupts the request that sends tests results, meaning that no data will be available for your test suite. It's recommended to use --detectOpenHandles to find any hanging process and clean them up and remove the use of --forceExit.

🔍 Debugging

To enable debugging output, set the BUILDKITE_ANALYTICS_DEBUG_ENABLED environment variable to true.

🔜 Roadmap

See the GitHub 'enhancement' issues for planned features. Pull requests are always welcome, and we’ll give you feedback and guidance if you choose to contribute 💚

⚒ Developing

After cloning the repository, install the dependencies:

npm install

And run the tests:

npm test

Useful resources for developing collectors include the Buildkite Test Engine docs and the RSpec and Minitest collectors.

👩‍💻 Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/buildkite/test-collector-javascript

🚀 Releasing

# Version bump the code, tag and push
git switch --create prepare-v1-2-3
npm version --no-git-tag-version v1.2.3
git push

# Open a pull request, get it merged
git switch main
git tag v1.2.3
git push --tags

# Publish to the NPM registry
npm publish

# Create a new GitHub release
open "https://github.com/buildkite/test-collector-javascript/releases"

📜 License

The package is available as open source under the terms of the MIT License.