cypress-fail-fast

What is cypress-fail-fast?

The cypress-fail-fast npm package is a plugin for Cypress that allows you to stop the test suite execution as soon as a test fails. This can save time during development and continuous integration by not running tests that are likely to fail due to a known issue.

What are cypress-fail-fast's main functionalities?

Stop test execution on first failure

This feature allows you to stop the entire test suite execution as soon as a single test fails. You need to add the plugin to your Cypress configuration file.

module.exports = (on, config) => {
  require('cypress-fail-fast/plugin')(on, config);
};

Command-line option to enable/disable fail-fast

You can enable or disable the fail-fast functionality via command-line options. This is useful for different environments or stages in your CI/CD pipeline.

cypress run --env failFast=true

Fail-fast configuration in cypress.json

You can also configure the fail-fast option directly in your cypress.json file, making it easy to manage the setting across different projects or environments.

{
  "env": {
    "failFast": true
  }
}

0

README

Cypress Fail Fast

Enables fail fast in Cypress, skipping the rest of tests on first failure.

It can be configured to skip all remaining tests in current spec file, in current run, or even in parallel runs.

Table of Contents

• Installation
• Limitations and notes
• Configuration
  • Environment variables
  • Configuration by test
  • Configuration examples for usual scenarios
  • Configuration for parallel runs
• Usage with TypeScript

Installation

Add the plugin to devDependencies

npm i --save-dev cypress-fail-fast

Now, depending on your Cypress version, use one of the next methods:

Installation on Cypress 10 and higher

Inside cypress.config.ts file:

import cypressFailFast from "cypress-fail-fast/plugin";

export default defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      cypressFailFast(on, config);
    },
  },
});

In case you are using JavaScript, you may explicit the file extension in some cases:

import cypressFailFast from "cypress-fail-fast/plugin.js"

Note: This example shows how to install the plugin for e2e testing type. Read Cypress configuration docs for further info.

At the top of your support file (usually cypress/support/e2e.js for e2e testing type)

import "cypress-fail-fast";

Installation on Cypress versions lower than 10

Inside cypress/plugins/index.js:

module.exports = (on, config) => {
  require("cypress-fail-fast/plugin")(on, config);
  return config;
};

At the top of cypress/support/index.js:

import "cypress-fail-fast";

From now, if one test fail after its last retry, the rest of tests will be skipped:

Limitations and notes

• All spec files will be loaded, even after entering "skip mode", but every tests and hooks inside them will be skipped.
• The spec strategy does not work in headed mode, because for Cypress events it is like running a single spec, so all remaining tests will be skipped.

Configuration

Environment variables

• FAIL_FAST_STRATEGY: 'spec'|'run'|'parallel'
  • If spec, only remaining tests in current spec file are skipped. This mode only works in "headless" mode.
  • If run, all remaining tests in all spec files are skipped (default value).
  • Use parallel to provide your own callbacks allowing to notify from one run to the others when remaining tests should be skipped.
• FAIL_FAST_ENABLED: boolean = true Allows disabling the "fail-fast" feature globally, but it could be still enabled for specific tests or describes using configuration by test.
• FAIL_FAST_PLUGIN: boolean = true If false, it disables the "fail-fast" feature totally, ignoring even plugin configurations by test.
• FAIL_FAST_BAIL: Number = 1 Enable the skip mode immediately upon n number of failing test suite. Defaults to 1.

Examples

CYPRESS_FAIL_FAST_PLUGIN=false npm run cypress

or set the "env" key in the cypress.json configuration file:

{
  "env":
  {
    "FAIL_FAST_STRATEGY": "run",
    "FAIL_FAST_ENABLED": true,
    "FAIL_FAST_BAIL": 2,
  }
}

Configuration by test

If you want to configure the plugin on a specific test, you can set this by using the failFast property in test configuration. The plugin allows next config values:

• failFast: Configuration for the plugin, containing any of next properties:
  • enabled : Indicates wheter a failure of the current test or children tests (if configuration is applied to a suite) should produce to skip the rest of tests or not. Note that the value defined in this property has priority over the value of the environment variable CYPRESS_FAIL_FAST_ENABLED (but not over CYPRESS_FAIL_FAST_PLUGIN, which disables the plugin totally).

Example

In the next example, tests are configured to "fail-fast" only in case the test with the "sanity test" description fails. If any of the other tests fails, "fail-fast" will not be applied.

describe("All tests", {
  failFast: {
    enabled: false, // Children tests and describes will inherit this configuration
  },
}, () => {
  it("sanity test", {
    failFast: {
      enabled: true, // Overwrite configuration defined in parents
    },
  }, () => {
    // Will skip the rest of tests if this one fails
    expect(true).to.be.true;
  });

  it("second test",() => {
    // Will continue executing tests if this one fails
    expect(true).to.be.true;
  });
});

Configuration examples for usual scenarios

You want to disable "fail-fast" in all specs except one:

Set the FAIL_FAST_ENABLED key in the cypress.json configuration file:

{
  "env":
  {
    "FAIL_FAST_ENABLED": false
  }
}

Enable "fail-fast" in those specs you want using configurations by test:

describe("All tests", { failFast: { enabled: true } }, () => {
  // If any test in this describe fails, the rest of tests and specs will be skipped
});

You want to totally disable "fail-fast" in your local environment:

Set the FAIL_FAST_PLUGIN key in your local cypress.env.json configuration file:

{
  "env":
  {
    "FAIL_FAST_PLUGIN": false
  }
}

Configuration for parallel runs

The plugin configuration supports defining two callbacks that, used in combination, allow to skip tests in one run when other run starts skipping tests also. Where, or how do you store the "flag" that allows to communicate your runs is in your hands, the plugin does not care about it.

To implement it, the plugin can receive an object with extra configuration as third argument when it is registered in the cypress/plugins/index.js file:

• parallelCallbacks: Object containing next properties:
  • onCancel: function() This callback is executed on first test failure that produces the plugin starts skipping tests.
  • isCancelled: function(): boolean If this callback returns true, the plugin skips remaining tests.

These callbacks are executed only when the environment variable FAIL_FAST_STRATEGY is set to parallel.

Here is an example of configuration that would skip tests on many parallel runs when one of them starts skipping tests. It would only work if all parallel runs have access to the folder where the isCancelled flag is being stored as a file (easy to achieve if all of your parallel runs are being executed on Docker images on a same machine, for example). Note that this is only an example, you could also implement it storing the flag in a REST API, etc.

// Example valid for Cypress versions lower than 10. Use config file on Cypress 10

const fs = require("fs");
const path = require("path");

// Flag file is stored in the /cypress folder
const isCancelledFlagFile = path.resolve(__dirname, "..", ".run-is-cancelled");

module.exports = (on, config) => {
  require("cypress-fail-fast/plugin")(on, config, {
    parallelCallbacks: {
      onCancel: () => {
        // Create flag file when the plugin starts skipping tests
        fs.writeFileSync(isCancelledFlagFile, "");
      },
      isCancelled: () => {
        // If any other run has created the file, start skipping tests
        return fs.existsSync(isCancelledFlagFile);
      },
    },
  });

  return config;
};

Note that this example requires to remove the created file when all of the runs have finished, or tests will always be skipped whenever any run starts again. So, the FAIL_FAST_STRATEGY environment variable should be set to parallel only in CI pipelines where the workspace is cleaned on finish, for example.

Usage with TypeScript

If you are using TypeScript in the Cypress plugins file, this plugin includes TypeScript declarations and can be imported like the following:

import cypressFailFast = require("cypress-fail-fast/plugin");

export default (on: Cypress.PluginEvents, config: Cypress.PluginConfigOptions): Cypress.PluginConfigOptions => {
  cypressFailFast(on, config);
  return config;
};

Note: The example above is only valid for Cypress versions lower than 10. Use the configuration file in Cypress 10.

Tests

To ensure the plugin stability, the current major version is being tested with Cypress major versions 9.x, 10.x, 11.x, 12.x and 13.x, and new releases will be published for each new Cypress minor or major releases, updating the E2E tests.

Minor versions used in the E2E tests can be checked in the devDependencies of the package.json files of the E2E tests:

• Cypress v9.x
• Cypress v10.x
• Cypress v11.x
• Cypress v12.x
• Cypress v13.x

Even when current major version may work with previous Cypress versions, it is not currently tested, so, to be sure it works you should use:

• Cypress 8.x may work, but it was tested until cypress-fail-fast 7.0.x
• If you need Cypress 7 support, use cypress-fail-fast 6.x
• If you need Cypress 6 support, use cypress-fail-fast 5.x
• If you need Cypress 5 or lower, use cypress-fail-fast <= 4.x

If you find any issue for a specific Cypress version, please report it at https://github.com/javierbrea/cypress-fail-fast/issues.

Acknowledgements

This plugin has been developed based on the solutions proposed by the community on this Cypress issue, so thanks to all! I hope this plugin can be deprecated soon, as soon as the Cypress team adds native support for this feature. 😃

Contributing

Contributors are welcome. Please read the contributing guidelines and code of conduct.

License

MIT, see LICENSE for details.

Changelog

[7.1.1] - 2024-08-04

Fixed

• fix(#296): Force to set the shouldSkip flag to true when a before hook fails in Cypress 13.

Changed

• docs(#295): Change installation example to ES6 import syntax
• chore(deps): Update devDependencies