@storybook/test-runner

What is @storybook/test-runner?

@storybook/test-runner is a tool designed to run automated tests for Storybook stories. It allows developers to ensure that their UI components behave as expected by running tests in a headless browser environment. This package integrates seamlessly with Storybook, making it easier to test components in isolation.

What are @storybook/test-runner's main functionalities?

Running Tests for Stories

This configuration allows you to run tests for all your Storybook stories. By specifying the testRunner as '@storybook/test-runner', you enable the test runner to execute tests for the stories defined in your project.

module.exports = {
  stories: ['../src/**/*.stories.@(js|jsx|ts|tsx)'],
  addons: ['@storybook/addon-essentials'],
  testRunner: '@storybook/test-runner',
};

Custom Test Configuration

You can customize the test setup by using the 'setup' function. The 'getStoryContext' function from '@storybook/test-runner' provides the context of the story being tested, allowing you to perform custom setup actions before running the tests.

const { getStoryContext } = require('@storybook/test-runner');

module.exports = {
  async setup(page) {
    const context = await getStoryContext(page);
    // Custom setup code here
  },
};

Integration with CI/CD

You can integrate @storybook/test-runner with your CI/CD pipeline to automate the testing of your Storybook stories. This example shows a GitHub Actions workflow that installs dependencies and runs the Storybook tests.

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Install dependencies
        run: npm install
      - name: Run Storybook tests
        run: npm run test-storybook

Other packages similar to @storybook/test-runner

jest

Jest is a popular JavaScript testing framework developed by Facebook. It provides a comprehensive solution for testing JavaScript applications, including support for mocking, code coverage, and snapshot testing. While Jest is not specifically designed for Storybook, it can be used to test Storybook stories with additional configuration.

cypress

Cypress is an end-to-end testing framework that allows developers to write tests for web applications. It provides a powerful and user-friendly interface for writing and running tests. Cypress can be used to test Storybook stories by navigating to the Storybook instance and interacting with the components.

puppeteer

Puppeteer is a Node library that provides a high-level API to control Chrome or Chromium over the DevTools Protocol. It is often used for web scraping, automated testing, and generating screenshots. Puppeteer can be used to test Storybook stories by programmatically interacting with the Storybook UI.

README

Storybook Test Runner

Storybook test runner turns all of your stories into executable tests.

Table of Contents

• 1. Features
• 2. Getting Started
• 3. Configuration
• 3. Running against a deployed Storybook
• 4. Running in CI
• 5. Troubleshooting
• 6. Future work

Features

• ⚡️ Zero config setup
• 💨 Smoke test all stories
• ▶️ Test stories with play functions
• 🏃 Test your stories in parallel in a headless browser
• 👷 Get feedback from error with a link directly to the story
• 🐛 Debug them visually and interactively in a live browser with addon-interactions
• 🎭 Powered by Jest and Playwright
• 👀 Watch mode, filters, and the conveniences you'd expect

Getting started

1. Install the test runner and the interactions addon in Storybook:

yarn add @storybook/test-runner -D

1.1 Optional instructions to install the Interactions addon for visual debugging of play functions

yarn add @storybook/addon-interactions @storybook/jest @storybook/testing-library -D

Then add it to your .storybook/main.js config and enable debugging:

module.exports = {
  stories: ['@storybook/addon-interactions'],
  features: {
    interactionsDebugger: true,
  }
};

1. Add a test-storybook script to your package.json

{
  "scripts": {
    "test-storybook": "test-storybook"
  }
}

1. Run Storybook (the test runner runs against a running Storybook instance):

yarn storybook

1. Run the test runner:

yarn test-storybook

NOTE: The runner assumes that your Storybook is running on port 6006. If you're running Storybook in another port, set the TARGET_URL before running your command like:

TARGET_URL=http://localhost:9009 yarn test-storybook

Configuration

The test runner is based on Jest and will accept the CLI options that Jest does, like --watch, --marWorkers, etc.

The test runner works out of the box, but you can override its Jest configuration by adding a test-runner-jest.config.js that sets up your environment in the root folder of your project.

// test-runner-jest.config.js
module.exports = {
  cacheDirectory: 'node_modules/.cache/storybook/test-runner',
  testMatch: ['**/*.stories.[jt]s(x)?'],
  transform: {
    '^.+\\.stories\\.[jt]sx?$': '@storybook/test-runner/playwright/transform',
    '^.+\\.[jt]sx?$': 'babel-jest',
  },
  preset: 'jest-playwright-preset',
  testEnvironment: '@storybook/test-runner/playwright/custom-environment.js',
  testEnvironmentOptions: {
    'jest-playwright': {
      browsers: ['chromium', 'firefox', 'webkit'], // run tests against all browsers
    },
  },
};

The runner uses jest-playwright and you can pass testEnvironmentOptions to further configure it, such as how it's done above to run tests against all browsers instead of just chromium.

Running against a deployed Storybook

By default, the test runner assumes that you're running it against a locally served Storybook. If you want to define a target url so it runs against deployed Storybooks, you can do so by passing the TARGET_URL environment variable:

TARGET_URL=https://the-storybook-url-here.com yarn test-storybook

Running in CI

If you want to add the test-runner to CI, there are a couple of ways to do so:

1. Running against deployed Storybooks on Github Actions deployment

On Github actions, once services like Vercel, Netlify and others do deployment runs, they follow a pattern of emitting a deployment_status event containing the newly generated URL under deployment_status.target_url. You can use that URL and set it as TARGET_URL for the test-runner.

Here's an example of an action to run tests based on that:

name: Storybook Tests
on: deployment_status
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    if: github.event.deployment_status.state == 'success'
    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-node@v2
      with:
        node-version: '14.x'
    - name: Install dependencies
      run: yarn
    - name: Run Storybook tests
      run: yarn test-storybook
      env:
        TARGET_URL: '${{ github.event.deployment_status.target_url }}'

NOTE: If you're running the test-runner against a TARGET_URL of a remotely deployed Storybook (e.g. Chromatic), make sure that the URL loads a publicly available Storybook. Does it load correctly when opened in incognito mode on your browser? If your deployed Storybook is private and has authentication layers, the test-runner will hit them and thus not be able to access your stories. If that is the case, use the next option instead.

2. Running against locally built Storybooks in CI

In order to build and run tests against your Storybook in CI, you might need to use a combination of commands involving the concurrently, http-server and wait-on libraries. Here's a recipe that does the following: Storybook is built and served locally, and once it is ready, the test runner will run against it.

{
  "test-storybook:ci": "concurrently -k -s first -n \"SB,TEST\" -c \"magenta,blue\" \"yarn build-storybook --quiet && npx http-server storybook-static --port 6006 --silent\" \"wait-on tcp:6006 && yarn test-storybook\""
}

And then you can essentially run test-storybook:ci in your CI:

name: Storybook Tests
on: push
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: actions/setup-node@v2
      with:
        node-version: '14.x'
    - name: Install dependencies
      run: yarn
    - name: Run Storybook tests
      run: yarn test-storybook:ci

NOTE: Building Storybook locally makes it simple to test Storybooks that could be available remotely, but are under authentication layers. If you also deploy your Storybooks somewhere (e.g. Chromatic, Vercel, etc.), the Storybook URL can still be useful with the test-runner. You can pass it to the REFERENCE_URL environment variable when running the test-storybook command, and if a story fails, the test-runner will provide a helpful message with the link to the story in your published Storybook instead.

Troubleshooting

The test runner seems flaky and keeps timing out

If your tests are timing out with Timeout - Async callback was not invoked within the 15000 ms timeout specified by jest.setTimeout, it might be that playwright couldn't handle to test the amount of stories you have in your project. Maybe you have a large amount of stories or your CI has a really low RAM configuration.

In either way, to fix it you should limit the amount of workers that run in parallel by passing the --maxWorkers option to your command:

{
  "test-storybook:ci": "concurrently -k -s first -n \"SB,TEST\" -c \"magenta,blue\" \"yarn build-storybook --quiet && npx http-server storybook-static --port 6006 --silent\" \"wait-on tcp:6006 && yarn test-storybook --maxWorkers=2\""
}

Adding the test runner to other CI environments

As the test runner is based on playwright, depending on your CI setup you might need to use specific docker images or other configuration. In that case, you can refer to the Playwright CI docs for more information.

Future work

Future plans involve adding support for the following features:

• 🧪 Custom test functions
• 📄 Run addon reports