Security News
ESLint is Now Language-Agnostic: Linting JSON, Markdown, and Beyond
ESLint has added JSON and Markdown linting support with new officially-supported plugins, expanding its versatility beyond JavaScript.
@storybook/test-runner
Advanced tools
@storybook/test-runner is a tool designed to run automated tests for Storybook stories. It allows developers to ensure that their UI components behave as expected by running tests in a headless browser environment. This package integrates seamlessly with Storybook, making it easier to test components in isolation.
Running Tests for Stories
This configuration allows you to run tests for all your Storybook stories. By specifying the testRunner as '@storybook/test-runner', you enable the test runner to execute tests for the stories defined in your project.
module.exports = {
stories: ['../src/**/*.stories.@(js|jsx|ts|tsx)'],
addons: ['@storybook/addon-essentials'],
testRunner: '@storybook/test-runner',
};
Custom Test Configuration
You can customize the test setup by using the 'setup' function. The 'getStoryContext' function from '@storybook/test-runner' provides the context of the story being tested, allowing you to perform custom setup actions before running the tests.
const { getStoryContext } = require('@storybook/test-runner');
module.exports = {
async setup(page) {
const context = await getStoryContext(page);
// Custom setup code here
},
};
Integration with CI/CD
You can integrate @storybook/test-runner with your CI/CD pipeline to automate the testing of your Storybook stories. This example shows a GitHub Actions workflow that installs dependencies and runs the Storybook tests.
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Install dependencies
run: npm install
- name: Run Storybook tests
run: npm run test-storybook
Jest is a popular JavaScript testing framework developed by Facebook. It provides a comprehensive solution for testing JavaScript applications, including support for mocking, code coverage, and snapshot testing. While Jest is not specifically designed for Storybook, it can be used to test Storybook stories with additional configuration.
Cypress is an end-to-end testing framework that allows developers to write tests for web applications. It provides a powerful and user-friendly interface for writing and running tests. Cypress can be used to test Storybook stories by navigating to the Storybook instance and interacting with the components.
Puppeteer is a Node library that provides a high-level API to control Chrome or Chromium over the DevTools Protocol. It is often used for web scraping, automated testing, and generating screenshots. Puppeteer can be used to test Storybook stories by programmatically interacting with the Storybook UI.
Storybook test runner turns all of your stories into executable tests.
See the announcement of Interaction Testing with Storybook in detail in this blog post or watch this video to see it in action.
The Storybook test runner uses Jest as a runner, and Playwright as a testing framework. Each one of your .stories
files is transformed into a spec file, and each story becomes a test, which is run in a headless browser.
The test runner is simple in design – it just visits each story from a running Storybook instance and makes sure the component is not failing:
play
function, it verifies whether the story rendered without any errors. This is essentially a smoke test.play
function, it also checks for errors in the play
function and that all assertions passed. This is essentially an interaction test.If there are any failures, the test runner will provide an output with the error, alongside with a link to the failing story, so you can see the error yourself and debug it directly in the browser:
yarn add @storybook/test-runner -D
Jest is a peer dependency. If you don't have it, also install it
yarn add jest@27 -D
yarn add @storybook/addon-interactions @storybook/jest @storybook/testing-library -D
Then add it to your .storybook/main.js
config and enable debugging:
module.exports = {
addons: ['@storybook/addon-interactions'],
features: {
interactionsDebugger: true,
},
};
test-storybook
script to your package.json{
"scripts": {
"test-storybook": "test-storybook"
}
}
yarn storybook
yarn test-storybook
NOTE: The runner assumes that your Storybook is running on port
6006
. If you're running Storybook in another port, either use --url or set the TARGET_URL before running your command like:
yarn test-storybook --url http://localhost:9009 or TARGET_URL=http://localhost:9009 yarn test-storybook
Usage: test-storybook [options]
Options | Description |
---|---|
--help | Output usage information test-storybook --help |
-i , --index-json | Run in index json mode. Automatically detected (requires a compatible Storybook) test-storybook --index-json |
--no-index-json | Disables index json mode test-storybook --no-index-json |
-c , --config-dir [dir-name] | Directory where to load Storybook configurations from test-storybook -c .storybook |
--watch | Watch files for changes and rerun tests related to changed files.test-storybook --watch |
--watchAll | Watch files for changes and rerun all tests when something changes.test-storybook --watchAll |
--coverage | Indicates that test coverage information should be collected and reported in the output test-storybook --coverage |
--url | Define the URL to run tests in. Useful for custom Storybook URLs test-storybook --url http://the-storybook-url-here.com |
--browsers | Define browsers to run tests in. One or multiple of: chromium, firefox, webkit test-storybook --browsers firefox chromium |
--maxWorkers [amount] | Specifies the maximum number of workers the worker-pool will spawn for running tests test-storybook --maxWorkers=2 |
--no-cache | Disable the cache test-storybook --no-cache |
--clearCache | Deletes the Jest cache directory and then exits without running tests test-storybook --clearCache |
--verbose | Display individual test results with the test suite hierarchy test-storybook --verbose |
-u , --updateSnapshot | Use this flag to re-record every snapshot that fails during this test run test-storybook -u |
--eject | Creates a local configuration file to override defaults of the test-runner test-storybook --eject |
The test runner is based on Jest and will accept the CLI options that Jest does, like --watch
, --watchAll
, --maxWorkers
, etc.
The test runner works out of the box, but if you want better control over its configuration, you can run test-storybook --eject
to create a local test-runner-jest.config.js
file in the root folder of your project, which will be used by the test runner.
The test runner uses jest-playwright and you can pass testEnvironmentOptions to further configure it, such as how it's done above to run tests against all browsers instead of just chromium. For this you must eject the test runner configuration.
By default, the test runner assumes that you're running it against a locally served Storybook on port 6006.
If you want to define a target url so it runs against deployed Storybooks, you can do so by passing the TARGET_URL
environment variable:
TARGET_URL=https://the-storybook-url-here.com yarn test-storybook
Or by using the --url
flag:
yarn test-storybook --url https://the-storybook-url-here.com
By default, the test runner transforms your story files into tests. It also supports a secondary "index.json mode" which runs directly against your Storybook's index data, which dependending on your Storybook version is located in a stories.json
or index.json
, a static index of all the stories.
This is particularly useful for running against a deployed storybook because index.json
is guaranteed to be in sync with the Storybook you are testing. In the default, story file-based mode, your local story files may be out of sync – or you might not even have access to the source code. Furthermore, it is not possible to run the test-runner directly against .mdx
stories, and index.json
mode must be used.
To run in index.json
mode, first make sure your Storybook has a v4 index.json
file. You can find it when navigating to:
https://your-storybook-url-here.com/index.json
It should be a JSON file and the first key should be "v": 4
followed by a key called "entries"
containing a map of story IDs to JSON objects.
In Storybok 7.0, index.json
is enabled by default, unless you are using the storiesOf()
syntax, in which case it is not supported.
On Storybook 6.4 and 6.5, to run in index.json
mode, first make sure your Storybook has a file called stories.json
that has "v": 3
, available at:
https://your-storybook-url-here.com/stories.json
If your Storybook does not have a stories.json
file, you can generate one, provided:
storiesOf
storiesTo enable stories.json
in your Storybook, set the buildStoriesJson
feature flag in .storybook/main.js
:
module.exports = {
features: { buildStoriesJson: true },
};
Once you have a valid stories.json
file, your Storybook will be compatible with the "index.json mode".
By default, the test runner will detect whether your Storybook URL is local or remote, and if it is remote, it will run in "index.json mode" automatically. To disable it, you can pass the --no-index-json
flag:
yarn test-storybook --no-index-json
If you are running tests against a local Storybook but for some reason want to run in "index.json mode", you can pass the --index-json
flag:
yarn test-storybook --index-json
NOTE: index.json mode is not compatible with watch mode.
If you want to add the test-runner to CI, there are a couple of ways to do so:
On Github actions, once services like Vercel, Netlify and others do deployment runs, they follow a pattern of emitting a deployment_status
event containing the newly generated URL under deployment_status.target_url
. You can use that URL and set it as TARGET_URL
for the test-runner.
Here's an example of an action to run tests based on that:
name: Storybook Tests
on: deployment_status
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
if: github.event.deployment_status.state == 'success'
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: '14.x'
- name: Install dependencies
run: yarn
- name: Run Storybook tests
run: yarn test-storybook
env:
TARGET_URL: '${{ github.event.deployment_status.target_url }}'
NOTE: If you're running the test-runner against a
TARGET_URL
of a remotely deployed Storybook (e.g. Chromatic), make sure that the URL loads a publicly available Storybook. Does it load correctly when opened in incognito mode on your browser? If your deployed Storybook is private and has authentication layers, the test-runner will hit them and thus not be able to access your stories. If that is the case, use the next option instead.
In order to build and run tests against your Storybook in CI, you might need to use a combination of commands involving the concurrently, http-server and wait-on libraries. Here's a recipe that does the following: Storybook is built and served locally, and once it is ready, the test runner will run against it.
{
"test-storybook:ci": "concurrently -k -s first -n \"SB,TEST\" -c \"magenta,blue\" \"yarn build-storybook --quiet && npx http-server storybook-static --port 6006 --silent\" \"wait-on tcp:6006 && yarn test-storybook\""
}
And then you can essentially run test-storybook:ci
in your CI:
name: Storybook Tests
on: push
jobs:
test:
timeout-minutes: 60
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/setup-node@v2
with:
node-version: '14.x'
- name: Install dependencies
run: yarn
- name: Run Storybook tests
run: yarn test-storybook:ci
NOTE: Building Storybook locally makes it simple to test Storybooks that could be available remotely, but are under authentication layers. If you also deploy your Storybooks somewhere (e.g. Chromatic, Vercel, etc.), the Storybook URL can still be useful with the test-runner. You can pass it to the
REFERENCE_URL
environment variable when running the test-storybook command, and if a story fails, the test-runner will provide a helpful message with the link to the story in your published Storybook instead.
The test runner supports code coverage with the --coverage
flag or STORYBOOK_COLLECT_COVERAGE
environment variable. The pre-requisite is that your components are instrumented using istanbul.
Given that your components' code runs in the context of a real browser, they have to be instrumented so that the test runner is able to collect coverage. This is done by configuring istanbul in your Storybook. You can achieve that in two different ways:
For select frameworks (React, Preact, HTML, Web components and Vue) you can use the @storybook/addon-coverage addon, which will automatically configure the plugin for you.
Install @storybook/addon-coverage
:
yarn add -D @storybook/addon-coverage
And register it in your .storybook/main.js
file:
// .storybook/main.js
module.exports = {
// ...rest of your code here
addons: [
"@storybook/addon-coverage",
]
};
The addon has default options that might suffice to your project, however if you want to customize the addon you can see how it's done here.
Some frameworks or Storybook builders might not automatically accept babel plugins. In that case, you will have to manually configure whatever flavor of istanbul (rollup, vite, webpack loader) your project might require.
After setting up instrumentation, run Storybook then run the test-runner with --coverage
:
yarn test-storybook --coverage
The test runner will report the results in the CLI and generate a coverage/storybook/coverage-storybook.json
file which can be used by nyc
.
If you want to generate reports with different reporters, you can use nyc
and point it to the folder which contains the Storybook coverage file. nyc
is a dependency of the test runner so you will already have it in your project.
Here's an example generating an lcov
report:
npx nyc report --reporter=lcov -t coverage/storybook --report-dir coverage/storybook
This will generate a more detailed, interactive coverage summary that you can access at coverage/storybook/index.html
file which can be explored and will show the coverage in detail:
The nyc
command will respect nyc configuration files if you have them in your project.
If you want certain parts of your code to be deliberately ignored, you can use istanbul parsing hints.
The test runner reports coverage related to the coverage/storybook/coverage-storybook.json
file. This is by design, showing you the coverage which is tested while running Storybook.
Now, you might have other tests (e.g. unit tests) which are not covered in Storybook but are covered when running tests with Jest, which you might also generate coverage files from, for instance. In such cases, if you are using tools like Codecov to automate reporting, the coverage files will be detected automatically and if there are multiple files in the coverage folder, they will be merged automatically.
Alternatively, in case you want to merge coverages from other tools, you should:
1 - move or copy the coverage/storybook/coverage-storybook.json
into coverage/coverage-storybook.json
;
2 - run nyc report
against the coverage
folder.
Here's an example on how to achieve that:
{
"scripts": {
"test:coverage": "jest --coverage",
"test-storybook:coverage": "test-storybook --coverage",
"coverage-report": "cp coverage/storybook/coverage-storybook.json coverage/coverage-storybook.json && nyc report --reporter=html -t coverage --report-dir coverage"
}
}
The test runner renders a story and executes its play function if one exists. However, there are certain behaviors that are not possible to achieve via the play function, which executes in the browser. For example, if you want the test runner to take visual snapshots for you, this is something that is possible via Playwright/Jest, but must be executed in Node.
To enable use cases like visual or DOM snapshots, the test runner exports test hooks that can be overridden globally. These hooks give you access to the test lifecycle before and after the story is rendered.
There are three hooks: setup
, preRender
, and postRender
. setup
executes once before all the tests run. preRender
and postRender
execute within a test before and after a story is rendered.
The render functions are async functions that receive a Playwright Page and a context object with the current story id
, title
, and name
. They are globally settable by @storybook/test-runner
's setPreRender
and setPostRender
APIs.
All three functions can be set up in the configuration file .storybook/test-runner.js
which can optionally export any of these functions.
NOTE: These test hooks are experimental and may be subject to breaking changes. We encourage you to test as much as possible within the story's play function.
The postRender
function provides a Playwright page instance, of which you can use for DOM snapshot testing:
// .storybook/test-runner.js
module.exports = {
async postRender(page, context) {
// the #root element wraps the story
const elementHandler = await page.$('#root');
const innerHTML = await elementHandler.innerHTML();
expect(innerHTML).toMatchSnapshot();
},
};
When running with --stories-json
, tests get generated in a temporary folder and snapshots get stored alongside. You will need to --eject
and configure a custom snapshotResolver
to store them elsewhere, e.g. in your working directory:
const path = require('path');
module.exports = {
resolveSnapshotPath: (testPath, snapshotExtension) =>
path.join(process.cwd(), '__snapshots__', path.basename(testPath) + snapshotExtension),
resolveTestPath: (snapshotFilePath, snapshotExtension) =>
path.join(process.env.TEST_ROOT, path.basename(snapshotFilePath, snapshotExtension)),
testPathForConsistencyCheck: path.join(process.env.TEST_ROOT, 'example.test.js'),
};
Here's a slightly different recipe for image snapshot testing:
// .storybook/test-runner.js
const { toMatchImageSnapshot } = require('jest-image-snapshot');
const customSnapshotsDir = `${process.cwd()}/__snapshots__`;
module.exports = {
setup() {
expect.extend({ toMatchImageSnapshot });
},
async postRender(page, context) {
const image = await page.screenshot();
expect(image).toMatchImageSnapshot({
customSnapshotsDir,
customSnapshotIdentifier: context.id,
});
},
};
There is also an exported TestRunnerConfig
type available for TypeScript users.
To visualize the test lifecycle, consider a simplified version of the test code automatically generated for each story in your Storybook:
it('button--basic', async () => {
// filled in with data for the current story
const context = { id: 'button--basic', title: 'Button', name: 'Basic' };
// playwright page https://playwright.dev/docs/pages
await page.goto(STORYBOOK_URL);
// pre-render hook
if (preRender) await preRender(page, context);
// render the story and run its play function (if applicable)
await page.execute('render', context);
// post-render hook
if (postRender) await postRender(page, context);
});
While running tests using the hooks, you might want to get information from a story, such as the parameters passed to it, or its args. The test runner now provides a getStoryContext
utility function that fetches the story context for the current story:
await getStoryContext(page, context);
You can use it for multiple use cases, and here's an example that combines the story context and accessibility testing:
// .storybook/test-runner.js
const { getStoryContext } = require('@storybook/test-runner');
const { injectAxe, checkA11y } = require('axe-playwright');
module.exports = {
async preRender(page, context) {
await injectAxe(page);
},
async postRender(page, context) {
// Get entire context of a story, including parameters, args, argTypes, etc.
const storyContext = await getStoryContext(page, context);
// Do not test a11y for stories that disable a11y
if (storyContext.parameters?.a11y?.disable) {
return;
}
await checkA11y(page, '#root', {
detailedReport: true,
detailedReportOptions: {
html: true,
},
// pass axe options defined in @storybook/addon-a11y
axeOptions: storyContext.parameters?.a11y?.options
})
},
};
Jest 28 has been released, but unfortunately jest-playwright
is not yet compatible with it, therefore the test-runner is also not compatible. You likely are having an issue that looks like this:
TypeError: Jest: Got error running globalSetup
reason: Class extends value #<Object> is not a constructor or null
As soon as jest-playwright
is compatible, so the test-runner will be too. Please follow this issue for updates.
By default, the test runner truncates error outputs at 1000 characters, and you can check the full output directly in Storybook, in the browser. If you do want to change that limit, however, you can do so by setting the DEBUG_PRINT_LIMIT
environment variable to a number of your choosing, for example, DEBUG_PRINT_LIMIT=5000 yarn test-storybook
.
If your tests are timing out with Timeout - Async callback was not invoked within the 15000 ms timeout specified by jest.setTimeout
, it might be that playwright couldn't handle to test the amount of stories you have in your project. Maybe you have a large amount of stories or your CI has a really low RAM configuration.
In either way, to fix it you should limit the amount of workers that run in parallel by passing the --maxWorkers option to your command:
{
"test-storybook:ci": "concurrently -k -s first -n \"SB,TEST\" -c \"magenta,blue\" \"yarn build-storybook --quiet && npx http-server storybook-static --port 6006 --silent\" \"wait-on tcp:6006 && yarn test-storybook --maxWorkers=2\""
}
There is currently a bug in Jest which means tests cannot be on a separate drive than the project. To work around this you will need to set the TEMP
environment variable to a temporary folder on the same drive as your project. Here's what that would look like on GitHub Actions:
env:
# Workaround for https://github.com/facebook/jest/issues/8536
TEMP: ${{ runner.temp }}
As the test runner is based on playwright, depending on your CI setup you might need to use specific docker images or other configuration. In that case, you can refer to the Playwright CI docs for more information.
Future plans involve adding support for the following features:
v0.5.0 (Mon Jul 18 2022)
FAQs
Test runner for Storybook stories
The npm package @storybook/test-runner receives a total of 667,672 weekly downloads. As such, @storybook/test-runner popularity was classified as popular.
We found that @storybook/test-runner demonstrated a healthy version release cadence and project activity because the last version was released less than a year ago. It has 11 open source maintainers collaborating on the project.
Did you know?
Socket for GitHub automatically highlights issues in each pull request and monitors the health of all your open source dependencies. Discover the contents of your packages and block harmful activity before you install or update your dependencies.
Security News
ESLint has added JSON and Markdown linting support with new officially-supported plugins, expanding its versatility beyond JavaScript.
Security News
Members Hub is conducting large-scale campaigns to artificially boost Discord server metrics, undermining community trust and platform integrity.
Security News
NIST has failed to meet its self-imposed deadline of clearing the NVD's backlog by the end of the fiscal year. Meanwhile, CVE's awaiting analysis have increased by 33% since June.