Folio
A customizable test framework to build your own test frameworks. Foundation for @playwright/test.
Docs
Fixtures
Base concepts
Folio is based on the concept of the test fixtures. Test fixtures are used to establish environment for each test, giving the test everything it needs and nothing else. Here is how typical test environment setup differs between traditional BDD and the fixture-based one:
Without fixtures
describe('database', () => {
let database;
let table;
beforeAll(async () => {
database = await connect();
});
afterAll(async () => {
await database.dispose();
});
beforeEach(async ()=> {
table = await database.createTable();
});
afterEach(async () => {
await database.dropTable(table);
});
it('create user', () => {
table.insert();
});
it('update user', () => {
table.insert();
table.update();
});
it('delete user', () => {
table.insert();
table.delete();
});
});
With fixtures
import { folio } from 'folio';
const fixtures = folio.extend<{ table: Table }, { database: Database }>();
fixtures.database.init(async ({}, run) => {
const database = await connect();
await run(database);
await database.dispose();
}, { scope: 'worker' });
fixtures.table.init(async ({ database }, run) => {
const table = await database.createTable();
await run(table);
await database.dropTable(table);
});
const { it } = fixtures.build();
it('create user', ({ table }) => {
table.insert();
});
it('update user', ({ table }) => {
table.insert();
table.update();
});
it('delete user', ({ table }) => {
table.insert();
table.delete();
});
You declare exact fixtures that the test needs and the runner initializes them for each test individually. Tests can use any combinations of the fixtures to tailor precise environment they need. You no longer need to wrap tests in describe
s that set up environment, everything is declarative and typed.
There are two types of fixtures: test
and worker
. Test fixtures are set up for each test and worker fixtures are set up for each process that runs test files.
Test fixtures
Test fixtures are set up for each test. Consider the following test file:
import { it, expect } from './hello.folio';
it('hello world', ({ hello, world }) => {
expect(`${hello}, ${world}!`).toBe('Hello, World!');
});
it('hello test', ({ hello, test }) => {
expect(`${hello}, ${test}!`).toBe('Hello, Test!');
});
It uses fixtures hello
, world
and test
that are set up by the framework for each test run.
Here is how test fixtures are declared and defined:
import { folio as base } from 'folio';
export { expect } from 'folio';
type TestFixtures = {
hello: string;
world: string;
test: string;
};
const fixtures = base.extend<TestFixtures>();
fixtures.hello.init(async ({}, run) => {
const value = 'Hello';
await run(value);
});
fixtures.world.init(async ({}, run) => {
await run('World');
});
fixtures.test.init(async ({}, run) => {
await run('Test');
});
const folio = fixtures.build();
export const it = folio.it;
Fixtures can use other fixtures.
...
helloWorld: async ({hello, world}, run) => {
await run(`${hello}, ${world}!`);
}
...
With fixtures, test organization becomes flexible - you can put tests that make sense next to each other based on what they test, not based on the environment they need.
Worker fixtures
Folio uses worker processes to run test files. You can specify the maximum number of workers using --workers
command line option. Similarly to how test fixtures are set up for individual test runs, worker fixtures are set up for each worker process. That's where you can set up services, run servers, etc. Folio will reuse the worker process for as many test files as it can, provided their worker fixtures match and hence environments are identical.
Here is how the test looks:
import { it, expect } from './express.folio';
import fetch from 'node-fetch';
it('fetch 1', async ({ port }) => {
const result = await fetch(`http://localhost:${port}/1`);
expect(await result.text()).toBe('Hello World 1!');
});
it('fetch 2', async ({ port }) => {
const result = await fetch(`http://localhost:${port}/2`);
expect(await result.text()).toBe('Hello World 2!');
});
And here is how fixtures are declared and defined:
import { folio as base } from 'folio';
export { expect } from 'folio';
import express from 'express';
import type { Express } from 'express';
type ExpressWorkerFixtures = {
port: number;
express: Express;
};
const fixtures = base.extend<{}, ExpressWorkerFixtures>();
fixtures.port.init(async ({ testWorkerIndex }, run) => {
await run(3000 + testWorkerIndex);
}, { scope: 'worker' });
fixtures.express.init(async ({ port }, run) => {
const app = express();
app.get('/1', (req, res) => {
res.send('Hello World 1!')
});
app.get('/2', (req, res) => {
res.send('Hello World 2!')
});
let server;
console.log('Starting server...');
await new Promise(f => {
server = app.listen(port, f);
});
console.log('Server ready');
await run(server);
console.log('Stopping server...');
await new Promise(f => server.close(f));
console.log('Server stopped');
}, { scope: 'worker', auto: true });
const folio = fixtures.build();
export const it = folio.it;
Parameters
It is common to run tests in different configurations, for example when running web app tests against multiple browsers or testing two different versions of api endpoint. Folio supports this via parameters - define the parameter and start using it in a test or a fixture.
Consider the following test that uses an API url endpoint:
import { it, expect } from './api.folio';
import fetch from 'node-fetch';
it('fetch 1', async ({ apiUrl }) => {
const result = await fetch(`${apiUrl}/hello`);
expect(await result.text()).toBe('Hello');
});
Here is how to define the api version parameter:
import { folio as base } from 'folio';
export { expect } from 'folio';
const fixtures = base.extend<{}, { apiUrl: string }, { version: string }>();
fixtures.version.initParameter('API version', 'v1');
fixtures.apiUrl.init(async ({ version }, runTest) => {
const server = await startServer();
await runTest(`http://localhost/api/${version}`);
await server.close();
}, { scope: 'worker' });
const folio = fixtures.build();
export const it = folio.it;
In the command line
Given the example above, it is possible to run tests against the specific api version.
TODO: do not assume this is read top-bottom, each section should be self-contained
npx folio tests
npx folio tests -p version=v2
Generating tests
TODO: do not assume this is read top-bottom, each section should be self-contained
It is also possible to run tests against multiple api versions.
folio.generateParametrizedTests('version', ['v1', 'v2', 'v3']);
npx folio tests -p version=v1 -p version=v2