@fluid-internal/mocha-test-setup

@fluid-internal/mocha-test-setup

IMPORTANT: This package is intended strictly as an implementation detail of the Fluid Framework and is not intended for public consumption. We make no stability guarantees regarding its APIs.

This package has a few main purposes:

• Expose/generate a default .mocharc.cjs configuration for running mocha tests, which other packages can extend.
• Map paths for required packages/modules to account for Lerna's dependency hoisting.
• Add mocha beforeAll, beforeEach and afterEach root hook plugins to add some special behavior when we run tests.

Base mocharc.cjs configuration

To leverage the base mocha configuration exposed by this package, first add it as a devDependency to your package.json (the rest of the file has been omitted):

{
	"devDependencies": {
		"@fluid-internal/mocha-test-setup": "version-that-matches-the-rest-of-the-release-group"
	}
}

Then put this in a .mocharc.cjs file at the root of your package:

"use strict";

const getFluidTestMochaConfig = require("@fluid-internal/mocha-test-setup/mocharc-common");

const packageDir = __dirname;
const config = getFluidTestMochaConfig(packageDir);
module.exports = config;

If you need extra configuration, make changes to the config object before assigning it to module.exports.

Mocha will use this file by default if no path to a configuration file is provided explicitly.

The default configuration generated by getFluidTestMochaConfig() looks like this:

{
  "exit": true,
  "recursive": true,
  "require": ["path1", "path2"], // Mapped paths for packages/modules indicated as required when calling getFluidTestMochaConfig()
  "unhandled-rejections": "strict",
};

The configuration will have additional settings if the following environment variables are present:

FLUID_TEST_TIMEOUT

If it exists, the default configuration will also include this:

{
	"timeout": "<value-of-FLUID_TEST_TIMEOUT>"
}

FLUID_TEST_FORBID_ONLY

If it exists, the default configuration will also include this:

{
	"forbid-only": "true"
}

FLUID_LOGGER_PROPS

In case there is a need to override telemetry metrics, one can make use of an environment variable FLUID_LOGGER_PROPS, that to override them during execution time:

FLUID_LOGGER_PROPS='{ "hostName": "Benchmark" }'

Mapping of package paths to account for Lerna hoisting

The way we use Lerna to manage our monorepo, package dependencies are sometimes moved out of the node_modules folder of the package that declares them, and into a node_modules folder at the root of the release group. This applies to external dependencies (i.e. not between packages in our repo) that are present in more than one of our packages, including mocha itself as well as other test runners, which means that their path when they execute might not be the one you'd normally expect. This can cause issues when test scripts point to other dependencies of the package being tested (e.g. with the -r/--require flag) when running tests. This utility maps the paths of packages/modules passed to the getFluidTestMochaConfig() function, to ensure they are imported from whichever location they end up at.

NOTE: once we move to pnpm this functionality might become irrelevant because pnpm keeps all dependencies in the expected paths and creates the necessary symlinks, instead of changing where dependency folders are located.

Considerations for non-package/module arguments

Sometimes you'll want to provide a package/module/path as an argument when executing your tests but outside of the required packages that you can pass to getFluidTestMochaConfig(). For example, the path to a reporter file:

mocha --require @fluid-internal/mocha-test-setup --reporter @fluid-tools/benchmark/dist/MochaMemoryTestReporter.js

Depending on where that file is coming from, you'll need to be careful with how that path is specified. E.g. @fluid-tools/benchmark, while being part of the repository, is not part of any release group, so it will always be downloaded from the public npm feed and treated as an external dependency. Since several packages declare it as a dependency, Lerna will hoist it to node_modules at the root of the release group. In general, the safest solution is to specify it starting with the package name (including scope if applicable). This might break in a few edge cases as long as we use Lerna with hoisting, so we'll have to deal with them individually.

A particular way of specifying a package/module/path (e.g. starting with the package name vs starting with node_modules/) might behave differently locally and in CI builds, so that's another factor to keep in mind.

Special behavior

Supress console output by default

console.log(), console.warn() and console.error() are disabled by default when using this package. They can be re-enabled by setting the FLUID_TEST_VERBOSE environment variable to any non-empty value, e.g. FLUID_TEST_VERBOSE=1.

Telemetry events

A couple of telemetry events fluid:telemetry:Test_start and fluid:telemetry:Test_end will be generated for each test. The Test_end event includes fields with the test's state (passing/failed), its duration, if it timed out. If a FLUID_TEST_VARIANT environment variable exists, the testVariant field in the events will have its value.

Support for a custom logger

This utility supports using a custom logger implementation. This can be useful to send the operational telemetry generated by the FluidFramework while the tests are running, to a particular log sink.

It is expected that the package with the custom logger implementation adds a getTestLogger() => ITelemetryBufferedLogger function to the global object, which this package will call in order to obtain an instance of the custom logger.

If a custom logger implementation will be used when executing tests in a package that leverages this utility, the path to the module with that implementation should not be specified with the --require/-r flag, but through the FLUID_TEST_LOGGER_PKG_PATH environment variable. This package has logic that needs the custom logger to be imported/executed at a particular time, and we ensure that by requiring that the path be provided through that environment variable.